summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2017-07-12kvm: nVMX: Don't set vmcs12 to "launched" when VMLAUNCH failsJim Mattson
The VMCS launch state is not set to "launched" unless the VMLAUNCH actually succeeds. VMLAUNCH failure includes VM-exits with bit 31 set. Note that this change does not address the general problem that a failure to launch/resume vmcs02 (i.e. vmx->fail) is not handled correctly. Signed-off-by: Jim Mattson <jmattson@google.com> Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
2017-07-12Merge branch 'overlayfs-linus' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs Pull overlayfs updates from Miklos Szeredi: "This work from Amir introduces the inodes index feature, which provides: - hardlinks are not broken on copy up - infrastructure for overlayfs NFS export This also fixes constant st_ino for samefs case for lower hardlinks" * 'overlayfs-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mszeredi/vfs: (33 commits) ovl: mark parent impure and restore timestamp on ovl_link_up() ovl: document copying layers restrictions with inodes index ovl: cleanup orphan index entries ovl: persistent overlay inode nlink for indexed inodes ovl: implement index dir copy up ovl: move copy up lock out ovl: rearrange copy up ovl: add flag for upper in ovl_entry ovl: use struct copy_up_ctx as function argument ovl: base tmpfile in workdir too ovl: factor out ovl_copy_up_inode() helper ovl: extract helper to get temp file in copy up ovl: defer upper dir lock to tempfile link ovl: hash overlay non-dir inodes by copy up origin ovl: cleanup bad and stale index entries on mount ovl: lookup index entry for copy up origin ovl: verify index dir matches upper dir ovl: verify upper root dir matches lower root dir ovl: introduce the inodes index dir feature ovl: generalize ovl_create_workdir() ...
2017-07-12fbdev: make get_fb_unmapped_area depends of !MMUBenjamin Gaignard
Even if CONFIG_FB_PROVIDE_GET_FB_UNMAPPED_AREA flag is selected do not compile and use get_fb_unmapped_area() if CONFIG_MMU is also set. This will avoid mmap errors when compiling multi architectures at same time. Signed-off-by: Benjamin Gaignard <benjamin.gaignard@linaro.org> Reviewed-by: Daniel Vetter <daniel.vetter@ffwll.ch> Cc: Noralf Trønnes <noralf@tronnes.org> Cc: Emil Velikov <emil.l.velikov@gmail.com> Cc: Yannick Fertre <yannick.fertre@st.com> Cc: Eric Engestrom <eric.engestrom@imgtec.com> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
2017-07-12atyfb: hide unused variableArnd Bergmann
The vdisplay variable is now only accessed inside of an #ifdef, producing a harmless warning: drivers/video/fbdev/aty/atyfb_base.c: In function 'aty_var_to_crtc': drivers/video/fbdev/aty/atyfb_base.c:805:19: error: unused variable 'vdisplay' [-Werror=unused-variable] This moves the declaration into the ifdef as well. Fixes: dd7d958ae912 ("video: fbdev: aty: remove useless variable assignments in aty_var_to_crtc()") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Cc: "Gustavo A. R. Silva" <garsilva@embeddedor.com> Signed-off-by: Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>
2017-07-12raid5-ppl: use BIOSET_NEED_BVECS when creating biosetArtur Paszkiewicz
This bioset is used for allocating bios with nr_iovecs > 0 so this flag must be set. Fixes: 011067b05668 ("blk: replace bioset_create_nobvec() with a flags arg to bioset_create()") Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com> Signed-off-by: Shaohua Li <shli@fb.com>
2017-07-12fix a braino in compat_sys_getrlimit()Al Viro
Reported-and-tested-by: Meelis Roos <mroos@linux.ee> Fixes: commit d9e968cb9f84 "getrlimit()/setrlimit(): move compat to native" Signed-off-by: Al Viro <viro@zeniv.linux.org.uk> Acked-by: David S. Miller <davem@davemloft.net> Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2017-07-12brcmfmac: fix possible buffer overflow in brcmf_cfg80211_mgmt_tx()Arend van Spriel
The lower level nl80211 code in cfg80211 ensures that "len" is between 25 and NL80211_ATTR_FRAME (2304). We subtract DOT11_MGMT_HDR_LEN (24) from "len" so thats's max of 2280. However, the action_frame->data[] buffer is only BRCMF_FIL_ACTION_FRAME_SIZE (1800) bytes long so this memcpy() can overflow. memcpy(action_frame->data, &buf[DOT11_MGMT_HDR_LEN], le16_to_cpu(action_frame->len)); Cc: stable@vger.kernel.org # 3.9.x Fixes: 18e2f61db3b70 ("brcmfmac: P2P action frame tx.") Reported-by: "freenerguo(郭大兴)" <freenerguo@tencent.com> Signed-off-by: Arend van Spriel <arend.vanspriel@broadcom.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12net: hns: Bugfix for Tx timeout handling in hns driverLin Yun Sheng
When hns port type is not debug mode, netif_tx_disable is called when there is a tx timeout, which requires system reboot to return to normal state. This patch fix this problem by resetting the net dev. Fixes: b5996f11ea54 ("net: add Hisilicon Network Subsystem basic ethernet support") Signed-off-by: Lin Yun Sheng <linyunsheng@huawei.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12net: ipmr: ipmr_get_table() returns NULLDan Carpenter
The ipmr_get_table() function doesn't return error pointers it returns NULL on error. Fixes: 4f75ba6982bc ("net: ipmr: Add ipmr_rtm_getroute") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Acked-by: Nikolay Aleksandrov <nikolay@cumulusnetworks.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12nfp: freeing the wrong variableDan Carpenter
We accidentally free a NULL pointer and leak the pointer we want to free. Also you can tell from the label name what was intended. :) Fixes: abfcdc1de9bf ("nfp: add a stats handler for flower offloads") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12Merge branch 'mlxsw-spectrum-Various-fixes'David S. Miller
Jiri Pirko says: ==================== mlxsw: spectrum: Various fixes First patch adds a missing rollback in error path. Second patch prevents a use-after-free during IPv4 route replace. Last two patches fix warnings from static checkers. ==================== Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12mlxsw: spectrum_switchdev: Check status of memory allocationIdo Schimmel
We can't rely on kzalloc() always succeeding, so check its return value. Suppresses the following smatch error: mlxsw_sp_switchdev_event() error: potential null dereference 'switchdev_work->fdb_info.addr'. (kzalloc returns null) Fixes: af061378924f ("mlxsw: spectrum_switchdev: Add support for learning FDB through notification") Signed-off-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: Jiri Pirko <jiri@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12mlxsw: spectrum_switchdev: Remove unused variableIdo Schimmel
Commit 10e23eb299fa ("mlxsw: spectrum: Remove support for bypass bridge port attributes/vlan set") removed statements that used 'bridge_vlan', but didn't remove the variable itself resulting in the following warning with W=1: warning: variable ‘bridge_vlan’ set but not used [-Wunused-but-set-variable] Remove the variable and suppress the warning. Fixes: 10e23eb299fa ("mlxsw: spectrum: Remove support for bypass bridge port attributes/vlan set") Signed-off-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: Jiri Pirko <jiri@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12mlxsw: spectrum_router: Fix use-after-free in route replaceIdo Schimmel
While working on IPv6 route replace I realized we can have a use-after-free in IPv4 in case the replaced route is offloaded and the only one using its FIB info. The problem is that fib_table_insert() drops the reference on the FIB info of the replaced routes which is eventually freed via call_rcu(). Since the driver doesn't hold a reference on this FIB info it can cause a use-after-free when it tries to clear the RTNH_F_OFFLOAD flag stored in fi->fib_flags. After running the following commands in a loop for enough time with a KASAN enabled kernel I finally got the below trace. $ ip route add 192.168.50.0/24 via 192.168.200.1 dev enp3s0np3 $ ip route replace 192.168.50.0/24 dev enp3s0np5 $ ip route del 192.168.50.0/24 dev enp3s0np5 BUG: KASAN: use-after-free in mlxsw_sp_fib_entry_offload_unset+0xa7/0x120 [mlxsw_spectrum] Read of size 4 at addr ffff8803717d9820 by task kworker/u4:2/55 [...] ? mlxsw_sp_fib_entry_offload_unset+0xa7/0x120 [mlxsw_spectrum] ? mlxsw_sp_fib_entry_offload_unset+0xa7/0x120 [mlxsw_spectrum] ? mlxsw_sp_router_neighs_update_work+0x1cd0/0x1ce0 [mlxsw_spectrum] ? mlxsw_sp_fib_entry_offload_unset+0xa7/0x120 [mlxsw_spectrum] __asan_load4+0x61/0x80 mlxsw_sp_fib_entry_offload_unset+0xa7/0x120 [mlxsw_spectrum] mlxsw_sp_fib_entry_offload_refresh+0xb6/0x370 [mlxsw_spectrum] mlxsw_sp_router_fib_event_work+0xd1c/0x2780 [mlxsw_spectrum] [...] Freed by task 5131: save_stack_trace+0x16/0x20 save_stack+0x46/0xd0 kasan_slab_free+0x70/0xc0 kfree+0x144/0x570 free_fib_info_rcu+0x2e7/0x410 rcu_process_callbacks+0x4f8/0xe30 __do_softirq+0x1d3/0x9e2 Fix this by taking a reference on the FIB info when creating the nexthop group it represents and drop it when the group is destroyed. Fixes: 599cf8f95f22 ("mlxsw: spectrum_router: Add support for route replace") Signed-off-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: Jiri Pirko <jiri@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12mlxsw: spectrum_router: Add missing rollbackIdo Schimmel
With this patch the error path of mlxsw_sp_nexthop_init() is symmetric with mlxsw_sp_nexthop_fini(). Noticed during code review. Fixes: a8c970142798 ("mlxsw: spectrum_router: Refactor nexthop init routine") Signed-off-by: Ido Schimmel <idosch@mellanox.com> Signed-off-by: Jiri Pirko <jiri@mellanox.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12SPARC64: Fix sun4v DMA panicTushar Dave
64bit DMA only supported on sun4v equipped with ATU IOMMU HW. 'Commit b02c2b0bfd7ae ("sparc: remove arch specific dma_supported implementations")' introduced a code that incorrectly allow dma_supported() to succeed for 64bit dma mask even if system doesn't have ATU IOMMU. This results into panic. Fix it. Reported-by: Meelis Roos <mroos@linux.ee> Signed-off-by: Tushar Dave <tushar.n.dave@oracle.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12perf symbols: Accept zero as the kernel base addressArnaldo Carvalho de Melo
Which is the case in S/390, where symbols were not being resolved because machine__get_kernel_start was only setting machine->kernel_start when the just successfully loaded kernel symtab had its map->start set to !0, when it was left at (1ULL << 63) assuming a partitioning of the address space for user/kernel, which is not the case in S/390 nor in Sparc. So just check if map__load() was successfull and set machine->kernel_start to zero, fixing kernel symbol resolution on S/390. Test performed by Thomas: ---- I like this patch. I have done a new build and removed all my debug output to start from scratch. Without your patch I get this: # Samples: 4 of event 'cpu-clock' # Event count (approx.): 1000000 # # Children Self Command Shared Object Symbol # ........ ........ ....... ................ ........................ 75.00% 0.00% true [unknown] [k] 0x00000000004bedda | ---0x4bedda | |--50.00%--0x42693a | | | --25.00%--0x2a72e0 | 0x2af0ca | 0x3d1003fe4c0 | --25.00%--0x4272bc 0x26fa84 and with your patch (I just rebuilt the perf tool, nothing else and used the same perf.data file as input): # Samples: 4 of event 'cpu-clock' # Event count (approx.): 1000000 # # Children Self Command Shared Object Symbol # ........ ........ ....... .......................... .................................. 75.00% 0.00% true [kernel.vmlinux] [k] pgm_check_handler | ---pgm_check_handler do_dat_exception handle_mm_fault __handle_mm_fault filemap_map_pages | |--25.00%--rcu_read_lock_held | rcu_lockdep_current_cpu_online | 0x3d1003ff4c0 | --25.00%--lock_release Looks good to me.... ---- Reported-and-Tested-by: Thomas-Mich Richter <tmricht@linux.vnet.ibm.com> Cc: Adrian Hunter <adrian.hunter@intel.com> Cc: David Ahern <dsahern@gmail.com> Cc: Hendrik Brueckner <brueckner@linux.vnet.ibm.com> Cc: Jiri Olsa <jolsa@kernel.org> Cc: Namhyung Kim <namhyung@kernel.org> Cc: Wang Nan <wangnan0@huawei.com> Cc: Zvonko Kosic <zvonko.kosic@de.ibm.com> Link: http://lkml.kernel.org/n/tip-dk0n1uzmbe0tbthrpfqlx6bz@git.kernel.org Signed-off-by: Arnaldo Carvalho de Melo <acme@redhat.com>
2017-07-12bfq: dispatch request to prevent queue stalling after the request completionHou Tao
There are mq devices (eg., virtio-blk, nbd and loopback) which don't invoke blk_mq_run_hw_queues() after the completion of a request. If bfq is enabled on these devices and the slice_idle attribute or strict_guarantees attribute is set as zero, it is possible that after a request completion the remaining requests of busy bfq queue will stalled in the bfq schedule until a new request arrives. To fix the scheduler latency problem, we need to check whether or not all issued requests have completed and dispatch more requests to driver if there is no request in driver. The problem can be reproduced by running the following script on a virtio-blk device with nr_hw_queues as 1: #!/bin/sh dev=vdb # mount point for dev mp=/tmp/mnt cd $mp job=strict.job cat <<EOF > $job [global] direct=1 bs=4k size=256M rw=write ioengine=libaio iodepth=128 runtime=5 time_based [1] filename=1.data [2] new_group filename=2.data EOF echo bfq > /sys/block/$dev/queue/scheduler echo 1 > /sys/block/$dev/queue/iosched/strict_guarantees fio $job Signed-off-by: Hou Tao <houtao1@huawei.com> Reviewed-by: Paolo Valente <paolo.valente@linaro.org> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2017-07-12bfq: fix typos in comments about B-WF2Q+ algorithmHou Tao
The start time of eligible entity should be less than or equal to the current virtual time, and the entity in idle tree has a finish time being greater than the current virtual time. Signed-off-by: Hou Tao <houtao1@huawei.com> Reviewed-by: Paolo Valente <paolo.valente@linaro.org> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2017-07-12ftrace: Fix uninitialized variable in match_records()Dan Carpenter
My static checker complains that if "func" is NULL then "clear_filter" is uninitialized. This seems like it could be true, although it's possible something subtle is happening that I haven't seen. kernel/trace/ftrace.c:3844 match_records() error: uninitialized symbol 'clear_filter'. Link: http://lkml.kernel.org/r/20170712073556.h6tkpjcdzjaozozs@mwanda Cc: stable@vger.kernel.org Fixes: f0a3b154bd7 ("ftrace: Clarify code for mod command") Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-07-12ftrace: Remove an unneeded NULL checkDan Carpenter
"func" can't be NULL and it doesn't make sense to check because we've already derefenced it. Link: http://lkml.kernel.org/r/20170712073340.4enzeojeoupuds5a@mwanda Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-07-12cpufreq: schedutil: Fix sugov_start() versus sugov_update_shared() raceVikram Mulukutla
With a shared policy in place, when one of the CPUs in the policy is hotplugged out and then brought back online, sugov_stop() and sugov_start() are called in order. sugov_stop() removes utilization hooks for each CPU in the policy and does nothing else in the for_each_cpu() loop. sugov_start() on the other hand iterates through the CPUs in the policy and re-initializes the per-cpu structure _and_ adds the utilization hook. This implies that the scheduler is allowed to invoke a CPU's utilization update hook when the rest of the per-cpu structures have yet to be re-inited. Apart from some strange values in tracepoints this doesn't cause a problem, but if we do end up accessing a pointer from the per-cpu sugov_cpu structure somewhere in the sugov_update_shared() path, we will likely see crashes since the memset for another CPU in the policy is free to race with sugov_update_shared from the CPU that is ready to go. So let's fix this now to first init all per-cpu structures, and then add the per-cpu utilization update hooks all at once. Signed-off-by: Vikram Mulukutla <markivx@codeaurora.org> Acked-by: Viresh Kumar <viresh.kumar@linaro.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12PM / QoS: return -EINVAL for bogus stringsDan Carpenter
In the current code, if the user accidentally writes a bogus command to this sysfs file, then we set the latency tolerance to an uninitialized variable. Fixes: 2d984ad132a8 (PM / QoS: Introcuce latency tolerance device PM QoS type) Signed-off-by: Dan Carpenter <dan.carpenter@oracle.com> Acked-by: Pavel Machek <pavel@ucw.cz> Cc: 3.15+ <stable@vger.kernel.org> # 3.15+ Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12cpufreq: intel_pstate: Fix ratio setting for min_perf_pctSrinivas Pandruvada
When the minimum performance limit percentage is set to the power-up default, it is possible that minimum performance ratio is off by one. In the set_policy() callback the minimum ratio is calculated by applying global.min_perf_pct to turbo_ratio and rounding up, but the power-up default global.min_perf_pct is already rounded up to the next percent in min_perf_pct_min(). That results in two round up operations, so for the default min_perf_pct one of them is not required. It is better to remove rounding up in min_perf_pct_min() as this matches the displayed min_perf_pct prior to commit c5a2ee7dde89 (cpufreq: intel_pstate: Active mode P-state limits rework) in 4.12. For example on a platform with max turbo ratio of 37 and minimum ratio of 10, the min_perf_pct resulted in 28 with the above commit. Before this commit it was 27 and it will be the same after this change. Fixes: 1a4fe38add8b (cpufreq: intel_pstate: Remove max/min fractions to limit performance) Reported-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com> Signed-off-by: Srinivas Pandruvada <srinivas.pandruvada@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12powerpc/64: Fix atomic64_inc_not_zero() to return an intMichael Ellerman
Although it's not documented anywhere, there is an expectation that atomic64_inc_not_zero() returns a result which fits in an int. This is the behaviour implemented on all arches except powerpc. This has caused at least one bug in practice, in the percpu-refcount code, where the long result from our atomic64_inc_not_zero() was truncated to an int leading to lost references and stuck systems. That was worked around in that code in commit 966d2b04e070 ("percpu-refcount: fix reference leak during percpu-atomic transition"). To the best of my grepping abilities there are no other callers in-tree which truncate the value, but we should fix it anyway. Because the breakage is subtle and potentially very harmful I'm also tagging it for stable. Code generation is largely unaffected because in most cases the callers are just using the result for a test anyway. In particular the case of fget() that was mentioned in commit a6cf7ed5119f ("powerpc/atomic: Implement atomic*_inc_not_zero") generates exactly the same code. Fixes: a6cf7ed5119f ("powerpc/atomic: Implement atomic*_inc_not_zero") Cc: stable@vger.kernel.org # v3.4 Noticed-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-07-12device property: Introduce fwnode_call_bool_op() for ops that return boolSakari Ailus
fwnode_call_int_op() isn't suitable for calling ops that return bool since it effectively causes the result returned to the user to be true when an op hasn't been defined or the fwnode is NULL. Address this by introducing fwnode_call_bool_op() for calling ops that return bool. Fixes: 3708184afc77 "device property: Move FW type specific functionality to FW specific files" Fixes: 2294b3af05e9 "device property: Introduce fwnode_device_is_available()" Reported-by: Dan Carpenter <dan.carpenter@oracle.com> Signed-off-by: Sakari Ailus <sakari.ailus@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12ACPI / x86: Add KIOX000A accelerometer on GPD win to always_present_ids arrayHans de Goede
The GPD win BIOS dated 20170320 has disabled the accelerometer, the drivers sometimes cause crashes under Windows and this is how the manufacturer has solved this :| I see no other way to keep the accelerometer working under Windows then adding it to the always_present_ids array. Signed-off-by: Hans de Goede <hdegoede@redhat.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12ACPI / x86: Add Dell Venue 11 Pro 7130 touchscreen to always_present_idsHans de Goede
The _STA method of the Venue 11 Pro 7130 touchscreen has this ugliness: Method (_STA, 0, NotSerialized) // _STA: Status { If ((SDS1 & One) == One) { If (RST1 == Zero) { Return (0x0F) } ElseIf (RST2 == Zero) { RST2 = One TMRV = Timer } Else { Local0 = ((Timer - TMRV) / 0x2710) If (Local0 > TMRI) { RST2 = Zero RST1 = Zero } } } Else { Return (Zero) } } Whereby RST1 gets set by _SB.PCI0.GFX0.LCD.LCD1._ON, this means that after RST1 has been set first _STA must be called to set TIMER and then after enough time has elapsed _STA must be called twice more, once to clear RST1 and once to finally return 0xf before the touchscreen will show up. Which is just crazy. This commit adds an always_present_ids entry for the SYNA7500 touchscreen ACPI node, together with a DMI match for the Venue 11 Pro 7130, fixing the touchscreen not working on this device. Signed-off-by: Hans de Goede <hdegoede@redhat.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12ACPI / x86: Allow matching always_present_id array entries by DMIHans de Goede
On some x86 systems the DSDT hides APCI devices to work around Windows driver bugs. On one such system the device is even hidden until a certain time after _SB.PCI0.GFX0.LCD.LCD1._ON gets called has passed *and* _STA has been called at least 3 times since. TL;DR: it is a mess. Until now the always_present_id matching was used to force status for a whole class of devices, e.g. always enable PWM1 on CHerry Trail devices. This commit extends the always_present_id matching code to optionally also check for a DMI match so that we can also add system specific quirks to the always_present_id array. Signed-off-by: Hans de Goede <hdegoede@redhat.com> Reviewed-by: Andy Shevchenko <andriy.shevchenko@linux.intel.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12Revert "ACPI / EC: Enable event freeze mode..." to fix a regressionLv Zheng
On Lenovo ThinkPad X1 Carbon - the 5th Generation, enabling an earlier EC event freezing timing causes acpitz-virtual-0 to report a stuck 48C temparature. And with EC firmware revisioned as 1.14, without reverting back to old EC event freezing timing, the fan still blows up after a system resume. This reverts the culprit change so that the regression can be fixed without upgrading the EC firmware. Fixes: d30283057ecd (ACPI / EC: Enable event freeze mode to improve event handling) Link: https://bugzilla.kernel.org/show_bug.cgi?id=191181#c168 Tested-by: Damjan Georgievski <gdamjan@gmail.com> Signed-off-by: Lv Zheng <lv.zheng@intel.com> Cc: 4.9+ <stable@vger.kernel.org> # 4.9+ Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12ACPI / EC: Drop EC noirq hooks to fix a regressionLv Zheng
According to bug reports, although the busy polling mode can make noirq stages execute faster, it causes abnormal fan blowing up after system resume (see the first link below for a video demonstration) on Lenovo ThinkPad X1 Carbon - the 5th Generation. The problem can be fixed by upgrading the EC firmware on that machine. However, many reporters confirm that the problem can be fixed by stopping busy polling during suspend/resume and for some of them upgrading the EC firmware is not an option. For this reason, drop the noirq stage hooks from the EC driver to fix the regression. Fixes: c3a696b6e8f8 (ACPI / EC: Use busy polling mode when GPE is not enabled) Link: https://youtu.be/9NQ9x-Jm99Q Link: https://bugzilla.kernel.org/show_bug.cgi?id=196129 Reported-by: Andreas Lindhe <andreas@lindhe.io> Tested-by: Gjorgji Jankovski <j.gjorgji@gmail.com> Tested-by: Damjan Georgievski <gdamjan@gmail.com> Tested-by: Fernando Chaves <nanochaves@gmail.com> Tested-by: Tomislav Ivek <tomislav.ivek@gmail.com> Tested-by: Denis P. <theoriginal.skullburner@gmail.com> Signed-off-by: Lv Zheng <lv.zheng@intel.com> Cc: All applicable <stable@vger.kernel.org> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12ACPI / irq: Fix return code of acpi_gsi_to_irq()Mark Salter
The function acpi_gsi_to_irq() must return 0 on success as the caller ghes_probe expects an 0 for success. This change also matches x86 implementation. This patch was submitted around 4.5 timeframe but wasn't pushed because it didn't fix a real problem. Now that RAS/GHES patches are in kernel, this fixes an error seen on a Mustang (arm64) platform: GHES: Failed to map GSI to IRQ for generic hardware error source: 2 GHES: probe of GHES.2 failed with error 81 Signed-off-by: Tuan Phan <tphan@apm.com> Signed-off-by: Loc Ho <lho@apm.com> Signed-off-by: Mark Salter <msalter@redhat.com> Acked-by: Lorenzo Pieralisi <lorenzo.pieralisi@arm.com> Signed-off-by: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
2017-07-12crypto: cavium - make several functions staticColin Ian King
The functions cvm_encrypt, cvm_decrypt, cvm_xts_setkey and cvm_enc_dec_init does not need to be in global scope, so make them static. Signed-off-by: Colin Ian King <colin.king@canonical.com> Acked-by: David S. Miller <davem@davemloft.net> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-07-12crypto: chcr - Avoid algo allocation in softirq.Harsh Jain
Thsi patch fixes calling "crypto_alloc_cipher" call in bottom halves. Pre allocate aes cipher required to update Tweak value for XTS. Signed-off-by: Harsh Jain <harsh@chelsio.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-07-12crypto: caam - properly set IV after {en,de}cryptDavid Gstir
Certain cipher modes like CTS expect the IV (req->info) of ablkcipher_request (or equivalently req->iv of skcipher_request) to contain the last ciphertext block when the {en,de}crypt operation is done. This is currently not the case for the CAAM driver which in turn breaks e.g. cts(cbc(aes)) when the CAAM driver is enabled. This patch fixes the CAAM driver to properly set the IV after the {en,de}crypt operation of ablkcipher finishes. This issue was revealed by the changes in the SW CTS mode in commit 0605c41cc53ca ("crypto: cts - Convert to skcipher") Cc: <stable@vger.kernel.org> # 4.8+ Signed-off-by: David Gstir <david@sigma-star.at> Reviewed-by: Horia Geantă <horia.geanta@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-07-12crypto: atmel - only treat EBUSY as transient if backlogGilad Ben-Yossef
The Atmel SHA driver was treating -EBUSY as indication of queueing to backlog without checking that backlog is enabled for the request. Fix it by checking request flags. Cc: <stable@vger.kernel.org> Signed-off-by: Gilad Ben-Yossef <gilad@benyossef.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-07-12NetworkingHerbert Xu
crypto: af_alg - Avoid sock_graft call warning The newly added sock_graft warning triggers in af_alg_accept. It's harmless as we're essentially doing sock->sk = sock->sk. The sock_graft call is actually redundant because all the work it does is subsumed by sock_init_data. However, it was added to placate SELinux as it uses it to initialise its internal state. This patch avoisd the warning by making the SELinux call directly. Reported-by: Linus Torvalds <torvalds@linux-foundation.org> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au> Acked-by: David S. Miller <davem@davemloft.net>
2017-07-12crypto: caam - fix signals handlingHoria Geantă
Driver does not properly handle the case when signals interrupt wait_for_completion_interruptible(): -it does not check for return value -completion structure is allocated on stack; in case a signal interrupts the sleep, it will go out of scope, causing the worker thread (caam_jr_dequeue) to fail when it accesses it wait_for_completion_interruptible() is replaced with uninterruptable wait_for_completion(). We choose to block all signals while waiting for I/O (device executing the split key generation job descriptor) since the alternative - in order to have a deterministic device state - would be to flush the job ring (aborting *all* in-progress jobs). Cc: <stable@vger.kernel.org> Fixes: 045e36780f115 ("crypto: caam - ahash hmac support") Fixes: 4c1ec1f930154 ("crypto: caam - refactor key_gen, sg") Signed-off-by: Horia Geantă <horia.geanta@nxp.com> Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
2017-07-12genirq: Keep chip buslock across irq_request/release_resources()Thomas Gleixner
Moving the irq_request/release_resources() callbacks out of the spinlocked, irq disabled and bus locked region, unearthed an interesting abuse of the irq_bus_lock/irq_bus_sync_unlock() callbacks. The OMAP GPIO driver does merily power management inside of them. The irq_request_resources() callback of this GPIO irqchip calls a function which reads a GPIO register. That read aborts now because the clock of the GPIO block is not magically enabled via the irq_bus_lock() callback. Move the callbacks under the bus lock again to prevent this. In the free_irq() path this requires to drop the bus_lock before calling synchronize_irq() and reaquiring it before calling the irq_release_resources() callback. The bus lock can't be held because: 1) The data which has been changed between bus_lock/un_lock is cached in the irq chip driver private data and needs to go out to the irq chip via the slow bus (usually SPI or I2C) before calling synchronize_irq(). That's the reason why this bus_lock/unlock magic exists in the first place, as you cannot do SPI/I2C transactions while holding desc->lock with interrupts disabled. 2) synchronize_irq() will actually deadlock, if there is a handler on flight. These chips use threaded handlers for obvious reasons, as they allow to do SPI/I2C communication. When the threaded handler returns then bus_lock needs to be taken in irq_finalize_oneshot() as we need to talk to the actual irq chip once more. After that the threaded handler is marked done, which makes synchronize_irq() return. So if we hold bus_lock accross the synchronize_irq() call, the handler cannot mark itself done because it blocks on the bus lock. That in turn makes synchronize_irq() wait forever on the threaded handler to complete.... Add the missing unlock of desc->request_mutex in the error path of __free_irq() and add a bunch of comments to explain the locking and protection rules. Fixes: 46e48e257360 ("genirq: Move irq resource handling out of spinlocked region") Reported-and-tested-by: Sebastian Reichel <sebastian.reichel@collabora.co.uk> Reported-and-tested-by: Tony Lindgren <tony@atomide.com> Reported-by: Pavel Machek <pavel@ucw.cz> Signed-off-by: Thomas Gleixner <tglx@linutronix.de> Not-longer-ranted-at-by: Linus Torvalds <torvalds@linux-foundation.org> Cc: Linus Walleij <linus.walleij@linaro.org> Cc: Grygorii Strashko <grygorii.strashko@ti.com> Cc: Marc Zyngier <marc.zyngier@arm.com>
2017-07-12powerpc: Fix emulation of mfocrf in emulate_step()Anton Blanchard
From POWER4 onwards, mfocrf() only places the specified CR field into the destination GPR, and the rest of it is set to 0. The PowerPC AS from version 3.0 now requires this behaviour. The emulation code currently puts the entire CR into the destination GPR. Fix it. Fixes: 6888199f7fe5 ("[POWERPC] Emulate more instructions in software") Cc: stable@vger.kernel.org # v2.6.22+ Signed-off-by: Anton Blanchard <anton@samba.org> Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-07-12powerpc: Fix emulation of mcrf in emulate_step()Anton Blanchard
The mcrf emulation code was using the CR field number directly as the shift value, without taking into account that CR fields are numbered from 0-7 starting at the high bits. That meant it was looking at the CR fields in the reverse order. Fixes: cf87c3f6b647 ("powerpc: Emulate icbi, mcrf and conditional-trap instructions") Cc: stable@vger.kernel.org # v3.18+ Signed-off-by: Anton Blanchard <anton@samba.org> Acked-by: Naveen N. Rao <naveen.n.rao@linux.vnet.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-07-11Merge git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparcLinus Torvalds
Pull sparc fixes from David Miller: - Fix symbol version generation for assembler on sparc, from Nagarathnam Muthusamy. - Fix compound page handling in gup_huge_pmd(), from Nitin Gupta. * git://git.kernel.org/pub/scm/linux/kernel/git/davem/sparc: sparc64: Fix gup_huge_pmd Adding the type of exported symbols sed regex in Makefile.build requires line break between exported symbols Adding asm-prototypes.h for genksyms to generate crc
2017-07-11samples/bpf: fix a build issueYonghong Song
With latest net-next: ==== clang -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/6.3.1/include -I./arch/x86/include -I./arch/x86/include/generated/uapi -I./arch/x86/include/generated -I./include -I./arch/x86/include/uapi -I./include/uapi -I./include/generated/uapi -include ./include/linux/kconfig.h -Isamples/bpf \ -D__KERNEL__ -D__ASM_SYSREG_H -Wno-unused-value -Wno-pointer-sign \ -Wno-compare-distinct-pointer-types \ -Wno-gnu-variable-sized-type-not-at-end \ -Wno-address-of-packed-member -Wno-tautological-compare \ -Wno-unknown-warning-option \ -O2 -emit-llvm -c samples/bpf/tcp_synrto_kern.c -o -| llc -march=bpf -filetype=obj -o samples/bpf/tcp_synrto_kern.o samples/bpf/tcp_synrto_kern.c:20:10: fatal error: 'bpf_endian.h' file not found ^~~~~~~~~~~~~~ 1 error generated. ==== net has the same issue. Add support for ntohl and htonl in tools/testing/selftests/bpf/bpf_endian.h. Also move bpf_helpers.h from samples/bpf to selftests/bpf and change compiler include logic so that programs in samples/bpf can access the headers in selftests/bpf, but not the other way around. Signed-off-by: Yonghong Song <yhs@fb.com> Acked-by: Daniel Borkmann <daniel@iogearbox.net> Acked-by: Lawrence Brakmo <brakmo@fb.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-11bridge: mdb: fix leak on complete_info ptr on fail pathEduardo Valentin
We currently get the following kmemleak report: unreferenced object 0xffff8800039d9820 (size 32): comm "softirq", pid 0, jiffies 4295212383 (age 792.416s) hex dump (first 32 bytes): 00 0c e0 03 00 88 ff ff ff 02 00 00 00 00 00 00 ................ 00 00 00 01 ff 11 00 02 86 dd 00 00 ff ff ff ff ................ backtrace: [<ffffffff8152b4aa>] kmemleak_alloc+0x4a/0xa0 [<ffffffff811d8ec8>] kmem_cache_alloc_trace+0xb8/0x1c0 [<ffffffffa0389683>] __br_mdb_notify+0x2a3/0x300 [bridge] [<ffffffffa038a0ce>] br_mdb_notify+0x6e/0x70 [bridge] [<ffffffffa0386479>] br_multicast_add_group+0x109/0x150 [bridge] [<ffffffffa0386518>] br_ip6_multicast_add_group+0x58/0x60 [bridge] [<ffffffffa0387fb5>] br_multicast_rcv+0x1d5/0xdb0 [bridge] [<ffffffffa037d7cf>] br_handle_frame_finish+0xcf/0x510 [bridge] [<ffffffffa03a236b>] br_nf_hook_thresh.part.27+0xb/0x10 [br_netfilter] [<ffffffffa03a3738>] br_nf_hook_thresh+0x48/0xb0 [br_netfilter] [<ffffffffa03a3fb9>] br_nf_pre_routing_finish_ipv6+0x109/0x1d0 [br_netfilter] [<ffffffffa03a4400>] br_nf_pre_routing_ipv6+0xd0/0x14c [br_netfilter] [<ffffffffa03a3c27>] br_nf_pre_routing+0x197/0x3d0 [br_netfilter] [<ffffffff814a2952>] nf_iterate+0x52/0x60 [<ffffffff814a29bc>] nf_hook_slow+0x5c/0xb0 [<ffffffffa037ddf4>] br_handle_frame+0x1a4/0x2c0 [bridge] This happens when switchdev_port_obj_add() fails. This patch frees complete_info object in the fail path. Reviewed-by: Vallish Vaidyeshwara <vallish@amazon.com> Signed-off-by: Eduardo Valentin <eduval@amazon.com> Signed-off-by: David S. Miller <davem@davemloft.net>
2017-07-12powerpc/perf: Add POWER9 alternate PM_RUN_CYC and PM_RUN_INST_CMPL eventsAnton Blanchard
Similar to POWER8, POWER9 can count run cycles and run instructions completed on more than one PMU. Signed-off-by: Anton Blanchard <anton@samba.org> Acked-by: Madhavan Srinivasan <maddy@linux.vnet.ibm.com> Signed-off-by: Michael Ellerman <mpe@ellerman.id.au>
2017-07-11ftrace: Hide cached module code for !CONFIG_MODULESArnd Bergmann
When modules are disabled, we get a harmless build warning: kernel/trace/ftrace.c:4051:13: error: 'process_cached_mods' defined but not used [-Werror=unused-function] This adds the same #ifdef around the new code that exists around its caller. Link: http://lkml.kernel.org/r/20170710084413.1820568-1-arnd@arndb.de Fixes: d7fbf8df7ca0 ("ftrace: Implement cached modules tracing on module load") Signed-off-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-07-11tracing: Do note expose stack_trace_filter without DYNAMIC_FTRACESteven Rostedt (VMware)
The "stack_trace_filter" file only makes sense if DYNAMIC_FTRACE is configured in. If it is not, then the user can not filter any functions. Not only that, the open function causes warnings when DYNAMIC_FTRACE is not set. Link: http://lkml.kernel.org/r/20170710110521.600806-1-arnd@arndb.de Reported-by: Arnd Bergmann <arnd@arndb.de> Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-07-11tracing: Update Documentation/trace/ftrace.txtSteven Rostedt (VMware)
The documentation of ftrace.txt has become rather outdated. Bring it closer to reality of todays kernel. Signed-off-by: Steven Rostedt (VMware) <rostedt@goodmis.org>
2017-07-11Merge branch 'for-linus' of git://git.kernel.dk/linux-blockLinus Torvalds
Pull more block updates from Jens Axboe: "This is a followup for block changes, that didn't make the initial pull request. It's a bit of a mixed bag, this contains: - A followup pull request from Sagi for NVMe. Outside of fixups for NVMe, it also includes a series for ensuring that we properly quiesce hardware queues when browsing live tags. - Set of integrity fixes from Dmitry (mostly), fixing various issues for folks using DIF/DIX. - Fix for a bug introduced in cciss, with the req init changes. From Christoph. - Fix for a bug in BFQ, from Paolo. - Two followup fixes for lightnvm/pblk from Javier. - Depth fix from Ming for blk-mq-sched. - Also from Ming, performance fix for mtip32xx that was introduced with the dynamic initialization of commands" * 'for-linus' of git://git.kernel.dk/linux-block: (44 commits) block: call bio_uninit in bio_endio nvmet: avoid unneeded assignment of submit_bio return value nvme-pci: add module parameter for io queue depth nvme-pci: compile warnings in nvme_alloc_host_mem() nvmet_fc: Accept variable pad lengths on Create Association LS nvme_fc/nvmet_fc: revise Create Association descriptor length lightnvm: pblk: remove unnecessary checks lightnvm: pblk: control I/O flow also on tear down cciss: initialize struct scsi_req null_blk: fix error flow for shared tags during module_init block: Fix __blkdev_issue_zeroout loop nvme-rdma: unconditionally recycle the request mr nvme: split nvme_uninit_ctrl into stop and uninit virtio_blk: quiesce/unquiesce live IO when entering PM states mtip32xx: quiesce request queues to make sure no submissions are inflight nbd: quiesce request queues to make sure no submissions are inflight nvme: kick requeue list when requeueing a request instead of when starting the queues nvme-pci: quiesce/unquiesce admin_q instead of start/stop its hw queues nvme-loop: quiesce/unquiesce admin_q instead of start/stop its hw queues nvme-fc: quiesce/unquiesce admin_q instead of start/stop its hw queues ...
2017-07-11Merge tag 'smb3-security-fixes-for-4.13' of git://git.samba.org/sfrench/cifs-2.6Linus Torvalds
Pull cifs fixes and sane default from Steve French: "Upgrade default dialect to more secure SMB3 from older cifs dialect" * tag 'smb3-security-fixes-for-4.13' of git://git.samba.org/sfrench/cifs-2.6: cifs: Clean up unused variables in smb2pdu.c [SMB3] Improve security, move default dialect to SMB3 from old CIFS [SMB3] Remove ifdef since SMB3 (and later) now STRONGLY preferred CIFS: Reconnect expired SMB sessions CIFS: Display SMB2 error codes in the hex format cifs: Use smb 2 - 3 and cifsacl mount options setacl function cifs: prototype declaration and definition to set acl for smb 2 - 3 and cifsacl mount options
able summary='file diffstat' width='100%'> -rw-r--r--drivers/base/devtmpfs.c6
-rw-r--r--drivers/base/firmware_loader/main.c2
-rw-r--r--drivers/base/memory.c101
-rw-r--r--drivers/base/node.c26
-rw-r--r--drivers/base/platform-msi.c3
-rw-r--r--drivers/base/platform.c11
-rw-r--r--drivers/base/power/clock_ops.c2
-rw-r--r--drivers/base/power/domain.c36
-rw-r--r--drivers/base/power/runtime.c123
-rw-r--r--drivers/base/power/wakeup.c17
-rw-r--r--drivers/base/power/wakeup_stats.c4
-rw-r--r--drivers/base/regmap/regmap-debugfs.c1
-rw-r--r--drivers/base/regmap/regmap-irq.c126
-rw-r--r--drivers/base/swnode.c135
-rw-r--r--drivers/base/test/Kconfig2
-rw-r--r--drivers/base/test/Makefile2
-rw-r--r--drivers/base/test/property-entry-test.c61
-rw-r--r--drivers/bcma/driver_mips.c7
-rw-r--r--drivers/block/Kconfig25
-rw-r--r--drivers/block/Makefile2
-rw-r--r--drivers/block/ataflop.c16
-rw-r--r--drivers/block/brd.c20
-rw-r--r--drivers/block/drbd/drbd_int.h2
-rw-r--r--drivers/block/drbd/drbd_interval.c8
-rw-r--r--drivers/block/drbd/drbd_main.c35
-rw-r--r--drivers/block/drbd/drbd_nl.c17
-rw-r--r--drivers/block/drbd/drbd_receiver.c27
-rw-r--r--drivers/block/drbd/drbd_req.c1
-rw-r--r--drivers/block/drbd/drbd_state.c7
-rw-r--r--drivers/block/floppy.c160
-rw-r--r--drivers/block/loop.c1
-rw-r--r--drivers/block/mtip32xx/mtip32xx.c13
-rw-r--r--drivers/block/nbd.c10
-rw-r--r--drivers/block/null_blk/main.c38
-rw-r--r--drivers/block/null_blk/null_blk.h2
-rw-r--r--drivers/block/null_blk/zoned.c1
-rw-r--r--drivers/block/paride/pd.c11
-rw-r--r--drivers/block/pktcdvd.c36
-rw-r--r--drivers/block/rnbd/rnbd-clt-sysfs.c84
-rw-r--r--drivers/block/rnbd/rnbd-clt.c217
-rw-r--r--drivers/block/rnbd/rnbd-clt.h8
-rw-r--r--drivers/block/rnbd/rnbd-srv-sysfs.c5
-rw-r--r--drivers/block/rnbd/rnbd-srv.c71
-rw-r--r--drivers/block/rnbd/rnbd-srv.h3
-rw-r--r--drivers/block/rsxx/core.c3
-rw-r--r--drivers/block/swim.c2
-rw-r--r--drivers/block/swim3.c34
-rw-r--r--drivers/block/umem.c1127
-rw-r--r--drivers/block/umem.h132
-rw-r--r--drivers/block/xen-blkback/blkback.c2
-rw-r--r--drivers/block/xen-blkback/common.h1
-rw-r--r--drivers/block/xen-blkback/xenbus.c38
-rw-r--r--drivers/block/xen-blkfront.c8
-rw-r--r--drivers/block/xsysace.c1273
-rw-r--r--drivers/block/zram/zram_drv.c17
-rw-r--r--drivers/bluetooth/Kconfig10
-rw-r--r--drivers/bluetooth/Makefile2
-rw-r--r--drivers/bluetooth/btintel.c232
-rw-r--r--drivers/bluetooth/btintel.h19
-rw-r--r--drivers/bluetooth/btrsi.c1
-rw-r--r--drivers/bluetooth/btusb.c415
-rw-r--r--drivers/bluetooth/hci_bcm.c19
-rw-r--r--drivers/bluetooth/hci_intel.c7
-rw-r--r--drivers/bluetooth/hci_ldisc.c1
-rw-r--r--drivers/bluetooth/hci_qca.c17
-rw-r--r--drivers/bluetooth/virtio_bt.c401
-rw-r--r--drivers/bus/mhi/core/boot.c64
-rw-r--r--drivers/bus/mhi/core/debugfs.c2
-rw-r--r--drivers/bus/mhi/core/init.c72
-rw-r--r--drivers/bus/mhi/core/internal.h20
-rw-r--r--drivers/bus/mhi/core/main.c416
-rw-r--r--drivers/bus/mhi/core/pm.c119
-rw-r--r--drivers/bus/mhi/pci_generic.c330
-rw-r--r--drivers/bus/moxtet.c4
-rw-r--r--drivers/bus/mvebu-mbus.c2
-rw-r--r--drivers/bus/omap_l3_noc.c4
-rw-r--r--drivers/bus/qcom-ebi2.c4
-rw-r--r--drivers/bus/ti-sysc.c91
-rw-r--r--drivers/cdrom/gdrom.c5
-rw-r--r--drivers/char/Kconfig10
-rw-r--r--drivers/char/agp/Kconfig2
-rw-r--r--drivers/char/applicom.c5
-rw-r--r--drivers/char/hw_random/ba431-rng.c16
-rw-r--r--drivers/char/hw_random/bcm2835-rng.c27
-rw-r--r--drivers/char/hw_random/cctrng.c20
-rw-r--r--drivers/char/hw_random/core.c2
-rw-r--r--drivers/char/hw_random/intel-rng.c2
-rw-r--r--drivers/char/hw_random/omap-rng.c14
-rw-r--r--drivers/char/hw_random/pic32-rng.c3
-rw-r--r--drivers/char/hw_random/pseries-rng.c3
-rw-r--r--drivers/char/hw_random/xiphera-trng.c4
-rw-r--r--drivers/char/ipmi/ipmi_msghandler.c60
-rw-r--r--drivers/char/ipmi/ipmi_plat_data.c2
-rw-r--r--drivers/char/ipmi/ipmi_si.h8
-rw-r--r--drivers/char/ipmi/ipmi_si_hardcode.c73
-rw-r--r--drivers/char/ipmi/ipmi_si_hotmod.c24
-rw-r--r--drivers/char/ipmi/ipmi_si_intf.c38
-rw-r--r--drivers/char/ipmi/ipmi_si_pci.c22
-rw-r--r--drivers/char/ipmi/ipmi_si_platform.c95
-rw-r--r--drivers/char/ipmi/ipmi_ssif.c81
-rw-r--r--drivers/char/ipmi/kcs_bmc_aspeed.c27
-rw-r--r--drivers/char/lp.c4
-rw-r--r--drivers/char/mem.c231
-rw-r--r--drivers/char/mwave/tp3780i.c6
-rw-r--r--drivers/char/mwave/tp3780i.h2
-rw-r--r--drivers/char/pcmcia/synclink_cs.c2
-rw-r--r--drivers/char/random.c21
-rw-r--r--drivers/char/toshiba.c1
-rw-r--r--drivers/char/tpm/eventlog/acpi.c33
-rw-r--r--drivers/char/tpm/eventlog/common.c3
-rw-r--r--drivers/char/tpm/eventlog/efi.c29
-rw-r--r--drivers/char/tpm/tpm2-cmd.c1
-rw-r--r--drivers/char/tpm/tpm_ibmvtpm.c4
-rw-r--r--drivers/char/tpm/tpm_tis_core.c22
-rw-r--r--drivers/char/tpm/tpm_tis_i2c_cr50.c1
-rw-r--r--drivers/char/ttyprintk.c11
-rw-r--r--drivers/char/virtio_console.c23
-rw-r--r--drivers/clk/Kconfig2
-rw-r--r--drivers/clk/Makefile5
-rw-r--r--drivers/clk/at91/sama7g5.c6
-rw-r--r--drivers/clk/bcm/clk-raspberrypi.c2
-rw-r--r--drivers/clk/clk-ast2600.c4
-rw-r--r--drivers/clk/clk-bd718x7.c9
-rw-r--r--drivers/clk/clk-fixed-factor.c9
-rw-r--r--drivers/clk/clk-mux.c35
-rw-r--r--drivers/clk/clk-scmi.c28
-rw-r--r--drivers/clk/clk.c78
-rw-r--r--drivers/clk/imx/clk-imx25.c12
-rw-r--r--drivers/clk/imx/clk-imx27.c13
-rw-r--r--drivers/clk/imx/clk-imx35.c10
-rw-r--r--drivers/clk/imx/clk-imx5.c30
-rw-r--r--drivers/clk/imx/clk-imx6q.c16
-rw-r--r--drivers/clk/imx/clk-imx6sl.c16
-rw-r--r--drivers/clk/imx/clk-imx6sll.c24
-rw-r--r--drivers/clk/imx/clk-imx6sx.c16
-rw-r--r--drivers/clk/imx/clk-imx7d.c22
-rw-r--r--drivers/clk/imx/clk-imx7ulp.c31
-rw-r--r--drivers/clk/imx/clk-imx8mm.c18
-rw-r--r--drivers/clk/imx/clk-imx8mn.c18
-rw-r--r--drivers/clk/imx/clk-imx8mp.c32
-rw-r--r--drivers/clk/imx/clk-imx8mq.c22
-rw-r--r--drivers/clk/imx/clk-lpcg-scu.c1
-rw-r--r--drivers/clk/imx/clk-scu.c1
-rw-r--r--drivers/clk/imx/clk.c41
-rw-r--r--drivers/clk/imx/clk.h4
-rw-r--r--drivers/clk/keystone/sci-clk.c4
-rw-r--r--drivers/clk/mvebu/armada-37xx-periph.c83
-rw-r--r--drivers/clk/qcom/a53-pll.c1
-rw-r--r--drivers/clk/qcom/a7-pll.c1
-rw-r--r--drivers/clk/qcom/apss-ipq-pll.c1
-rw-r--r--drivers/clk/qcom/camcc-sc7180.c50
-rw-r--r--drivers/clk/qcom/clk-rcg2.c17
-rw-r--r--drivers/clk/qcom/clk-rpmh.c9
-rw-r--r--drivers/clk/qcom/dispcc-sc7180.c70
-rw-r--r--drivers/clk/qcom/dispcc-sdm845.c217
-rw-r--r--drivers/clk/qcom/dispcc-sm8250.c124
-rw-r--r--drivers/clk/qcom/gcc-sc7180.c200
-rw-r--r--drivers/clk/qcom/gcc-sc7280.c248
-rw-r--r--drivers/clk/qcom/gcc-sc8180x.c122
-rw-r--r--drivers/clk/qcom/gcc-sdm845.c650
-rw-r--r--drivers/clk/qcom/gcc-sdx55.c8
-rw-r--r--drivers/clk/qcom/gcc-sm8150.c100
-rw-r--r--drivers/clk/qcom/gcc-sm8250.c345
-rw-r--r--drivers/clk/qcom/gcc-sm8350.c418
-rw-r--r--drivers/clk/qcom/gpucc-sc7180.c3
-rw-r--r--drivers/clk/qcom/gpucc-sdm845.c45
-rw-r--r--drivers/clk/qcom/gpucc-sm8150.c9
-rw-r--r--drivers/clk/qcom/gpucc-sm8250.c9
-rw-r--r--drivers/clk/qcom/videocc-sc7180.c4
-rw-r--r--drivers/clk/qcom/videocc-sdm845.c55
-rw-r--r--drivers/clk/qcom/videocc-sm8150.c20
-rw-r--r--drivers/clk/qcom/videocc-sm8250.c30
-rw-r--r--drivers/clk/ralink/Kconfig11
-rw-r--r--drivers/clk/ralink/Makefile2
-rw-r--r--drivers/clk/ralink/clk-mt7621.c495
-rw-r--r--drivers/clk/renesas/clk-div6.c3
-rw-r--r--drivers/clk/renesas/clk-mstp.c2
-rw-r--r--drivers/clk/renesas/r8a7795-cpg-mssr.c6
-rw-r--r--drivers/clk/renesas/r8a77965-cpg-mssr.c1
-rw-r--r--drivers/clk/renesas/r8a77990-cpg-mssr.c1
-rw-r--r--drivers/clk/renesas/r8a779a0-cpg-mssr.c11
-rw-r--r--drivers/clk/renesas/r9a06g032-clocks.c12
-rw-r--r--drivers/clk/renesas/rcar-cpg-lib.c2
-rw-r--r--drivers/clk/renesas/rcar-gen2-cpg.c3
-rw-r--r--drivers/clk/renesas/rcar-gen3-cpg.c2
-rw-r--r--drivers/clk/renesas/rcar-usb2-clock-sel.c5
-rw-r--r--drivers/clk/renesas/renesas-cpg-mssr.c2
-rw-r--r--drivers/clk/rockchip/Kconfig29
-rw-r--r--drivers/clk/rockchip/Makefile1
-rw-r--r--drivers/clk/rockchip/clk-cpu.c53
-rw-r--r--drivers/clk/rockchip/clk-px30.c7
-rw-r--r--drivers/clk/rockchip/clk-rk3036.c7
-rw-r--r--drivers/clk/rockchip/clk-rk3128.c7
-rw-r--r--drivers/clk/rockchip/clk-rk3188.c14
-rw-r--r--drivers/clk/rockchip/clk-rk3228.c7
-rw-r--r--drivers/clk/rockchip/clk-rk3288.c7
-rw-r--r--drivers/clk/rockchip/clk-rk3308.c7
-rw-r--r--drivers/clk/rockchip/clk-rk3328.c7
-rw-r--r--drivers/clk/rockchip/clk-rk3368.c14
-rw-r--r--drivers/clk/rockchip/clk-rk3399.c15
-rw-r--r--drivers/clk/rockchip/clk-rk3568.c1725
-rw-r--r--drivers/clk/rockchip/clk-rv1108.c7
-rw-r--r--drivers/clk/rockchip/clk.h54
-rw-r--r--drivers/clk/samsung/clk-exynos4412-isp.c4
-rw-r--r--drivers/clk/samsung/clk-exynos7.c7
-rw-r--r--drivers/clk/samsung/clk-s5pv210-audss.c4
-rw-r--r--drivers/clk/sifive/Kconfig2
-rw-r--r--drivers/clk/sifive/fu740-prci.c11
-rw-r--r--drivers/clk/sifive/fu740-prci.h2
-rw-r--r--drivers/clk/sifive/sifive-prci.c54
-rw-r--r--drivers/clk/sifive/sifive-prci.h13
-rw-r--r--drivers/clk/socfpga/Kconfig19
-rw-r--r--drivers/clk/socfpga/Makefile11
-rw-r--r--drivers/clk/socfpga/clk-agilex.c117
-rw-r--r--drivers/clk/socfpga/clk-gate-a10.c9
-rw-r--r--drivers/clk/socfpga/clk-gate-s10.c17
-rw-r--r--drivers/clk/socfpga/clk-gate.c13
-rw-r--r--drivers/clk/socfpga/clk-periph-a10.c11
-rw-r--r--drivers/clk/socfpga/clk-periph-s10.c42
-rw-r--r--drivers/clk/socfpga/clk-periph.c8
-rw-r--r--drivers/clk/socfpga/clk-pll-a10.c12
-rw-r--r--drivers/clk/socfpga/clk-pll-s10.c45
-rw-r--r--drivers/clk/socfpga/clk-pll.c18
-rw-r--r--drivers/clk/socfpga/clk-s10.c68
-rw-r--r--drivers/clk/socfpga/stratix10-clk.h24
-rw-r--r--drivers/clk/sunxi-ng/ccu-sun8i-v3s.c33
-rw-r--r--drivers/clk/sunxi/clk-sun6i-ar100.c2
-rw-r--r--drivers/clk/sunxi/clk-sun9i-core.c8
-rw-r--r--drivers/clk/sunxi/clk-usb.c2
-rw-r--r--drivers/clk/tegra/clk-pll.c12
-rw-r--r--drivers/clk/tegra/clk-tegra210.c53
-rw-r--r--drivers/clk/ti/clk-54xx.c2
-rw-r--r--drivers/clk/uniphier/clk-uniphier-mux.c4
-rw-r--r--drivers/clk/zynqmp/Kconfig4
-rw-r--r--drivers/clk/zynqmp/pll.c24
-rw-r--r--drivers/clocksource/arm_arch_timer.c83
-rw-r--r--drivers/clocksource/clksrc-dbx500-prcmu.c8
-rw-r--r--drivers/clocksource/dw_apb_timer_of.c28
-rw-r--r--drivers/clocksource/hyperv_timer.c251
-rw-r--r--drivers/clocksource/ingenic-ost.c9
-rw-r--r--drivers/clocksource/ingenic-timer.c2
-rw-r--r--drivers/clocksource/sh_cmt.c5
-rw-r--r--drivers/clocksource/timer-atmel-tcb.c4
-rw-r--r--drivers/clocksource/timer-fsl-ftm.c2
-rw-r--r--drivers/clocksource/timer-microchip-pit64b.c2
-rw-r--r--drivers/clocksource/timer-npcm7xx.c1
-rw-r--r--drivers/clocksource/timer-of.c4
-rw-r--r--drivers/clocksource/timer-pistachio.c4
-rw-r--r--drivers/clocksource/timer-ti-dm-systimer.c157
-rw-r--r--drivers/clocksource/timer-vf-pit.c2
-rw-r--r--drivers/comedi/Kconfig (renamed from drivers/staging/comedi/Kconfig)33
-rw-r--r--drivers/comedi/Makefile (renamed from drivers/staging/comedi/Makefile)0
-rw-r--r--drivers/comedi/TODO (renamed from drivers/staging/comedi/TODO)0
-rw-r--r--drivers/comedi/comedi.h (renamed from drivers/staging/comedi/comedi.h)0
-rw-r--r--drivers/comedi/comedi_buf.c (renamed from drivers/staging/comedi/comedi_buf.c)52
-rw-r--r--drivers/comedi/comedi_fops.c (renamed from drivers/staging/comedi/comedi_fops.c)0
-rw-r--r--drivers/comedi/comedi_internal.h (renamed from drivers/staging/comedi/comedi_internal.h)0
-rw-r--r--drivers/comedi/comedi_pci.c (renamed from drivers/staging/comedi/comedi_pci.c)0
-rw-r--r--drivers/comedi/comedi_pci.h (renamed from drivers/staging/comedi/comedi_pci.h)0
-rw-r--r--drivers/comedi/comedi_pcmcia.c (renamed from drivers/staging/comedi/comedi_pcmcia.c)0
-rw-r--r--drivers/comedi/comedi_pcmcia.h (renamed from drivers/staging/comedi/comedi_pcmcia.h)0
-rw-r--r--drivers/comedi/comedi_usb.c (renamed from drivers/staging/comedi/comedi_usb.c)0
-rw-r--r--drivers/comedi/comedi_usb.h (renamed from drivers/staging/comedi/comedi_usb.h)0
-rw-r--r--drivers/comedi/comedidev.h (renamed from drivers/staging/comedi/comedidev.h)0
-rw-r--r--drivers/comedi/comedilib.h (renamed from drivers/staging/comedi/comedilib.h)0
-rw-r--r--drivers/comedi/drivers.c (renamed from drivers/staging/comedi/drivers.c)0
-rw-r--r--drivers/comedi/drivers/8255.c (renamed from drivers/staging/comedi/drivers/8255.c)0
-rw-r--r--drivers/comedi/drivers/8255.h (renamed from drivers/staging/comedi/drivers/8255.h)0
-rw-r--r--drivers/comedi/drivers/8255_pci.c (renamed from drivers/staging/comedi/drivers/8255_pci.c)0
-rw-r--r--drivers/comedi/drivers/Makefile (renamed from drivers/staging/comedi/drivers/Makefile)0
-rw-r--r--drivers/comedi/drivers/addi_apci_1032.c (renamed from drivers/staging/comedi/drivers/addi_apci_1032.c)4
-rw-r--r--drivers/comedi/drivers/addi_apci_1500.c (renamed from drivers/staging/comedi/drivers/addi_apci_1500.c)18
-rw-r--r--drivers/comedi/drivers/addi_apci_1516.c (renamed from drivers/staging/comedi/drivers/addi_apci_1516.c)0
-rw-r--r--drivers/comedi/drivers/addi_apci_1564.c (renamed from drivers/staging/comedi/drivers/addi_apci_1564.c)0
-rw-r--r--drivers/comedi/drivers/addi_apci_16xx.c (renamed from drivers/staging/comedi/drivers/addi_apci_16xx.c)2
-rw-r--r--drivers/comedi/drivers/addi_apci_2032.c (renamed from drivers/staging/comedi/drivers/addi_apci_2032.c)0
-rw-r--r--drivers/comedi/drivers/addi_apci_2200.c (renamed from drivers/staging/comedi/drivers/addi_apci_2200.c)0
-rw-r--r--drivers/comedi/drivers/addi_apci_3120.c (renamed from drivers/staging/comedi/drivers/addi_apci_3120.c)0
-rw-r--r--drivers/comedi/drivers/addi_apci_3501.c (renamed from drivers/staging/comedi/drivers/addi_apci_3501.c)0
-rw-r--r--drivers/comedi/drivers/addi_apci_3xxx.c (renamed from drivers/staging/comedi/drivers/addi_apci_3xxx.c)0
-rw-r--r--drivers/comedi/drivers/addi_tcw.h (renamed from drivers/staging/comedi/drivers/addi_tcw.h)0
-rw-r--r--drivers/comedi/drivers/addi_watchdog.c (renamed from drivers/staging/comedi/drivers/addi_watchdog.c)0
-rw-r--r--drivers/comedi/drivers/addi_watchdog.h (renamed from drivers/staging/comedi/drivers/addi_watchdog.h)0
-rw-r--r--drivers/comedi/drivers/adl_pci6208.c (renamed from drivers/staging/comedi/drivers/adl_pci6208.c)0
-rw-r--r--drivers/comedi/drivers/adl_pci7x3x.c (renamed from drivers/staging/comedi/drivers/adl_pci7x3x.c)0
-rw-r--r--drivers/comedi/drivers/adl_pci8164.c (renamed from drivers/staging/comedi/drivers/adl_pci8164.c)0
-rw-r--r--drivers/comedi/drivers/adl_pci9111.c (renamed from drivers/staging/comedi/drivers/adl_pci9111.c)0
-rw-r--r--drivers/comedi/drivers/adl_pci9118.c (renamed from drivers/staging/comedi/drivers/adl_pci9118.c)0
-rw-r--r--drivers/comedi/drivers/adq12b.c (renamed from drivers/staging/comedi/drivers/adq12b.c)0
-rw-r--r--drivers/comedi/drivers/adv_pci1710.c (renamed from drivers/staging/comedi/drivers/adv_pci1710.c)10
-rw-r--r--drivers/comedi/drivers/adv_pci1720.c (renamed from drivers/staging/comedi/drivers/adv_pci1720.c)0
-rw-r--r--drivers/comedi/drivers/adv_pci1723.c (renamed from drivers/staging/comedi/drivers/adv_pci1723.c)0
-rw-r--r--drivers/comedi/drivers/adv_pci1724.c (renamed from drivers/staging/comedi/drivers/adv_pci1724.c)0
-rw-r--r--drivers/comedi/drivers/adv_pci1760.c (renamed from drivers/staging/comedi/drivers/adv_pci1760.c)0
-rw-r--r--drivers/comedi/drivers/adv_pci_dio.c (renamed from drivers/staging/comedi/drivers/adv_pci_dio.c)1
-rw-r--r--drivers/comedi/drivers/aio_aio12_8.c (renamed from drivers/staging/comedi/drivers/aio_aio12_8.c)0
-rw-r--r--drivers/comedi/drivers/aio_iiro_16.c (renamed from drivers/staging/comedi/drivers/aio_iiro_16.c)0
-rw-r--r--drivers/comedi/drivers/amcc_s5933.h (renamed from drivers/staging/comedi/drivers/amcc_s5933.h)0
-rw-r--r--drivers/comedi/drivers/amplc_dio200.c (renamed from drivers/staging/comedi/drivers/amplc_dio200.c)0
-rw-r--r--drivers/comedi/drivers/amplc_dio200.h (renamed from drivers/staging/comedi/drivers/amplc_dio200.h)0
-rw-r--r--drivers/comedi/drivers/amplc_dio200_common.c (renamed from drivers/staging/comedi/drivers/amplc_dio200_common.c)0
-rw-r--r--drivers/comedi/drivers/amplc_dio200_pci.c (renamed from drivers/staging/comedi/drivers/amplc_dio200_pci.c)0
-rw-r--r--drivers/comedi/drivers/amplc_pc236.c (renamed from drivers/staging/comedi/drivers/amplc_pc236.c)0
-rw-r--r--drivers/comedi/drivers/amplc_pc236.h (renamed from drivers/staging/comedi/drivers/amplc_pc236.h)0
-rw-r--r--drivers/comedi/drivers/amplc_pc236_common.c (renamed from drivers/staging/comedi/drivers/amplc_pc236_common.c)4
-rw-r--r--drivers/comedi/drivers/amplc_pc263.c (renamed from drivers/staging/comedi/drivers/amplc_pc263.c)0
-rw-r--r--drivers/comedi/drivers/amplc_pci224.c (renamed from drivers/staging/comedi/drivers/amplc_pci224.c)0
-rw-r--r--drivers/comedi/drivers/amplc_pci230.c (renamed from drivers/staging/comedi/drivers/amplc_pci230.c)0
-rw-r--r--drivers/comedi/drivers/amplc_pci236.c (renamed from drivers/staging/comedi/drivers/amplc_pci236.c)0
-rw-r--r--drivers/comedi/drivers/amplc_pci263.c (renamed from drivers/staging/comedi/drivers/amplc_pci263.c)0
-rw-r--r--drivers/comedi/drivers/c6xdigio.c (renamed from drivers/staging/comedi/drivers/c6xdigio.c)0
-rw-r--r--drivers/comedi/drivers/cb_das16_cs.c (renamed from drivers/staging/comedi/drivers/cb_das16_cs.c)0
-rw-r--r--drivers/comedi/drivers/cb_pcidas.c (renamed from drivers/staging/comedi/drivers/cb_pcidas.c)2
-rw-r--r--drivers/comedi/drivers/cb_pcidas64.c (renamed from drivers/staging/comedi/drivers/cb_pcidas64.c)7
-rw-r--r--drivers/comedi/drivers/cb_pcidda.c (renamed from drivers/staging/comedi/drivers/cb_pcidda.c)0
-rw-r--r--drivers/comedi/drivers/cb_pcimdas.c (renamed from drivers/staging/comedi/drivers/cb_pcimdas.c)0
-rw-r--r--drivers/comedi/drivers/cb_pcimdda.c (renamed from drivers/staging/comedi/drivers/cb_pcimdda.c)0
-rw-r--r--drivers/comedi/drivers/comedi_8254.c (renamed from drivers/staging/comedi/drivers/comedi_8254.c)0
-rw-r--r--drivers/comedi/drivers/comedi_8254.h (renamed from drivers/staging/comedi/drivers/comedi_8254.h)0
-rw-r--r--drivers/comedi/drivers/comedi_8255.c (renamed from drivers/staging/comedi/drivers/comedi_8255.c)0
-rw-r--r--drivers/comedi/drivers/comedi_bond.c (renamed from drivers/staging/comedi/drivers/comedi_bond.c)0
-rw-r--r--drivers/comedi/drivers/comedi_isadma.c (renamed from drivers/staging/comedi/drivers/comedi_isadma.c)0
-rw-r--r--drivers/comedi/drivers/comedi_isadma.h (renamed from drivers/staging/comedi/drivers/comedi_isadma.h)0
-rw-r--r--drivers/comedi/drivers/comedi_parport.c (renamed from drivers/staging/comedi/drivers/comedi_parport.c)3
-rw-r--r--drivers/comedi/drivers/comedi_test.c (renamed from drivers/staging/comedi/drivers/comedi_test.c)0
-rw-r--r--drivers/comedi/drivers/contec_pci_dio.c (renamed from drivers/staging/comedi/drivers/contec_pci_dio.c)0
-rw-r--r--drivers/comedi/drivers/dac02.c (renamed from drivers/staging/comedi/drivers/dac02.c)0
-rw-r--r--drivers/comedi/drivers/daqboard2000.c (renamed from drivers/staging/comedi/drivers/daqboard2000.c)0
-rw-r--r--drivers/comedi/drivers/das08.c (renamed from drivers/staging/comedi/drivers/das08.c)0
-rw-r--r--drivers/comedi/drivers/das08.h (renamed from drivers/staging/comedi/drivers/das08.h)0
-rw-r--r--drivers/comedi/drivers/das08_cs.c (renamed from drivers/staging/comedi/drivers/das08_cs.c)0
-rw-r--r--drivers/comedi/drivers/das08_isa.c (renamed from drivers/staging/comedi/drivers/das08_isa.c)0
-rw-r--r--drivers/comedi/drivers/das08_pci.c (renamed from drivers/staging/comedi/drivers/das08_pci.c)0
-rw-r--r--drivers/comedi/drivers/das16.c (renamed from drivers/staging/comedi/drivers/das16.c)0
-rw-r--r--drivers/comedi/drivers/das16m1.c (renamed from drivers/staging/comedi/drivers/das16m1.c)0
-rw-r--r--drivers/comedi/drivers/das1800.c (renamed from drivers/staging/comedi/drivers/das1800.c)0
-rw-r--r--drivers/comedi/drivers/das6402.c (renamed from drivers/staging/comedi/drivers/das6402.c)2
-rw-r--r--drivers/comedi/drivers/das800.c (renamed from drivers/staging/comedi/drivers/das800.c)4
-rw-r--r--drivers/comedi/drivers/dmm32at.c (renamed from drivers/staging/comedi/drivers/dmm32at.c)2
-rw-r--r--drivers/comedi/drivers/dt2801.c (renamed from drivers/staging/comedi/drivers/dt2801.c)0
-rw-r--r--drivers/comedi/drivers/dt2811.c (renamed from drivers/staging/comedi/drivers/dt2811.c)0
-rw-r--r--drivers/comedi/drivers/dt2814.c (renamed from drivers/staging/comedi/drivers/dt2814.c)152
-rw-r--r--drivers/comedi/drivers/dt2815.c (renamed from drivers/staging/comedi/drivers/dt2815.c)0
-rw-r--r--drivers/comedi/drivers/dt2817.c (renamed from drivers/staging/comedi/drivers/dt2817.c)0
-rw-r--r--drivers/comedi/drivers/dt282x.c (renamed from drivers/staging/comedi/drivers/dt282x.c)0
-rw-r--r--drivers/comedi/drivers/dt3000.c (renamed from drivers/staging/comedi/drivers/dt3000.c)0
-rw-r--r--drivers/comedi/drivers/dt9812.c (renamed from drivers/staging/comedi/drivers/dt9812.c)0
-rw-r--r--drivers/comedi/drivers/dyna_pci10xx.c (renamed from drivers/staging/comedi/drivers/dyna_pci10xx.c)0
-rw-r--r--drivers/comedi/drivers/fl512.c (renamed from drivers/staging/comedi/drivers/fl512.c)0
-rw-r--r--drivers/comedi/drivers/gsc_hpdi.c (renamed from drivers/staging/comedi/drivers/gsc_hpdi.c)0
-rw-r--r--drivers/comedi/drivers/icp_multi.c (renamed from drivers/staging/comedi/drivers/icp_multi.c)0
-rw-r--r--drivers/comedi/drivers/ii_pci20kc.c (renamed from drivers/staging/comedi/drivers/ii_pci20kc.c)0
-rw-r--r--drivers/comedi/drivers/jr3_pci.c (renamed from drivers/staging/comedi/drivers/jr3_pci.c)0
-rw-r--r--drivers/comedi/drivers/jr3_pci.h (renamed from drivers/staging/comedi/drivers/jr3_pci.h)0
-rw-r--r--drivers/comedi/drivers/ke_counter.c (renamed from drivers/staging/comedi/drivers/ke_counter.c)0
-rw-r--r--drivers/comedi/drivers/me4000.c (renamed from drivers/staging/comedi/drivers/me4000.c)2
-rw-r--r--drivers/comedi/drivers/me_daq.c (renamed from drivers/staging/comedi/drivers/me_daq.c)0
-rw-r--r--drivers/comedi/drivers/mf6x4.c (renamed from drivers/staging/comedi/drivers/mf6x4.c)0
-rw-r--r--drivers/comedi/drivers/mite.c (renamed from drivers/staging/comedi/drivers/mite.c)0
-rw-r--r--drivers/comedi/drivers/mite.h (renamed from drivers/staging/comedi/drivers/mite.h)0
-rw-r--r--drivers/comedi/drivers/mpc624.c (renamed from drivers/staging/comedi/drivers/mpc624.c)0
-rw-r--r--drivers/comedi/drivers/multiq3.c (renamed from drivers/staging/comedi/drivers/multiq3.c)0
-rw-r--r--drivers/comedi/drivers/ni_6527.c (renamed from drivers/staging/comedi/drivers/ni_6527.c)4
-rw-r--r--drivers/comedi/drivers/ni_65xx.c (renamed from drivers/staging/comedi/drivers/ni_65xx.c)3
-rw-r--r--drivers/comedi/drivers/ni_660x.c (renamed from drivers/staging/comedi/drivers/ni_660x.c)0
-rw-r--r--drivers/comedi/drivers/ni_670x.c (renamed from drivers/staging/comedi/drivers/ni_670x.c)0
-rw-r--r--drivers/comedi/drivers/ni_at_a2150.c (renamed from drivers/staging/comedi/drivers/ni_at_a2150.c)0
-rw-r--r--drivers/comedi/drivers/ni_at_ao.c (renamed from drivers/staging/comedi/drivers/ni_at_ao.c)0
-rw-r--r--drivers/comedi/drivers/ni_atmio.c (renamed from drivers/staging/comedi/drivers/ni_atmio.c)0
-rw-r--r--drivers/comedi/drivers/ni_atmio16d.c (renamed from drivers/staging/comedi/drivers/ni_atmio16d.c)0
-rw-r--r--drivers/comedi/drivers/ni_daq_700.c (renamed from drivers/staging/comedi/drivers/ni_daq_700.c)0
-rw-r--r--drivers/comedi/drivers/ni_daq_dio24.c (renamed from drivers/staging/comedi/drivers/ni_daq_dio24.c)0
-rw-r--r--drivers/comedi/drivers/ni_labpc.c (renamed from drivers/staging/comedi/drivers/ni_labpc.c)0
-rw-r--r--drivers/comedi/drivers/ni_labpc.h (renamed from drivers/staging/comedi/drivers/ni_labpc.h)0
-rw-r--r--drivers/comedi/drivers/ni_labpc_common.c (renamed from drivers/staging/comedi/drivers/ni_labpc_common.c)0
-rw-r--r--drivers/comedi/drivers/ni_labpc_cs.c (renamed from drivers/staging/comedi/drivers/ni_labpc_cs.c)0
-rw-r--r--drivers/comedi/drivers/ni_labpc_isadma.c (renamed from drivers/staging/comedi/drivers/ni_labpc_isadma.c)0
-rw-r--r--drivers/comedi/drivers/ni_labpc_isadma.h (renamed from drivers/staging/comedi/drivers/ni_labpc_isadma.h)0
-rw-r--r--drivers/comedi/drivers/ni_labpc_pci.c (renamed from drivers/staging/comedi/drivers/ni_labpc_pci.c)0
-rw-r--r--drivers/comedi/drivers/ni_labpc_regs.h (renamed from drivers/staging/comedi/drivers/ni_labpc_regs.h)0
-rw-r--r--drivers/comedi/drivers/ni_mio_common.c (renamed from drivers/staging/comedi/drivers/ni_mio_common.c)0
-rw-r--r--drivers/comedi/drivers/ni_mio_cs.c (renamed from drivers/staging/comedi/drivers/ni_mio_cs.c)0
-rw-r--r--drivers/comedi/drivers/ni_pcidio.c (renamed from drivers/staging/comedi/drivers/ni_pcidio.c)0
-rw-r--r--drivers/comedi/drivers/ni_pcimio.c (renamed from drivers/staging/comedi/drivers/ni_pcimio.c)0
-rw-r--r--drivers/comedi/drivers/ni_routes.c (renamed from drivers/staging/comedi/drivers/ni_routes.c)0
-rw-r--r--drivers/comedi/drivers/ni_routes.h (renamed from drivers/staging/comedi/drivers/ni_routes.h)0
-rw-r--r--drivers/comedi/drivers/ni_routing/README (renamed from drivers/staging/comedi/drivers/ni_routing/README)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes.h (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes.h)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/all.h (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/all.h)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6070e.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6070e.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6220.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6220.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6221.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6221.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6229.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6229.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6251.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6251.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6254.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6254.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6259.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6259.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6534.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6534.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6602.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6602.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6713.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6713.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6723.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6723.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6733.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6733.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6030e.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6030e.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6224.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6224.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6225.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6225.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6251.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6251.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6733.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6733.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6251.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6251.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6535.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6535.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6738.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6738.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_route_values.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_route_values.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_route_values.h (renamed from drivers/staging/comedi/drivers/ni_routing/ni_route_values.h)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_route_values/all.h (renamed from drivers/staging/comedi/drivers/ni_routing/ni_route_values/all.h)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_route_values/ni_660x.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_660x.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_route_values/ni_eseries.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_eseries.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/ni_route_values/ni_mseries.c (renamed from drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_mseries.c)0
-rw-r--r--drivers/comedi/drivers/ni_routing/tools/.gitignore (renamed from drivers/staging/comedi/drivers/ni_routing/tools/.gitignore)0
-rw-r--r--drivers/comedi/drivers/ni_routing/tools/Makefile (renamed from drivers/staging/comedi/drivers/ni_routing/tools/Makefile)0
-rw-r--r--drivers/comedi/drivers/ni_routing/tools/convert_c_to_py.c (renamed from drivers/staging/comedi/drivers/ni_routing/tools/convert_c_to_py.c)0
-rwxr-xr-xdrivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py (renamed from drivers/staging/comedi/drivers/ni_routing/tools/convert_csv_to_c.py)0
-rwxr-xr-xdrivers/comedi/drivers/ni_routing/tools/convert_py_to_csv.py (renamed from drivers/staging/comedi/drivers/ni_routing/tools/convert_py_to_csv.py)0
-rw-r--r--drivers/comedi/drivers/ni_routing/tools/csv_collection.py (renamed from drivers/staging/comedi/drivers/ni_routing/tools/csv_collection.py)0
-rwxr-xr-xdrivers/comedi/drivers/ni_routing/tools/make_blank_csv.py (renamed from drivers/staging/comedi/drivers/ni_routing/tools/make_blank_csv.py)0
-rw-r--r--drivers/comedi/drivers/ni_routing/tools/ni_names.py (renamed from drivers/staging/comedi/drivers/ni_routing/tools/ni_names.py)0
-rw-r--r--drivers/comedi/drivers/ni_stc.h (renamed from drivers/staging/comedi/drivers/ni_stc.h)0
-rw-r--r--drivers/comedi/drivers/ni_tio.c (renamed from drivers/staging/comedi/drivers/ni_tio.c)0
-rw-r--r--drivers/comedi/drivers/ni_tio.h (renamed from drivers/staging/comedi/drivers/ni_tio.h)0
-rw-r--r--drivers/comedi/drivers/ni_tio_internal.h (renamed from drivers/staging/comedi/drivers/ni_tio_internal.h)0
-rw-r--r--drivers/comedi/drivers/ni_tiocmd.c (renamed from drivers/staging/comedi/drivers/ni_tiocmd.c)0
-rw-r--r--drivers/comedi/drivers/ni_usb6501.c (renamed from drivers/staging/comedi/drivers/ni_usb6501.c)0
-rw-r--r--drivers/comedi/drivers/pcl711.c (renamed from drivers/staging/comedi/drivers/pcl711.c)2
-rw-r--r--drivers/comedi/drivers/pcl724.c (renamed from drivers/staging/comedi/drivers/pcl724.c)0
-rw-r--r--drivers/comedi/drivers/pcl726.c (renamed from drivers/staging/comedi/drivers/pcl726.c)4
-rw-r--r--drivers/comedi/drivers/pcl730.c (renamed from drivers/staging/comedi/drivers/pcl730.c)0
-rw-r--r--drivers/comedi/drivers/pcl812.c (renamed from drivers/staging/comedi/drivers/pcl812.c)0
-rw-r--r--drivers/comedi/drivers/pcl816.c (renamed from drivers/staging/comedi/drivers/pcl816.c)0
-rw-r--r--drivers/comedi/drivers/pcl818.c (renamed from drivers/staging/comedi/drivers/pcl818.c)2
-rw-r--r--drivers/comedi/drivers/pcm3724.c (renamed from drivers/staging/comedi/drivers/pcm3724.c)0
-rw-r--r--drivers/comedi/drivers/pcmad.c (renamed from drivers/staging/comedi/drivers/pcmad.c)0
-rw-r--r--drivers/comedi/drivers/pcmda12.c (renamed from drivers/staging/comedi/drivers/pcmda12.c)0
-rw-r--r--drivers/comedi/drivers/pcmmio.c (renamed from drivers/staging/comedi/drivers/pcmmio.c)0
-rw-r--r--drivers/comedi/drivers/pcmuio.c (renamed from drivers/staging/comedi/drivers/pcmuio.c)0
-rw-r--r--drivers/comedi/drivers/plx9052.h (renamed from drivers/staging/comedi/drivers/plx9052.h)0
-rw-r--r--drivers/comedi/drivers/plx9080.h (renamed from drivers/staging/comedi/drivers/plx9080.h)0
-rw-r--r--drivers/comedi/drivers/quatech_daqp_cs.c (renamed from drivers/staging/comedi/drivers/quatech_daqp_cs.c)0
-rw-r--r--drivers/comedi/drivers/rtd520.c (renamed from drivers/staging/comedi/drivers/rtd520.c)0
-rw-r--r--drivers/comedi/drivers/rti800.c (renamed from drivers/staging/comedi/drivers/rti800.c)0
-rw-r--r--drivers/comedi/drivers/rti802.c (renamed from drivers/staging/comedi/drivers/rti802.c)0
-rw-r--r--drivers/comedi/drivers/s526.c (renamed from drivers/staging/comedi/drivers/s526.c)0
-rw-r--r--drivers/comedi/drivers/s626.c (renamed from drivers/staging/comedi/drivers/s626.c)0
-rw-r--r--drivers/comedi/drivers/s626.h (renamed from drivers/staging/comedi/drivers/s626.h)0
-rw-r--r--drivers/comedi/drivers/ssv_dnp.c (renamed from drivers/staging/comedi/drivers/ssv_dnp.c)0
-rw-r--r--drivers/comedi/drivers/tests/Makefile (renamed from drivers/staging/comedi/drivers/tests/Makefile)3
-rw-r--r--drivers/comedi/drivers/tests/comedi_example_test.c (renamed from drivers/staging/comedi/drivers/tests/example_test.c)10
-rw-r--r--drivers/comedi/drivers/tests/ni_routes_test.c (renamed from drivers/staging/comedi/drivers/tests/ni_routes_test.c)120
-rw-r--r--drivers/comedi/drivers/tests/unittest.h (renamed from drivers/staging/comedi/drivers/tests/unittest.h)2
-rw-r--r--drivers/comedi/drivers/usbdux.c (renamed from drivers/staging/comedi/drivers/usbdux.c)0
-rw-r--r--drivers/comedi/drivers/usbduxfast.c (renamed from drivers/staging/comedi/drivers/usbduxfast.c)0
-rw-r--r--drivers/comedi/drivers/usbduxsigma.c (renamed from drivers/staging/comedi/drivers/usbduxsigma.c)0
-rw-r--r--drivers/comedi/drivers/vmk80xx.c (renamed from drivers/staging/comedi/drivers/vmk80xx.c)1
-rw-r--r--drivers/comedi/drivers/z8536.h (renamed from drivers/staging/comedi/drivers/z8536.h)0
-rw-r--r--drivers/comedi/kcomedilib/Makefile (renamed from drivers/staging/comedi/kcomedilib/Makefile)0
-rw-r--r--drivers/comedi/kcomedilib/kcomedilib_main.c (renamed from drivers/staging/comedi/kcomedilib/kcomedilib_main.c)0
-rw-r--r--drivers/comedi/proc.c (renamed from drivers/staging/comedi/proc.c)0
-rw-r--r--drivers/comedi/range.c (renamed from drivers/staging/comedi/range.c)0
-rw-r--r--drivers/counter/104-quad-8.c653
-rw-r--r--drivers/counter/Kconfig14
-rw-r--r--drivers/counter/Makefile1
-rw-r--r--drivers/counter/interrupt-cnt.c244
-rw-r--r--drivers/counter/stm32-lptimer-cnt.c297
-rw-r--r--drivers/counter/stm32-timer-cnt.c55
-rw-r--r--drivers/cpufreq/Kconfig23
-rw-r--r--drivers/cpufreq/Kconfig.arm10
-rw-r--r--drivers/cpufreq/acpi-cpufreq.c6
-rw-r--r--drivers/cpufreq/armada-37xx-cpufreq.c111
-rw-r--r--drivers/cpufreq/cppc_cpufreq.c259
-rw-r--r--drivers/cpufreq/cpufreq-dt-platdev.c2
-rw-r--r--drivers/cpufreq/cpufreq-dt.c9
-rw-r--r--drivers/cpufreq/cpufreq.c3
-rw-r--r--drivers/cpufreq/freq_table.c4
-rw-r--r--drivers/cpufreq/ia64-acpi-cpufreq.c4
-rw-r--r--drivers/cpufreq/intel_pstate.c121
-rw-r--r--drivers/cpufreq/qcom-cpufreq-hw.c6
-rw-r--r--drivers/cpufreq/s5pv210-cpufreq.c14
-rw-r--r--drivers/cpufreq/scmi-cpufreq.c107
-rw-r--r--drivers/cpuidle/Kconfig.arm2
-rw-r--r--drivers/cpuidle/cpuidle-tegra.c19
-rw-r--r--drivers/cpuidle/driver.c4
-rw-r--r--drivers/cpuidle/governors/menu.c17
-rw-r--r--drivers/cpuidle/governors/teo.c54
-rw-r--r--drivers/crypto/allwinner/Kconfig14
-rw-r--r--drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c2
-rw-r--r--drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c23
-rw-r--r--drivers/crypto/allwinner/sun4i-ss/sun4i-ss-hash.c2
-rw-r--r--drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c2
-rw-r--r--drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c9
-rw-r--r--drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c2
-rw-r--r--drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c3
-rw-r--r--drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c1
-rw-r--r--drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c11
-rw-r--r--drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c2
-rw-r--r--drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c12
-rw-r--r--drivers/crypto/allwinner/sun8i-ss/sun8i-ss-prng.c4
-rw-r--r--drivers/crypto/amcc/crypto4xx_alg.c12
-rw-r--r--drivers/crypto/amcc/crypto4xx_core.c18
-rw-r--r--drivers/crypto/amcc/crypto4xx_core.h4
-rw-r--r--drivers/crypto/amcc/crypto4xx_reg_def.h8
-rw-r--r--drivers/crypto/amcc/crypto4xx_sa.h18
-rw-r--r--drivers/crypto/amcc/crypto4xx_trng.h2
-rw-r--r--drivers/crypto/amlogic/amlogic-gxl-cipher.c6
-rw-r--r--drivers/crypto/amlogic/amlogic-gxl-core.c3
-rw-r--r--drivers/crypto/atmel-ecc.c30
-rw-r--r--drivers/crypto/atmel-i2c.c2
-rw-r--r--drivers/crypto/atmel-sha.c4
-rw-r--r--drivers/crypto/atmel-tdes.c1
-rw-r--r--drivers/crypto/bcm/cipher.c7
-rw-r--r--drivers/crypto/bcm/spu.c16
-rw-r--r--drivers/crypto/bcm/spu2.c43
-rw-r--r--drivers/crypto/bcm/util.c4
-rw-r--r--drivers/crypto/caam/caamalg_qi2.c3
-rw-r--r--drivers/crypto/caam/caampkc.c3
-rw-r--r--drivers/crypto/cavium/cpt/cptpf_main.c1
-rw-r--r--drivers/crypto/cavium/nitrox/nitrox_isr.c4
-rw-r--r--drivers/crypto/cavium/nitrox/nitrox_reqmgr.c9
-rw-r--r--drivers/crypto/cavium/zip/common.h1
-rw-r--r--drivers/crypto/ccp/ccp-crypto-main.c3
-rw-r--r--drivers/crypto/ccp/ccp-dev.c12
-rw-r--r--drivers/crypto/ccp/ccp-ops.c1
-rw-r--r--drivers/crypto/ccp/sev-dev.c199
-rw-r--r--drivers/crypto/ccp/sev-dev.h4
-rw-r--r--drivers/crypto/ccp/sp-dev.c12
-rw-r--r--drivers/crypto/ccp/sp-dev.h15
-rw-r--r--drivers/crypto/ccp/sp-pci.c1
-rw-r--r--drivers/crypto/ccp/tee-dev.c57
-rw-r--r--drivers/crypto/ccp/tee-dev.h20
-rw-r--r--drivers/crypto/ccree/cc_driver.c4
-rw-r--r--drivers/crypto/chelsio/chcr_algo.c32
-rw-r--r--drivers/crypto/chelsio/chcr_core.c5
-rw-r--r--drivers/crypto/chelsio/chcr_core.h1
-rw-r--r--drivers/crypto/geode-aes.c4
-rw-r--r--drivers/crypto/hisilicon/Kconfig2
-rw-r--r--drivers/crypto/hisilicon/hpre/hpre.h18
-rw-r--r--drivers/crypto/hisilicon/hpre/hpre_crypto.c921
-rw-r--r--drivers/crypto/hisilicon/hpre/hpre_main.c158
-rw-r--r--drivers/crypto/hisilicon/qm.c396
-rw-r--r--drivers/crypto/hisilicon/qm.h29
-rw-r--r--drivers/crypto/hisilicon/sec/sec_algs.c2
-rw-r--r--drivers/crypto/hisilicon/sec/sec_drv.c13
-rw-r--r--drivers/crypto/hisilicon/sec/sec_drv.h2
-rw-r--r--drivers/crypto/hisilicon/sec2/sec.h10
-rw-r--r--drivers/crypto/hisilicon/sec2/sec_crypto.c137
-rw-r--r--drivers/crypto/hisilicon/sec2/sec_crypto.h6
-rw-r--r--drivers/crypto/hisilicon/sec2/sec_main.c267
-rw-r--r--drivers/crypto/hisilicon/sgl.c37
-rw-r--r--drivers/crypto/hisilicon/trng/trng.c13
-rw-r--r--drivers/crypto/hisilicon/zip/zip.h50
-rw-r--r--drivers/crypto/hisilicon/zip/zip_crypto.c698
-rw-r--r--drivers/crypto/hisilicon/zip/zip_main.c99
-rw-r--r--drivers/crypto/img-hash.c3
-rw-r--r--drivers/crypto/inside-secure/safexcel.c2
-rw-r--r--drivers/crypto/ixp4xx_crypto.c7
-rw-r--r--drivers/crypto/keembay/keembay-ocs-aes-core.c8
-rw-r--r--drivers/crypto/keembay/keembay-ocs-hcu-core.c8
-rw-r--r--drivers/crypto/keembay/ocs-hcu.c8
-rw-r--r--drivers/crypto/marvell/octeontx2/otx2_cpt_common.h10
-rw-r--r--drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c14
-rw-r--r--drivers/crypto/marvell/octeontx2/otx2_cptlf.c8
-rw-r--r--drivers/crypto/marvell/octeontx2/otx2_cptlf.h1
-rw-r--r--drivers/crypto/marvell/octeontx2/otx2_cptpf.h1
-rw-r--r--drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c33
-rw-r--r--drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c144
-rw-r--r--drivers/crypto/nx/nx-842-pseries.c4
-rw-r--r--drivers/crypto/nx/nx-aes-cbc.c2
-rw-r--r--drivers/crypto/nx/nx-aes-ccm.c2
-rw-r--r--drivers/crypto/nx/nx-aes-ctr.c2
-rw-r--r--drivers/crypto/nx/nx-aes-ecb.c2
-rw-r--r--drivers/crypto/nx/nx-aes-gcm.c2
-rw-r--r--drivers/crypto/nx/nx-aes-xcbc.c2
-rw-r--r--drivers/crypto/nx/nx-common-powernv.c4
-rw-r--r--drivers/crypto/nx/nx-sha256.c2
-rw-r--r--drivers/crypto/nx/nx-sha512.c2
-rw-r--r--drivers/crypto/nx/nx.c9
-rw-r--r--drivers/crypto/nx/nx_debugfs.c2
-rw-r--r--drivers/crypto/omap-aes.c7
-rw-r--r--drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c1
-rw-r--r--drivers/crypto/qat/qat_c3xxx/adf_c3xxx_hw_data.c1
-rw-r--r--drivers/crypto/qat/qat_c3xxxvf/adf_drv.c4
-rw-r--r--drivers/crypto/qat/qat_c62x/adf_c62x_hw_data.c1
-rw-r--r--drivers/crypto/qat/qat_c62xvf/adf_drv.c4
-rw-r--r--drivers/crypto/qat/qat_common/adf_accel_devices.h1
-rw-r--r--drivers/crypto/qat/qat_common/adf_gen2_hw_data.c25
-rw-r--r--drivers/crypto/qat/qat_common/adf_gen2_hw_data.h13
-rw-r--r--drivers/crypto/qat/qat_common/adf_gen4_hw_data.c40
-rw-r--r--drivers/crypto/qat/qat_common/adf_gen4_hw_data.h14
-rw-r--r--drivers/crypto/qat/qat_common/adf_init.c4
-rw-r--r--drivers/crypto/qat/qat_common/adf_isr.c29
-rw-r--r--drivers/crypto/qat/qat_common/adf_pf2vf_msg.c2
-rw-r--r--drivers/crypto/qat/qat_common/adf_transport.c1
-rw-r--r--drivers/crypto/qat/qat_common/adf_vf2pf_msg.c4
-rw-r--r--drivers/crypto/qat/qat_common/adf_vf_isr.c17
-rw-r--r--drivers/crypto/qat/qat_common/qat_algs.c32
-rw-r--r--drivers/crypto/qat/qat_dh895xccvf/adf_drv.c4
-rw-r--r--drivers/crypto/qce/cipher.h1
-rw-r--r--drivers/crypto/qce/common.c25
-rw-r--r--drivers/crypto/qce/common.h3
-rw-r--r--drivers/crypto/qce/sha.c143
-rw-r--r--drivers/crypto/qce/skcipher.c69
-rw-r--r--drivers/crypto/rockchip/rk3288_crypto_ahash.c2
-rw-r--r--drivers/crypto/s5p-sss.c17
-rw-r--r--drivers/crypto/sa2ul.c143
-rw-r--r--drivers/crypto/sa2ul.h4
-rw-r--r--drivers/crypto/stm32/stm32-cryp.c4
-rw-r--r--drivers/crypto/stm32/stm32-hash.c8
-rw-r--r--drivers/crypto/ux500/cryp/cryp.c5
-rw-r--r--drivers/crypto/ux500/cryp/cryp.h2
-rw-r--r--drivers/crypto/ux500/cryp/cryp_core.c10
-rw-r--r--drivers/crypto/ux500/cryp/cryp_irq.c2
-rw-r--r--drivers/crypto/ux500/cryp/cryp_irq.h4
-rw-r--r--drivers/crypto/ux500/cryp/cryp_irqp.h4
-rw-r--r--drivers/crypto/ux500/cryp/cryp_p.h15
-rw-r--r--drivers/crypto/ux500/hash/hash_core.c18
-rw-r--r--drivers/crypto/vmx/aes.c2
-rw-r--r--drivers/crypto/vmx/aes_cbc.c2
-rw-r--r--drivers/crypto/vmx/aes_ctr.c2
-rw-r--r--drivers/crypto/vmx/aes_xts.c2
-rw-r--r--drivers/crypto/vmx/ghash.c2
-rw-r--r--drivers/crypto/vmx/vmx.c2
-rw-r--r--drivers/cxl/mem.c152
-rw-r--r--drivers/dax/bus.c6
-rw-r--r--drivers/devfreq/Kconfig2
-rw-r--r--drivers/devfreq/devfreq.c14
-rw-r--r--drivers/devfreq/governor.h5
-rw-r--r--drivers/devfreq/imx-bus.c2
-rw-r--r--drivers/devfreq/imx8m-ddrc.c16
-rw-r--r--drivers/devfreq/rk3399_dmc.c20
-rw-r--r--drivers/dma-buf/dma-fence.c27
-rw-r--r--drivers/dma-buf/dma-heap.c12
-rw-r--r--drivers/dma-buf/heaps/cma_heap.c1
-rw-r--r--drivers/dma-buf/heaps/system_heap.c1
-rw-r--r--drivers/dma/Kconfig14
-rw-r--r--drivers/dma/at_xdmac.c11
-rw-r--r--drivers/dma/dmaengine.c1
-rw-r--r--drivers/dma/dw-edma/dw-edma-core.c178
-rw-r--r--drivers/dma/dw-edma/dw-edma-core.h37
-rw-r--r--drivers/dma/dw-edma/dw-edma-pcie.c277
-rw-r--r--drivers/dma/dw-edma/dw-edma-v0-core.c300
-rw-r--r--drivers/dma/dw-edma/dw-edma-v0-core.h2
-rw-r--r--drivers/dma/dw-edma/dw-edma-v0-debugfs.c77
-rw-r--r--drivers/dma/dw-edma/dw-edma-v0-debugfs.h4
-rw-r--r--drivers/dma/dw-edma/dw-edma-v0-regs.h291
-rw-r--r--drivers/dma/dw/Kconfig2
-rw-r--r--drivers/dma/idxd/Makefile2
-rw-r--r--drivers/dma/idxd/cdev.c132
-rw-r--r--drivers/dma/idxd/device.c346
-rw-r--r--drivers/dma/idxd/dma.c77
-rw-r--r--drivers/dma/idxd/idxd.h171
-rw-r--r--drivers/dma/idxd/init.c496
-rw-r--r--drivers/dma/idxd/irq.c33
-rw-r--r--drivers/dma/idxd/perfmon.c662
-rw-r--r--drivers/dma/idxd/perfmon.h119
-rw-r--r--drivers/dma/idxd/registers.h120
-rw-r--r--drivers/dma/idxd/submit.c42
-rw-r--r--drivers/dma/idxd/sysfs.c795
-rw-r--r--drivers/dma/k3dma.c4
-rw-r--r--drivers/dma/plx_dma.c18
-rw-r--r--drivers/dma/qcom/gpi.c1
-rw-r--r--drivers/dma/qcom/hidma.c6
-rw-r--r--drivers/dma/tegra20-apb-dma.c4
-rw-r--r--drivers/dma/xilinx/xilinx_dma.c8
-rw-r--r--drivers/dma/xilinx/xilinx_dpdma.c31
-rw-r--r--drivers/edac/Kconfig2
-rw-r--r--drivers/edac/altera_edac.c17
-rw-r--r--drivers/edac/amd64_edac.c2
-rw-r--r--drivers/extcon/Kconfig8
-rw-r--r--drivers/extcon/Makefile1
-rw-r--r--drivers/extcon/extcon-arizona.c1816
-rw-r--r--drivers/extcon/extcon-gpio.c15
-rw-r--r--drivers/extcon/extcon-intel-int3496.c16
-rw-r--r--drivers/extcon/extcon-max8997.c4
-rw-r--r--drivers/extcon/extcon-palmas.c17
-rw-r--r--drivers/extcon/extcon-qcom-spmi-misc.c116
-rw-r--r--drivers/extcon/extcon-sm5502.c22
-rw-r--r--drivers/extcon/extcon.c1
-rw-r--r--drivers/firewire/nosy.c9
-rw-r--r--drivers/firewire/ohci.c4
-rw-r--r--drivers/firmware/Kconfig3
-rw-r--r--drivers/firmware/arm_scmi/base.c142
-rw-r--r--drivers/firmware/arm_scmi/bus.c100
-rw-r--r--drivers/firmware/arm_scmi/clock.c129
-rw-r--r--drivers/firmware/arm_scmi/common.h133
-rw-r--r--drivers/firmware/arm_scmi/driver.c798
-rw-r--r--drivers/firmware/arm_scmi/notify.c328
-rw-r--r--drivers/firmware/arm_scmi/notify.h40
-rw-r--r--drivers/firmware/arm_scmi/perf.c262
-rw-r--r--drivers/firmware/arm_scmi/power.c134
-rw-r--r--drivers/firmware/arm_scmi/reset.c146
-rw-r--r--drivers/firmware/arm_scmi/scmi_pm_domain.c26
-rw-r--r--drivers/firmware/arm_scmi/sensors.c232
-rw-r--r--drivers/firmware/arm_scmi/system.c63
-rw-r--r--drivers/firmware/arm_scmi/voltage.c126
-rw-r--r--drivers/firmware/broadcom/bcm47xx_nvram.c92
-rw-r--r--drivers/firmware/dmi-id.c1
-rw-r--r--drivers/firmware/dmi_scan.c1
-rw-r--r--drivers/firmware/efi/efi.c3
-rw-r--r--drivers/firmware/efi/libstub/Makefile3
-rw-r--r--drivers/firmware/efi/libstub/arm64-stub.c2
-rw-r--r--drivers/firmware/efi/libstub/efi-stub.c16
-rw-r--r--drivers/firmware/efi/vars.c4
-rw-r--r--drivers/firmware/google/gsmi.c14
-rw-r--r--drivers/firmware/imx/scu-pd.c41
-rw-r--r--drivers/firmware/psci/psci.c11
-rw-r--r--drivers/firmware/qcom_scm-legacy.c4
-rw-r--r--drivers/firmware/qcom_scm-smc.c12
-rw-r--r--drivers/firmware/qcom_scm.c89
-rw-r--r--drivers/firmware/qcom_scm.h7
-rw-r--r--drivers/firmware/raspberrypi.c69
-rw-r--r--drivers/firmware/smccc/Makefile2
-rw-r--r--drivers/firmware/smccc/kvm_guest.c50
-rw-r--r--drivers/firmware/smccc/smccc.c1
-rw-r--r--drivers/firmware/turris-mox-rwtm.c4
-rw-r--r--drivers/firmware/xilinx/zynqmp.c119
-rw-r--r--drivers/fpga/Kconfig17
-rw-r--r--drivers/fpga/dfl-afu-error.c10
-rw-r--r--drivers/fpga/dfl-afu-main.c35
-rw-r--r--drivers/fpga/dfl-afu.h2
-rw-r--r--drivers/fpga/dfl-pci.c18
-rw-r--r--drivers/fpga/xilinx-pr-decoupler.c45
-rw-r--r--drivers/fpga/xilinx-spi.c24
-rw-r--r--drivers/gpio/Kconfig34
-rw-r--r--drivers/gpio/Makefile2
-rw-r--r--drivers/gpio/gpio-104-dio-48e.c50
-rw-r--r--drivers/gpio/gpio-aggregator.c39
-rw-r--r--drivers/gpio/gpio-bd71815.c185
-rw-r--r--drivers/gpio/gpio-ich.c2
-rw-r--r--drivers/gpio/gpio-it87.c8
-rw-r--r--drivers/gpio/gpio-mockup.c9
-rw-r--r--drivers/gpio/gpio-moxtet.c4
-rw-r--r--drivers/gpio/gpio-mpc8xxx.c47
-rw-r--r--drivers/gpio/gpio-mxs.c5
-rw-r--r--drivers/gpio/gpio-omap.c14
-rw-r--r--drivers/gpio/gpio-pca953x.c78
-rw-r--r--drivers/gpio/gpio-raspberrypi-exp.c2
-rw-r--r--drivers/gpio/gpio-realtek-otto.c325
-rw-r--r--drivers/gpio/gpio-regmap.c5
-rw-r--r--drivers/gpio/gpio-sch.c198
-rw-r--r--drivers/gpio/gpiolib-acpi.c42
-rw-r--r--drivers/gpio/gpiolib-acpi.h4
-rw-r--r--drivers/gpio/gpiolib-of.c6
-rw-r--r--drivers/gpio/gpiolib-sysfs.c8
-rw-r--r--drivers/gpio/gpiolib.c91
-rw-r--r--drivers/gpu/drm/Kconfig5
-rw-r--r--drivers/gpu/drm/Makefile7
-rw-r--r--drivers/gpu/drm/amd/amdgpu/Kconfig9
-rw-r--r--drivers/gpu/drm/amd/amdgpu/Makefile21
-rw-r--r--drivers/gpu/drm/amd/amdgpu/aldebaran.c407
-rw-r--r--drivers/gpu/drm/amd/amdgpu/aldebaran.h32
-rw-r--r--drivers/gpu/drm/amd/amdgpu/aldebaran_reg_init.c54
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu.h82
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c20
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h5
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_aldebaran.c47
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c16
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h30
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c6
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c100
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c196
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h12
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c73
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c16
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c14
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c184
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.h14
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_device.c720
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c28
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_display.c346
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_display.h3
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c33
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c231
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c10
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c101
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c30
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h3
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c28
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c54
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h21
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c224
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h34
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c6
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c25
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c19
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c54
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h6
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c54
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_job.c6
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c147
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.h12
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h8
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.h19
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_object.c143
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_object.h24
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c148
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h18
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_rap.c7
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c485
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h13
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c51
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h105
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c98
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_reset.h85
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c6
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h6
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_rlc.h3
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h5
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c13
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_smuio.h2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_test.c1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c500
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h12
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c17
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h9
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c8
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h14
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c11
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h28
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c41
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c70
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c43
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.h7
-rw-r--r--drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h17
-rw-r--r--drivers/gpu/drm/amd/amdgpu/athub_v2_1.c6
-rw-r--r--drivers/gpu/drm/amd/amdgpu/atom.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/atom.h2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/cik.c77
-rw-r--r--drivers/gpu/drm/amd/amdgpu/cik_sdma.c5
-rw-r--r--drivers/gpu/drm/amd/amdgpu/dce_v10_0.c14
-rw-r--r--drivers/gpu/drm/amd/amdgpu/dce_v11_0.c14
-rw-r--r--drivers/gpu/drm/amd/amdgpu/dce_v6_0.c13
-rw-r--r--drivers/gpu/drm/amd/amdgpu/dce_v8_0.c14
-rw-r--r--drivers/gpu/drm/amd/amdgpu/dce_virtual.c15
-rw-r--r--drivers/gpu/drm/amd/amdgpu/df_v3_6.c4
-rw-r--r--[-rwxr-xr-x]drivers/gpu/drm/amd/amdgpu/dimgrey_cavefish_reg_init.c0
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c153
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c26
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c15
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c225
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c36
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v9_4.h12
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c1301
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.h35
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c69
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfxhub_v1_1.c54
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c3
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c3
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c12
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c14
-rw-r--r--drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c195
-rw-r--r--drivers/gpu/drm/amd/amdgpu/hdp_v4_0.c3
-rw-r--r--drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c67
-rw-r--r--drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mes_v10_1.c3
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c9
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.c1333
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.h29
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c19
-rw-r--r--drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c128
-rw-r--r--drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c189
-rw-r--r--drivers/gpu/drm/amd/amdgpu/nbio_v7_4.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/nv.c299
-rw-r--r--drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h37
-rw-r--r--drivers/gpu/drm/amd/amdgpu/psp_v11_0.c8
-rw-r--r--drivers/gpu/drm/amd/amdgpu/psp_v13_0.c378
-rw-r--r--drivers/gpu/drm/amd/amdgpu/psp_v13_0.h30
-rw-r--r--drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c8
-rw-r--r--drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c8
-rw-r--r--drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c185
-rw-r--r--drivers/gpu/drm/amd/amdgpu/sdma_v4_4.c253
-rw-r--r--drivers/gpu/drm/amd/amdgpu/sdma_v4_4.h28
-rw-r--r--drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c8
-rw-r--r--drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c37
-rw-r--r--drivers/gpu/drm/amd/amdgpu/si.c111
-rw-r--r--drivers/gpu/drm/amd/amdgpu/si_dma.c5
-rw-r--r--drivers/gpu/drm/amd/amdgpu/smuio_v13_0.c121
-rw-r--r--drivers/gpu/drm/amd/amdgpu/smuio_v13_0.h30
-rw-r--r--drivers/gpu/drm/amd/amdgpu/soc15.c386
-rw-r--r--drivers/gpu/drm/amd/amdgpu/soc15.h12
-rw-r--r--drivers/gpu/drm/amd/amdgpu/soc15_common.h63
-rw-r--r--drivers/gpu/drm/amd/amdgpu/ta_secureDisplay_if.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/umc_v6_1.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/umc_v6_1.h2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/umc_v6_7.c281
-rw-r--r--drivers/gpu/drm/amd/amdgpu/umc_v6_7.h37
-rw-r--r--drivers/gpu/drm/amd/amdgpu/umc_v8_7.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/umc_v8_7.h2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vce_v2_0.c5
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vce_v3_0.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vce_v4_0.c2
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c17
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c4
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c103
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vcn_v2_5.h1
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c171
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vega10_ih.c32
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vega20_ih.c55
-rw-r--r--drivers/gpu/drm/amd/amdgpu/vi.c383
-rw-r--r--drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c5
-rw-r--r--drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler.h492
-rw-r--r--drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx8.asm1
-rw-r--r--drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx9.asm204
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_chardev.c122
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_crat.c17
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c2
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_debugfs.c7
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_device.c27
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c63
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h7
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c1
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c134
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_iommu.c18
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_iommu.h9
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h1
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c8
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c8
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c8
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c8
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c3
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c2
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c2
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_priv.h37
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_process.c201
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c6
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c2
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_topology.c1
-rw-r--r--drivers/gpu/drm/amd/amdkfd/kfd_topology.h5
-rw-r--r--drivers/gpu/drm/amd/display/Kconfig14
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c1090
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h84
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c295
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h37
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c731
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.h5
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c15
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c79
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c185
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h9
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c26
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c134
-rw-r--r--drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h40
-rw-r--r--drivers/gpu/drm/amd/display/dc/Makefile3
-rw-r--r--drivers/gpu/drm/amd/display/dc/basics/dc_common.c20
-rw-r--r--drivers/gpu/drm/amd/display/dc/basics/dc_common.h4
-rw-r--r--drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c195
-rw-r--r--drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.c5
-rw-r--r--drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.h2
-rw-r--r--drivers/gpu/drm/amd/display/dc/bios/command_table.c21
-rw-r--r--drivers/gpu/drm/amd/display/dc/bios/command_table2.c93
-rw-r--r--drivers/gpu/drm/amd/display/dc/bios/command_table2.h3
-rw-r--r--drivers/gpu/drm/amd/display/dc/calcs/dce_calcs.c1154
-rw-r--r--drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c137
-rw-r--r--drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c71
-rw-r--r--drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c11
-rw-r--r--drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c29
-rw-r--r--drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c105
-rw-r--r--drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.h28
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc.c378
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc_link.c136
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c5
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c256
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c303
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c28
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc_resource.c83
-rw-r--r--drivers/gpu/drm/amd/display/dc/core/dc_stat.c64
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc.h40
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_bios_types.h9
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_ddc_types.h10
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c34
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h8
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_dsc.h11
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_hw_types.h2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_link.h39
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_stat.h42
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_stream.h16
-rw-r--r--drivers/gpu/drm/amd/display/dc/dc_types.h19
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/Makefile2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dce_abm.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dce_aux.c40
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dce_aux.h4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c52
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c90
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dce_opp.c4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dmub_abm.c10
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.c60
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.h33
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c34
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c106
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c16
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.c11
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.h6
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c331
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.h15
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c16
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.h5
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dccg.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c16
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c55
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c3
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_link_encoder.c3
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c140
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h3
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c45
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hwseq.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c28
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_cm_common.c26
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp_cm.c4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb_cm.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.h1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c100
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c26
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c54
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.h1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c6
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c103
-rw-r--r--drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c7
-rw-r--r--drivers/gpu/drm/amd/display/dc/dm_helpers.h4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20.c1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20v2.c1
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20.c28
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20v2.c28
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c28
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dcn30/display_rq_dlg_calc_30.c28
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.c2
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/display_mode_structs.h4
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.c11
-rw-r--r--drivers/gpu/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c28
-rw-r--r--drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c105
-rw-r--r--drivers/gpu/drm/amd/display/dc/gpio/dce110/hw_factory_dce110.c4
-rw-r--r--drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c9
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/clock_source.h5
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/core_types.h42
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/dc_link_ddc.h3
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h8
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw/aux_engine.h4
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h10
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h21
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h3
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h1
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h13
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h12
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h12
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h86
-rw-r--r--drivers/gpu/drm/amd/display/dc/inc/resource.h5
-rw-r--r--drivers/gpu/drm/amd/display/dc/irq/dcn20/irq_service_dcn20.c31
-rw-r--r--drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c63
-rw-r--r--drivers/gpu/drm/amd/display/dc/irq/dcn30/irq_service_dcn30.c63
-rw-r--r--drivers/gpu/drm/amd/display/dc/irq/dcn302/irq_service_dcn302.c60
-rw-r--r--drivers/gpu/drm/amd/display/dc/irq_types.h3
-rw-r--r--drivers/gpu/drm/amd/display/dmub/dmub_srv.h55
-rw-r--r--drivers/gpu/drm/amd/display/dmub/dmub_srv_stat.h41
-rw-r--r--drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h1496
-rw-r--r--drivers/gpu/drm/amd/display/dmub/inc/dmub_trace_buffer.h3
-rw-r--r--drivers/gpu/drm/amd/display/dmub/src/Makefile2
-rw-r--r--drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.c57
-rw-r--r--drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.h35
-rw-r--r--drivers/gpu/drm/amd/display/dmub/src/dmub_dcn30.c7
-rw-r--r--drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c182
-rw-r--r--drivers/gpu/drm/amd/display/dmub/src/dmub_srv_stat.c105
-rw-r--r--drivers/gpu/drm/amd/display/include/logger_types.h3
-rw-r--r--drivers/gpu/drm/amd/display/modules/color/color_gamma.c6
-rw-r--r--drivers/gpu/drm/amd/display/modules/color/color_gamma.h4
-rw-r--r--drivers/gpu/drm/amd/display/modules/freesync/freesync.c69
-rw-r--r--drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c10
-rw-r--r--drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h4
-rw-r--r--drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c35
-rw-r--r--drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_transition.c6
-rw-r--r--drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_execution.c8
-rw-r--r--drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_transition.c10
-rw-r--r--drivers/gpu/drm/amd/display/modules/hdcp/hdcp_psp.c12
-rw-r--r--drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h10
-rw-r--r--drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h12
-rw-r--r--drivers/gpu/drm/amd/include/aldebaran_ip_offset.h1738
-rw-r--r--drivers/gpu/drm/amd/include/amd_shared.h6
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h2
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_offset.h6
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_sh_mask.h9
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_1_sh_mask.h16
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_2_offset.h7683
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_2_sh_mask.h32949
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/mmhub/mmhub_1_7_offset.h5125
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/mmhub/mmhub_1_7_sh_mask.h32178
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/mp/mp_13_0_2_offset.h361
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/mp/mp_13_0_2_sh_mask.h531
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/sdma/sdma_4_4_0_offset.h5224
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/sdma/sdma_4_4_0_sh_mask.h13922
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_13_0_2_offset.h516
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/smuio/smuio_13_0_2_sh_mask.h1163
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/thm/thm_13_0_2_offset.h346
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/thm/thm_13_0_2_sh_mask.h1297
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/umc/umc_6_7_0_offset.h2620
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/umc/umc_6_7_0_sh_mask.h10796
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_6_0_offset.h1462
-rw-r--r--drivers/gpu/drm/amd/include/asic_reg/vcn/vcn_2_6_0_sh_mask.h4535
-rw-r--r--drivers/gpu/drm/amd/include/atombios.h4
-rw-r--r--drivers/gpu/drm/amd/include/atomfirmware.h208
-rw-r--r--drivers/gpu/drm/amd/include/ivsrcid/dcn/irqsrcs_dcn_1_0.h4
-rw-r--r--drivers/gpu/drm/amd/include/kgd_pp_interface.h141
-rw-r--r--drivers/gpu/drm/amd/include/soc15_ih_clientid.h9
-rw-r--r--drivers/gpu/drm/amd/include/vi_structs.h32
-rw-r--r--drivers/gpu/drm/amd/pm/Makefile1
-rw-r--r--drivers/gpu/drm/amd/pm/amdgpu_dpm.c236
-rw-r--r--drivers/gpu/drm/amd/pm/amdgpu_pm.c1000
-rw-r--r--drivers/gpu/drm/amd/pm/inc/aldebaran_ppsmc.h127
-rw-r--r--drivers/gpu/drm/amd/pm/inc/amdgpu_pm.h2
-rw-r--r--drivers/gpu/drm/amd/pm/inc/amdgpu_smu.h141
-rw-r--r--drivers/gpu/drm/amd/pm/inc/arcturus_ppsmc.h7
-rw-r--r--drivers/gpu/drm/amd/pm/inc/rv_ppsmc.h1
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu11_driver_if_navi10.h98
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu11_driver_if_vangogh.h40
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu13_driver_if_aldebaran.h519
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu_types.h29
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu_v11_0.h21
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu_v12_0.h2
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu_v13_0.h275
-rw-r--r--drivers/gpu/drm/amd/pm/inc/smu_v13_0_pptable.h165
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/amd_powerplay.c12
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu10_hwmgr.c45
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu7_hwmgr.c70
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/hwmgr/smu8_hwmgr.c9
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_hwmgr.c126
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega10_pptable.h6
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega12_hwmgr.c96
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/hwmgr/vega20_hwmgr.c79
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/kv_dpm.c2
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/si_dpm.c182
-rw-r--r--drivers/gpu/drm/amd/pm/powerplay/sislands_smc.h44
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/Makefile2
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/amdgpu_smu.c723
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu11/arcturus_ppt.c19
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu11/navi10_ppt.c651
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu11/sienna_cichlid_ppt.c66
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c27
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu11/vangogh_ppt.c442
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu12/renoir_ppt.c31
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu12/smu_v12_0.c123
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu13/Makefile30
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.c1859
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu13/aldebaran_ppt.h72
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c1837
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu_cmn.c50
-rw-r--r--drivers/gpu/drm/amd/pm/swsmu/smu_cmn.h5
-rw-r--r--drivers/gpu/drm/arc/Kconfig10
-rw-r--r--drivers/gpu/drm/arc/Makefile3
-rw-r--r--drivers/gpu/drm/arc/arcpgu.h37
-rw-r--r--drivers/gpu/drm/arc/arcpgu_crtc.c217
-rw-r--r--drivers/gpu/drm/arc/arcpgu_drv.c224
-rw-r--r--drivers/gpu/drm/arc/arcpgu_hdmi.c48
-rw-r--r--drivers/gpu/drm/arc/arcpgu_regs.h31
-rw-r--r--drivers/gpu/drm/arc/arcpgu_sim.c108
-rw-r--r--drivers/gpu/drm/arm/display/include/malidp_utils.h3
-rw-r--r--drivers/gpu/drm/arm/display/komeda/komeda_dev.c6
-rw-r--r--drivers/gpu/drm/arm/display/komeda/komeda_format_caps.h11
-rw-r--r--drivers/gpu/drm/arm/display/komeda/komeda_framebuffer.c4
-rw-r--r--drivers/gpu/drm/arm/display/komeda/komeda_kms.c3
-rw-r--r--drivers/gpu/drm/arm/display/komeda/komeda_pipeline.c16
-rw-r--r--drivers/gpu/drm/arm/display/komeda/komeda_pipeline_state.c19
-rw-r--r--drivers/gpu/drm/arm/display/komeda/komeda_plane.c21
-rw-r--r--drivers/gpu/drm/arm/hdlcd_crtc.c30
-rw-r--r--drivers/gpu/drm/arm/malidp_drv.c3
-rw-r--r--drivers/gpu/drm/arm/malidp_mw.c7
-rw-r--r--drivers/gpu/drm/arm/malidp_planes.c79
-rw-r--r--drivers/gpu/drm/armada/armada_overlay.c113
-rw-r--r--drivers/gpu/drm/armada/armada_plane.c115
-rw-r--r--drivers/gpu/drm/armada/armada_plane.h2
-rw-r--r--drivers/gpu/drm/aspeed/aspeed_gfx.h8
-rw-r--r--drivers/gpu/drm/aspeed/aspeed_gfx_crtc.c15
-rw-r--r--drivers/gpu/drm/aspeed/aspeed_gfx_drv.c69
-rw-r--r--drivers/gpu/drm/ast/Makefile3
-rw-r--r--drivers/gpu/drm/ast/ast_cursor.c286
-rw-r--r--drivers/gpu/drm/ast/ast_drv.c2
-rw-r--r--drivers/gpu/drm/ast/ast_drv.h47
-rw-r--r--drivers/gpu/drm/ast/ast_mode.c388
-rw-r--r--drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.c107
-rw-r--r--drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_dc.h7
-rw-r--r--drivers/gpu/drm/atmel-hlcdc/atmel_hlcdc_plane.c135
-rw-r--r--drivers/gpu/drm/bridge/Kconfig33
-rw-r--r--drivers/gpu/drm/bridge/Makefile2
-rw-r--r--drivers/gpu/drm/bridge/analogix/Kconfig1
-rw-r--r--drivers/gpu/drm/bridge/analogix/analogix-anx6345.c27
-rw-r--r--drivers/gpu/drm/bridge/analogix/analogix-anx78xx.c27
-rw-r--r--drivers/gpu/drm/bridge/analogix/analogix_dp_core.c1
-rw-r--r--drivers/gpu/drm/bridge/analogix/anx7625.c34
-rw-r--r--drivers/gpu/drm/bridge/analogix/anx7625.h1
-rw-r--r--drivers/gpu/drm/bridge/cadence/cdns-mhdp8546-core.c2
-rw-r--r--drivers/gpu/drm/bridge/chipone-icn6211.c293
-rw-r--r--drivers/gpu/drm/bridge/lontium-lt8912b.c768
-rw-r--r--drivers/gpu/drm/bridge/lontium-lt9611.c8
-rw-r--r--drivers/gpu/drm/bridge/lontium-lt9611uxc.c2
-rw-r--r--drivers/gpu/drm/bridge/panel.c12
-rw-r--r--drivers/gpu/drm/bridge/tc358767.c20
-rw-r--r--drivers/gpu/drm/bridge/ti-sn65dsi86.c18
-rw-r--r--drivers/gpu/drm/drm_atomic.c51
-rw-r--r--drivers/gpu/drm/drm_atomic_helper.c90
-rw-r--r--drivers/gpu/drm/drm_connector.c1
-rw-r--r--drivers/gpu/drm/drm_crtc.c7
-rw-r--r--drivers/gpu/drm/drm_displayid.c132
-rw-r--r--drivers/gpu/drm/drm_dp_helper.c24
-rw-r--r--drivers/gpu/drm/drm_dp_mst_topology.c99
-rw-r--r--drivers/gpu/drm/drm_drv.c2
-rw-r--r--drivers/gpu/drm/drm_edid.c171
-rw-r--r--drivers/gpu/drm/drm_fb_helper.c2
-rw-r--r--drivers/gpu/drm/drm_framebuffer.c11
-rw-r--r--drivers/gpu/drm/drm_gem.c2
-rw-r--r--drivers/gpu/drm/drm_gem_atomic_helper.c432
-rw-r--r--drivers/gpu/drm/drm_gem_framebuffer_helper.c63
-rw-r--r--drivers/gpu/drm/drm_gem_shmem_helper.c32
-rw-r--r--drivers/gpu/drm/drm_gem_vram_helper.c42
-rw-r--r--drivers/gpu/drm/drm_internal.h1
-rw-r--r--drivers/gpu/drm/drm_ioc32.c26
-rw-r--r--drivers/gpu/drm/drm_mipi_dbi.c5
-rw-r--r--drivers/gpu/drm/drm_modes.c6
-rw-r--r--drivers/gpu/drm/drm_panel_orientation_quirks.c14
-rw-r--r--drivers/gpu/drm/drm_plane.c66
-rw-r--r--drivers/gpu/drm/drm_probe_helper.c7
-rw-r--r--drivers/gpu/drm/drm_property.c4
-rw-r--r--drivers/gpu/drm/drm_simple_kms_helper.c50
-rw-r--r--drivers/gpu/drm/drm_syncobj.c37
-rw-r--r--drivers/gpu/drm/drm_sysfs.c9
-rw-r--r--drivers/gpu/drm/drm_vblank.c28
-rw-r--r--drivers/gpu/drm/etnaviv/etnaviv_gem.c3
-rw-r--r--drivers/gpu/drm/etnaviv/etnaviv_sched.c9
-rw-r--r--drivers/gpu/drm/exynos/exynos5433_drm_decon.c5
-rw-r--r--drivers/gpu/drm/exynos/exynos_drm_dsi.c7
-rw-r--r--drivers/gpu/drm/exynos/exynos_drm_plane.c20
-rw-r--r--drivers/gpu/drm/fsl-dcu/fsl_dcu_drm_plane.c24
-rw-r--r--drivers/gpu/drm/gma500/Kconfig11
-rw-r--r--drivers/gpu/drm/gma500/Makefile17
-rw-r--r--drivers/gpu/drm/gma500/accel_2d.c60
-rw-r--r--drivers/gpu/drm/gma500/blitter.c43
-rw-r--r--drivers/gpu/drm/gma500/blitter.h16
-rw-r--r--drivers/gpu/drm/gma500/cdv_device.c2
-rw-r--r--drivers/gpu/drm/gma500/cdv_device.h1
-rw-r--r--drivers/gpu/drm/gma500/cdv_intel_crt.c15
-rw-r--r--drivers/gpu/drm/gma500/cdv_intel_display.c23
-rw-r--r--drivers/gpu/drm/gma500/gtt.c11
-rw-r--r--drivers/gpu/drm/gma500/intel_gmbus.c4
-rw-r--r--drivers/gpu/drm/gma500/oaktrail_device.c2
-rw-r--r--drivers/gpu/drm/gma500/power.c3
-rw-r--r--drivers/gpu/drm/gma500/psb_device.c2
-rw-r--r--drivers/gpu/drm/gma500/psb_drv.c38
-rw-r--r--drivers/gpu/drm/gma500/psb_drv.h6
-rw-r--r--drivers/gpu/drm/gma500/psb_intel_display.c2
-rw-r--r--drivers/gpu/drm/gma500/psb_intel_reg.h32
-rw-r--r--drivers/gpu/drm/gma500/psb_irq.c110
-rw-r--r--drivers/gpu/drm/gma500/psb_irq.h4
-rw-r--r--drivers/gpu/drm/gud/Kconfig14
-rw-r--r--drivers/gpu/drm/gud/Makefile4
-rw-r--r--drivers/gpu/drm/gud/gud_connector.c729
-rw-r--r--drivers/gpu/drm/gud/gud_drv.c661
-rw-r--r--drivers/gpu/drm/gud/gud_internal.h154
-rw-r--r--drivers/gpu/drm/gud/gud_pipe.c552
-rw-r--r--drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_de.c39
-rw-r--r--drivers/gpu/drm/hisilicon/kirin/kirin_drm_ade.c47
-rw-r--r--drivers/gpu/drm/i915/Kconfig2
-rw-r--r--drivers/gpu/drm/i915/Kconfig.profile14
-rw-r--r--drivers/gpu/drm/i915/Makefile10
-rw-r--r--drivers/gpu/drm/i915/TODO.txt41
-rw-r--r--drivers/gpu/drm/i915/display/g4x_dp.c1432
-rw-r--r--drivers/gpu/drm/i915/display/g4x_dp.h30
-rw-r--r--drivers/gpu/drm/i915/display/g4x_hdmi.c616
-rw-r--r--drivers/gpu/drm/i915/display/g4x_hdmi.h19
-rw-r--r--drivers/gpu/drm/i915/display/i9xx_plane.c196
-rw-r--r--drivers/gpu/drm/i915/display/i9xx_plane.h4
-rw-r--r--drivers/gpu/drm/i915/display/icl_dsi.c42
-rw-r--r--drivers/gpu/drm/i915/display/intel_acpi.c22
-rw-r--r--drivers/gpu/drm/i915/display/intel_atomic.c9
-rw-r--r--drivers/gpu/drm/i915/display/intel_atomic_plane.c5
-rw-r--r--drivers/gpu/drm/i915/display/intel_audio.c48
-rw-r--r--drivers/gpu/drm/i915/display/intel_bios.c1248
-rw-r--r--drivers/gpu/drm/i915/display/intel_bios.h19
-rw-r--r--drivers/gpu/drm/i915/display/intel_bw.c32
-rw-r--r--drivers/gpu/drm/i915/display/intel_cdclk.c88
-rw-r--r--drivers/gpu/drm/i915/display/intel_color.c32
-rw-r--r--drivers/gpu/drm/i915/display/intel_combo_phy.c31
-rw-r--r--drivers/gpu/drm/i915/display/intel_crt.c18
-rw-r--r--drivers/gpu/drm/i915/display/intel_crt.h1
-rw-r--r--drivers/gpu/drm/i915/display/intel_crtc.c290
-rw-r--r--drivers/gpu/drm/i915/display/intel_csr.c16
-rw-r--r--drivers/gpu/drm/i915/display/intel_cursor.c29
-rw-r--r--drivers/gpu/drm/i915/display/intel_ddi.c2785
-rw-r--r--drivers/gpu/drm/i915/display/intel_ddi.h23
-rw-r--r--drivers/gpu/drm/i915/display/intel_ddi_buf_trans.c1394
-rw-r--r--drivers/gpu/drm/i915/display/intel_ddi_buf_trans.h100
-rw-r--r--drivers/gpu/drm/i915/display/intel_display.c3863
-rw-r--r--drivers/gpu/drm/i915/display/intel_display.h56
-rw-r--r--drivers/gpu/drm/i915/display/intel_display_debugfs.c209
-rw-r--r--drivers/gpu/drm/i915/display/intel_display_power.c283
-rw-r--r--drivers/gpu/drm/i915/display/intel_display_power.h32
-rw-r--r--drivers/gpu/drm/i915/display/intel_display_types.h179
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp.c1736
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp.h11
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp_aux.c19
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp_aux_backlight.c1
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp_hdcp.c115
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp_link_training.c124
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp_link_training.h5
-rw-r--r--drivers/gpu/drm/i915/display/intel_dp_mst.c23
-rw-r--r--drivers/gpu/drm/i915/display/intel_dpll.c519
-rw-r--r--drivers/gpu/drm/i915/display/intel_dpll.h18
-rw-r--r--drivers/gpu/drm/i915/display/intel_dpll_mgr.c117
-rw-r--r--drivers/gpu/drm/i915/display/intel_dpll_mgr.h9
-rw-r--r--drivers/gpu/drm/i915/display/intel_dsb.c2
-rw-r--r--drivers/gpu/drm/i915/display/intel_dsi_vbt.c6
-rw-r--r--drivers/gpu/drm/i915/display/intel_fb.c962
-rw-r--r--drivers/gpu/drm/i915/display/intel_fb.h54
-rw-r--r--drivers/gpu/drm/i915/display/intel_fbc.c68
-rw-r--r--drivers/gpu/drm/i915/display/intel_fbdev.c4
-rw-r--r--drivers/gpu/drm/i915/display/intel_fdi.c144
-rw-r--r--drivers/gpu/drm/i915/display/intel_fdi.h3
-rw-r--r--drivers/gpu/drm/i915/display/intel_fifo_underrun.c8
-rw-r--r--drivers/gpu/drm/i915/display/intel_frontbuffer.c2
-rw-r--r--drivers/gpu/drm/i915/display/intel_gmbus.c4
-rw-r--r--drivers/gpu/drm/i915/display/intel_hdcp.c54
-rw-r--r--drivers/gpu/drm/i915/display/intel_hdmi.c714
-rw-r--r--drivers/gpu/drm/i915/display/intel_hdmi.h3
-rw-r--r--drivers/gpu/drm/i915/display/intel_lpe_audio.c5
-rw-r--r--drivers/gpu/drm/i915/display/intel_lvds.c12
-rw-r--r--drivers/gpu/drm/i915/display/intel_opregion.c6
-rw-r--r--drivers/gpu/drm/i915/display/intel_overlay.c46
-rw-r--r--drivers/gpu/drm/i915/display/intel_panel.c22
-rw-r--r--drivers/gpu/drm/i915/display/intel_pipe_crc.c16
-rw-r--r--drivers/gpu/drm/i915/display/intel_pps.c8
-rw-r--r--drivers/gpu/drm/i915/display/intel_psr.c653
-rw-r--r--drivers/gpu/drm/i915/display/intel_psr.h10
-rw-r--r--drivers/gpu/drm/i915/display/intel_quirks.c2
-rw-r--r--drivers/gpu/drm/i915/display/intel_sdvo.c10
-rw-r--r--drivers/gpu/drm/i915/display/intel_sprite.c1714
-rw-r--r--drivers/gpu/drm/i915/display/intel_sprite.h8
-rw-r--r--drivers/gpu/drm/i915/display/intel_tc.c8
-rw-r--r--drivers/gpu/drm/i915/display/intel_tv.c8
-rw-r--r--drivers/gpu/drm/i915/display/intel_vbt_defs.h27
-rw-r--r--drivers/gpu/drm/i915/display/intel_vdsc.c22
-rw-r--r--drivers/gpu/drm/i915/display/intel_vga.c12
-rw-r--r--drivers/gpu/drm/i915/display/intel_vrr.h1
-rw-r--r--drivers/gpu/drm/i915/display/skl_scaler.c556
-rw-r--r--drivers/gpu/drm/i915/display/skl_scaler.h29
-rw-r--r--drivers/gpu/drm/i915/display/skl_universal_plane.c2218
-rw-r--r--drivers/gpu/drm/i915/display/skl_universal_plane.h35
-rw-r--r--drivers/gpu/drm/i915/display/vlv_dsi.c5
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_clflush.c15
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_context.c79
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_context_types.h4
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_dmabuf.c67
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_domain.c72
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c335
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_fence.c95
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_internal.c8
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_ioctls.h2
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_lmem.c4
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_mman.c46
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_object.c10
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_object.h122
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_object_blt.c6
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_object_types.h23
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_pages.c109
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_phys.c116
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_pm.c2
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_region.c16
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_region.h3
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_shmem.c41
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_shrinker.c40
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_shrinker.h4
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_stolen.c18
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_tiling.c2
-rw-r--r--drivers/gpu/drm/i915/gem/i915_gem_userptr.c918
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/huge_gem_object.c4
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/huge_pages.c38
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_client_blt.c8
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_coherency.c14
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_context.c10
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_dmabuf.c2
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_execbuffer.c2
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_mman.c21
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_object.c2
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_object_blt.c10
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/i915_gem_phys.c10
-rw-r--r--drivers/gpu/drm/i915/gem/selftests/igt_gem_utils.c2
-rw-r--r--drivers/gpu/drm/i915/gt/debugfs_gt.c2
-rw-r--r--drivers/gpu/drm/i915/gt/gen2_engine_cs.c2
-rw-r--r--drivers/gpu/drm/i915/gt/gen6_engine_cs.c8
-rw-r--r--drivers/gpu/drm/i915/gt/gen6_ppgtt.h4
-rw-r--r--drivers/gpu/drm/i915/gt/gen6_renderstate.c20
-rw-r--r--drivers/gpu/drm/i915/gt/gen7_renderstate.c20
-rw-r--r--drivers/gpu/drm/i915/gt/gen8_engine_cs.c15
-rw-r--r--drivers/gpu/drm/i915/gt/gen8_ppgtt.c14
-rw-r--r--drivers/gpu/drm/i915/gt/gen8_renderstate.c20
-rw-r--r--drivers/gpu/drm/i915/gt/gen9_renderstate.c20
-rw-r--r--drivers/gpu/drm/i915/gt/intel_breadcrumbs.c23
-rw-r--r--drivers/gpu/drm/i915/gt/intel_context.c3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_context.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_context_param.h11
-rw-r--r--drivers/gpu/drm/i915/gt/intel_context_types.h7
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_cs.c80
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_heartbeat.c27
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_heartbeat.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_pm.c7
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_pm.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_types.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_user.c6
-rw-r--r--drivers/gpu/drm/i915/gt/intel_engine_user.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_execlists_submission.c146
-rw-r--r--drivers/gpu/drm/i915/gt/intel_execlists_submission.h2
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ggtt.c102
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ggtt_fencing.c40
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ggtt_fencing.h21
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gpu_commands.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt.c47
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.c47
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_buffer_pool.h5
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_buffer_pool_types.h4
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_clock_utils.c1
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_irq.c3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_irq.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_pm.c3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_pm.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_pm_irq.c3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_pm_irq.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_requests.c31
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_requests.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gt_types.h11
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gtt.c52
-rw-r--r--drivers/gpu/drm/i915/gt/intel_gtt.h13
-rw-r--r--drivers/gpu/drm/i915/gt/intel_llc.c3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_llc.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_llc_types.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_lrc.c55
-rw-r--r--drivers/gpu/drm/i915/gt/intel_lrc_reg.h5
-rw-r--r--drivers/gpu/drm/i915/gt/intel_mocs.c21
-rw-r--r--drivers/gpu/drm/i915/gt/intel_mocs.h22
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ppgtt.c9
-rw-r--r--drivers/gpu/drm/i915/gt/intel_rc6.c8
-rw-r--r--drivers/gpu/drm/i915/gt/intel_rc6.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_rc6_types.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_region_lmem.c105
-rw-r--r--drivers/gpu/drm/i915/gt/intel_region_lmem.h6
-rw-r--r--drivers/gpu/drm/i915/gt/intel_renderstate.c29
-rw-r--r--drivers/gpu/drm/i915/gt/intel_renderstate.h20
-rw-r--r--drivers/gpu/drm/i915/gt/intel_reset.c33
-rw-r--r--drivers/gpu/drm/i915/gt/intel_reset.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_reset_types.h2
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ring.c8
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ring.h4
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ring_submission.c330
-rw-r--r--drivers/gpu/drm/i915/gt/intel_ring_types.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_rps.c3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_rps.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_rps_types.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_sseu.c3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_sseu.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_timeline.c429
-rw-r--r--drivers/gpu/drm/i915/gt/intel_timeline.h23
-rw-r--r--drivers/gpu/drm/i915/gt/intel_timeline_types.h20
-rw-r--r--drivers/gpu/drm/i915/gt/intel_workarounds.c105
-rw-r--r--drivers/gpu/drm/i915/gt/intel_workarounds.h3
-rw-r--r--drivers/gpu/drm/i915/gt/intel_workarounds_types.h3
-rw-r--r--drivers/gpu/drm/i915/gt/mock_engine.c51
-rw-r--r--drivers/gpu/drm/i915/gt/mock_engine.h21
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_context.c7
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_engine.c3
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_engine.h3
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_engine_cs.c12
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_engine_heartbeat.c17
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_engine_pm.c11
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_execlists.c77
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_gt_pm.c4
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_hangcheck.c29
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_llc.c3
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_llc.h3
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_lrc.c19
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_mocs.c40
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_rc6.c3
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_rc6.h3
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_ring_submission.c4
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_timeline.c180
-rw-r--r--drivers/gpu/drm/i915/gt/selftest_workarounds.c101
-rw-r--r--drivers/gpu/drm/i915/gt/shmem_utils.c2
-rw-r--r--drivers/gpu/drm/i915/gt/uc/intel_guc.c2
-rw-r--r--drivers/gpu/drm/i915/gt/uc/intel_guc_log.c4
-rw-r--r--drivers/gpu/drm/i915/gt/uc/intel_guc_submission.c16
-rw-r--r--drivers/gpu/drm/i915/gt/uc/intel_huc.c2
-rw-r--r--drivers/gpu/drm/i915/gt/uc/intel_uc_fw.c6
-rw-r--r--drivers/gpu/drm/i915/gvt/cfg_space.c5
-rw-r--r--drivers/gpu/drm/i915/gvt/cmd_parser.c24
-rw-r--r--drivers/gpu/drm/i915/gvt/debugfs.c2
-rw-r--r--drivers/gpu/drm/i915/gvt/display.c107
-rw-r--r--drivers/gpu/drm/i915/gvt/display.h14
-rw-r--r--drivers/gpu/drm/i915/gvt/dmabuf.c2
-rw-r--r--drivers/gpu/drm/i915/gvt/firmware.c10
-rw-r--r--drivers/gpu/drm/i915/gvt/gtt.c22
-rw-r--r--drivers/gpu/drm/i915/gvt/gvt.c78
-rw-r--r--drivers/gpu/drm/i915/gvt/gvt.h17
-rw-r--r--drivers/gpu/drm/i915/gvt/handlers.c261
-rw-r--r--drivers/gpu/drm/i915/gvt/interrupt.c37
-rw-r--r--drivers/gpu/drm/i915/gvt/interrupt.h7
-rw-r--r--drivers/gpu/drm/i915/gvt/kvmgt.c11
-rw-r--r--drivers/gpu/drm/i915/gvt/vgpu.c2
-rw-r--r--drivers/gpu/drm/i915/i915_active.c23
-rw-r--r--drivers/gpu/drm/i915/i915_buddy.c3
-rw-r--r--drivers/gpu/drm/i915/i915_buddy.h7
-rw-r--r--drivers/gpu/drm/i915/i915_cmd_parser.c123
-rw-r--r--drivers/gpu/drm/i915/i915_debugfs.c22
-rw-r--r--drivers/gpu/drm/i915/i915_drv.c95
-rw-r--r--drivers/gpu/drm/i915/i915_drv.h209
-rw-r--r--drivers/gpu/drm/i915/i915_gem.c258
-rw-r--r--drivers/gpu/drm/i915/i915_gem.h2
-rw-r--r--drivers/gpu/drm/i915/i915_gem_gtt.c7
-rw-r--r--drivers/gpu/drm/i915/i915_getparam.c5
-rw-r--r--drivers/gpu/drm/i915/i915_gpu_error.c2
-rw-r--r--drivers/gpu/drm/i915/i915_irq.c168
-rw-r--r--drivers/gpu/drm/i915/i915_memcpy.c2
-rw-r--r--drivers/gpu/drm/i915/i915_memcpy.h2
-rw-r--r--drivers/gpu/drm/i915/i915_mm.c117
-rw-r--r--drivers/gpu/drm/i915/i915_params.c5
-rw-r--r--drivers/gpu/drm/i915/i915_params.h5
-rw-r--r--drivers/gpu/drm/i915/i915_pci.c23
-rw-r--r--drivers/gpu/drm/i915/i915_perf.c153
-rw-r--r--drivers/gpu/drm/i915/i915_perf_types.h8
-rw-r--r--drivers/gpu/drm/i915/i915_pmu.c2
-rw-r--r--drivers/gpu/drm/i915/i915_priolist_types.h11
-rw-r--r--drivers/gpu/drm/i915/i915_reg.h78
-rw-r--r--drivers/gpu/drm/i915/i915_request.c143
-rw-r--r--drivers/gpu/drm/i915/i915_request.h49
-rw-r--r--drivers/gpu/drm/i915/i915_scheduler.c52
-rw-r--r--drivers/gpu/drm/i915/i915_scheduler.h16
-rw-r--r--drivers/gpu/drm/i915/i915_selftest.h2
-rw-r--r--drivers/gpu/drm/i915/i915_suspend.c4
-rw-r--r--drivers/gpu/drm/i915/i915_switcheroo.c4
-rw-r--r--drivers/gpu/drm/i915/i915_vgpu.c2
-rw-r--r--drivers/gpu/drm/i915/i915_vma.c30
-rw-r--r--drivers/gpu/drm/i915/i915_vma.h20
-rw-r--r--drivers/gpu/drm/i915/i915_vma_types.h12
-rw-r--r--drivers/gpu/drm/i915/intel_device_info.c13
-rw-r--r--drivers/gpu/drm/i915/intel_device_info.h12
-rw-r--r--drivers/gpu/drm/i915/intel_dram.c6
-rw-r--r--drivers/gpu/drm/i915/intel_memory_region.c54
-rw-r--r--drivers/gpu/drm/i915/intel_memory_region.h22
-rw-r--r--drivers/gpu/drm/i915/intel_pch.c11
-rw-r--r--drivers/gpu/drm/i915/intel_pch.h3
-rw-r--r--drivers/gpu/drm/i915/intel_pm.c564
-rw-r--r--drivers/gpu/drm/i915/intel_pm.h5
-rw-r--r--drivers/gpu/drm/i915/intel_runtime_pm.c31
-rw-r--r--drivers/gpu/drm/i915/intel_runtime_pm.h5
-rw-r--r--drivers/gpu/drm/i915/intel_step.c106
-rw-r--r--drivers/gpu/drm/i915/intel_step.h40
-rw-r--r--drivers/gpu/drm/i915/intel_uncore.c20
-rw-r--r--drivers/gpu/drm/i915/selftests/i915_buddy.c48
-rw-r--r--drivers/gpu/drm/i915/selftests/i915_gem_gtt.c97
-rw-r--r--drivers/gpu/drm/i915/selftests/i915_request.c211
-rw-r--r--drivers/gpu/drm/i915/selftests/i915_vma.c219
-rw-r--r--drivers/gpu/drm/i915/selftests/igt_spinner.c136
-rw-r--r--drivers/gpu/drm/i915/selftests/igt_spinner.h5
-rw-r--r--drivers/gpu/drm/i915/selftests/intel_memory_region.c95
-rw-r--r--drivers/gpu/drm/i915/selftests/mock_gtt.c2
-rw-r--r--drivers/gpu/drm/i915/selftests/mock_region.c4
-rw-r--r--drivers/gpu/drm/imx/dcss/dcss-plane.c64
-rw-r--r--drivers/gpu/drm/imx/imx-drm-core.c2
-rw-r--r--drivers/gpu/drm/imx/imx-ldb.c12
-rw-r--r--drivers/gpu/drm/imx/ipuv3-plane.c92
-rw-r--r--drivers/gpu/drm/ingenic/ingenic-drm-drv.c83
-rw-r--r--drivers/gpu/drm/ingenic/ingenic-ipu.c77
-rw-r--r--drivers/gpu/drm/kmb/kmb_plane.c50
-rw-r--r--drivers/gpu/drm/lima/lima_devfreq.c58
-rw-r--r--drivers/gpu/drm/lima/lima_devfreq.h5
-rw-r--r--drivers/gpu/drm/lima/lima_sched.c6
-rw-r--r--drivers/gpu/drm/mcde/mcde_display.c10
-rw-r--r--drivers/gpu/drm/mcde/mcde_dsi.c2
-rw-r--r--drivers/gpu/drm/mediatek/mtk_cec.c6
-rw-r--r--drivers/gpu/drm/mediatek/mtk_dpi.c32
-rw-r--r--drivers/gpu/drm/mediatek/mtk_drm_crtc.c23
-rw-r--r--drivers/gpu/drm/mediatek/mtk_drm_crtc.h2
-rw-r--r--drivers/gpu/drm/mediatek/mtk_drm_drv.c1
-rw-r--r--drivers/gpu/drm/mediatek/mtk_drm_plane.c103
-rw-r--r--drivers/gpu/drm/mediatek/mtk_dsi.c9
-rw-r--r--drivers/gpu/drm/mediatek/mtk_hdmi.c175
-rw-r--r--drivers/gpu/drm/mediatek/mtk_hdmi_ddc.c1
-rw-r--r--drivers/gpu/drm/meson/meson_drv.c11
-rw-r--r--drivers/gpu/drm/meson/meson_overlay.c31
-rw-r--r--drivers/gpu/drm/meson/meson_plane.c51
-rw-r--r--drivers/gpu/drm/meson/meson_venc.c2
-rw-r--r--drivers/gpu/drm/mgag200/mgag200_mode.c25
-rw-r--r--drivers/gpu/drm/msm/Kconfig9
-rw-r--r--drivers/gpu/drm/msm/Makefile9
-rw-r--r--drivers/gpu/drm/msm/adreno/a5xx_gpu.c4
-rw-r--r--drivers/gpu/drm/msm/adreno/a5xx_power.c2
-rw-r--r--drivers/gpu/drm/msm/adreno/a6xx_gmu.c14
-rw-r--r--drivers/gpu/drm/msm/adreno/a6xx_gpu.c117
-rw-r--r--drivers/gpu/drm/msm/adreno/adreno_gpu.c8
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_core_irq.c4
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_core_perf.c1
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_crtc.c104
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.c30
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_encoder.h11
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys.h1
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_encoder_phys_vid.c26
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.c195
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_catalog.h10
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.c31
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_ctl.h3
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.c793
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_interrupts.h5
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.c12
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_intf.h1
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_hw_top.h4
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_kms.c19
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_mdss.c54
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_plane.c66
-rw-r--r--drivers/gpu/drm/msm/disp/dpu1/dpu_plane.h6
-rw-r--r--drivers/gpu/drm/msm/disp/mdp4/mdp4_plane.c18
-rw-r--r--drivers/gpu/drm/msm/disp/mdp5/mdp5_cmd_encoder.c19
-rw-r--r--drivers/gpu/drm/msm/disp/mdp5/mdp5_plane.c57
-rw-r--r--drivers/gpu/drm/msm/dp/dp_audio.c1
-rw-r--r--drivers/gpu/drm/msm/dp/dp_aux.c7
-rw-r--r--drivers/gpu/drm/msm/dp/dp_debug.c33
-rw-r--r--drivers/gpu/drm/msm/dp/dp_display.c26
-rw-r--r--drivers/gpu/drm/msm/dp/dp_display.h1
-rw-r--r--drivers/gpu/drm/msm/dp/dp_hpd.c4
-rw-r--r--drivers/gpu/drm/msm/dp/dp_power.c2
-rw-r--r--drivers/gpu/drm/msm/dsi/dsi.h60
-rw-r--r--drivers/gpu/drm/msm/dsi/dsi_cfg.c6
-rw-r--r--drivers/gpu/drm/msm/dsi/dsi_host.c6
-rw-r--r--drivers/gpu/drm/msm/dsi/dsi_manager.c30
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy.c161
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy.h41
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy_10nm.c747
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy_14nm.c939
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy_20nm.c16
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm.c654
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy_28nm_8960.c483
-rw-r--r--drivers/gpu/drm/msm/dsi/phy/dsi_phy_7nm.c774
-rw-r--r--drivers/gpu/drm/msm/dsi/pll/dsi_pll.c184
-rw-r--r--drivers/gpu/drm/msm/dsi/pll/dsi_pll.h130
-rw-r--r--drivers/gpu/drm/msm/dsi/pll/dsi_pll_10nm.c881
-rw-r--r--drivers/gpu/drm/msm/dsi/pll/dsi_pll_14nm.c1096
-rw-r--r--drivers/gpu/drm/msm/dsi/pll/dsi_pll_28nm.c643
-rw-r--r--drivers/gpu/drm/msm/dsi/pll/dsi_pll_28nm_8960.c526
-rw-r--r--drivers/gpu/drm/msm/dsi/pll/dsi_pll_7nm.c912
-rw-r--r--drivers/gpu/drm/msm/msm_atomic.c11
-rw-r--r--drivers/gpu/drm/msm/msm_debugfs.c14
-rw-r--r--drivers/gpu/drm/msm/msm_drv.c38
-rw-r--r--drivers/gpu/drm/msm/msm_drv.h29
-rw-r--r--drivers/gpu/drm/msm/msm_fb.c3
-rw-r--r--drivers/gpu/drm/msm/msm_fence.c2
-rw-r--r--drivers/gpu/drm/msm/msm_gem.c228
-rw-r--r--drivers/gpu/drm/msm/msm_gem.h126
-rw-r--r--drivers/gpu/drm/msm/msm_gem_shrinker.c166
-rw-r--r--drivers/gpu/drm/msm/msm_gpu.c2
-rw-r--r--drivers/gpu/drm/msm/msm_gpu.h2
-rw-r--r--drivers/gpu/drm/msm/msm_gpu_trace.h13
-rw-r--r--drivers/gpu/drm/msm/msm_kms.h8
-rw-r--r--drivers/gpu/drm/mxsfb/mxsfb_kms.c23
-rw-r--r--drivers/gpu/drm/nouveau/dispnv50/disp.c13
-rw-r--r--drivers/gpu/drm/nouveau/dispnv50/wndw.c8
-rw-r--r--drivers/gpu/drm/nouveau/nouveau_bo.c54
-rw-r--r--drivers/gpu/drm/nouveau/nouveau_bo.h2
-rw-r--r--drivers/gpu/drm/nouveau/nouveau_display.c9
-rw-r--r--drivers/gpu/drm/nouveau/nouveau_drv.h3
-rw-r--r--drivers/gpu/drm/nouveau/nouveau_sgdma.c6
-rw-r--r--drivers/gpu/drm/nouveau/nouveau_ttm.c14
-rw-r--r--drivers/gpu/drm/nouveau/nouveau_ttm.h8
-rw-r--r--drivers/gpu/drm/omapdrm/dss/dispc.c5
-rw-r--r--drivers/gpu/drm/omapdrm/dss/dsi.c10
-rw-r--r--drivers/gpu/drm/omapdrm/dss/omapdss.h9
-rw-r--r--drivers/gpu/drm/omapdrm/omap_drv.c9
-rw-r--r--drivers/gpu/drm/omapdrm/omap_plane.c56
-rw-r--r--drivers/gpu/drm/panel/panel-dsi-cm.c12
-rw-r--r--drivers/gpu/drm/panel/panel-lvds.c2
-rw-r--r--drivers/gpu/drm/panel/panel-novatek-nt35510.c3
-rw-r--r--drivers/gpu/drm/panel/panel-samsung-s6d16d0.c4
-rw-r--r--drivers/gpu/drm/panel/panel-samsung-s6e63m0-dsi.c1
-rw-r--r--drivers/gpu/drm/panel/panel-seiko-43wvf1g.c4
-rw-r--r--drivers/gpu/drm/panel/panel-simple.c87
-rw-r--r--drivers/gpu/drm/panel/panel-sony-acx424akp.c3
-rw-r--r--drivers/gpu/drm/panel/panel-tpo-td043mtea1.c4
-rw-r--r--drivers/gpu/drm/panfrost/panfrost_devfreq.c47
-rw-r--r--drivers/gpu/drm/panfrost/panfrost_devfreq.h4
-rw-r--r--drivers/gpu/drm/panfrost/panfrost_job.c11
-rw-r--r--drivers/gpu/drm/panfrost/panfrost_mmu.c39
-rw-r--r--drivers/gpu/drm/pl111/pl111_display.c4
-rw-r--r--drivers/gpu/drm/qxl/qxl_cmd.c3
-rw-r--r--drivers/gpu/drm/qxl/qxl_display.c369
-rw-r--r--drivers/gpu/drm/qxl/qxl_draw.c8
-rw-r--r--drivers/gpu/drm/qxl/qxl_drv.c1
-rw-r--r--drivers/gpu/drm/qxl/qxl_drv.h6
-rw-r--r--drivers/gpu/drm/qxl/qxl_dumb.c2
-rw-r--r--drivers/gpu/drm/qxl/qxl_gem.c2
-rw-r--r--drivers/gpu/drm/qxl/qxl_image.c2
-rw-r--r--drivers/gpu/drm/qxl/qxl_irq.c1
-rw-r--r--drivers/gpu/drm/qxl/qxl_kms.c30
-rw-r--r--drivers/gpu/drm/qxl/qxl_object.c57
-rw-r--r--drivers/gpu/drm/qxl/qxl_object.h7
-rw-r--r--drivers/gpu/drm/qxl/qxl_prime.c4
-rw-r--r--drivers/gpu/drm/qxl/qxl_release.c89
-rw-r--r--drivers/gpu/drm/qxl/qxl_ttm.c35
-rw-r--r--drivers/gpu/drm/radeon/atombios_crtc.c10
-rw-r--r--drivers/gpu/drm/radeon/cik.c4
-rw-r--r--drivers/gpu/drm/radeon/ni_dpm.c144
-rw-r--r--drivers/gpu/drm/radeon/nislands_smc.h44
-rw-r--r--drivers/gpu/drm/radeon/r100.c84
-rw-r--r--drivers/gpu/drm/radeon/r300.c33
-rw-r--r--drivers/gpu/drm/radeon/r420.c28
-rw-r--r--drivers/gpu/drm/radeon/r600.c29
-rw-r--r--drivers/gpu/drm/radeon/r600_cs.c6
-rw-r--r--drivers/gpu/drm/radeon/radeon.h34
-rw-r--r--drivers/gpu/drm/radeon/radeon_asic.h8
-rw-r--r--drivers/gpu/drm/radeon/radeon_atombios.c26
-rw-r--r--drivers/gpu/drm/radeon/radeon_cs.c16
-rw-r--r--drivers/gpu/drm/radeon/radeon_device.c47
-rw-r--r--drivers/gpu/drm/radeon/radeon_dp_mst.c23
-rw-r--r--drivers/gpu/drm/radeon/radeon_fence.c42
-rw-r--r--drivers/gpu/drm/radeon/radeon_gem.c24
-rw-r--r--drivers/gpu/drm/radeon/radeon_ib.c26
-rw-r--r--drivers/gpu/drm/radeon/radeon_kms.c1
-rw-r--r--drivers/gpu/drm/radeon/radeon_mode.h2
-rw-r--r--drivers/gpu/drm/radeon/radeon_object.c12
-rw-r--r--drivers/gpu/drm/radeon/radeon_pm.c76
-rw-r--r--drivers/gpu/drm/radeon/radeon_prime.c2
-rw-r--r--drivers/gpu/drm/radeon/radeon_ring.c80
-rw-r--r--drivers/gpu/drm/radeon/radeon_ttm.c139
-rw-r--r--drivers/gpu/drm/radeon/rs400.c25
-rw-r--r--drivers/gpu/drm/radeon/rs600.c8
-rw-r--r--drivers/gpu/drm/radeon/rv515.c56
-rw-r--r--drivers/gpu/drm/radeon/si.c2
-rw-r--r--drivers/gpu/drm/radeon/si_dpm.c182
-rw-r--r--drivers/gpu/drm/radeon/sislands_smc.h44
-rw-r--r--drivers/gpu/drm/rcar-du/rcar_du_encoder.c31
-rw-r--r--drivers/gpu/drm/rcar-du/rcar_du_plane.c17
-rw-r--r--drivers/gpu/drm/rcar-du/rcar_du_vsp.c20
-rw-r--r--drivers/gpu/drm/rockchip/rockchip_drm_vop.c79
-rw-r--r--drivers/gpu/drm/rockchip/rockchip_lvds.c2
-rw-r--r--drivers/gpu/drm/scheduler/sched_entity.c8
-rw-r--r--drivers/gpu/drm/scheduler/sched_main.c125
-rw-r--r--drivers/gpu/drm/sti/sti_cursor.c41
-rw-r--r--drivers/gpu/drm/sti/sti_gdp.c77
-rw-r--r--drivers/gpu/drm/sti/sti_hqvdp.c75
-rw-r--r--drivers/gpu/drm/stm/dw_mipi_dsi-stm.c9
-rw-r--r--drivers/gpu/drm/stm/ltdc.c96
-rw-r--r--drivers/gpu/drm/sun4i/sun4i_backend.c6
-rw-r--r--drivers/gpu/drm/sun4i/sun4i_layer.c15
-rw-r--r--drivers/gpu/drm/sun4i/sun8i_ui_layer.c59
-rw-r--r--drivers/gpu/drm/sun4i/sun8i_ui_layer.h5
-rw-r--r--drivers/gpu/drm/sun4i/sun8i_vi_layer.c78
-rw-r--r--drivers/gpu/drm/sun4i/sun8i_vi_layer.h11
-rw-r--r--drivers/gpu/drm/tegra/dc.c253
-rw-r--r--drivers/gpu/drm/tegra/dc.h6
-rw-r--r--drivers/gpu/drm/tegra/dpaux.c11
-rw-r--r--drivers/gpu/drm/tegra/drm.c30
-rw-r--r--drivers/gpu/drm/tegra/drm.h5
-rw-r--r--drivers/gpu/drm/tegra/fb.c10
-rw-r--r--drivers/gpu/drm/tegra/gem.h6
-rw-r--r--drivers/gpu/drm/tegra/gr2d.c4
-rw-r--r--drivers/gpu/drm/tegra/gr3d.c4
-rw-r--r--drivers/gpu/drm/tegra/hub.c103
-rw-r--r--drivers/gpu/drm/tegra/plane.c36
-rw-r--r--drivers/gpu/drm/tegra/sor.c7
-rw-r--r--drivers/gpu/drm/tegra/vic.c4
-rw-r--r--drivers/gpu/drm/tidss/tidss_kms.c4
-rw-r--r--drivers/gpu/drm/tidss/tidss_plane.c53
-rw-r--r--drivers/gpu/drm/tilcdc/tilcdc_crtc.c39
-rw-r--r--drivers/gpu/drm/tilcdc/tilcdc_panel.c1
-rw-r--r--drivers/gpu/drm/tilcdc/tilcdc_plane.c46
-rw-r--r--drivers/gpu/drm/tiny/Kconfig10
-rw-r--r--drivers/gpu/drm/tiny/Makefile1
-rw-r--r--drivers/gpu/drm/tiny/arcpgu.c434
-rw-r--r--drivers/gpu/drm/tiny/cirrus.c43
-rw-r--r--drivers/gpu/drm/tiny/gm12u320.c72
-rw-r--r--drivers/gpu/drm/tiny/hx8357d.c4
-rw-r--r--drivers/gpu/drm/tiny/ili9225.c4
-rw-r--r--drivers/gpu/drm/tiny/ili9341.c4
-rw-r--r--drivers/gpu/drm/tiny/ili9486.c4
-rw-r--r--drivers/gpu/drm/tiny/mi0283qt.c4
-rw-r--r--drivers/gpu/drm/tiny/repaper.c3
-rw-r--r--drivers/gpu/drm/tiny/st7586.c4
-rw-r--r--drivers/gpu/drm/tiny/st7735r.c4
-rw-r--r--drivers/gpu/drm/ttm/Makefile7
-rw-r--r--drivers/gpu/drm/ttm/ttm_agp_backend.c2
-rw-r--r--drivers/gpu/drm/ttm/ttm_bo.c435
-rw-r--r--drivers/gpu/drm/ttm/ttm_bo_util.c40
-rw-r--r--drivers/gpu/drm/ttm/ttm_bo_vm.c24
-rw-r--r--drivers/gpu/drm/ttm/ttm_device.c268
-rw-r--r--drivers/gpu/drm/ttm/ttm_execbuf_util.c8
-rw-r--r--drivers/gpu/drm/ttm/ttm_module.c54
-rw-r--r--drivers/gpu/drm/ttm/ttm_module.h8
-rw-r--r--drivers/gpu/drm/ttm/ttm_pool.c161
-rw-r--r--drivers/gpu/drm/ttm/ttm_range_manager.c4
-rw-r--r--drivers/gpu/drm/ttm/ttm_resource.c11
-rw-r--r--drivers/gpu/drm/ttm/ttm_tt.c116
-rw-r--r--drivers/gpu/drm/tve200/tve200_display.c4
-rw-r--r--drivers/gpu/drm/udl/udl_drv.c17
-rw-r--r--drivers/gpu/drm/udl/udl_drv.h1
-rw-r--r--drivers/gpu/drm/udl/udl_main.c10
-rw-r--r--drivers/gpu/drm/udl/udl_modeset.c34
-rw-r--r--drivers/gpu/drm/v3d/v3d_sched.c42
-rw-r--r--drivers/gpu/drm/vboxvideo/vbox_mode.c82
-rw-r--r--drivers/gpu/drm/vboxvideo/vbox_ttm.c7
-rw-r--r--drivers/gpu/drm/vc4/vc4_crtc.c17
-rw-r--r--drivers/gpu/drm/vc4/vc4_kms.c17
-rw-r--r--drivers/gpu/drm/vc4/vc4_plane.c75
-rw-r--r--drivers/gpu/drm/vc4/vc4_vec.c6
-rw-r--r--drivers/gpu/drm/virtio/virtgpu_ioctl.c2
-rw-r--r--drivers/gpu/drm/virtio/virtgpu_object.c1
-rw-r--r--drivers/gpu/drm/virtio/virtgpu_plane.c19
-rw-r--r--drivers/gpu/drm/vkms/vkms_crtc.c11
-rw-r--r--drivers/gpu/drm/vkms/vkms_plane.c30
-rw-r--r--drivers/gpu/drm/vkms/vkms_writeback.c7
-rw-r--r--drivers/gpu/drm/vmwgfx/Makefile2
-rw-r--r--drivers/gpu/drm/vmwgfx/ttm_memory.c (renamed from drivers/gpu/drm/ttm/ttm_memory.c)23
-rw-r--r--drivers/gpu/drm/vmwgfx/ttm_memory.h96
-rw-r--r--drivers/gpu/drm/vmwgfx/ttm_object.c25
-rw-r--r--drivers/gpu/drm/vmwgfx/ttm_object.h3
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_binding.c9
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_blit.c5
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_bo.c28
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_cmd.c2
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf.c14
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_cmdbuf_res.c8
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_context.c6
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_cotable.c7
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_drv.c62
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_drv.h16
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_execbuf.c20
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_fence.c18
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_ioctl.c2
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_kms.c103
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_kms.h10
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_ldu.c14
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_mob.c15
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_msg.c2
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_overlay.c16
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_page_dirty.c4
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_resource.c21
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_scrn.c20
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_shader.c10
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_so.c1
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_stdu.c21
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_surface.c17
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_thp.c1
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_ttm_buffer.c69
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_ttm_glue.c2
-rw-r--r--drivers/gpu/drm/vmwgfx/vmwgfx_validation.c5
-rw-r--r--drivers/gpu/drm/xen/Kconfig10
-rw-r--r--drivers/gpu/drm/xen/xen_drm_front.c6
-rw-r--r--drivers/gpu/drm/xen/xen_drm_front_conn.h1
-rw-r--r--drivers/gpu/drm/xen/xen_drm_front_kms.c3
-rw-r--r--drivers/gpu/drm/xlnx/zynqmp_disp.c32
-rw-r--r--drivers/gpu/drm/xlnx/zynqmp_dp.c2
-rw-r--r--drivers/gpu/drm/zte/zx_plane.c49
-rw-r--r--drivers/gpu/host1x/bus.c41
-rw-r--r--drivers/gpu/host1x/cdma.c11
-rw-r--r--drivers/gpu/host1x/debug.c14
-rw-r--r--drivers/gpu/host1x/dev.c6
-rw-r--r--drivers/gpu/host1x/dev.h13
-rw-r--r--drivers/gpu/host1x/hw/cdma_hw.c2
-rw-r--r--drivers/gpu/host1x/hw/channel_hw.c10
-rw-r--r--drivers/gpu/host1x/hw/debug_hw.c2
-rw-r--r--drivers/gpu/host1x/hw/hw_host1x07_vm.h2
-rw-r--r--drivers/gpu/host1x/intr.c28
-rw-r--r--drivers/gpu/host1x/intr.h4
-rw-r--r--drivers/gpu/host1x/job.c5
-rw-r--r--drivers/gpu/host1x/syncpt.c202
-rw-r--r--drivers/gpu/host1x/syncpt.h4
-rw-r--r--drivers/greybus/es2.c8
-rw-r--r--drivers/hid/Kconfig20
-rw-r--r--drivers/hid/Makefile6
-rw-r--r--drivers/hid/amd-sfh-hid/amd_sfh_pcie.c40
-rw-r--r--drivers/hid/amd-sfh-hid/amd_sfh_pcie.h1
-rw-r--r--drivers/hid/hid-alps.c3
-rw-r--r--drivers/hid/hid-asus.c3
-rw-r--r--drivers/hid/hid-core.c2
-rw-r--r--drivers/hid/hid-cp2112.c22
-rw-r--r--drivers/hid/hid-debug.c1
-rw-r--r--drivers/hid/hid-elan.c17
-rw-r--r--drivers/hid/hid-ft260.c1054
-rw-r--r--drivers/hid/hid-google-hammer.c2
-rw-r--r--drivers/hid/hid-ids.h16
-rw-r--r--drivers/hid/hid-input.c22
-rw-r--r--drivers/hid/hid-kye.c2
-rw-r--r--drivers/hid/hid-lenovo.c147
-rw-r--r--drivers/hid/hid-lg.c24
-rw-r--r--drivers/hid/hid-logitech-dj.c131
-rw-r--r--drivers/hid/hid-logitech-hidpp.c7
-rw-r--r--drivers/hid/hid-magicmouse.c158
-rw-r--r--drivers/hid/hid-picolcd_core.c5
-rw-r--r--drivers/hid/hid-plantronics.c60
-rw-r--r--drivers/hid/hid-quirks.c5
-rw-r--r--drivers/hid/hid-sensor-custom.c5
-rw-r--r--drivers/hid/hid-sensor-hub.c4
-rw-r--r--drivers/hid/hid-thrustmaster.c371
-rw-r--r--drivers/hid/hid-uclogic-params.c8
-rw-r--r--drivers/hid/hid-uclogic-rdesc.c2
-rw-r--r--drivers/hid/i2c-hid/i2c-hid-acpi.c52
-rw-r--r--drivers/hid/surface-hid/Kconfig42
-rw-r--r--drivers/hid/surface-hid/Makefile7
-rw-r--r--drivers/hid/surface-hid/surface_hid.c253
-rw-r--r--drivers/hid/surface-hid/surface_hid_core.c272
-rw-r--r--drivers/hid/surface-hid/surface_hid_core.h77
-rw-r--r--drivers/hid/surface-hid/surface_kbd.c300
-rw-r--r--drivers/hid/usbhid/hid-pidff.c4
-rw-r--r--drivers/hid/usbhid/hiddev.c6
-rw-r--r--drivers/hid/usbhid/usbkbd.c18
-rw-r--r--drivers/hid/wacom_sys.c2
-rw-r--r--drivers/hid/wacom_wac.c58
-rw-r--r--drivers/hid/wacom_wac.h1
-rw-r--r--drivers/hsi/hsi_core.c3
-rw-r--r--drivers/hv/Kconfig1
-rw-r--r--drivers/hv/channel.c103
-rw-r--r--drivers/hv/channel_mgmt.c86
-rw-r--r--drivers/hv/connection.c7
-rw-r--r--drivers/hv/hv.c152
-rw-r--r--drivers/hv/hv_balloon.c89
-rw-r--r--drivers/hv/hv_trace.h15
-rw-r--r--drivers/hv/ring_buffer.c10
-rw-r--r--drivers/hv/vmbus_drv.c97
-rw-r--r--drivers/hwmon/Kconfig20
-rw-r--r--drivers/hwmon/Makefile1
-rw-r--r--drivers/hwmon/acpi_power_meter.c29
-rw-r--r--drivers/hwmon/adc128d818.c2
-rw-r--r--drivers/hwmon/adm9240.c982
-rw-r--r--drivers/hwmon/amd_energy.c379
-rw-r--r--drivers/hwmon/applesmc.c34
-rw-r--r--drivers/hwmon/corsair-psu.c355
-rw-r--r--drivers/hwmon/dell-smm-hwmon.c8
-rw-r--r--drivers/hwmon/ds1621.c2
-rw-r--r--drivers/hwmon/ftsteutates.c4
-rw-r--r--drivers/hwmon/hwmon.c2
-rw-r--r--drivers/hwmon/ina209.c6
-rw-r--r--drivers/hwmon/ina2xx.c11
-rw-r--r--drivers/hwmon/ina3221.c2
-rw-r--r--drivers/hwmon/intel-m10-bmc-hwmon.c122
-rw-r--r--drivers/hwmon/it87.c12
-rw-r--r--drivers/hwmon/lineage-pem.c8
-rw-r--r--drivers/hwmon/lm63.c2
-rw-r--r--drivers/hwmon/ltc2945.c4
-rw-r--r--drivers/hwmon/ltc2990.c4
-rw-r--r--drivers/hwmon/ltc2992.c8
-rw-r--r--drivers/hwmon/ltc4151.c2
-rw-r--r--drivers/hwmon/ltc4215.c8
-rw-r--r--drivers/hwmon/ltc4222.c4
-rw-r--r--drivers/hwmon/ltc4260.c4
-rw-r--r--drivers/hwmon/ltc4261.c4
-rw-r--r--drivers/hwmon/max16065.c18
-rw-r--r--drivers/hwmon/max6697.c2
-rw-r--r--drivers/hwmon/mlxreg-fan.c51
-rw-r--r--drivers/hwmon/nct6683.c22
-rw-r--r--drivers/hwmon/ntc_thermistor.c27
-rw-r--r--drivers/hwmon/nzxt-kraken2.c234
-rw-r--r--drivers/hwmon/occ/common.c74
-rw-r--r--drivers/hwmon/occ/common.h2
-rw-r--r--drivers/hwmon/occ/sysfs.c4
-rw-r--r--drivers/hwmon/pmbus/Kconfig51
-rw-r--r--drivers/hwmon/pmbus/Makefile5
-rw-r--r--drivers/hwmon/pmbus/adm1266.c1
-rw-r--r--drivers/hwmon/pmbus/adm1275.c1
-rw-r--r--drivers/hwmon/pmbus/bel-pfe.c1
-rw-r--r--drivers/hwmon/pmbus/bpa-rs600.c173
-rw-r--r--drivers/hwmon/pmbus/fsp-3y.c277
-rw-r--r--drivers/hwmon/pmbus/ibm-cffps.c1
-rw-r--r--drivers/hwmon/pmbus/inspur-ipsps.c29
-rw-r--r--drivers/hwmon/pmbus/ir35221.c1
-rw-r--r--drivers/hwmon/pmbus/ir36021.c80
-rw-r--r--drivers/hwmon/pmbus/ir38064.c1
-rw-r--r--drivers/hwmon/pmbus/irps5401.c1
-rw-r--r--drivers/hwmon/pmbus/isl68137.c1
-rw-r--r--drivers/hwmon/pmbus/lm25066.c1
-rw-r--r--drivers/hwmon/pmbus/ltc2978.c1
-rw-r--r--drivers/hwmon/pmbus/ltc3815.c1
-rw-r--r--drivers/hwmon/pmbus/max15301.c190
-rw-r--r--drivers/hwmon/pmbus/max16064.c1
-rw-r--r--drivers/hwmon/pmbus/max16601.c1
-rw-r--r--drivers/hwmon/pmbus/max20730.c1
-rw-r--r--drivers/hwmon/pmbus/max20751.c1
-rw-r--r--drivers/hwmon/pmbus/max31785.c1
-rw-r--r--drivers/hwmon/pmbus/max34440.c1
-rw-r--r--drivers/hwmon/pmbus/max8688.c1
-rw-r--r--drivers/hwmon/pmbus/mp2975.c1
-rw-r--r--drivers/hwmon/pmbus/pm6764tr.c1
-rw-r--r--drivers/hwmon/pmbus/pmbus.c1
-rw-r--r--drivers/hwmon/pmbus/pmbus.h1
-rw-r--r--drivers/hwmon/pmbus/pmbus_core.c63
-rw-r--r--drivers/hwmon/pmbus/pxe1610.c10
-rw-r--r--drivers/hwmon/pmbus/q54sj108a2.c1
-rw-r--r--drivers/hwmon/pmbus/stpddc60.c249
-rw-r--r--drivers/hwmon/pmbus/tps40422.c1
-rw-r--r--drivers/hwmon/pmbus/tps53679.c52
-rw-r--r--drivers/hwmon/pmbus/ucd9000.c1
-rw-r--r--drivers/hwmon/pmbus/ucd9200.c1
-rw-r--r--drivers/hwmon/pmbus/xdpe12284.c1
-rw-r--r--drivers/hwmon/pmbus/zl6100.c1
-rw-r--r--drivers/hwmon/raspberrypi-hwmon.c17
-rw-r--r--drivers/hwmon/s3c-hwmon.c4
-rw-r--r--drivers/hwmon/sch5627.c521
-rw-r--r--drivers/hwmon/sch5636.c20
-rw-r--r--drivers/hwmon/scmi-hwmon.c24
-rw-r--r--drivers/hwmon/smm665.c4
-rw-r--r--drivers/hwmon/stts751.c20
-rw-r--r--drivers/hwmon/vexpress-hwmon.c12
-rw-r--r--drivers/hwmon/xgene-hwmon.c14
-rw-r--r--drivers/hwspinlock/Kconfig11
-rw-r--r--drivers/hwspinlock/Makefile1
-rw-r--r--drivers/hwspinlock/sirf_hwspinlock.c105
-rw-r--r--drivers/hwtracing/coresight/Kconfig24
-rw-r--r--drivers/hwtracing/coresight/Makefile1
-rw-r--r--drivers/hwtracing/coresight/coresight-core.c31
-rw-r--r--drivers/hwtracing/coresight/coresight-etm-perf.c123
-rw-r--r--drivers/hwtracing/coresight/coresight-etm4x-core.c162
-rw-r--r--drivers/hwtracing/coresight/coresight-etm4x-sysfs.c19
-rw-r--r--drivers/hwtracing/coresight/coresight-etm4x.h83
-rw-r--r--drivers/hwtracing/coresight/coresight-platform.c6
-rw-r--r--drivers/hwtracing/coresight/coresight-priv.h3
-rw-r--r--drivers/hwtracing/coresight/coresight-trbe.c1157
-rw-r--r--drivers/hwtracing/coresight/coresight-trbe.h152
-rw-r--r--drivers/hwtracing/intel_th/core.c2
-rw-r--r--drivers/hwtracing/intel_th/gth.c4
-rw-r--r--drivers/hwtracing/intel_th/intel_th.h8
-rw-r--r--drivers/hwtracing/intel_th/msu.c2
-rw-r--r--drivers/hwtracing/intel_th/pci.c12
-rw-r--r--drivers/hwtracing/intel_th/pti.c4
-rw-r--r--drivers/hwtracing/stm/p_sys-t.c6
-rw-r--r--drivers/hwtracing/stm/policy.c5
-rw-r--r--drivers/i2c/busses/Kconfig24
-rw-r--r--drivers/i2c/busses/Makefile2
-rw-r--r--drivers/i2c/busses/i2c-amd8111.c268
-rw-r--r--drivers/i2c/busses/i2c-brcmstb.c1
-rw-r--r--drivers/i2c/busses/i2c-cadence.c9
-rw-r--r--drivers/i2c/busses/i2c-cht-wc.c6
-rw-r--r--drivers/i2c/busses/i2c-cp2615.c330
-rw-r--r--drivers/i2c/busses/i2c-designware-common.c3
-rw-r--r--drivers/i2c/busses/i2c-designware-core.h8
-rw-r--r--drivers/i2c/busses/i2c-designware-master.c156
-rw-r--r--drivers/i2c/busses/i2c-designware-pcidrv.c61
-rw-r--r--drivers/i2c/busses/i2c-designware-platdrv.c7
-rw-r--r--drivers/i2c/busses/i2c-emev2.c5
-rw-r--r--drivers/i2c/busses/i2c-exynos5.c2
-rw-r--r--drivers/i2c/busses/i2c-hisi.c504
-rw-r--r--drivers/i2c/busses/i2c-hix5hd2.c2
-rw-r--r--drivers/i2c/busses/i2c-i801.c4
-rw-r--r--drivers/i2c/busses/i2c-icy.c32
-rw-r--r--drivers/i2c/busses/i2c-img-scb.c4
-rw-r--r--drivers/i2c/busses/i2c-imx-lpi2c.c2
-rw-r--r--drivers/i2c/busses/i2c-imx.c6
-rw-r--r--drivers/i2c/busses/i2c-iop3xx.c28
-rw-r--r--drivers/i2c/busses/i2c-jz4780.c9
-rw-r--r--drivers/i2c/busses/i2c-mlxbf.c16
-rw-r--r--drivers/i2c/busses/i2c-mpc.c579
-rw-r--r--drivers/i2c/busses/i2c-mt65xx.c17
-rw-r--r--drivers/i2c/busses/i2c-mv64xxx.c4
-rw-r--r--drivers/i2c/busses/i2c-nomadik.c4
-rw-r--r--drivers/i2c/busses/i2c-nvidia-gpu.c6
-rw-r--r--drivers/i2c/busses/i2c-omap.c8
-rw-r--r--drivers/i2c/busses/i2c-powermac.c5
-rw-r--r--drivers/i2c/busses/i2c-qcom-cci.c4
-rw-r--r--drivers/i2c/busses/i2c-rcar.c89
-rw-r--r--drivers/i2c/busses/i2c-s3c2410.c9
-rw-r--r--drivers/i2c/busses/i2c-scmi.c2
-rw-r--r--drivers/i2c/busses/i2c-sh7760.c5
-rw-r--r--drivers/i2c/busses/i2c-sprd.c5
-rw-r--r--drivers/i2c/busses/i2c-stm32f4.c2
-rw-r--r--drivers/i2c/busses/i2c-stm32f7.c82
-rw-r--r--drivers/i2c/busses/i2c-tegra-bpmp.c79
-rw-r--r--drivers/i2c/busses/i2c-xgene-slimpro.c1
-rw-r--r--drivers/i2c/busses/i2c-xiic.c4
-rw-r--r--drivers/i2c/i2c-boardinfo.c11
-rw-r--r--drivers/i2c/i2c-core-base.c121
-rw-r--r--drivers/i2c/i2c-dev.c9
-rw-r--r--drivers/i3c/master.c5
-rw-r--r--drivers/i3c/master/svc-i3c-master.c1
-rw-r--r--drivers/idle/intel_idle.c5
-rw-r--r--drivers/iio/Kconfig2
-rw-r--r--drivers/iio/Makefile2
-rw-r--r--drivers/iio/accel/Kconfig19
-rw-r--r--drivers/iio/accel/Makefile2
-rw-r--r--drivers/iio/accel/adis16201.c2
-rw-r--r--drivers/iio/accel/adxl372.c2
-rw-r--r--drivers/iio/accel/bma180.c3
-rw-r--r--drivers/iio/accel/bma220_spi.c2
-rw-r--r--drivers/iio/accel/bmc150-accel-core.c19
-rw-r--r--drivers/iio/accel/bmi088-accel-core.c567
-rw-r--r--drivers/iio/accel/bmi088-accel-spi.c83
-rw-r--r--drivers/iio/accel/bmi088-accel.h18
-rw-r--r--drivers/iio/accel/cros_ec_accel_legacy.c2
-rw-r--r--drivers/iio/accel/da280.c2
-rw-r--r--drivers/iio/accel/da311.c2
-rw-r--r--drivers/iio/accel/dmard10.c2
-rw-r--r--drivers/iio/accel/hid-sensor-accel-3d.c23
-rw-r--r--drivers/iio/accel/kxcjk-1013.c16
-rw-r--r--drivers/iio/accel/mc3230.c2
-rw-r--r--drivers/iio/accel/mma7660.c2
-rw-r--r--drivers/iio/accel/mma8452.c47
-rw-r--r--drivers/iio/accel/mxc4005.c1
-rw-r--r--drivers/iio/accel/sca3000.c27
-rw-r--r--drivers/iio/accel/ssp_accel_sensor.c14
-rw-r--r--drivers/iio/accel/stk8312.c3
-rw-r--r--drivers/iio/accel/stk8ba50.c3
-rw-r--r--drivers/iio/adc/Kconfig19
-rw-r--r--drivers/iio/adc/Makefile1
-rw-r--r--drivers/iio/adc/ab8500-gpadc.c2
-rw-r--r--drivers/iio/adc/ad7124.c459
-rw-r--r--drivers/iio/adc/ad7292.c2
-rw-r--r--drivers/iio/adc/ad7298.c6
-rw-r--r--drivers/iio/adc/ad7476.c18
-rw-r--r--drivers/iio/adc/ad7606.c1
-rw-r--r--drivers/iio/adc/ad7766.c16
-rw-r--r--drivers/iio/adc/ad7768-1.c1
-rw-r--r--drivers/iio/adc/ad7887.c6
-rw-r--r--drivers/iio/adc/ad7923.c47
-rw-r--r--drivers/iio/adc/ad7949.c2
-rw-r--r--drivers/iio/adc/ad799x.c6
-rw-r--r--drivers/iio/adc/ad_sigma_delta.c13
-rw-r--r--drivers/iio/adc/adi-axi-adc.c14
-rw-r--r--drivers/iio/adc/at91_adc.c3
-rw-r--r--drivers/iio/adc/cpcap-adc.c2
-rw-r--r--drivers/iio/adc/exynos_adc.c4
-rw-r--r--drivers/iio/adc/ina2xx-adc.c14
-rw-r--r--drivers/iio/adc/max1027.c1
-rw-r--r--drivers/iio/adc/mt6360-adc.c3
-rw-r--r--drivers/iio/adc/nau7802.c6
-rw-r--r--drivers/iio/adc/npcm_adc.c15
-rw-r--r--drivers/iio/adc/palmas_gpadc.c18
-rw-r--r--drivers/iio/adc/qcom-spmi-vadc.c2
-rw-r--r--drivers/iio/adc/spear_adc.c20
-rw-r--r--drivers/iio/adc/stm32-adc.c39
-rw-r--r--drivers/iio/adc/stm32-dfsdm-adc.c10
-rw-r--r--drivers/iio/adc/sun4i-gpadc-iio.c4
-rw-r--r--drivers/iio/adc/ti-adc084s021.c6
-rw-r--r--drivers/iio/adc/ti-ads131e08.c954
-rw-r--r--drivers/iio/adc/ti_am335x_adc.c18
-rw-r--r--drivers/iio/adc/xilinx-xadc-core.c1
-rw-r--r--drivers/iio/buffer/industrialio-buffer-dmaengine.c35
-rw-r--r--drivers/iio/buffer/industrialio-triggered-buffer.c10
-rw-r--r--drivers/iio/buffer/kfifo_buf.c45
-rw-r--r--drivers/iio/cdc/Kconfig17
-rw-r--r--drivers/iio/cdc/Makefile6
-rw-r--r--drivers/iio/cdc/ad7150.c673
-rw-r--r--drivers/iio/chemical/atlas-sensor.c1
-rw-r--r--drivers/iio/chemical/bme680_i2c.c3
-rw-r--r--drivers/iio/chemical/bme680_spi.c3
-rw-r--r--drivers/iio/chemical/ccs811.c1
-rw-r--r--drivers/iio/chemical/scd30_core.c17
-rw-r--r--drivers/iio/chemical/scd30_serial.c2
-rw-r--r--drivers/iio/common/Kconfig1
-rw-r--r--drivers/iio/common/Makefile1
-rw-r--r--drivers/iio/common/cros_ec_sensors/cros_ec_lid_angle.c3
-rw-r--r--drivers/iio/common/cros_ec_sensors/cros_ec_sensors.c3
-rw-r--r--drivers/iio/common/cros_ec_sensors/cros_ec_sensors_core.c33
-rw-r--r--drivers/iio/common/hid-sensors/Kconfig1
-rw-r--r--drivers/iio/common/hid-sensors/hid-sensor-attributes.c83
-rw-r--r--drivers/iio/common/hid-sensors/hid-sensor-trigger.c4
-rw-r--r--drivers/iio/common/scmi_sensors/Kconfig18
-rw-r--r--drivers/iio/common/scmi_sensors/Makefile5
-rw-r--r--drivers/iio/common/scmi_sensors/scmi_iio.c672
-rw-r--r--drivers/iio/common/st_sensors/st_sensors_buffer.c2
-rw-r--r--drivers/iio/common/st_sensors/st_sensors_trigger.c4
-rw-r--r--drivers/iio/dac/Kconfig5
-rw-r--r--drivers/iio/dac/ad5064.c2
-rw-r--r--drivers/iio/dac/ad5360.c2
-rw-r--r--drivers/iio/dac/ad5380.c2
-rw-r--r--drivers/iio/dac/ad5446.c2
-rw-r--r--drivers/iio/dac/ad5504.c6
-rw-r--r--drivers/iio/dac/ad5624r_spi.c4
-rw-r--r--drivers/iio/dac/ad5686.c14
-rw-r--r--drivers/iio/dac/ad5686.h2
-rw-r--r--drivers/iio/dac/ad5696-i2c.c6
-rw-r--r--drivers/iio/dac/ad5755.c4
-rw-r--r--drivers/iio/dac/ad5758.c2
-rw-r--r--drivers/iio/dac/ad5766.c2
-rw-r--r--drivers/iio/dac/ad5770r.c4
-rw-r--r--drivers/iio/dac/ad5791.c4
-rw-r--r--drivers/iio/dac/ad7303.c2
-rw-r--r--drivers/iio/dac/ltc2632.c4
-rw-r--r--drivers/iio/dac/max517.c10
-rw-r--r--drivers/iio/dac/max5821.c2
-rw-r--r--drivers/iio/dac/mcp4725.c2
-rw-r--r--drivers/iio/dac/stm32-dac.c2
-rw-r--r--drivers/iio/dac/ti-dac082s085.c2
-rw-r--r--drivers/iio/dac/ti-dac5571.c2
-rw-r--r--drivers/iio/dac/ti-dac7311.c2
-rw-r--r--drivers/iio/dummy/iio_simple_dummy_buffer.c68
-rw-r--r--drivers/iio/gyro/Kconfig1
-rw-r--r--drivers/iio/gyro/adxrs290.c1
-rw-r--r--drivers/iio/gyro/bmg160_core.c2
-rw-r--r--drivers/iio/gyro/fxas21002c_core.c1
-rw-r--r--drivers/iio/gyro/hid-sensor-gyro-3d.c19
-rw-r--r--drivers/iio/gyro/itg3200_buffer.c3
-rw-r--r--drivers/iio/gyro/mpu3050-core.c15
-rw-r--r--drivers/iio/gyro/ssp_gyro_sensor.c14
-rw-r--r--drivers/iio/health/afe4403.c1
-rw-r--r--drivers/iio/health/afe4404.c1
-rw-r--r--drivers/iio/health/max30100.c16
-rw-r--r--drivers/iio/health/max30102.c16
-rw-r--r--drivers/iio/humidity/Kconfig1
-rw-r--r--drivers/iio/humidity/am2315.c2
-rw-r--r--drivers/iio/humidity/hid-sensor-humidity.c28
-rw-r--r--drivers/iio/humidity/hts221_buffer.c1
-rw-r--r--drivers/iio/iio_core.h32
-rw-r--r--drivers/iio/iio_core_trigger.h4
-rw-r--r--drivers/iio/imu/adis16400.c25
-rw-r--r--drivers/iio/imu/adis16460.c4
-rw-r--r--drivers/iio/imu/adis16475.c123
-rw-r--r--drivers/iio/imu/adis16480.c133
-rw-r--r--drivers/iio/imu/adis_trigger.c21
-rw-r--r--drivers/iio/imu/fxos8700_i2c.c3
-rw-r--r--drivers/iio/imu/fxos8700_spi.c3
-rw-r--r--drivers/iio/imu/inv_icm42600/inv_icm42600_accel.c14
-rw-r--r--drivers/iio/imu/inv_icm42600/inv_icm42600_gyro.c13
-rw-r--r--drivers/iio/imu/inv_mpu6050/inv_mpu_core.c72
-rw-r--r--drivers/iio/imu/inv_mpu6050/inv_mpu_trigger.c10
-rw-r--r--drivers/iio/imu/kmx61.c1
-rw-r--r--drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_buffer.c15
-rw-r--r--drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_i2c.c3
-rw-r--r--drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_i3c.c3
-rw-r--r--drivers/iio/imu/st_lsm6dsx/st_lsm6dsx_spi.c3
-rw-r--r--drivers/iio/industrialio-buffer.c517
-rw-r--r--drivers/iio/industrialio-core.c207
-rw-r--r--drivers/iio/industrialio-event.c9
-rw-r--r--drivers/iio/industrialio-trigger.c72
-rw-r--r--drivers/iio/inkern.c16
-rw-r--r--drivers/iio/light/Kconfig2
-rw-r--r--drivers/iio/light/acpi-als.c121
-rw-r--r--drivers/iio/light/apds9960.c16
-rw-r--r--drivers/iio/light/cros_ec_light_prox.c3
-rw-r--r--drivers/iio/light/gp2ap002.c8
-rw-r--r--drivers/iio/light/gp2ap020a00f.c1
-rw-r--r--drivers/iio/light/hid-sensor-als.c34
-rw-r--r--drivers/iio/light/hid-sensor-prox.c35
-rw-r--r--drivers/iio/light/opt3001.c2
-rw-r--r--drivers/iio/light/rpr0521.c1
-rw-r--r--drivers/iio/light/si1145.c1
-rw-r--r--drivers/iio/light/st_uvis25_core.c1
-rw-r--r--drivers/iio/light/stk3310.c2
-rw-r--r--drivers/iio/light/tsl2583.c8
-rw-r--r--drivers/iio/light/vcnl4000.c1
-rw-r--r--drivers/iio/light/vcnl4035.c1
-rw-r--r--drivers/iio/magnetometer/Kconfig1
-rw-r--r--drivers/iio/magnetometer/bmc150_magn.c1
-rw-r--r--drivers/iio/magnetometer/hid-sensor-magn-3d.c32
-rw-r--r--drivers/iio/magnetometer/rm3100-core.c1
-rw-r--r--drivers/iio/magnetometer/st_magn.h1
-rw-r--r--drivers/iio/magnetometer/st_magn_core.c1
-rw-r--r--drivers/iio/magnetometer/st_magn_i2c.c5
-rw-r--r--drivers/iio/magnetometer/st_magn_spi.c5
-rw-r--r--drivers/iio/magnetometer/yamaha-yas530.c4
-rw-r--r--drivers/iio/orientation/Kconfig2
-rw-r--r--drivers/iio/orientation/hid-sensor-incl-3d.c20
-rw-r--r--drivers/iio/orientation/hid-sensor-rotation.c37
-rw-r--r--drivers/iio/position/hid-sensor-custom-intel-hinge.c20
-rw-r--r--drivers/iio/potentiometer/max5481.c4
-rw-r--r--drivers/iio/potentiometer/max5487.c4
-rw-r--r--drivers/iio/potentiostat/lmp91000.c3
-rw-r--r--drivers/iio/pressure/Kconfig1
-rw-r--r--drivers/iio/pressure/cros_ec_baro.c3
-rw-r--r--drivers/iio/pressure/hid-sensor-press.c20
-rw-r--r--drivers/iio/pressure/zpa2326.c5
-rw-r--r--drivers/iio/proximity/Kconfig11
-rw-r--r--drivers/iio/proximity/Makefile1
-rw-r--r--drivers/iio/proximity/as3935.c1
-rw-r--r--drivers/iio/proximity/cros_ec_mkbp_proximity.c271
-rw-r--r--drivers/iio/proximity/pulsedlight-lidar-lite-v2.c1
-rw-r--r--drivers/iio/proximity/sx9310.c53
-rw-r--r--drivers/iio/proximity/sx9500.c3
-rw-r--r--drivers/iio/proximity/vcnl3020.c97
-rw-r--r--drivers/iio/temperature/Kconfig1
-rw-r--r--drivers/iio/temperature/hid-sensor-temperature.c30
-rw-r--r--drivers/iio/temperature/tmp007.c36
-rw-r--r--drivers/iio/test/Kconfig9
-rw-r--r--drivers/iio/test/Makefile7
-rw-r--r--drivers/iio/test/iio-test-format.c198
-rw-r--r--drivers/iio/trigger/iio-trig-hrtimer.c37
-rw-r--r--drivers/iio/trigger/iio-trig-interrupt.c2
-rw-r--r--drivers/iio/trigger/iio-trig-loop.c2
-rw-r--r--drivers/iio/trigger/iio-trig-sysfs.c3
-rw-r--r--drivers/infiniband/core/addr.c4
-rw-r--r--drivers/infiniband/core/cache.c87
-rw-r--r--drivers/infiniband/core/cm.c58
-rw-r--r--drivers/infiniband/core/cm_msgs.h4
-rw-r--r--drivers/infiniband/core/cma.c116
-rw-r--r--drivers/infiniband/core/cma_configfs.c8
-rw-r--r--drivers/infiniband/core/cma_priv.h10
-rw-r--r--drivers/infiniband/core/core_priv.h28
-rw-r--r--drivers/infiniband/core/counters.c62
-rw-r--r--drivers/infiniband/core/device.c37
-rw-r--r--drivers/infiniband/core/iwpm_msg.c3
-rw-r--r--drivers/infiniband/core/mad.c79
-rw-r--r--drivers/infiniband/core/mad_rmpp.c10
-rw-r--r--drivers/infiniband/core/multicast.c8
-rw-r--r--drivers/infiniband/core/nldev.c176
-rw-r--r--drivers/infiniband/core/opa_smi.h4
-rw-r--r--drivers/infiniband/core/rdma_core.c4
-rw-r--r--drivers/infiniband/core/restrack.c3
-rw-r--r--drivers/infiniband/core/roce_gid_mgmt.c52
-rw-r--r--drivers/infiniband/core/rw.c25
-rw-r--r--drivers/infiniband/core/sa.h2
-rw-r--r--drivers/infiniband/core/sa_query.c22
-rw-r--r--drivers/infiniband/core/security.c8
-rw-r--r--drivers/infiniband/core/smi.c12
-rw-r--r--drivers/infiniband/core/smi.h4
-rw-r--r--drivers/infiniband/core/sysfs.c29
-rw-r--r--drivers/infiniband/core/ucma.c8
-rw-r--r--drivers/infiniband/core/umem.c20
-rw-r--r--drivers/infiniband/core/umem_dmabuf.c4
-rw-r--r--drivers/infiniband/core/user_mad.c34
-rw-r--r--drivers/infiniband/core/uverbs_cmd.c25
-rw-r--r--drivers/infiniband/core/uverbs_ioctl.c32
-rw-r--r--drivers/infiniband/core/verbs.c43
-rw-r--r--drivers/infiniband/hw/bnxt_re/Kconfig4
-rw-r--r--drivers/infiniband/hw/bnxt_re/bnxt_re.h1
-rw-r--r--drivers/infiniband/hw/bnxt_re/hw_counters.c4
-rw-r--r--drivers/infiniband/hw/bnxt_re/hw_counters.h4
-rw-r--r--drivers/infiniband/hw/bnxt_re/ib_verbs.c10
-rw-r--r--drivers/infiniband/hw/bnxt_re/ib_verbs.h10
-rw-r--r--drivers/infiniband/hw/bnxt_re/main.c63
-rw-r--r--drivers/infiniband/hw/bnxt_re/qplib_fp.c1
-rw-r--r--drivers/infiniband/hw/bnxt_re/qplib_rcfw.c4
-rw-r--r--drivers/infiniband/hw/bnxt_re/qplib_rcfw.h2
-rw-r--r--drivers/infiniband/hw/bnxt_re/qplib_res.c1
-rw-r--r--drivers/infiniband/hw/cxgb4/cm.c8
-rw-r--r--drivers/infiniband/hw/cxgb4/iw_cxgb4.h11
-rw-r--r--drivers/infiniband/hw/cxgb4/provider.c12
-rw-r--r--drivers/infiniband/hw/cxgb4/resource.c2
-rw-r--r--drivers/infiniband/hw/cxgb4/t4.h33
-rw-r--r--drivers/infiniband/hw/efa/efa.h14
-rw-r--r--drivers/infiniband/hw/efa/efa_main.c10
-rw-r--r--drivers/infiniband/hw/efa/efa_verbs.c14
-rw-r--r--drivers/infiniband/hw/hfi1/affinity.c29
-rw-r--r--drivers/infiniband/hw/hfi1/chip.c10
-rw-r--r--drivers/infiniband/hw/hfi1/chip.h5
-rw-r--r--drivers/infiniband/hw/hfi1/driver.c2
-rw-r--r--drivers/infiniband/hw/hfi1/exp_rcv.c6
-rw-r--r--drivers/infiniband/hw/hfi1/firmware.c1
-rw-r--r--drivers/infiniband/hw/hfi1/hfi.h21
-rw-r--r--drivers/infiniband/hw/hfi1/init.c17
-rw-r--r--drivers/infiniband/hw/hfi1/iowait.h2
-rw-r--r--drivers/infiniband/hw/hfi1/ipoib.h15
-rw-r--r--drivers/infiniband/hw/hfi1/ipoib_main.c13
-rw-r--r--drivers/infiniband/hw/hfi1/ipoib_tx.c71
-rw-r--r--drivers/infiniband/hw/hfi1/mad.c128
-rw-r--r--drivers/infiniband/hw/hfi1/mad.h2
-rw-r--r--drivers/infiniband/hw/hfi1/mmu_rb.c2
-rw-r--r--drivers/infiniband/hw/hfi1/msix.c12
-rw-r--r--drivers/infiniband/hw/hfi1/netdev.h39
-rw-r--r--drivers/infiniband/hw/hfi1/netdev_rx.c175
-rw-r--r--drivers/infiniband/hw/hfi1/sdma.c2
-rw-r--r--drivers/infiniband/hw/hfi1/sdma.h18
-rw-r--r--drivers/infiniband/hw/hfi1/sysfs.c2
-rw-r--r--drivers/infiniband/hw/hfi1/trace_tx.h179
-rw-r--r--drivers/infiniband/hw/hfi1/user_sdma.c12
-rw-r--r--drivers/infiniband/hw/hfi1/user_sdma.h1
-rw-r--r--drivers/infiniband/hw/hfi1/verbs.c8
-rw-r--r--drivers/infiniband/hw/hfi1/verbs.h4
-rw-r--r--drivers/infiniband/hw/hfi1/verbs_txreq.h5
-rw-r--r--drivers/infiniband/hw/hfi1/vnic.h2
-rw-r--r--drivers/infiniband/hw/hfi1/vnic_main.c2
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_alloc.c3
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_cmd.c114
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_common.h25
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_cq.c92
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_device.h91
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_hw_v1.c55
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_hw_v2.c2271
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_hw_v2.h578
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_main.c74
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_pd.c59
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_qp.c124
-rw-r--r--drivers/infiniband/hw/hns/hns_roce_srq.c5
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw.h9
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_cm.c4
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_hmc.c4
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_main.c5
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_osdep.h22
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_pble.c6
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_puda.c2
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_utils.c2
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_verbs.c14
-rw-r--r--drivers/infiniband/hw/i40iw/i40iw_virtchnl.c2
-rw-r--r--drivers/infiniband/hw/mlx4/alias_GUID.c16
-rw-r--r--drivers/infiniband/hw/mlx4/mad.c46
-rw-r--r--drivers/infiniband/hw/mlx4/main.c47
-rw-r--r--drivers/infiniband/hw/mlx4/mlx4_ib.h26
-rw-r--r--drivers/infiniband/hw/mlx4/qp.c3
-rw-r--r--drivers/infiniband/hw/mlx5/Makefile1
-rw-r--r--drivers/infiniband/hw/mlx5/cmd.c101
-rw-r--r--drivers/infiniband/hw/mlx5/cmd.h3
-rw-r--r--drivers/infiniband/hw/mlx5/cong.c8
-rw-r--r--drivers/infiniband/hw/mlx5/counters.c10
-rw-r--r--drivers/infiniband/hw/mlx5/counters.h2
-rw-r--r--drivers/infiniband/hw/mlx5/devx.c66
-rw-r--r--drivers/infiniband/hw/mlx5/dm.c587
-rw-r--r--drivers/infiniband/hw/mlx5/dm.h68
-rw-r--r--drivers/infiniband/hw/mlx5/fs.c11
-rw-r--r--drivers/infiniband/hw/mlx5/ib_rep.c9
-rw-r--r--drivers/infiniband/hw/mlx5/ib_rep.h4
-rw-r--r--drivers/infiniband/hw/mlx5/ib_virt.c16
-rw-r--r--drivers/infiniband/hw/mlx5/mad.c16
-rw-r--r--drivers/infiniband/hw/mlx5/main.c346
-rw-r--r--drivers/infiniband/hw/mlx5/mlx5_ib.h182
-rw-r--r--drivers/infiniband/hw/mlx5/mr.c163
-rw-r--r--drivers/infiniband/hw/mlx5/odp.c185
-rw-r--r--drivers/infiniband/hw/mlx5/qp.c35
-rw-r--r--drivers/infiniband/hw/mlx5/std_types.c173
-rw-r--r--drivers/infiniband/hw/mthca/mthca_av.c6
-rw-r--r--drivers/infiniband/hw/mthca/mthca_dev.h8
-rw-r--r--drivers/infiniband/hw/mthca/mthca_mad.c4
-rw-r--r--drivers/infiniband/hw/mthca/mthca_provider.c10
-rw-r--r--drivers/infiniband/hw/mthca/mthca_qp.c2
-rw-r--r--drivers/infiniband/hw/ocrdma/ocrdma_ah.c2
-rw-r--r--drivers/infiniband/hw/ocrdma/ocrdma_ah.h2
-rw-r--r--drivers/infiniband/hw/ocrdma/ocrdma_main.c4
-rw-r--r--drivers/infiniband/hw/ocrdma/ocrdma_verbs.c4
-rw-r--r--drivers/infiniband/hw/ocrdma/ocrdma_verbs.h7
-rw-r--r--drivers/infiniband/hw/qedr/main.c8
-rw-r--r--drivers/infiniband/hw/qedr/qedr_iw_cm.c4
-rw-r--r--drivers/infiniband/hw/qedr/verbs.c12
-rw-r--r--drivers/infiniband/hw/qedr/verbs.h11
-rw-r--r--drivers/infiniband/hw/qib/qib.h34
-rw-r--r--drivers/infiniband/hw/qib/qib_common.h7
-rw-r--r--drivers/infiniband/hw/qib/qib_file_ops.c5
-rw-r--r--drivers/infiniband/hw/qib/qib_fs.c68
-rw-r--r--drivers/infiniband/hw/qib/qib_iba6120.c2
-rw-r--r--drivers/infiniband/hw/qib/qib_iba7220.c4
-rw-r--r--drivers/infiniband/hw/qib/qib_iba7322.c26
-rw-r--r--drivers/infiniband/hw/qib/qib_init.c2
-rw-r--r--drivers/infiniband/hw/qib/qib_mad.c4
-rw-r--r--drivers/infiniband/hw/qib/qib_qp.c4
-rw-r--r--drivers/infiniband/hw/qib/qib_sd7220.c1
-rw-r--r--drivers/infiniband/hw/qib/qib_sysfs.c2
-rw-r--r--drivers/infiniband/hw/qib/qib_verbs.c6
-rw-r--r--drivers/infiniband/hw/qib/qib_verbs.h6
-rw-r--r--drivers/infiniband/hw/usnic/usnic_ib_main.c2
-rw-r--r--drivers/infiniband/hw/usnic/usnic_ib_verbs.c6
-rw-r--r--drivers/infiniband/hw/usnic/usnic_ib_verbs.h6
-rw-r--r--drivers/infiniband/hw/usnic/usnic_uiom_interval_tree.c3
-rw-r--r--drivers/infiniband/hw/vmw_pvrdma/pvrdma.h10
-rw-r--r--drivers/infiniband/hw/vmw_pvrdma/pvrdma_main.c2
-rw-r--r--drivers/infiniband/hw/vmw_pvrdma/pvrdma_qp.c2
-rw-r--r--drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.c12
-rw-r--r--drivers/infiniband/hw/vmw_pvrdma/pvrdma_verbs.h45
-rw-r--r--drivers/infiniband/sw/rdmavt/mad.c5
-rw-r--r--drivers/infiniband/sw/rdmavt/mad.h2
-rw-r--r--drivers/infiniband/sw/rdmavt/vt.c34
-rw-r--r--drivers/infiniband/sw/rdmavt/vt.h11
-rw-r--r--drivers/infiniband/sw/rxe/rxe_av.c2
-rw-r--r--drivers/infiniband/sw/rxe/rxe_comp.c5
-rw-r--r--drivers/infiniband/sw/rxe/rxe_hw_counters.c4
-rw-r--r--drivers/infiniband/sw/rxe/rxe_hw_counters.h4
-rw-r--r--drivers/infiniband/sw/rxe/rxe_loc.h30
-rw-r--r--drivers/infiniband/sw/rxe/rxe_mr.c271
-rw-r--r--drivers/infiniband/sw/rxe/rxe_pool.c14
-rw-r--r--drivers/infiniband/sw/rxe/rxe_req.c10
-rw-r--r--drivers/infiniband/sw/rxe/rxe_resp.c52
-rw-r--r--drivers/infiniband/sw/rxe/rxe_verbs.c32
-rw-r--r--drivers/infiniband/sw/rxe/rxe_verbs.h60
-rw-r--r--drivers/infiniband/sw/siw/iwarp.h13
-rw-r--r--drivers/infiniband/sw/siw/siw_cm.c19
-rw-r--r--drivers/infiniband/sw/siw/siw_mem.c4
-rw-r--r--drivers/infiniband/sw/siw/siw_mem.h5
-rw-r--r--drivers/infiniband/sw/siw/siw_verbs.c8
-rw-r--r--drivers/infiniband/sw/siw/siw_verbs.h10
-rw-r--r--drivers/infiniband/ulp/ipoib/ipoib.h6
-rw-r--r--drivers/infiniband/ulp/ipoib/ipoib_cm.c6
-rw-r--r--drivers/infiniband/ulp/ipoib/ipoib_ib.c2
-rw-r--r--drivers/infiniband/ulp/ipoib/ipoib_main.c26
-rw-r--r--drivers/infiniband/ulp/iser/iscsi_iser.h1
-rw-r--r--drivers/infiniband/ulp/isert/ib_isert.c16
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs-clt-sysfs.c48
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs-clt.c200
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs-clt.h2
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs-pri.h4
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs-srv-sysfs.c20
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs-srv.c40
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs.c35
-rw-r--r--drivers/infiniband/ulp/rtrs/rtrs.h16
-rw-r--r--drivers/infiniband/ulp/srpt/ib_srpt.c18
-rw-r--r--drivers/input/Makefile1
-rw-r--r--drivers/input/joydev.c1
-rw-r--r--drivers/input/joystick/n64joy.c4
-rw-r--r--drivers/input/joystick/xpad.c2
-rw-r--r--drivers/input/keyboard/gpio_keys.c105
-rw-r--r--drivers/input/keyboard/imx_keypad.c13
-rw-r--r--drivers/input/keyboard/nspire-keypad.c56
-rw-r--r--drivers/input/keyboard/tca6416-keypad.c3
-rw-r--r--drivers/input/keyboard/tegra-kbc.c5
-rw-r--r--drivers/input/misc/Kconfig22
-rw-r--r--drivers/input/misc/Makefile3
-rw-r--r--drivers/input/misc/atc260x-onkey.c305
-rw-r--r--drivers/input/misc/ims-pcu.c1
-rw-r--r--drivers/input/misc/iqs626a.c1838
-rw-r--r--drivers/input/misc/max8997_haptic.c9
-rw-r--r--drivers/input/mouse/elan_i2c.h7
-rw-r--r--drivers/input/mouse/elan_i2c_core.c58
-rw-r--r--drivers/input/mouse/elantech.c6
-rw-r--r--drivers/input/serio/apbps2.c3
-rw-r--r--drivers/input/serio/i8042-x86ia64io.h1
-rw-r--r--drivers/input/touchscreen.c (renamed from drivers/input/touchscreen/of_touchscreen.c)13
-rw-r--r--drivers/input/touchscreen/Kconfig39
-rw-r--r--drivers/input/touchscreen/Makefile4
-rw-r--r--drivers/input/touchscreen/ar1021_i2c.c5
-rw-r--r--drivers/input/touchscreen/atmel_mxt_ts.c83
-rw-r--r--drivers/input/touchscreen/bu21029_ts.c4
-rw-r--r--drivers/input/touchscreen/cyttsp_core.c39
-rw-r--r--drivers/input/touchscreen/cyttsp_core.h1
-rw-r--r--drivers/input/touchscreen/elants_i2c.c49
-rw-r--r--drivers/input/touchscreen/exc3000.c253
-rw-r--r--drivers/input/touchscreen/hycon-hy46xx.c591
-rw-r--r--drivers/input/touchscreen/ili210x.c2
-rw-r--r--drivers/input/touchscreen/ilitek_ts_i2c.c690
-rw-r--r--drivers/input/touchscreen/iqs5xx.c171
-rw-r--r--drivers/input/touchscreen/lpc32xx_ts.c10
-rw-r--r--drivers/input/touchscreen/melfas_mip4.c5
-rw-r--r--drivers/input/touchscreen/mms114.c26
-rw-r--r--drivers/input/touchscreen/msg2638.c337
-rw-r--r--drivers/input/touchscreen/raspberrypi-ts.c2
-rw-r--r--drivers/input/touchscreen/s6sy761.c4
-rw-r--r--drivers/input/touchscreen/silead.c46
-rw-r--r--drivers/input/touchscreen/stmfts.c3
-rw-r--r--drivers/input/touchscreen/tsc2007.h4
-rw-r--r--drivers/input/touchscreen/tsc2007_core.c60
-rw-r--r--drivers/input/touchscreen/wacom_i2c.c56
-rw-r--r--drivers/input/touchscreen/wm831x-ts.c3
-rw-r--r--drivers/input/touchscreen/zinitix.c4
-rw-r--r--drivers/interconnect/bulk.c2
-rw-r--r--drivers/interconnect/core.c2
-rw-r--r--drivers/interconnect/qcom/Kconfig18
-rw-r--r--drivers/interconnect/qcom/Makefile4
-rw-r--r--drivers/interconnect/qcom/bcm-voter.c2
-rw-r--r--drivers/interconnect/qcom/icc-rpm.c4
-rw-r--r--drivers/interconnect/qcom/msm8939.c16
-rw-r--r--drivers/interconnect/qcom/sdm660.c923
-rw-r--r--drivers/interconnect/qcom/sm8350.c633
-rw-r--r--drivers/interconnect/qcom/sm8350.h168
-rw-r--r--drivers/iommu/Kconfig16
-rw-r--r--drivers/iommu/Makefile3
-rw-r--r--drivers/iommu/amd/amd_iommu.h2
-rw-r--r--drivers/iommu/amd/amd_iommu_types.h1
-rw-r--r--drivers/iommu/amd/init.c95
-rw-r--r--drivers/iommu/amd/iommu.c201
-rw-r--r--drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c247
-rw-r--r--drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h18
-rw-r--r--drivers/iommu/arm/arm-smmu/arm-smmu.c117
-rw-r--r--drivers/iommu/arm/arm-smmu/arm-smmu.h2
-rw-r--r--drivers/iommu/arm/arm-smmu/qcom_iommu.c5
-rw-r--r--drivers/iommu/dma-iommu.c149
-rw-r--r--drivers/iommu/exynos-iommu.c7
-rw-r--r--drivers/iommu/fsl_pamu.c293
-rw-r--r--drivers/iommu/fsl_pamu.h12
-rw-r--r--drivers/iommu/fsl_pamu_domain.c693
-rw-r--r--drivers/iommu/fsl_pamu_domain.h46
-rw-r--r--drivers/iommu/intel/dmar.c72
-rw-r--r--drivers/iommu/intel/iommu.c233
-rw-r--r--drivers/iommu/intel/irq_remapping.c5
-rw-r--r--drivers/iommu/intel/pasid.c75
-rw-r--r--drivers/iommu/intel/pasid.h6
-rw-r--r--drivers/iommu/intel/svm.c82
-rw-r--r--drivers/iommu/io-pgfault.c461
-rw-r--r--drivers/iommu/iommu-sva-lib.h53
-rw-r--r--drivers/iommu/iommu.c161
-rw-r--r--drivers/iommu/iova.c96
-rw-r--r--drivers/iommu/ipmmu-vmsa.c6
-rw-r--r--drivers/iommu/msm_iommu.c5
-rw-r--r--drivers/iommu/mtk_iommu.c41
-rw-r--r--drivers/iommu/mtk_iommu_v1.c98
-rw-r--r--drivers/iommu/of_iommu.c5
-rw-r--r--drivers/iommu/omap-iommu.c5
-rw-r--r--drivers/iommu/rockchip-iommu.c5
-rw-r--r--drivers/iommu/s390-iommu.c4
-rw-r--r--drivers/iommu/sprd-iommu.c575
-rw-r--r--drivers/iommu/sun50i-iommu.c5
-rw-r--r--drivers/iommu/tegra-gart.c5
-rw-r--r--drivers/iommu/tegra-smmu.c12
-rw-r--r--drivers/iommu/virtio-iommu.c6
-rw-r--r--drivers/irqchip/Kconfig35
-rw-r--r--drivers/irqchip/Makefile3
-rw-r--r--drivers/irqchip/irq-apple-aic.c852
-rw-r--r--drivers/irqchip/irq-aspeed-vic.c4
-rw-r--r--drivers/irqchip/irq-bcm7120-l2.c2
-rw-r--r--drivers/irqchip/irq-csky-apb-intc.c2
-rw-r--r--drivers/irqchip/irq-gic-v2m.c2
-rw-r--r--drivers/irqchip/irq-gic-v3-its.c28
-rw-r--r--drivers/irqchip/irq-gic-v3-mbi.c2
-rw-r--r--drivers/irqchip/irq-gic-v3.c10
-rw-r--r--drivers/irqchip/irq-gic-v4.c27
-rw-r--r--drivers/irqchip/irq-hip04.c4
-rw-r--r--drivers/irqchip/irq-idt3243x.c124
-rw-r--r--drivers/irqchip/irq-ingenic-tcu.c1
-rw-r--r--drivers/irqchip/irq-ingenic.c1
-rw-r--r--drivers/irqchip/irq-jcore-aic.c4
-rw-r--r--drivers/irqchip/irq-loongson-liointc.c60
-rw-r--r--drivers/irqchip/irq-loongson-pch-pic.c2
-rw-r--r--drivers/irqchip/irq-mbigen.c4
-rw-r--r--drivers/irqchip/irq-meson-gpio.c2
-rw-r--r--drivers/irqchip/irq-mst-intc.c98
-rw-r--r--drivers/irqchip/irq-mtk-cirq.c2
-rw-r--r--drivers/irqchip/irq-mxs.c4
-rw-r--r--drivers/irqchip/irq-sifive-plic.c4
-rw-r--r--drivers/irqchip/irq-stm32-exti.c7
-rw-r--r--drivers/irqchip/irq-sun4i.c2
-rw-r--r--drivers/irqchip/irq-tb10x.c1
-rw-r--r--drivers/irqchip/irq-ti-sci-inta.c2
-rw-r--r--drivers/irqchip/irq-vic.c4
-rw-r--r--drivers/irqchip/irq-wpcm450-aic.c161
-rw-r--r--drivers/irqchip/irq-xilinx-intc.c2
-rw-r--r--drivers/isdn/capi/kcapi.c4
-rw-r--r--drivers/isdn/capi/kcapi_proc.c1
-rw-r--r--drivers/isdn/hardware/mISDN/hfcmulti.c9
-rw-r--r--drivers/isdn/hardware/mISDN/iohelper.h14
-rw-r--r--drivers/isdn/hardware/mISDN/mISDNipac.c2
-rw-r--r--drivers/isdn/mISDN/dsp_core.c13
-rw-r--r--drivers/isdn/mISDN/l1oip_core.c9
-rw-r--r--drivers/leds/Kconfig7
-rw-r--r--drivers/leds/Makefile2
-rw-r--r--drivers/leds/blink/Kconfig33
-rw-r--r--drivers/leds/blink/Makefile2
-rw-r--r--drivers/leds/blink/leds-lgm-sso.c2
-rw-r--r--drivers/leds/flash/Kconfig11
-rw-r--r--drivers/leds/flash/Makefile1
-rw-r--r--drivers/leds/flash/leds-rt4505.c430
-rw-r--r--drivers/leds/leds-lm3642.c4
-rw-r--r--drivers/leds/leds-pca9532.c2
-rw-r--r--drivers/leds/leds-turris-omnia.c4
-rw-r--r--drivers/leds/trigger/ledtrig-pattern.c2
-rw-r--r--drivers/leds/trigger/ledtrig-tty.c11
-rw-r--r--drivers/lightnvm/Kconfig4
-rw-r--r--drivers/lightnvm/core.c4
-rw-r--r--drivers/macintosh/via-pmu.c4
-rw-r--r--drivers/macintosh/windfarm_core.c2
-rw-r--r--drivers/macintosh/windfarm_pm121.c2
-rw-r--r--drivers/macintosh/windfarm_smu_controls.c2
-rw-r--r--drivers/mailbox/Kconfig2
-rw-r--r--drivers/mailbox/arm_mhu_db.c4
-rw-r--r--drivers/mailbox/armada-37xx-rwtm-mailbox.c4
-rw-r--r--drivers/mailbox/bcm-flexrm-mailbox.c4
-rw-r--r--drivers/mailbox/mailbox-xgene-slimpro.c6
-rw-r--r--drivers/mailbox/mailbox.h2
-rw-r--r--drivers/mailbox/pcc.c3
-rw-r--r--drivers/mailbox/pl320-ipc.c2
-rw-r--r--drivers/mailbox/sprd-mailbox.c147
-rw-r--r--drivers/mailbox/ti-msgmgr.c2
-rw-r--r--drivers/md/bcache/alloc.c5
-rw-r--r--drivers/md/bcache/bcache.h11
-rw-r--r--drivers/md/bcache/btree.c4
-rw-r--r--drivers/md/bcache/debug.c2
-rw-r--r--drivers/md/bcache/extents.c4
-rw-r--r--drivers/md/bcache/features.c2
-rw-r--r--drivers/md/bcache/io.c4
-rw-r--r--drivers/md/bcache/journal.c6
-rw-r--r--drivers/md/bcache/super.c28
-rw-r--r--drivers/md/bcache/util.h2
-rw-r--r--drivers/md/bcache/writeback.c11
-rw-r--r--drivers/md/dm-cache-target.c2
-rw-r--r--drivers/md/dm-clone-metadata.c6
-rw-r--r--drivers/md/dm-crypt.c8
-rw-r--r--drivers/md/dm-ebs-target.c6
-rw-r--r--drivers/md/dm-integrity.c85
-rw-r--r--drivers/md/dm-ioctl.c296
-rw-r--r--drivers/md/dm-raid.c44
-rw-r--r--drivers/md/dm-rq.c2
-rw-r--r--drivers/md/dm-snap-persistent.c6
-rw-r--r--drivers/md/dm-snap.c5
-rw-r--r--drivers/md/dm-table.c63
-rw-r--r--drivers/md/dm-thin.c2
-rw-r--r--drivers/md/dm-verity-fec.c11
-rw-r--r--drivers/md/dm-verity-fec.h1
-rw-r--r--drivers/md/dm-verity-target.c42
-rw-r--r--drivers/md/dm-writecache.c6
-rw-r--r--drivers/md/dm-zoned-target.c2
-rw-r--r--drivers/md/dm.c68
-rw-r--r--drivers/md/md-bitmap.c2
-rw-r--r--drivers/md/md.c206
-rw-r--r--drivers/md/md.h2
-rw-r--r--drivers/md/persistent-data/dm-btree-internal.h6
-rw-r--r--drivers/md/persistent-data/dm-btree-spine.c8
-rw-r--r--drivers/md/persistent-data/dm-space-map-common.c2
-rw-r--r--drivers/md/persistent-data/dm-space-map-common.h8
-rw-r--r--drivers/md/persistent-data/dm-space-map-disk.c9
-rw-r--r--drivers/md/raid0.c14
-rw-r--r--drivers/md/raid1.c2
-rw-r--r--drivers/md/raid10.c434
-rw-r--r--drivers/md/raid10.h1
-rw-r--r--drivers/md/raid5-cache.c4
-rw-r--r--drivers/md/raid5-ppl.c2
-rw-r--r--drivers/md/raid5.c3
-rw-r--r--drivers/media/Kconfig2
-rw-r--r--drivers/media/cec/core/cec-notifier.c33
-rw-r--r--drivers/media/cec/core/cec-pin-error-inj.c2
-rw-r--r--drivers/media/common/b2c2/flexcop-hw-filter.c2
-rw-r--r--drivers/media/common/b2c2/flexcop.h6
-rw-r--r--drivers/media/common/saa7146/saa7146_core.c2
-rw-r--r--drivers/media/common/saa7146/saa7146_video.c3
-rw-r--r--drivers/media/common/siano/smscoreapi.c42
-rw-r--r--drivers/media/common/siano/smscoreapi.h5
-rw-r--r--drivers/media/common/siano/smsdvb-main.c58
-rw-r--r--drivers/media/common/videobuf2/frame_vector.c1
-rw-r--r--drivers/media/common/videobuf2/videobuf2-dma-sg.c2
-rw-r--r--drivers/media/dvb-core/dvb_ca_en50221.c32
-rw-r--r--drivers/media/dvb-core/dvb_frontend.c93
-rw-r--r--drivers/media/dvb-core/dvbdev.c1
-rw-r--r--drivers/media/dvb-frontends/drx39xyj/drxj.c2
-rw-r--r--drivers/media/dvb-frontends/lgdt3306a.c38
-rw-r--r--drivers/media/dvb-frontends/m88ds3103.c4
-rw-r--r--drivers/media/dvb-frontends/mxl692.c2
-rw-r--r--drivers/media/firewire/firedtv-fw.c1
-rw-r--r--drivers/media/i2c/adv748x/adv748x.h10
-rw-r--r--drivers/media/i2c/adv7511-v4l2.c72
-rw-r--r--drivers/media/i2c/adv7604.c118
-rw-r--r--drivers/media/i2c/adv7842.c108
-rw-r--r--drivers/media/i2c/ccs/ccs-core.c6
-rw-r--r--drivers/media/i2c/ccs/ccs-data.h2
-rw-r--r--drivers/media/i2c/ccs/ccs-quirk.h4
-rw-r--r--drivers/media/i2c/et8ek8/et8ek8_driver.c2
-rw-r--r--drivers/media/i2c/hi556.c2
-rw-r--r--drivers/media/i2c/imx214.c2
-rw-r--r--drivers/media/i2c/imx219.c51
-rw-r--r--drivers/media/i2c/imx258.c30
-rw-r--r--drivers/media/i2c/imx274.c10
-rw-r--r--drivers/media/i2c/imx319.c2
-rw-r--r--drivers/media/i2c/imx334.c2
-rw-r--r--drivers/media/i2c/imx355.c2
-rw-r--r--drivers/media/i2c/m5mols/m5mols.h4
-rw-r--r--drivers/media/i2c/max2175.c3
-rw-r--r--drivers/media/i2c/ov02a10.c2
-rw-r--r--drivers/media/i2c/ov13858.c2
-rw-r--r--drivers/media/i2c/ov2740.c2
-rw-r--r--drivers/media/i2c/ov5640.c2
-rw-r--r--drivers/media/i2c/ov5648.c2
-rw-r--r--drivers/media/i2c/ov5670.c2
-rw-r--r--drivers/media/i2c/ov5675.c2
-rw-r--r--drivers/media/i2c/ov5695.c2
-rw-r--r--drivers/media/i2c/ov8856.c2
-rw-r--r--drivers/media/i2c/ov8865.c5
-rw-r--r--drivers/media/i2c/ov9734.c2
-rw-r--r--drivers/media/i2c/rdacm21.c2
-rw-r--r--drivers/media/i2c/s5k5baf.c2
-rw-r--r--drivers/media/i2c/s5k6aa.c2
-rw-r--r--drivers/media/i2c/tc358743.c2
-rw-r--r--drivers/media/i2c/tda1997x.c2
-rw-r--r--drivers/media/i2c/tvp514x_regs.h6
-rw-r--r--drivers/media/mc/mc-entity.c1
-rw-r--r--drivers/media/pci/b2c2/flexcop-pci.c6
-rw-r--r--drivers/media/pci/bt8xx/bttv-cards.c20
-rw-r--r--drivers/media/pci/cobalt/cobalt-driver.c12
-rw-r--r--drivers/media/pci/cx18/cx18-alsa-main.c1
-rw-r--r--drivers/media/pci/cx18/cx18-av-audio.c2
-rw-r--r--drivers/media/pci/cx18/cx18-av-core.c4
-rw-r--r--drivers/media/pci/cx18/cx18-driver.c1
-rw-r--r--drivers/media/pci/cx18/cx18-firmware.c2
-rw-r--r--drivers/media/pci/cx23885/cx23885-alsa.c2
-rw-r--r--drivers/media/pci/cx23885/cx23885-cards.c28
-rw-r--r--drivers/media/pci/cx23885/cx23885-core.c22
-rw-r--r--drivers/media/pci/cx25821/cx25821-alsa.c3
-rw-r--r--drivers/media/pci/cx25821/cx25821-core.c10
-rw-r--r--drivers/media/pci/cx25821/cx25821.h1
-rw-r--r--drivers/media/pci/cx88/cx88-alsa.c1
-rw-r--r--drivers/media/pci/cx88/cx88-cards.c2
-rw-r--r--drivers/media/pci/intel/ipu3/cio2-bridge.c4
-rw-r--r--drivers/media/pci/intel/ipu3/ipu3-cio2-main.c2
-rw-r--r--drivers/media/pci/ivtv/ivtv-alsa-main.c1
-rw-r--r--drivers/media/pci/ivtv/ivtv-driver.c5
-rw-r--r--drivers/media/pci/mantis/mantis_ca.c1
-rw-r--r--drivers/media/pci/mantis/mantis_core.c200
-rw-r--r--drivers/media/pci/mantis/mantis_core.h2
-rw-r--r--drivers/media/pci/mantis/mantis_dma.c20
-rw-r--r--drivers/media/pci/mantis/mantis_pci.c2
-rw-r--r--drivers/media/pci/ngene/ngene-cards.c6
-rw-r--r--drivers/media/pci/ngene/ngene-core.c56
-rw-r--r--drivers/media/pci/pt1/pt1.c6
-rw-r--r--drivers/media/pci/saa7134/saa7134-core.c2
-rw-r--r--drivers/media/pci/saa7164/saa7164-encoder.c20
-rw-r--r--drivers/media/pci/saa7164/saa7164-types.h4
-rw-r--r--drivers/media/pci/solo6x10/solo6x10-core.c2
-rw-r--r--drivers/media/pci/sta2x11/Kconfig1
-rw-r--r--drivers/media/pci/sta2x11/sta2x11_vip.c1
-rw-r--r--drivers/media/pci/sta2x11/sta2x11_vip.h2
-rw-r--r--drivers/media/pci/tw68/tw68-core.c2
-rw-r--r--drivers/media/pci/tw68/tw68-risc.c3
-rw-r--r--drivers/media/pci/tw68/tw68-video.c2
-rw-r--r--drivers/media/pci/tw686x/tw686x-audio.c13
-rw-r--r--drivers/media/pci/tw686x/tw686x-core.c2
-rw-r--r--drivers/media/pci/tw686x/tw686x-video.c17
-rw-r--r--drivers/media/pci/tw686x/tw686x.h10
-rw-r--r--drivers/media/platform/Kconfig21
-rw-r--r--drivers/media/platform/Makefile1
-rw-r--r--drivers/media/platform/allegro-dvt/nal-h264.h10
-rw-r--r--drivers/media/platform/allegro-dvt/nal-hevc.h6
-rw-r--r--drivers/media/platform/aspeed-video.c9
-rw-r--r--drivers/media/platform/atmel/atmel-isi.c1
-rw-r--r--drivers/media/platform/atmel/atmel-sama5d2-isc.c1
-rw-r--r--drivers/media/platform/coda/coda-common.c6
-rw-r--r--drivers/media/platform/exynos-gsc/gsc-core.h21
-rw-r--r--drivers/media/platform/exynos4-is/fimc-core.h12
-rw-r--r--drivers/media/platform/exynos4-is/fimc-is.h21
-rw-r--r--drivers/media/platform/exynos4-is/fimc-isp.h17
-rw-r--r--drivers/media/platform/exynos4-is/fimc-lite.h2
-rw-r--r--drivers/media/platform/exynos4-is/fimc-reg.h1
-rw-r--r--drivers/media/platform/exynos4-is/media-dev.c6
-rw-r--r--drivers/media/platform/exynos4-is/media-dev.h7
-rw-r--r--drivers/media/platform/fsl-viu.c6
-rw-r--r--drivers/media/platform/imx-jpeg/Kconfig11
-rw-r--r--drivers/media/platform/imx-jpeg/Makefile3
-rw-r--r--drivers/media/platform/imx-jpeg/mxc-jpeg-hw.c168
-rw-r--r--drivers/media/platform/imx-jpeg/mxc-jpeg-hw.h140
-rw-r--r--drivers/media/platform/imx-jpeg/mxc-jpeg.c2126
-rw-r--r--drivers/media/platform/imx-jpeg/mxc-jpeg.h180
-rw-r--r--drivers/media/platform/imx-pxp.c9
-rw-r--r--drivers/media/platform/marvell-ccic/cafe-driver.c4
-rw-r--r--drivers/media/platform/meson/ge2d/ge2d.c4
-rw-r--r--drivers/media/platform/mtk-jpeg/mtk_jpeg_core.h14
-rw-r--r--drivers/media/platform/mtk-mdp/mtk_mdp_comp.h1
-rw-r--r--drivers/media/platform/mtk-mdp/mtk_mdp_core.h10
-rw-r--r--drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h21
-rw-r--r--drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c18
-rw-r--r--drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c108
-rw-r--r--drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c40
-rw-r--r--drivers/media/platform/mtk-vcodec/vdec/vdec_vp9_if.c4
-rw-r--r--drivers/media/platform/mtk-vcodec/vdec_drv_if.h8
-rw-r--r--drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c4
-rw-r--r--drivers/media/platform/mtk-vcodec/venc_ipi_msg.h14
-rw-r--r--drivers/media/platform/mtk-vpu/mtk_vpu.c4
-rw-r--r--drivers/media/platform/mtk-vpu/mtk_vpu.h18
-rw-r--r--drivers/media/platform/omap3isp/isp.c16
-rw-r--r--drivers/media/platform/pxa_camera.c4
-rw-r--r--drivers/media/platform/qcom/camss/Makefile6
-rw-r--r--drivers/media/platform/qcom/camss/camss-csid-170.c599
-rw-r--r--drivers/media/platform/qcom/camss/camss-csid-4-1.c328
-rw-r--r--drivers/media/platform/qcom/camss/camss-csid-4-7.c404
-rw-r--r--drivers/media/platform/qcom/camss/camss-csid-gen1.h27
-rw-r--r--drivers/media/platform/qcom/camss/camss-csid-gen2.h39
-rw-r--r--drivers/media/platform/qcom/camss/camss-csid.c661
-rw-r--r--drivers/media/platform/qcom/camss/camss-csid.h150
-rw-r--r--drivers/media/platform/qcom/camss/camss-csiphy-2ph-1-0.c22
-rw-r--r--drivers/media/platform/qcom/camss/camss-csiphy-3ph-1-0.c197
-rw-r--r--drivers/media/platform/qcom/camss/camss-csiphy.c104
-rw-r--r--drivers/media/platform/qcom/camss/camss-csiphy.h2
-rw-r--r--drivers/media/platform/qcom/camss/camss-ispif.c127
-rw-r--r--drivers/media/platform/qcom/camss/camss-ispif.h3
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe-170.c786
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe-4-1.c144
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe-4-7.c277
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe-4-8.c1195
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe-gen1.c742
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe-gen1.h117
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe.c847
-rw-r--r--drivers/media/platform/qcom/camss/camss-vfe.h128
-rw-r--r--drivers/media/platform/qcom/camss/camss-video.c52
-rw-r--r--drivers/media/platform/qcom/camss/camss.c435
-rw-r--r--drivers/media/platform/qcom/camss/camss.h19
-rw-r--r--drivers/media/platform/qcom/venus/core.c116
-rw-r--r--drivers/media/platform/qcom/venus/core.h52
-rw-r--r--drivers/media/platform/qcom/venus/firmware.c42
-rw-r--r--drivers/media/platform/qcom/venus/helpers.c71
-rw-r--r--drivers/media/platform/qcom/venus/helpers.h3
-rw-r--r--drivers/media/platform/qcom/venus/hfi_cmds.c59
-rw-r--r--drivers/media/platform/qcom/venus/hfi_helper.h39
-rw-r--r--drivers/media/platform/qcom/venus/hfi_msgs.c20
-rw-r--r--drivers/media/platform/qcom/venus/hfi_parser.c12
-rw-r--r--drivers/media/platform/qcom/venus/hfi_plat_bufs_v6.c32
-rw-r--r--drivers/media/platform/qcom/venus/hfi_platform_v6.c138
-rw-r--r--drivers/media/platform/qcom/venus/hfi_venus.c167
-rw-r--r--drivers/media/platform/qcom/venus/hfi_venus_io.h118
-rw-r--r--drivers/media/platform/qcom/venus/pm_helpers.c143
-rw-r--r--drivers/media/platform/qcom/venus/pm_helpers.h7
-rw-r--r--drivers/media/platform/qcom/venus/vdec.c36
-rw-r--r--drivers/media/platform/qcom/venus/vdec_ctrls.c23
-rw-r--r--drivers/media/platform/qcom/venus/venc.c54
-rw-r--r--drivers/media/platform/qcom/venus/venc_ctrls.c70
-rw-r--r--drivers/media/platform/rcar-vin/rcar-vin.h19
-rw-r--r--drivers/media/platform/rockchip/rkisp1/rkisp1-params.c446
-rw-r--r--drivers/media/platform/rockchip/rkisp1/rkisp1-resizer.c9
-rw-r--r--drivers/media/platform/s3c-camif/camif-core.h16
-rw-r--r--drivers/media/platform/s5p-jpeg/jpeg-core.h18
-rw-r--r--drivers/media/platform/s5p-mfc/s5p_mfc_common.h51
-rw-r--r--drivers/media/platform/s5p-mfc/s5p_mfc_dec.c16
-rw-r--r--drivers/media/platform/sti/bdisp/bdisp-debug.c2
-rw-r--r--drivers/media/platform/sti/bdisp/bdisp-filter.h4
-rw-r--r--drivers/media/platform/sti/c8sectpfe/c8sectpfe-core.c2
-rw-r--r--drivers/media/platform/sti/c8sectpfe/c8sectpfe-debugfs.h3
-rw-r--r--drivers/media/platform/sti/hva/hva-h264.c10
-rw-r--r--drivers/media/platform/sti/hva/hva.h2
-rw-r--r--drivers/media/platform/stm32/stm32-dcmi.c1
-rw-r--r--drivers/media/platform/sunxi/sun6i-csi/sun6i_video.c4
-rw-r--r--drivers/media/platform/sunxi/sun8i-di/sun8i-di.c2
-rw-r--r--drivers/media/platform/ti-vpe/cal-camerarx.c1
-rw-r--r--drivers/media/platform/ti-vpe/cal-video.c332
-rw-r--r--drivers/media/platform/ti-vpe/cal.c16
-rw-r--r--drivers/media/platform/ti-vpe/cal.h21
-rw-r--r--drivers/media/platform/ti-vpe/csc.c4
-rw-r--r--drivers/media/platform/ti-vpe/sc.c4
-rw-r--r--drivers/media/platform/ti-vpe/vpdma.c3
-rw-r--r--drivers/media/platform/vsp1/vsp1_dl.c2
-rw-r--r--drivers/media/platform/vsp1/vsp1_drm.c6
-rw-r--r--drivers/media/platform/vsp1/vsp1_drm.h4
-rw-r--r--drivers/media/platform/xilinx/xilinx-vipp.c1
-rw-r--r--drivers/media/radio/radio-maxiradio.c2
-rw-r--r--drivers/media/radio/radio-si476x.c6
-rw-r--r--drivers/media/rc/Kconfig11
-rw-r--r--drivers/media/rc/Makefile2
-rw-r--r--drivers/media/rc/img-ir/img-ir-hw.c2
-rw-r--r--drivers/media/rc/ir-hix5hd2.c2
-rw-r--r--drivers/media/rc/ite-cir.c351
-rw-r--r--drivers/media/rc/ite-cir.h51
-rw-r--r--drivers/media/rc/keymaps/Makefile5
-rw-r--r--drivers/media/rc/keymaps/rc-cec.c28
-rw-r--r--drivers/media/rc/keymaps/rc-mecool-kii-pro.c91
-rw-r--r--drivers/media/rc/keymaps/rc-mecool-kiii-pro.c88
-rw-r--r--drivers/media/rc/keymaps/rc-minix-neo.c55
-rw-r--r--drivers/media/rc/keymaps/rc-xbox-360.c83
-rw-r--r--drivers/media/rc/rc-main.c6
-rw-r--r--drivers/media/rc/zx-irdec.c181
-rw-r--r--drivers/media/spi/cxd2880-spi.c4
-rw-r--r--drivers/media/test-drivers/vidtv/vidtv_psi.c1
-rw-r--r--drivers/media/test-drivers/vim2m.c14
-rw-r--r--drivers/media/test-drivers/vivid/vivid-core.c6
-rw-r--r--drivers/media/test-drivers/vivid/vivid-core.h1
-rw-r--r--drivers/media/test-drivers/vivid/vivid-ctrls.c13
-rw-r--r--drivers/media/test-drivers/vivid/vivid-kthread-cap.c10
-rw-r--r--drivers/media/test-drivers/vivid/vivid-vid-out.c2
-rw-r--r--drivers/media/tuners/it913x.h2
-rw-r--r--drivers/media/tuners/m88rs6000t.c6
-rw-r--r--drivers/media/tuners/mt2063.c6
-rw-r--r--drivers/media/tuners/qt1010.h8
-rw-r--r--drivers/media/tuners/tda827x.h10
-rw-r--r--drivers/media/tuners/tuner-i2c.h4
-rw-r--r--drivers/media/usb/b2c2/flexcop-usb.c9
-rw-r--r--drivers/media/usb/cpia2/cpia2_v4l.c1
-rw-r--r--drivers/media/usb/cx231xx/cx231xx.h2
-rw-r--r--drivers/media/usb/dvb-usb-v2/anysee.c21
-rw-r--r--drivers/media/usb/dvb-usb-v2/dvb_usb.h34
-rw-r--r--drivers/media/usb/dvb-usb/dvb-usb-init.c90
-rw-r--r--drivers/media/usb/dvb-usb/dvb-usb.h91
-rw-r--r--drivers/media/usb/em28xx/em28xx-cards.c4
-rw-r--r--drivers/media/usb/em28xx/em28xx-dvb.c1
-rw-r--r--drivers/media/usb/em28xx/em28xx.h4
-rw-r--r--drivers/media/usb/gspca/gspca.c2
-rw-r--r--drivers/media/usb/gspca/gspca.h1
-rw-r--r--drivers/media/usb/gspca/sq905.c2
-rw-r--r--drivers/media/usb/gspca/stv06xx/stv06xx.c9
-rw-r--r--drivers/media/usb/gspca/w996Xcf.c3
-rw-r--r--drivers/media/usb/pwc/pwc-dec23.c2
-rw-r--r--drivers/media/usb/pwc/pwc-if.c8
-rw-r--r--drivers/media/usb/pwc/pwc-uncompress.c3
-rw-r--r--drivers/media/usb/tm6000/tm6000-alsa.c1
-rw-r--r--drivers/media/usb/tm6000/tm6000-dvb.c2
-rw-r--r--drivers/media/usb/ttusb-dec/ttusb_dec.c4
-rw-r--r--drivers/media/usb/usbtv/usbtv-audio.c2
-rw-r--r--drivers/media/usb/usbtv/usbtv-video.c6
-rw-r--r--drivers/media/usb/uvc/uvc_driver.c34
-rw-r--r--drivers/media/usb/uvc/uvc_video.c94
-rw-r--r--drivers/media/usb/uvc/uvcvideo.h5
-rw-r--r--drivers/media/usb/zr364xx/zr364xx.c13
-rw-r--r--drivers/media/v4l2-core/v4l2-common.c5
-rw-r--r--drivers/media/v4l2-core/v4l2-ctrls.c307
-rw-r--r--drivers/media/v4l2-core/v4l2-dev.c5
-rw-r--r--drivers/media/v4l2-core/v4l2-fwnode.c31
-rw-r--r--drivers/media/v4l2-core/v4l2-ioctl.c88
-rw-r--r--drivers/media/v4l2-core/v4l2-jpeg.c59
-rw-r--r--drivers/media/v4l2-core/v4l2-mc.c8
-rw-r--r--drivers/memory/.gitignore2
-rw-r--r--drivers/memory/fsl-corenet-cf.c4
-rw-r--r--drivers/memory/mtk-smi.c19
-rw-r--r--drivers/memory/omap-gpmc.c7
-rw-r--r--drivers/memory/pl353-smc.c2
-rw-r--r--drivers/memory/renesas-rpc-if.c2
-rw-r--r--drivers/memory/samsung/exynos5422-dmc.c17
-rw-r--r--drivers/memory/tegra/mc.c9
-rw-r--r--drivers/memory/tegra/mc.h4
-rw-r--r--drivers/memory/tegra/tegra124-emc.c16
-rw-r--r--drivers/memory/tegra/tegra20-emc.c20
-rw-r--r--drivers/memory/tegra/tegra20.c332
-rw-r--r--drivers/memory/tegra/tegra30-emc.c18
-rw-r--r--drivers/memstick/core/memstick.c21
-rw-r--r--drivers/memstick/core/mspro_block.c3
-rw-r--r--drivers/memstick/host/r592.c6
-rw-r--r--drivers/message/fusion/lsi/mpi.h4
-rw-r--r--drivers/message/fusion/lsi/mpi_ioc.h2
-rw-r--r--drivers/message/fusion/mptbase.c9
-rw-r--r--drivers/message/fusion/mptbase.h2
-rw-r--r--drivers/message/fusion/mptctl.c8
-rw-r--r--drivers/message/fusion/mptdebug.h7
-rw-r--r--drivers/message/fusion/mptlan.c9
-rw-r--r--drivers/message/fusion/mptsas.c10
-rw-r--r--drivers/mfd/Kconfig86
-rw-r--r--drivers/mfd/Makefile7
-rw-r--r--drivers/mfd/ab3100-core.c929
-rw-r--r--drivers/mfd/ab3100-otp.c240
-rw-r--r--drivers/mfd/ab8500-core.c37
-rw-r--r--drivers/mfd/arizona-core.c20
-rw-r--r--drivers/mfd/arizona-irq.c2
-rw-r--r--drivers/mfd/arizona-spi.c4
-rw-r--r--drivers/mfd/atc260x-core.c310
-rw-r--r--drivers/mfd/atc260x-i2c.c64
-rw-r--r--drivers/mfd/da9063-i2c.c10
-rw-r--r--drivers/mfd/ene-kb3930.c2
-rw-r--r--drivers/mfd/intel-lpss-acpi.c32
-rw-r--r--drivers/mfd/intel-lpss-pci.c46
-rw-r--r--drivers/mfd/intel-lpss.c2
-rw-r--r--drivers/mfd/intel-lpss.h4
-rw-r--r--drivers/mfd/intel-m10-bmc.c40
-rw-r--r--drivers/mfd/intel_pmt.c112
-rw-r--r--drivers/mfd/intel_quark_i2c_gpio.c118
-rw-r--r--drivers/mfd/lm3533-core.c2
-rw-r--r--drivers/mfd/lpc_sch.c32
-rw-r--r--drivers/mfd/max8997.c4
-rw-r--r--drivers/mfd/mfd-core.c29
-rw-r--r--drivers/mfd/ntxec.c271
-rw-r--r--drivers/mfd/rn5t618.c3
-rw-r--r--drivers/mfd/rohm-bd71828.c486
-rw-r--r--drivers/mfd/rohm-bd718x7.c43
-rw-r--r--drivers/mfd/rohm-bd9576.c189
-rw-r--r--drivers/mfd/sec-core.c14
-rw-r--r--drivers/mfd/stm32-timers.c7
-rw-r--r--drivers/mfd/stmpe.c14
-rw-r--r--drivers/misc/Kconfig19
-rw-r--r--drivers/misc/Makefile3
-rw-r--r--drivers/misc/ad525x_dpot.c3
-rw-r--r--drivers/misc/cxl/context.c2
-rw-r--r--drivers/misc/cxl/fault.c2
-rw-r--r--drivers/misc/dw-xdata-pcie.c420
-rw-r--r--drivers/misc/fastrpc.c5
-rw-r--r--drivers/misc/genwqe/card_ddcb.c10
-rw-r--r--drivers/misc/habanalabs/common/command_buffer.c12
-rw-r--r--drivers/misc/habanalabs/common/command_submission.c368
-rw-r--r--drivers/misc/habanalabs/common/context.c14
-rw-r--r--drivers/misc/habanalabs/common/debugfs.c229
-rw-r--r--drivers/misc/habanalabs/common/device.c259
-rw-r--r--drivers/misc/habanalabs/common/firmware_if.c238
-rw-r--r--drivers/misc/habanalabs/common/habanalabs.h186
-rw-r--r--drivers/misc/habanalabs/common/habanalabs_drv.c28
-rw-r--r--drivers/misc/habanalabs/common/habanalabs_ioctl.c47
-rw-r--r--drivers/misc/habanalabs/common/hw_queue.c10
-rw-r--r--drivers/misc/habanalabs/common/irq.c58
-rw-r--r--drivers/misc/habanalabs/common/memory.c182
-rw-r--r--drivers/misc/habanalabs/common/mmu/mmu.c41
-rw-r--r--drivers/misc/habanalabs/common/pci/pci.c52
-rw-r--r--drivers/misc/habanalabs/common/sysfs.c33
-rw-r--r--drivers/misc/habanalabs/gaudi/gaudi.c357
-rw-r--r--drivers/misc/habanalabs/gaudi/gaudiP.h3
-rw-r--r--drivers/misc/habanalabs/gaudi/gaudi_security.c8
-rw-r--r--drivers/misc/habanalabs/goya/goya.c140
-rw-r--r--drivers/misc/habanalabs/goya/goyaP.h2
-rw-r--r--drivers/misc/habanalabs/include/common/cpucp_if.h99
-rw-r--r--drivers/misc/habanalabs/include/common/hl_boot_if.h219
-rw-r--r--drivers/misc/habanalabs/include/gaudi/gaudi.h2
-rw-r--r--drivers/misc/habanalabs/include/gaudi/gaudi_async_events.h2
-rw-r--r--drivers/misc/habanalabs/include/gaudi/gaudi_async_ids_map_extended.h43
-rw-r--r--drivers/misc/habanalabs/include/gaudi/gaudi_fw_if.h14
-rw-r--r--drivers/misc/habanalabs/include/goya/goya.h2
-rw-r--r--drivers/misc/habanalabs/include/goya/goya_async_events.h1
-rw-r--r--drivers/misc/habanalabs/include/goya/goya_fw_if.h11
-rw-r--r--drivers/misc/ibmvmc.c4
-rw-r--r--drivers/misc/kgdbts.c74
-rw-r--r--drivers/misc/lis3lv02d/lis3lv02d.c27
-rw-r--r--drivers/misc/lkdtm/bugs.c17
-rw-r--r--drivers/misc/lkdtm/core.c1
-rw-r--r--drivers/misc/lkdtm/lkdtm.h1
-rw-r--r--drivers/misc/lkdtm/usercopy.c2
-rw-r--r--drivers/misc/mei/client.c17
-rw-r--r--drivers/misc/mei/hw-me-regs.h1
-rw-r--r--drivers/misc/mei/pci-me.c1
-rw-r--r--drivers/misc/pvpanic/Kconfig27
-rw-r--r--drivers/misc/pvpanic/Makefile8
-rw-r--r--drivers/misc/pvpanic/pvpanic-mmio.c (renamed from drivers/misc/pvpanic.c)90
-rw-r--r--drivers/misc/pvpanic/pvpanic-pci.c125
-rw-r--r--drivers/misc/pvpanic/pvpanic.c113
-rw-r--r--drivers/misc/pvpanic/pvpanic.h21
-rw-r--r--drivers/misc/sgi-xp/xp_main.c4
-rw-r--r--drivers/misc/sgi-xp/xpc_main.c2
-rw-r--r--drivers/misc/sram.c4
-rw-r--r--drivers/misc/ti-st/st_core.c1
-rw-r--r--drivers/misc/uacce/uacce.c41
-rw-r--r--drivers/misc/vmw_balloon.c11
-rw-r--r--drivers/misc/vmw_vmci/vmci_doorbell.c2
-rw-r--r--drivers/misc/vmw_vmci/vmci_guest.c2
-rw-r--r--drivers/misc/vmw_vmci/vmci_host.c2
-rw-r--r--drivers/mmc/core/block.c76
-rw-r--r--drivers/mmc/core/bus.c11
-rw-r--r--drivers/mmc/core/core.c186
-rw-r--r--drivers/mmc/core/core.h17
-rw-r--r--drivers/mmc/core/host.c90
-rw-r--r--drivers/mmc/core/mmc.c44
-rw-r--r--drivers/mmc/core/mmc_ops.c59
-rw-r--r--drivers/mmc/core/mmc_ops.h4
-rw-r--r--drivers/mmc/core/queue.c11
-rw-r--r--drivers/mmc/core/queue.h1
-rw-r--r--drivers/mmc/core/sd.c6
-rw-r--r--drivers/mmc/core/sdio.c28
-rw-r--r--drivers/mmc/host/Kconfig2
-rw-r--r--drivers/mmc/host/Makefile2
-rw-r--r--drivers/mmc/host/cavium.c3
-rw-r--r--drivers/mmc/host/dw_mmc-k3.c2
-rw-r--r--drivers/mmc/host/dw_mmc-rockchip.c2
-rw-r--r--drivers/mmc/host/dw_mmc.c16
-rw-r--r--drivers/mmc/host/meson-gx-mmc.c5
-rw-r--r--drivers/mmc/host/mmc_spi.c8
-rw-r--r--drivers/mmc/host/mmci.c10
-rw-r--r--drivers/mmc/host/moxart-mmc.c10
-rw-r--r--drivers/mmc/host/of_mmc_spi.c18
-rw-r--r--drivers/mmc/host/owl-mmc.c1
-rw-r--r--drivers/mmc/host/renesas_sdhi.h2
-rw-r--r--drivers/mmc/host/renesas_sdhi_core.c38
-rw-r--r--drivers/mmc/host/renesas_sdhi_internal_dmac.c4
-rw-r--r--drivers/mmc/host/renesas_sdhi_sys_dmac.c8
-rw-r--r--drivers/mmc/host/sdhci-acpi.c2
-rw-r--r--drivers/mmc/host/sdhci-brcmstb.c1
-rw-r--r--drivers/mmc/host/sdhci-esdhc-imx.c26
-rw-r--r--drivers/mmc/host/sdhci-esdhc-mcf.c8
-rw-r--r--drivers/mmc/host/sdhci-msm.c27
-rw-r--r--drivers/mmc/host/sdhci-of-aspeed.c2
-rw-r--r--drivers/mmc/host/sdhci-of-dwcmshc.c313
-rw-r--r--drivers/mmc/host/sdhci-of-esdhc.c2
-rw-r--r--drivers/mmc/host/sdhci-pci-core.c31
-rw-r--r--drivers/mmc/host/sdhci-pci-gli.c46
-rw-r--r--drivers/mmc/host/sdhci-pci-o2micro.c8
-rw-r--r--drivers/mmc/host/sdhci-pci.h2
-rw-r--r--drivers/mmc/host/sdhci-s3c.c22
-rw-r--r--drivers/mmc/host/sdhci-st.c23
-rw-r--r--drivers/mmc/host/sdhci-tegra.c66
-rw-r--r--drivers/mmc/host/sdhci.c113
-rw-r--r--drivers/mmc/host/sdhci_am654.c2
-rw-r--r--drivers/mmc/host/tmio_mmc.h3
-rw-r--r--drivers/mmc/host/tmio_mmc_core.c61
-rw-r--r--drivers/mmc/host/uniphier-sd.c5
-rw-r--r--drivers/mmc/host/via-sdmmc.c3
-rw-r--r--drivers/most/Kconfig10
-rw-r--r--drivers/most/Makefile1
-rw-r--r--drivers/most/most_cdev.c6
-rw-r--r--drivers/most/most_snd.c (renamed from drivers/staging/most/sound/sound.c)0
-rw-r--r--drivers/mtd/chips/cfi_cmdset_0001.c8
-rw-r--r--drivers/mtd/chips/cfi_cmdset_0002.c14
-rw-r--r--drivers/mtd/chips/cfi_cmdset_0020.c2
-rw-r--r--drivers/mtd/devices/mtd_dataflash.c2
-rw-r--r--drivers/mtd/ftl.c14
-rw-r--r--drivers/mtd/inftlcore.c13
-rw-r--r--drivers/mtd/maps/physmap-bt1-rom.c2
-rw-r--r--drivers/mtd/maps/physmap-core.c4
-rw-r--r--drivers/mtd/maps/sun_uflash.c1
-rw-r--r--drivers/mtd/mtdblock.c14
-rw-r--r--drivers/mtd/mtdblock_ro.c13
-rw-r--r--drivers/mtd/mtdchar.c40
-rw-r--r--drivers/mtd/mtdcore.c23
-rw-r--r--drivers/mtd/mtdoops.c17
-rw-r--r--drivers/mtd/mtdpart.c2
-rw-r--r--drivers/mtd/mtdswap.c14
-rw-r--r--drivers/mtd/nand/ecc-sw-bch.c10
-rw-r--r--drivers/mtd/nand/ecc-sw-hamming.c10
-rw-r--r--drivers/mtd/nand/onenand/onenand_base.c5
-rw-r--r--drivers/mtd/nand/onenand/onenand_samsung.c1
-rw-r--r--drivers/mtd/nand/raw/atmel/nand-controller.c6
-rw-r--r--drivers/mtd/nand/raw/brcmnand/brcmnand.c69
-rw-r--r--drivers/mtd/nand/raw/fsmc_nand.c3
-rw-r--r--drivers/mtd/nand/raw/gpmi-nand/gpmi-nand.c2
-rw-r--r--drivers/mtd/nand/raw/mtk_nand.c4
-rw-r--r--drivers/mtd/nand/raw/mxc_nand.c2
-rw-r--r--drivers/mtd/nand/raw/nand_base.c145
-rw-r--r--drivers/mtd/nand/raw/omap2.c14
-rw-r--r--drivers/mtd/nand/raw/qcom_nandc.c265
-rw-r--r--drivers/mtd/nand/raw/r852.c5
-rw-r--r--drivers/mtd/nand/raw/rockchip-nand-controller.c2
-rw-r--r--drivers/mtd/nand/raw/stm32_fmc2_nand.c2
-rw-r--r--drivers/mtd/nand/spi/core.c2
-rw-r--r--drivers/mtd/nand/spi/gigadevice.c69
-rw-r--r--drivers/mtd/nftlcore.c13
-rw-r--r--drivers/mtd/parsers/Kconfig24
-rw-r--r--drivers/mtd/parsers/Makefile3
-rw-r--r--drivers/mtd/parsers/ofpart_bcm4908.c64
-rw-r--r--drivers/mtd/parsers/ofpart_bcm4908.h15
-rw-r--r--drivers/mtd/parsers/ofpart_core.c (renamed from drivers/mtd/parsers/ofpart.c)38
-rw-r--r--drivers/mtd/parsers/ofpart_linksys_ns.c50
-rw-r--r--drivers/mtd/parsers/ofpart_linksys_ns.h18
-rw-r--r--drivers/mtd/parsers/qcomsmempart.c9
-rw-r--r--drivers/mtd/rfd_ftl.c13
-rw-r--r--drivers/mtd/spi-nor/Makefile2
-rw-r--r--drivers/mtd/spi-nor/controllers/intel-spi.c1
-rw-r--r--drivers/mtd/spi-nor/controllers/intel-spi.h2
-rw-r--r--drivers/mtd/spi-nor/core.c478
-rw-r--r--drivers/mtd/spi-nor/core.h71
-rw-r--r--drivers/mtd/spi-nor/issi.c3
-rw-r--r--drivers/mtd/spi-nor/macronix.c6
-rw-r--r--drivers/mtd/spi-nor/otp.c377
-rw-r--r--drivers/mtd/spi-nor/sfdp.c72
-rw-r--r--drivers/mtd/spi-nor/sfdp.h3
-rw-r--r--drivers/mtd/spi-nor/spansion.c12
-rw-r--r--drivers/mtd/spi-nor/swp.c427
-rw-r--r--drivers/mtd/spi-nor/winbond.c22
-rw-r--r--drivers/mtd/ubi/build.c1
-rw-r--r--drivers/mtd/ubi/ubi.h2
-rw-r--r--drivers/mux/gpio.c19
-rw-r--r--drivers/net/Kconfig5
-rw-r--r--drivers/net/Makefile3
-rw-r--r--drivers/net/Space.c3
-rw-r--r--drivers/net/arcnet/com20020-pci.c34
-rw-r--r--drivers/net/bareudp.c1
-rw-r--r--drivers/net/bonding/bond_alb.c2
-rw-r--r--drivers/net/bonding/bond_main.c9
-rw-r--r--drivers/net/bonding/bond_options.c9
-rw-r--r--drivers/net/caif/caif_serial.c18
-rw-r--r--drivers/net/can/Kconfig2
-rw-r--r--drivers/net/can/c_can/c_can.c177
-rw-r--r--drivers/net/can/c_can/c_can.h43
-rw-r--r--drivers/net/can/c_can/c_can_pci.c34
-rw-r--r--drivers/net/can/c_can/c_can_platform.c12
-rw-r--r--drivers/net/can/dev/bittiming.c28
-rw-r--r--drivers/net/can/dev/netlink.c28
-rw-r--r--drivers/net/can/dev/skb.c37
-rw-r--r--drivers/net/can/flexcan.c32
-rw-r--r--drivers/net/can/grcan.c2
-rw-r--r--drivers/net/can/kvaser_pciefd.c4
-rw-r--r--drivers/net/can/m_can/m_can.c175
-rw-r--r--drivers/net/can/m_can/m_can.h2
-rw-r--r--drivers/net/can/m_can/tcan4x5x-core.c6
-rw-r--r--drivers/net/can/m_can/tcan4x5x.h1
-rw-r--r--drivers/net/can/peak_canfd/peak_pciefd_main.c1
-rw-r--r--drivers/net/can/rcar/rcar_can.c2
-rw-r--r--drivers/net/can/rcar/rcar_canfd.c2
-rw-r--r--drivers/net/can/sja1000/ems_pci.c1
-rw-r--r--drivers/net/can/sja1000/ems_pcmcia.c1
-rw-r--r--drivers/net/can/sja1000/kvaser_pci.c1
-rw-r--r--drivers/net/can/sja1000/peak_pci.c2
-rw-r--r--drivers/net/can/sja1000/peak_pcmcia.c1
-rw-r--r--drivers/net/can/sja1000/plx_pci.c12
-rw-r--r--drivers/net/can/sja1000/sja1000.c2
-rw-r--r--drivers/net/can/slcan.c1
-rw-r--r--drivers/net/can/spi/hi311x.c2
-rw-r--r--drivers/net/can/spi/mcp251x.c61
-rw-r--r--drivers/net/can/spi/mcp251xfd/Kconfig1
-rw-r--r--drivers/net/can/spi/mcp251xfd/Makefile3
-rw-r--r--drivers/net/can/spi/mcp251xfd/mcp251xfd-core.c120
-rw-r--r--drivers/net/can/spi/mcp251xfd/mcp251xfd-dump.c285
-rw-r--r--drivers/net/can/spi/mcp251xfd/mcp251xfd-dump.h45
-rw-r--r--drivers/net/can/spi/mcp251xfd/mcp251xfd-regmap.c64
-rw-r--r--drivers/net/can/spi/mcp251xfd/mcp251xfd-timestamp.c71
-rw-r--r--drivers/net/can/spi/mcp251xfd/mcp251xfd.h28
-rw-r--r--drivers/net/can/usb/Kconfig11
-rw-r--r--drivers/net/can/usb/Makefile1
-rw-r--r--drivers/net/can/usb/ems_usb.c2
-rw-r--r--drivers/net/can/usb/esd_usb2.c4
-rw-r--r--drivers/net/can/usb/etas_es58x/Makefile3
-rw-r--r--drivers/net/can/usb/etas_es58x/es581_4.c507
-rw-r--r--drivers/net/can/usb/etas_es58x/es581_4.h207
-rw-r--r--drivers/net/can/usb/etas_es58x/es58x_core.c2301
-rw-r--r--drivers/net/can/usb/etas_es58x/es58x_core.h700
-rw-r--r--drivers/net/can/usb/etas_es58x/es58x_fd.c562
-rw-r--r--drivers/net/can/usb/etas_es58x/es58x_fd.h243
-rw-r--r--drivers/net/can/usb/gs_usb.c2
-rw-r--r--drivers/net/can/usb/kvaser_usb/kvaser_usb_core.c6
-rw-r--r--drivers/net/can/usb/mcba_usb.c2
-rw-r--r--drivers/net/can/usb/peak_usb/pcan_usb.c108
-rw-r--r--drivers/net/can/usb/peak_usb/pcan_usb_core.c70
-rw-r--r--drivers/net/can/usb/peak_usb/pcan_usb_core.h9
-rw-r--r--drivers/net/can/usb/peak_usb/pcan_usb_fd.c53
-rw-r--r--drivers/net/can/usb/peak_usb/pcan_usb_pro.c54
-rw-r--r--drivers/net/can/usb/peak_usb/pcan_usb_pro.h82
-rw-r--r--drivers/net/can/usb/ucan.c8
-rw-r--r--drivers/net/can/usb/usb_8dev.c2
-rw-r--r--drivers/net/can/xilinx_can.c10
-rw-r--r--drivers/net/dsa/Kconfig17
-rw-r--r--drivers/net/dsa/b53/Kconfig1
-rw-r--r--drivers/net/dsa/b53/b53_common.c37
-rw-r--r--drivers/net/dsa/b53/b53_mmap.c55
-rw-r--r--drivers/net/dsa/b53/b53_priv.h4
-rw-r--r--drivers/net/dsa/b53/b53_spi.c14
-rw-r--r--drivers/net/dsa/bcm_sf2.c138
-rw-r--r--drivers/net/dsa/bcm_sf2.h2
-rw-r--r--drivers/net/dsa/bcm_sf2_regs.h8
-rw-r--r--drivers/net/dsa/hirschmann/hellcreek.c378
-rw-r--r--drivers/net/dsa/hirschmann/hellcreek.h7
-rw-r--r--drivers/net/dsa/hirschmann/hellcreek_hwtstamp.c28
-rw-r--r--drivers/net/dsa/hirschmann/hellcreek_hwtstamp.h4
-rw-r--r--drivers/net/dsa/lantiq_gswip.c357
-rw-r--r--drivers/net/dsa/microchip/Kconfig10
-rw-r--r--drivers/net/dsa/microchip/Makefile1
-rw-r--r--drivers/net/dsa/microchip/ksz8.h69
-rw-r--r--drivers/net/dsa/microchip/ksz8795.c884
-rw-r--r--drivers/net/dsa/microchip/ksz8795_reg.h125
-rw-r--r--drivers/net/dsa/microchip/ksz8795_spi.c49
-rw-r--r--drivers/net/dsa/microchip/ksz8863_smi.c216
-rw-r--r--drivers/net/dsa/microchip/ksz_common.h5
-rw-r--r--drivers/net/dsa/mt7530.c220
-rw-r--r--drivers/net/dsa/mt7530.h15
-rw-r--r--drivers/net/dsa/mv88e6xxx/chip.c629
-rw-r--r--drivers/net/dsa/mv88e6xxx/chip.h44
-rw-r--r--drivers/net/dsa/mv88e6xxx/devlink.c58
-rw-r--r--drivers/net/dsa/mv88e6xxx/global1.c19
-rw-r--r--drivers/net/dsa/mv88e6xxx/global1.h2
-rw-r--r--drivers/net/dsa/mv88e6xxx/global2.c17
-rw-r--r--drivers/net/dsa/mv88e6xxx/global2.h12
-rw-r--r--drivers/net/dsa/mv88e6xxx/global2_scratch.c6
-rw-r--r--drivers/net/dsa/mv88e6xxx/hwtstamp.c26
-rw-r--r--drivers/net/dsa/mv88e6xxx/hwtstamp.h10
-rw-r--r--drivers/net/dsa/mv88e6xxx/port.c418
-rw-r--r--drivers/net/dsa/mv88e6xxx/port.h50
-rw-r--r--drivers/net/dsa/mv88e6xxx/serdes.c344
-rw-r--r--drivers/net/dsa/mv88e6xxx/serdes.h98
-rw-r--r--drivers/net/dsa/ocelot/felix.c23
-rw-r--r--drivers/net/dsa/ocelot/felix_vsc9959.c12
-rw-r--r--drivers/net/dsa/sja1105/sja1105_flower.c9
-rw-r--r--drivers/net/dsa/sja1105/sja1105_main.c24
-rw-r--r--drivers/net/dsa/sja1105/sja1105_ptp.c16
-rw-r--r--drivers/net/dsa/sja1105/sja1105_ptp.h4
-rw-r--r--drivers/net/dsa/xrs700x/xrs700x.c10
-rw-r--r--drivers/net/ethernet/3com/3c509.c1
-rw-r--r--drivers/net/ethernet/Kconfig5
-rw-r--r--drivers/net/ethernet/Makefile2
-rw-r--r--drivers/net/ethernet/actions/Kconfig26
-rw-r--r--drivers/net/ethernet/actions/Makefile6
-rw-r--r--drivers/net/ethernet/actions/owl-emac.c1625
-rw-r--r--drivers/net/ethernet/actions/owl-emac.h280
-rw-r--r--drivers/net/ethernet/adaptec/starfire.c8
-rw-r--r--drivers/net/ethernet/aeroflex/greth.c6
-rw-r--r--drivers/net/ethernet/allwinner/sun4i-emac.c10
-rw-r--r--drivers/net/ethernet/altera/altera_tse_main.c7
-rw-r--r--drivers/net/ethernet/amazon/ena/ena_com.c4
-rw-r--r--drivers/net/ethernet/amazon/ena/ena_com.h2
-rw-r--r--drivers/net/ethernet/amazon/ena/ena_ethtool.c25
-rw-r--r--drivers/net/ethernet/amazon/ena/ena_netdev.c23
-rw-r--r--drivers/net/ethernet/amd/amd8111e.c362
-rw-r--r--drivers/net/ethernet/amd/atarilance.c8
-rw-r--r--drivers/net/ethernet/amd/hplance.c3
-rw-r--r--drivers/net/ethernet/amd/pcnet32.c15
-rw-r--r--drivers/net/ethernet/amd/xgbe/xgbe.h6
-rw-r--r--drivers/net/ethernet/arc/emac_main.c8
-rw-r--r--drivers/net/ethernet/atheros/Kconfig1
-rw-r--r--drivers/net/ethernet/atheros/ag71xx.c31
-rw-r--r--drivers/net/ethernet/atheros/alx/main.c9
-rw-r--r--drivers/net/ethernet/atheros/atl1c/atl1c.h2
-rw-r--r--drivers/net/ethernet/atheros/atl1c/atl1c_main.c76
-rw-r--r--drivers/net/ethernet/atheros/atlx/atl2.c24
-rw-r--r--drivers/net/ethernet/broadcom/Kconfig2
-rw-r--r--drivers/net/ethernet/broadcom/bcm4908_enet.c147
-rw-r--r--drivers/net/ethernet/broadcom/bcmsysport.c7
-rw-r--r--drivers/net/ethernet/broadcom/bgmac-bcma.c10
-rw-r--r--drivers/net/ethernet/broadcom/bgmac-platform.c11
-rw-r--r--drivers/net/ethernet/broadcom/bnx2.c2
-rw-r--r--drivers/net/ethernet/broadcom/bnx2x/bnx2x_main.c6
-rw-r--r--drivers/net/ethernet/broadcom/bnx2x/bnx2x_sriov.c1
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt.c285
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt.h33
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt_ethtool.c154
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.c74
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt_sriov.h1
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt_ulp.c1
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt_vfr.c122
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt_vfr.h12
-rw-r--r--drivers/net/ethernet/broadcom/bnxt/bnxt_xdp.c20
-rw-r--r--drivers/net/ethernet/broadcom/genet/bcmgenet_wol.c1
-rw-r--r--drivers/net/ethernet/broadcom/tg3.c4
-rw-r--r--drivers/net/ethernet/brocade/bna/bnad.c2
-rw-r--r--drivers/net/ethernet/brocade/bna/bnad_ethtool.c266
-rw-r--r--drivers/net/ethernet/cadence/macb.h14
-rw-r--r--drivers/net/ethernet/cadence/macb_main.c87
-rw-r--r--drivers/net/ethernet/cavium/liquidio/cn23xx_pf_regs.h2
-rw-r--r--drivers/net/ethernet/cavium/liquidio/cn66xx_regs.h2
-rw-r--r--drivers/net/ethernet/cavium/octeon/octeon_mgmt.c8
-rw-r--r--drivers/net/ethernet/cavium/thunder/nicvf_queues.c2
-rw-r--r--drivers/net/ethernet/cavium/thunder/thunder_bgx.c5
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/clip_tbl.c3
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/cudbg_lib.c26
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/cxgb4_cudbg.c3
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/cxgb4_filter.c24
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_matchall.c11
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/cxgb4_tc_u32.c8
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/sge.c16
-rw-r--r--drivers/net/ethernet/chelsio/cxgb4/t4_hw.c5
-rw-r--r--drivers/net/ethernet/chelsio/inline_crypto/ch_ktls/chcr_ktls.c120
-rw-r--r--drivers/net/ethernet/cisco/enic/enic_main.c20
-rw-r--r--drivers/net/ethernet/davicom/dm9000.c40
-rw-r--r--drivers/net/ethernet/dec/tulip/de2104x.c13
-rw-r--r--drivers/net/ethernet/dec/tulip/winbond-840.c13
-rw-r--r--drivers/net/ethernet/dlink/sundance.c17
-rw-r--r--drivers/net/ethernet/ethoc.c6
-rw-r--r--drivers/net/ethernet/ezchip/nps_enet.c7
-rw-r--r--drivers/net/ethernet/faraday/ftgmac100.c1
-rw-r--r--drivers/net/ethernet/faraday/ftmac100.c13
-rw-r--r--drivers/net/ethernet/fealnx.c13
-rw-r--r--drivers/net/ethernet/freescale/Kconfig1
-rw-r--r--drivers/net/ethernet/freescale/Makefile4
-rw-r--r--drivers/net/ethernet/freescale/dpaa/dpaa_eth.c12
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/Kconfig8
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/Makefile2
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.c68
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpaa2-eth.h10
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpaa2-ethtool.c40
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpaa2-switch-ethtool.c (renamed from drivers/staging/fsl-dpaa2/ethsw/ethsw-ethtool.c)2
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpaa2-switch-flower.c492
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.c3394
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpaa2-switch.h246
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpkg.h5
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpmac.h24
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpni.c6
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpni.h162
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dprtc.h3
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpsw-cmd.h (renamed from drivers/staging/fsl-dpaa2/ethsw/dpsw-cmd.h)219
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpsw.c (renamed from drivers/staging/fsl-dpaa2/ethsw/dpsw.c)781
-rw-r--r--drivers/net/ethernet/freescale/dpaa2/dpsw.h755
-rw-r--r--drivers/net/ethernet/freescale/enetc/Kconfig9
-rw-r--r--drivers/net/ethernet/freescale/enetc/Makefile3
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc.c1393
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc.h126
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_cbdr.c82
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_ethtool.c40
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_hw.h36
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_ierb.c155
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_ierb.h20
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_pf.c295
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_qos.c21
-rw-r--r--drivers/net/ethernet/freescale/enetc/enetc_vf.c15
-rw-r--r--drivers/net/ethernet/freescale/fec_main.c17
-rw-r--r--drivers/net/ethernet/freescale/fec_mpc52xx.c7
-rw-r--r--drivers/net/ethernet/freescale/fec_ptp.c7
-rw-r--r--drivers/net/ethernet/freescale/fman/mac.c9
-rw-r--r--drivers/net/ethernet/freescale/fs_enet/fs_enet-main.c5
-rw-r--r--drivers/net/ethernet/freescale/gianfar.c199
-rw-r--r--drivers/net/ethernet/freescale/gianfar.h17
-rw-r--r--drivers/net/ethernet/freescale/ucc_geth.c5
-rw-r--r--drivers/net/ethernet/google/gve/gve_ethtool.c8
-rw-r--r--drivers/net/ethernet/hisilicon/hisi_femac.c7
-rw-r--r--drivers/net/ethernet/hisilicon/hix5hd2_gmac.c7
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hnae.h6
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_ae_adapt.c22
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_gmac.c27
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.c2
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_mac.h4
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_main.c16
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_misc.c2
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_ppe.c41
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_rcb.c95
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_dsaf_xgmac.c26
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_enet.c14
-rw-r--r--drivers/net/ethernet/hisilicon/hns/hns_ethtool.c106
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hclge_mbx.h3
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hnae3.h17
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3_debugfs.c5
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3_enet.c222
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3_enet.h9
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3_ethtool.c21
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.c14
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_cmd.h76
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_debugfs.c25
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.c13
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_err.h3
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c2155
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.h64
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mbx.c40
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.c41
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_mdio.h2
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_tm.c20
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.c1
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_cmd.h7
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.c109
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_main.h6
-rw-r--r--drivers/net/ethernet/hisilicon/hns3/hns3vf/hclgevf_mbx.c6
-rw-r--r--drivers/net/ethernet/hisilicon/hns_mdio.c4
-rw-r--r--drivers/net/ethernet/huawei/hinic/hinic_ethtool.c29
-rw-r--r--drivers/net/ethernet/huawei/hinic/hinic_hw_api_cmd.c8
-rw-r--r--drivers/net/ethernet/huawei/hinic/hinic_hw_if.c2
-rw-r--r--drivers/net/ethernet/huawei/hinic/hinic_hw_mgmt.c6
-rw-r--r--drivers/net/ethernet/huawei/hinic/hinic_hw_qp.c1
-rw-r--r--drivers/net/ethernet/huawei/hinic/hinic_rx.c8
-rw-r--r--drivers/net/ethernet/huawei/hinic/hinic_tx.c1
-rw-r--r--drivers/net/ethernet/ibm/ehea/ehea_main.c1
-rw-r--r--drivers/net/ethernet/ibm/ibmveth.c4
-rw-r--r--drivers/net/ethernet/ibm/ibmvnic.c171
-rw-r--r--drivers/net/ethernet/ibm/ibmvnic.h94
-rw-r--r--drivers/net/ethernet/intel/Kconfig1
-rw-r--r--drivers/net/ethernet/intel/e1000/e1000_hw.c1
-rw-r--r--drivers/net/ethernet/intel/e1000e/82571.c2
-rw-r--r--drivers/net/ethernet/intel/e1000e/hw.h6
-rw-r--r--drivers/net/ethernet/intel/e1000e/ich8lan.c4
-rw-r--r--drivers/net/ethernet/intel/e1000e/netdev.c24
-rw-r--r--drivers/net/ethernet/intel/e1000e/phy.c2
-rw-r--r--drivers/net/ethernet/intel/e1000e/ptp.c2
-rw-r--r--drivers/net/ethernet/intel/fm10k/fm10k_dcbnl.c4
-rw-r--r--drivers/net/ethernet/intel/fm10k/fm10k_debugfs.c2
-rw-r--r--drivers/net/ethernet/intel/fm10k/fm10k_main.c4
-rw-r--r--drivers/net/ethernet/intel/fm10k/fm10k_mbx.c4
-rw-r--r--drivers/net/ethernet/intel/fm10k/fm10k_pf.c2
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e.h4
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_adminq_cmd.h6
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_client.c1
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_common.c10
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_dcb.c4
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_dcb_nl.c2
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_ddp.c2
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_debugfs.c7
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_ethtool.c81
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_lan_hmc.c2
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_main.c124
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_nvm.c2
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_ptp.c2
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_txrx.c49
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_type.h7
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_virtchnl_pf.c117
-rw-r--r--drivers/net/ethernet/intel/i40e/i40e_xsk.c17
-rw-r--r--drivers/net/ethernet/intel/iavf/Makefile3
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf.h22
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_adv_rss.c218
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_adv_rss.h95
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_ethtool.c883
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_fdir.c779
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_fdir.h118
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_main.c65
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_txrx.c17
-rw-r--r--drivers/net/ethernet/intel/iavf/iavf_virtchnl.c360
-rw-r--r--drivers/net/ethernet/intel/ice/Makefile3
-rw-r--r--drivers/net/ethernet/intel/ice/ice.h115
-rw-r--r--drivers/net/ethernet/intel/ice/ice_adminq_cmd.h24
-rw-r--r--drivers/net/ethernet/intel/ice/ice_arfs.c6
-rw-r--r--drivers/net/ethernet/intel/ice/ice_base.c72
-rw-r--r--drivers/net/ethernet/intel/ice/ice_common.c199
-rw-r--r--drivers/net/ethernet/intel/ice/ice_common.h10
-rw-r--r--drivers/net/ethernet/intel/ice/ice_controlq.c8
-rw-r--r--drivers/net/ethernet/intel/ice/ice_controlq.h9
-rw-r--r--drivers/net/ethernet/intel/ice/ice_dcb.c48
-rw-r--r--drivers/net/ethernet/intel/ice/ice_dcb_lib.c2
-rw-r--r--drivers/net/ethernet/intel/ice/ice_dcb_nl.c2
-rw-r--r--drivers/net/ethernet/intel/ice/ice_ethtool.c377
-rw-r--r--drivers/net/ethernet/intel/ice/ice_ethtool_fdir.c6
-rw-r--r--drivers/net/ethernet/intel/ice/ice_fdir.c488
-rw-r--r--drivers/net/ethernet/intel/ice/ice_fdir.h58
-rw-r--r--drivers/net/ethernet/intel/ice/ice_flex_pipe.c571
-rw-r--r--drivers/net/ethernet/intel/ice/ice_flex_pipe.h3
-rw-r--r--drivers/net/ethernet/intel/ice/ice_flex_type.h91
-rw-r--r--drivers/net/ethernet/intel/ice/ice_flow.c835
-rw-r--r--drivers/net/ethernet/intel/ice/ice_flow.h166
-rw-r--r--drivers/net/ethernet/intel/ice/ice_hw_autogen.h19
-rw-r--r--drivers/net/ethernet/intel/ice/ice_lan_tx_rx.h22
-rw-r--r--drivers/net/ethernet/intel/ice/ice_lib.c448
-rw-r--r--drivers/net/ethernet/intel/ice/ice_lib.h7
-rw-r--r--drivers/net/ethernet/intel/ice/ice_main.c837
-rw-r--r--drivers/net/ethernet/intel/ice/ice_nvm.c1
-rw-r--r--drivers/net/ethernet/intel/ice/ice_protocol_type.h10
-rw-r--r--drivers/net/ethernet/intel/ice/ice_sched.c133
-rw-r--r--drivers/net/ethernet/intel/ice/ice_sriov.c400
-rw-r--r--drivers/net/ethernet/intel/ice/ice_sriov.h20
-rw-r--r--drivers/net/ethernet/intel/ice/ice_switch.c17
-rw-r--r--drivers/net/ethernet/intel/ice/ice_txrx.c355
-rw-r--r--drivers/net/ethernet/intel/ice/ice_txrx.h45
-rw-r--r--drivers/net/ethernet/intel/ice/ice_txrx_lib.c1
-rw-r--r--drivers/net/ethernet/intel/ice/ice_type.h118
-rw-r--r--drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.c171
-rw-r--r--drivers/net/ethernet/intel/ice/ice_virtchnl_allowlist.h13
-rw-r--r--drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.c2204
-rw-r--r--drivers/net/ethernet/intel/ice/ice_virtchnl_fdir.h55
-rw-r--r--drivers/net/ethernet/intel/ice/ice_virtchnl_pf.c774
-rw-r--r--drivers/net/ethernet/intel/ice/ice_virtchnl_pf.h21
-rw-r--r--drivers/net/ethernet/intel/ice/ice_xsk.c31
-rw-r--r--drivers/net/ethernet/intel/igb/e1000_defines.h8
-rw-r--r--drivers/net/ethernet/intel/igb/e1000_hw.h6
-rw-r--r--drivers/net/ethernet/intel/igb/e1000_mac.c27
-rw-r--r--drivers/net/ethernet/intel/igb/e1000_mbx.c2
-rw-r--r--drivers/net/ethernet/intel/igb/e1000_phy.c1
-rw-r--r--drivers/net/ethernet/intel/igb/igb.h4
-rw-r--r--drivers/net/ethernet/intel/igb/igb_ethtool.c41
-rw-r--r--drivers/net/ethernet/intel/igb/igb_main.c58
-rw-r--r--drivers/net/ethernet/intel/igb/igb_ptp.c32
-rw-r--r--drivers/net/ethernet/intel/igc/Makefile2
-rw-r--r--drivers/net/ethernet/intel/igc/igc.h31
-rw-r--r--drivers/net/ethernet/intel/igc/igc_defines.h68
-rw-r--r--drivers/net/ethernet/intel/igc/igc_ethtool.c9
-rw-r--r--drivers/net/ethernet/intel/igc/igc_i225.c6
-rw-r--r--drivers/net/ethernet/intel/igc/igc_main.c548
-rw-r--r--drivers/net/ethernet/intel/igc/igc_ptp.c382
-rw-r--r--drivers/net/ethernet/intel/igc/igc_regs.h10
-rw-r--r--drivers/net/ethernet/intel/igc/igc_xdp.c60
-rw-r--r--drivers/net/ethernet/intel/igc/igc_xdp.h13
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_82599.c4
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_common.c16
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_dcb_nl.c2
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_ethtool.c40
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_ipsec.c5
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_lib.c1
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_main.c38
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_phy.c8
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c1
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_type.h5
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_x540.c2
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_x550.c2
-rw-r--r--drivers/net/ethernet/intel/ixgbe/ixgbe_xsk.c11
-rw-r--r--drivers/net/ethernet/intel/ixgbevf/ipsec.c5
-rw-r--r--drivers/net/ethernet/intel/ixgbevf/ixgbevf_main.c1
-rw-r--r--drivers/net/ethernet/intel/ixgbevf/vf.c18
-rw-r--r--drivers/net/ethernet/intel/ixgbevf/vf.h3
-rw-r--r--drivers/net/ethernet/jme.c10
-rw-r--r--drivers/net/ethernet/jme.h2
-rw-r--r--drivers/net/ethernet/korina.c617
-rw-r--r--drivers/net/ethernet/lantiq_xrx200.c11
-rw-r--r--drivers/net/ethernet/marvell/Kconfig4
-rw-r--r--drivers/net/ethernet/marvell/mv643xx_eth.c13
-rw-r--r--drivers/net/ethernet/marvell/mvneta.c31
-rw-r--r--drivers/net/ethernet/marvell/mvpp2/mvpp2_main.c13
-rw-r--r--drivers/net/ethernet/marvell/mvpp2/mvpp2_prs.c107
-rw-r--r--drivers/net/ethernet/marvell/mvpp2/mvpp2_prs.h3
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/cgx.c64
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/mbox.h89
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/npc.h7
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/npc_profile.h2
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu.c6
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu.h18
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu_cgx.c18
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu_cpt.c192
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu_debugfs.c61
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu_nix.c12
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu_npc.c198
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu_npc_fs.c79
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/af/rvu_reg.h21
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/nic/Makefile2
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/nic/otx2_common.h20
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/nic/otx2_flows.c51
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/nic/otx2_pf.c42
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/nic/otx2_reg.h1
-rw-r--r--drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c787
-rw-r--r--drivers/net/ethernet/marvell/prestera/prestera_main.c14
-rw-r--r--drivers/net/ethernet/marvell/prestera/prestera_pci.c1
-rw-r--r--drivers/net/ethernet/marvell/prestera/prestera_switchdev.c2
-rw-r--r--drivers/net/ethernet/marvell/pxa168_eth.c11
-rw-r--r--drivers/net/ethernet/marvell/skge.c9
-rw-r--r--drivers/net/ethernet/marvell/sky2.c19
-rw-r--r--drivers/net/ethernet/mediatek/Kconfig2
-rw-r--r--drivers/net/ethernet/mediatek/Makefile2
-rw-r--r--drivers/net/ethernet/mediatek/mtk_eth_soc.c315
-rw-r--r--drivers/net/ethernet/mediatek/mtk_eth_soc.h73
-rw-r--r--drivers/net/ethernet/mediatek/mtk_ppe.c509
-rw-r--r--drivers/net/ethernet/mediatek/mtk_ppe.h288
-rw-r--r--drivers/net/ethernet/mediatek/mtk_ppe_debugfs.c217
-rw-r--r--drivers/net/ethernet/mediatek/mtk_ppe_offload.c495
-rw-r--r--drivers/net/ethernet/mediatek/mtk_ppe_regs.h144
-rw-r--r--drivers/net/ethernet/mediatek/mtk_star_emac.c5
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/cmd.c69
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_ethtool.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/en_netdev.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/mlx4.h1
-rw-r--r--drivers/net/ethernet/mellanox/mlx4/mlx4_en.h1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/Kconfig12
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/Makefile12
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/cmd.c32
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/dev.c7
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/devlink.c77
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/devlink.h1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c5
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en.h65
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/devlink.c43
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/devlink.h6
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/fs.h60
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.c605
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/fs_tt_redirect.h26
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/health.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/params.c548
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/params.h53
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/port.c23
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/ptp.c399
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/ptp.h32
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/qos.c4
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/rep/tc.c91
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/reporter_rx.c183
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/reporter_tx.c27
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_ct.c93
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.c8
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun.h20
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_encap.c27
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_geneve.c33
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_gre.c1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_mplsoudp.c1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/tc_tun_vxlan.c1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/trap.c203
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/txrx.h6
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c15
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en/xsk/setup.c56
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/en_accel.h7
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/ipsec_fs.c4
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls.h11
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_rx.c167
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_tx.c5
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/ktls_txrx.h20
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls.h3
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.c9
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_rxtx.h14
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_accel/tls_stats.c49
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_arfs.c99
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_common.c27
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_dcbnl.c34
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_ethtool.c455
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_fs.c147
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_main.c1199
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_rep.c314
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_rep.h10
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_rx.c10
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_stats.c302
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_stats.h21
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_tc.c285
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_tc.h1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_tx.c13
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/en_txrx.c5
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/eq.c40
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_lgcy.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/acl/egress_ofld.c4
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/acl/helper.c8
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/acl/helper.h2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_lgcy.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/acl/ingress_ofld.c4
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/devlink_port.c9
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.c10
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/indir_table.h6
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.c510
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/legacy.h22
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/sample.c585
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/sample.h42
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/esw/vporttbl.c140
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/eswitch.c979
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/eswitch.h277
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c791
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads_termtbl.c14
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/fpga/conn.c1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/fpga/ipsec.c9
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/fs_core.c102
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/fs_core.h7
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/fs_counters.c16
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c13
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/health.c6
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib.c44
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/ipoib/ipoib_vlan.c1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lag.c4
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lag_mp.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/clock.c33
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/clock.h11
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/crypto.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/eq.h15
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/fs_chains.c52
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/fs_chains.h6
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/gid.c19
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/lib/mlx5.h9
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/main.c149
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h20
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/pagealloc.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/pci_irq.c74
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/port.c110
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/rdma.c1
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/rl.c139
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.c16
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/dev/dev.h2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/dev/driver.c14
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c43
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c279
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/mlx5_ifc_vhca_event.h2
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/priv.h9
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/vhca_event.c23
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sf/vhca_event.h7
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/sriov.c48
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_action.c242
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_cmd.c70
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_matcher.c256
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_rule.c11
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_send.c66
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.c145
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste.h31
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v0.c368
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_ste_v1.c293
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_table.c4
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/dr_types.h205
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/steering/mlx5_ifc_dr.h16
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/vport.c14
-rw-r--r--drivers/net/ethernet/mellanox/mlx5/core/wq.c5
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/core.c6
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/core.h21
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.c131
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/core_acl_flex_actions.h11
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/core_thermal.c29
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/pci.c55
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/pci_hw.h71
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/reg.h131
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum.c215
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum.h91
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_acl.c25
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_acl_flex_actions.c83
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_dpipe.c21
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_ethtool.c153
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_flow.c2
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_flower.c23
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_ipip.c17
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_ipip.h3
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_matchall.c245
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_mr.c30
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.c7
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_nve.h1
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_nve_vxlan.c15
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_qdisc.c453
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_router.c689
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_router.h12
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_span.c21
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_span.h16
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_switchdev.c79
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/spectrum_trap.c213
-rw-r--r--drivers/net/ethernet/mellanox/mlxsw/switchx2.c3
-rw-r--r--drivers/net/ethernet/micrel/ks8851_common.c7
-rw-r--r--drivers/net/ethernet/microchip/encx24j600.c15
-rw-r--r--drivers/net/ethernet/microchip/lan743x_ethtool.c4
-rw-r--r--drivers/net/ethernet/microchip/lan743x_main.c15
-rw-r--r--drivers/net/ethernet/microsoft/Kconfig29
-rw-r--r--drivers/net/ethernet/microsoft/Makefile5
-rw-r--r--drivers/net/ethernet/microsoft/mana/Makefile6
-rw-r--r--drivers/net/ethernet/microsoft/mana/gdma.h673
-rw-r--r--drivers/net/ethernet/microsoft/mana/gdma_main.c1415
-rw-r--r--drivers/net/ethernet/microsoft/mana/hw_channel.c843
-rw-r--r--drivers/net/ethernet/microsoft/mana/hw_channel.h190
-rw-r--r--drivers/net/ethernet/microsoft/mana/mana.h533
-rw-r--r--drivers/net/ethernet/microsoft/mana/mana_en.c1895
-rw-r--r--drivers/net/ethernet/microsoft/mana/mana_ethtool.c250
-rw-r--r--drivers/net/ethernet/microsoft/mana/shm_channel.c291
-rw-r--r--drivers/net/ethernet/microsoft/mana/shm_channel.h21
-rw-r--r--drivers/net/ethernet/moxa/moxart_ether.c1
-rw-r--r--drivers/net/ethernet/mscc/Kconfig4
-rw-r--r--drivers/net/ethernet/mscc/ocelot.c188
-rw-r--r--drivers/net/ethernet/mscc/ocelot_flower.c8
-rw-r--r--drivers/net/ethernet/mscc/ocelot_mrp.c225
-rw-r--r--drivers/net/ethernet/mscc/ocelot_net.c234
-rw-r--r--drivers/net/ethernet/mscc/ocelot_ptp.c2
-rw-r--r--drivers/net/ethernet/mscc/ocelot_vcap.c1
-rw-r--r--drivers/net/ethernet/myricom/myri10ge/myri10ge.c2
-rw-r--r--drivers/net/ethernet/neterion/s2io.c2
-rw-r--r--drivers/net/ethernet/neterion/vxge/vxge-main.h14
-rw-r--r--drivers/net/ethernet/netronome/nfp/abm/main.c4
-rw-r--r--drivers/net/ethernet/netronome/nfp/bpf/cmsg.c1
-rw-r--r--drivers/net/ethernet/netronome/nfp/flower/main.h12
-rw-r--r--drivers/net/ethernet/netronome/nfp/flower/metadata.c40
-rw-r--r--drivers/net/ethernet/netronome/nfp/flower/offload.c66
-rw-r--r--drivers/net/ethernet/netronome/nfp/flower/qos_conf.c156
-rw-r--r--drivers/net/ethernet/netronome/nfp/flower/tunnel_conf.c15
-rw-r--r--drivers/net/ethernet/netronome/nfp/nfp_app.h1
-rw-r--r--drivers/net/ethernet/netronome/nfp/nfp_devlink.c1
-rw-r--r--drivers/net/ethernet/netronome/nfp/nfp_net_ethtool.c79
-rw-r--r--drivers/net/ethernet/netronome/nfp/nfp_port.h2
-rw-r--r--drivers/net/ethernet/nxp/lpc_eth.c13
-rw-r--r--drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe.h9
-rw-r--r--drivers/net/ethernet/oki-semi/pch_gbe/pch_gbe_main.c1
-rw-r--r--drivers/net/ethernet/pensando/ionic/Makefile1
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic.h6
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_bus_pci.c4
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_dev.c107
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_dev.h33
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_ethtool.c109
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_if.h242
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_lif.c536
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_lif.h104
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_main.c43
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_phc.c615
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_rx_filter.c21
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_rx_filter.h1
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_stats.c392
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_txrx.c882
-rw-r--r--drivers/net/ethernet/pensando/ionic/ionic_txrx.h3
-rw-r--r--drivers/net/ethernet/qlogic/qed/qed_l2.c1
-rw-r--r--drivers/net/ethernet/qlogic/qed/qed_sriov.c1
-rw-r--r--drivers/net/ethernet/qlogic/qede/qede_fp.c26
-rw-r--r--drivers/net/ethernet/qlogic/qede/qede_main.c1
-rw-r--r--drivers/net/ethernet/qlogic/qlcnic/qlcnic_minidump.c3
-rw-r--r--drivers/net/ethernet/qualcomm/emac/emac-mac.c4
-rw-r--r--drivers/net/ethernet/qualcomm/qca_spi.c10
-rw-r--r--drivers/net/ethernet/qualcomm/qca_uart.c9
-rw-r--r--drivers/net/ethernet/qualcomm/rmnet/rmnet_handlers.c10
-rw-r--r--drivers/net/ethernet/qualcomm/rmnet/rmnet_map.h12
-rw-r--r--drivers/net/ethernet/qualcomm/rmnet/rmnet_map_command.c11
-rw-r--r--drivers/net/ethernet/qualcomm/rmnet/rmnet_map_data.c56
-rw-r--r--drivers/net/ethernet/realtek/r8169_main.c89
-rw-r--r--drivers/net/ethernet/renesas/ravb.h1
-rw-r--r--drivers/net/ethernet/renesas/ravb_main.c62
-rw-r--r--drivers/net/ethernet/renesas/sh_eth.c16
-rw-r--r--drivers/net/ethernet/renesas/sh_eth.h114
-rw-r--r--drivers/net/ethernet/rocker/rocker_main.c4
-rw-r--r--drivers/net/ethernet/samsung/sxgbe/sxgbe_platform.c13
-rw-r--r--drivers/net/ethernet/sfc/ef10.c20
-rw-r--r--drivers/net/ethernet/sfc/efx.c2
-rw-r--r--drivers/net/ethernet/sfc/efx_channels.c2
-rw-r--r--drivers/net/ethernet/sfc/enum.h1
-rw-r--r--drivers/net/ethernet/sfc/ethtool.c10
-rw-r--r--drivers/net/ethernet/sfc/falcon/efx.c2
-rw-r--r--drivers/net/ethernet/sfc/falcon/net_driver.h2
-rw-r--r--drivers/net/ethernet/sfc/farch.c16
-rw-r--r--drivers/net/ethernet/sfc/net_driver.h3
-rw-r--r--drivers/net/ethernet/sfc/rx.c11
-rw-r--r--drivers/net/ethernet/sfc/tx.c15
-rw-r--r--drivers/net/ethernet/smsc/smc91x.c2
-rw-r--r--drivers/net/ethernet/smsc/smsc911x.c2
-rw-r--r--drivers/net/ethernet/socionext/netsec.c25
-rw-r--r--drivers/net/ethernet/socionext/sni_ave.c10
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/Kconfig4
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/Makefile1
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/common.h37
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-anarion.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-dwc-qos-eth.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-generic.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-imx.c62
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-intel-plat.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-intel.c459
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-intel.h1
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-ipq806x.c4
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-lpc18xx.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-mediatek.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-meson.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-oxnas.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-qcom-ethqos.c3
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-rk.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-socfpga.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-sti.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-stm32.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-sun8i.c28
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-sunxi.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac-visconti.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac1000_dma.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac4.h15
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac4_core.c9
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac4_descs.c59
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.c48
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac4_dma.h25
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac4_lib.c34
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac5.c136
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac5.h44
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac_dma.h22
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwmac_lib.c8
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwxgmac2.h6
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwxgmac2_descs.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/dwxgmac2_dma.c15
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/enh_desc.c9
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/hwif.h30
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/norm_desc.c9
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac.h92
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_ethtool.c124
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_hwtstamp.c50
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_main.c2868
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_mdio.c111
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_pci.c2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_platform.c44
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_platform.h2
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.c75
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_ptp.h24
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_tc.c124
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_xdp.c135
-rw-r--r--drivers/net/ethernet/stmicro/stmmac/stmmac_xdp.h15
-rw-r--r--drivers/net/ethernet/sun/cassini.c1
-rw-r--r--drivers/net/ethernet/sun/niu.c2
-rw-r--r--drivers/net/ethernet/sun/sungem.c4
-rw-r--r--drivers/net/ethernet/tehuti/tehuti.c1
-rw-r--r--drivers/net/ethernet/ti/am65-cpsw-nuss.c19
-rw-r--r--drivers/net/ethernet/ti/am65-cpsw-switchdev.c4
-rw-r--r--drivers/net/ethernet/ti/cpsw.c21
-rw-r--r--drivers/net/ethernet/ti/cpsw_new.c21
-rw-r--r--drivers/net/ethernet/ti/cpsw_priv.c11
-rw-r--r--drivers/net/ethernet/ti/cpsw_switchdev.c4
-rw-r--r--drivers/net/ethernet/ti/davinci_emac.c12
-rw-r--r--drivers/net/ethernet/ti/netcp_core.c7
-rw-r--r--drivers/net/ethernet/toshiba/spider_net.c42
-rw-r--r--drivers/net/ethernet/toshiba/tc35815.c3
-rw-r--r--drivers/net/ethernet/via/via-velocity.c2
-rw-r--r--drivers/net/ethernet/wiznet/w5100-spi.c8
-rw-r--r--drivers/net/ethernet/wiznet/w5100.c2
-rw-r--r--drivers/net/ethernet/xilinx/Kconfig3
-rw-r--r--drivers/net/ethernet/xilinx/ll_temac_main.c8
-rw-r--r--drivers/net/ethernet/xilinx/xilinx_axienet.h20
-rw-r--r--drivers/net/ethernet/xilinx/xilinx_axienet_main.c97
-rw-r--r--drivers/net/ethernet/xilinx/xilinx_axienet_mdio.c4
-rw-r--r--drivers/net/ethernet/xilinx/xilinx_emaclite.c8
-rw-r--r--drivers/net/ethernet/xircom/xirc2ps_cs.c2
-rw-r--r--drivers/net/ethernet/xscale/Kconfig1
-rw-r--r--drivers/net/ethernet/xscale/ixp4xx_eth.c215
-rw-r--r--drivers/net/fddi/Kconfig16
-rw-r--r--drivers/net/fddi/defxx.c96
-rw-r--r--drivers/net/fddi/defxx.h5
-rw-r--r--drivers/net/fddi/defza.c2
-rw-r--r--drivers/net/fddi/skfp/h/smc.h2
-rw-r--r--drivers/net/fddi/skfp/h/smt.h12
-rw-r--r--drivers/net/fddi/skfp/smt.c4
-rw-r--r--drivers/net/geneve.c31
-rw-r--r--drivers/net/hamradio/6pack.c6
-rw-r--r--drivers/net/hamradio/mkiss.c1
-rw-r--r--drivers/net/hamradio/scc.c1
-rw-r--r--drivers/net/hyperv/hyperv_net.h8
-rw-r--r--drivers/net/hyperv/netvsc.c55
-rw-r--r--drivers/net/hyperv/netvsc_drv.c78
-rw-r--r--drivers/net/hyperv/rndis_filter.c2
-rw-r--r--drivers/net/ieee802154/atusb.c1
-rw-r--r--drivers/net/ieee802154/mac802154_hwsim.c2
-rw-r--r--drivers/net/ipa/Kconfig5
-rw-r--r--drivers/net/ipa/Makefile6
-rw-r--r--drivers/net/ipa/gsi.c110
-rw-r--r--drivers/net/ipa/gsi.h4
-rw-r--r--drivers/net/ipa/gsi_private.h4
-rw-r--r--drivers/net/ipa/gsi_reg.h85
-rw-r--r--drivers/net/ipa/gsi_trans.c13
-rw-r--r--drivers/net/ipa/gsi_trans.h5
-rw-r--r--drivers/net/ipa/ipa.h7
-rw-r--r--drivers/net/ipa/ipa_cmd.c78
-rw-r--r--drivers/net/ipa/ipa_cmd.h19
-rw-r--r--drivers/net/ipa/ipa_data-v3.5.1.c (renamed from drivers/net/ipa/ipa_data-sdm845.c)229
-rw-r--r--drivers/net/ipa/ipa_data-v4.11.c382
-rw-r--r--drivers/net/ipa/ipa_data-v4.2.c (renamed from drivers/net/ipa/ipa_data-sc7180.c)158
-rw-r--r--drivers/net/ipa/ipa_data-v4.5.c437
-rw-r--r--drivers/net/ipa/ipa_data-v4.9.c430
-rw-r--r--drivers/net/ipa/ipa_data.h131
-rw-r--r--drivers/net/ipa/ipa_endpoint.c82
-rw-r--r--drivers/net/ipa/ipa_endpoint.h32
-rw-r--r--drivers/net/ipa/ipa_interrupt.c54
-rw-r--r--drivers/net/ipa/ipa_interrupt.h1
-rw-r--r--drivers/net/ipa/ipa_main.c330
-rw-r--r--drivers/net/ipa/ipa_mem.c15
-rw-r--r--drivers/net/ipa/ipa_mem.h21
-rw-r--r--drivers/net/ipa/ipa_modem.c34
-rw-r--r--drivers/net/ipa/ipa_qmi.c16
-rw-r--r--drivers/net/ipa/ipa_qmi.h14
-rw-r--r--drivers/net/ipa/ipa_qmi_msg.c78
-rw-r--r--drivers/net/ipa/ipa_qmi_msg.h6
-rw-r--r--drivers/net/ipa/ipa_reg.h495
-rw-r--r--drivers/net/ipa/ipa_resource.c176
-rw-r--r--drivers/net/ipa/ipa_resource.h23
-rw-r--r--drivers/net/ipa/ipa_smp2p.h2
-rw-r--r--drivers/net/ipa/ipa_table.c117
-rw-r--r--drivers/net/ipa/ipa_table.h27
-rw-r--r--drivers/net/ipa/ipa_uc.c5
-rw-r--r--drivers/net/ipa/ipa_version.h29
-rw-r--r--drivers/net/macvlan.c64
-rw-r--r--drivers/net/mdio.c2
-rw-r--r--drivers/net/mdio/Kconfig11
-rw-r--r--drivers/net/mdio/Makefile1
-rw-r--r--drivers/net/mdio/mdio-bcm-unimac.c16
-rw-r--r--drivers/net/mdio/mdio-bitbang.c12
-rw-r--r--drivers/net/mdio/mdio-cavium.c2
-rw-r--r--drivers/net/mdio/mdio-gpio.c18
-rw-r--r--drivers/net/mdio/mdio-ipq4019.c4
-rw-r--r--drivers/net/mdio/mdio-ipq8064.c4
-rw-r--r--drivers/net/mdio/mdio-mscc-miim.c8
-rw-r--r--drivers/net/mdio/mdio-mux-bcm-iproc.c14
-rw-r--r--drivers/net/mdio/mdio-mux-bcm6368.c184
-rw-r--r--drivers/net/mdio/mdio-mux-gpio.c8
-rw-r--r--drivers/net/mdio/mdio-mux-mmioreg.c6
-rw-r--r--drivers/net/mdio/mdio-mux-multiplexer.c2
-rw-r--r--drivers/net/mdio/mdio-mux.c6
-rw-r--r--drivers/net/mdio/mdio-octeon.c8
-rw-r--r--drivers/net/mdio/mdio-thunder.c10
-rw-r--r--drivers/net/mdio/mdio-xgene.c6
-rw-r--r--drivers/net/mdio/of_mdio.c10
-rw-r--r--drivers/net/mhi/mhi.h1
-rw-r--r--drivers/net/mhi/net.c7
-rw-r--r--drivers/net/mhi/proto_mbim.c62
-rw-r--r--drivers/net/netdevsim/Makefile4
-rw-r--r--drivers/net/netdevsim/dev.c17
-rw-r--r--drivers/net/netdevsim/ethtool.c36
-rw-r--r--drivers/net/netdevsim/fib.c147
-rw-r--r--drivers/net/netdevsim/health.c11
-rw-r--r--drivers/net/netdevsim/netdev.c1
-rw-r--r--drivers/net/netdevsim/netdevsim.h18
-rw-r--r--drivers/net/netdevsim/psample.c265
-rw-r--r--drivers/net/pcs/pcs-xpcs.c257
-rw-r--r--drivers/net/phy/Kconfig12
-rw-r--r--drivers/net/phy/Makefile2
-rw-r--r--drivers/net/phy/at803x.c100
-rw-r--r--drivers/net/phy/bcm-phy-lib.c13
-rw-r--r--drivers/net/phy/broadcom.c85
-rw-r--r--drivers/net/phy/dp83822.c9
-rw-r--r--drivers/net/phy/dp83tc811.c11
-rw-r--r--drivers/net/phy/intel-xway.c21
-rw-r--r--drivers/net/phy/marvell-88x2222.c621
-rw-r--r--drivers/net/phy/marvell.c647
-rw-r--r--drivers/net/phy/marvell10g.c386
-rw-r--r--drivers/net/phy/mdio-boardinfo.c2
-rw-r--r--drivers/net/phy/mdio_bus.c2
-rw-r--r--drivers/net/phy/mscc/mscc_main.c217
-rw-r--r--drivers/net/phy/nxp-c45-tja11xx.c621
-rw-r--r--drivers/net/phy/phy-c45.c51
-rw-r--r--drivers/net/phy/phy.c9
-rw-r--r--drivers/net/phy/phy_device.c58
-rw-r--r--drivers/net/phy/phylink.c7
-rw-r--r--drivers/net/phy/sfp-bus.c20
-rw-r--r--drivers/net/phy/sfp.c25
-rw-r--r--drivers/net/phy/sfp.h3
-rw-r--r--drivers/net/phy/smsc.c7
-rw-r--r--drivers/net/plip/plip.c2
-rw-r--r--drivers/net/ppp/ppp_async.c1
-rw-r--r--drivers/net/ppp/ppp_deflate.c1
-rw-r--r--drivers/net/ppp/ppp_generic.c22
-rw-r--r--drivers/net/ppp/ppp_synctty.c1
-rw-r--r--drivers/net/ppp/pppoe.c27
-rw-r--r--drivers/net/slip/slip.c1
-rw-r--r--drivers/net/thunderbolt.c56
-rw-r--r--drivers/net/tun.c64
-rw-r--r--drivers/net/usb/asix_devices.c12
-rw-r--r--drivers/net/usb/ax88179_178a.c6
-rw-r--r--drivers/net/usb/cdc-phonet.c2
-rw-r--r--drivers/net/usb/cdc_ether.c27
-rw-r--r--drivers/net/usb/cdc_ncm.c66
-rw-r--r--drivers/net/usb/dm9601.c4
-rw-r--r--drivers/net/usb/hso.c35
-rw-r--r--drivers/net/usb/lan78xx.c1
-rw-r--r--drivers/net/usb/mcs7830.c4
-rw-r--r--drivers/net/usb/qmi_wwan.c14
-rw-r--r--drivers/net/usb/r8152.c3017
-rw-r--r--drivers/net/usb/sierra_net.c4
-rw-r--r--drivers/net/usb/smsc75xx.c4
-rw-r--r--drivers/net/usb/sr9700.c4
-rw-r--r--drivers/net/usb/sr9800.c4
-rw-r--r--drivers/net/usb/usbnet.c40
-rw-r--r--drivers/net/veth.c202
-rw-r--r--drivers/net/virtio_net.c208
-rw-r--r--drivers/net/vmxnet3/vmxnet3_ethtool.c53
-rw-r--r--drivers/net/vrf.c10
-rw-r--r--drivers/net/vxlan.c20
-rw-r--r--drivers/net/wan/farsync.c3
-rw-r--r--drivers/net/wan/fsl_ucc_hdlc.c8
-rw-r--r--drivers/net/wan/hdlc_x25.c72
-rw-r--r--drivers/net/wan/lapbether.c86
-rw-r--r--drivers/net/wan/z85230.h39
-rw-r--r--drivers/net/wireless/admtek/adm8211.c1
-rw-r--r--drivers/net/wireless/ath/ath10k/htc.c2
-rw-r--r--drivers/net/wireless/ath/ath10k/mac.c2
-rw-r--r--drivers/net/wireless/ath/ath10k/snoc.c29
-rw-r--r--drivers/net/wireless/ath/ath10k/wmi-tlv.c3
-rw-r--r--drivers/net/wireless/ath/ath11k/ahb.c2
-rw-r--r--drivers/net/wireless/ath/ath11k/ce.c58
-rw-r--r--drivers/net/wireless/ath/ath11k/ce.h1
-rw-r--r--drivers/net/wireless/ath/ath11k/core.c45
-rw-r--r--drivers/net/wireless/ath/ath11k/core.h6
-rw-r--r--drivers/net/wireless/ath/ath11k/debugfs_htt_stats.c2
-rw-r--r--drivers/net/wireless/ath/ath11k/dp_rx.c476
-rw-r--r--drivers/net/wireless/ath/ath11k/dp_tx.c6
-rw-r--r--drivers/net/wireless/ath/ath11k/hal.c96
-rw-r--r--drivers/net/wireless/ath/ath11k/hal.h33
-rw-r--r--drivers/net/wireless/ath/ath11k/hal_desc.h13
-rw-r--r--drivers/net/wireless/ath/ath11k/hal_tx.c3
-rw-r--r--drivers/net/wireless/ath/ath11k/hal_tx.h1
-rw-r--r--drivers/net/wireless/ath/ath11k/hif.h10
-rw-r--r--drivers/net/wireless/ath/ath11k/hw.c796
-rw-r--r--drivers/net/wireless/ath/ath11k/hw.h53
-rw-r--r--drivers/net/wireless/ath/ath11k/mac.c107
-rw-r--r--drivers/net/wireless/ath/ath11k/mac.h2
-rw-r--r--drivers/net/wireless/ath/ath11k/mhi.c125
-rw-r--r--drivers/net/wireless/ath/ath11k/pci.c194
-rw-r--r--drivers/net/wireless/ath/ath11k/pci.h21
-rw-r--r--drivers/net/wireless/ath/ath11k/qmi.c122
-rw-r--r--drivers/net/wireless/ath/ath11k/qmi.h9
-rw-r--r--drivers/net/wireless/ath/ath11k/rx_desc.h212
-rw-r--r--drivers/net/wireless/ath/ath11k/wmi.c64
-rw-r--r--drivers/net/wireless/ath/ath5k/base.c1
-rw-r--r--drivers/net/wireless/ath/ath9k/ath9k.h3
-rw-r--r--drivers/net/wireless/ath/ath9k/beacon.c2
-rw-r--r--drivers/net/wireless/ath/ath9k/htc_drv_init.c2
-rw-r--r--drivers/net/wireless/ath/ath9k/hw.c3
-rw-r--r--drivers/net/wireless/ath/ath9k/init.c6
-rw-r--r--drivers/net/wireless/ath/ath9k/xmit.c6
-rw-r--r--drivers/net/wireless/ath/carl9170/carl9170.h7
-rw-r--r--drivers/net/wireless/ath/carl9170/tx.c2
-rw-r--r--drivers/net/wireless/ath/wil6210/wmi.c2
-rw-r--r--drivers/net/wireless/atmel/atmel.c1
-rw-r--r--drivers/net/wireless/atmel/atmel_cs.c1
-rw-r--r--drivers/net/wireless/atmel/atmel_pci.c1
-rw-r--r--drivers/net/wireless/broadcom/b43/main.c2
-rw-r--r--drivers/net/wireless/broadcom/b43legacy/main.c2
-rw-r--r--drivers/net/wireless/broadcom/brcm80211/brcmfmac/core.c2
-rw-r--r--drivers/net/wireless/broadcom/brcm80211/brcmfmac/debug.h1
-rw-r--r--drivers/net/wireless/broadcom/brcm80211/brcmfmac/p2p.c2
-rw-r--r--drivers/net/wireless/broadcom/brcm80211/brcmfmac/usb.h2
-rw-r--r--drivers/net/wireless/broadcom/brcm80211/brcmsmac/mac80211_if.c1
-rw-r--r--drivers/net/wireless/broadcom/brcm80211/brcmutil/utils.c1
-rw-r--r--drivers/net/wireless/cisco/airo.c118
-rw-r--r--drivers/net/wireless/cisco/airo_cs.c1
-rw-r--r--drivers/net/wireless/intel/ipw2x00/libipw_wx.c6
-rw-r--r--drivers/net/wireless/intel/iwlegacy/3945-mac.c2
-rw-r--r--drivers/net/wireless/intel/iwlegacy/common.c2
-rw-r--r--drivers/net/wireless/intel/iwlegacy/common.h2
-rw-r--r--drivers/net/wireless/intel/iwlwifi/cfg/22000.c72
-rw-r--r--drivers/net/wireless/intel/iwlwifi/cfg/9000.c2
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/acpi.c78
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/acpi.h13
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/api/location.h173
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/api/nvm-reg.h20
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/api/power.h22
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/api/rx.h30
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/api/scan.h8
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/dbg.c4
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/file.h3
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/img.h5
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/init.c59
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/notif-wait.c10
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/pnvm.c4
-rw-r--r--drivers/net/wireless/intel/iwlwifi/fw/runtime.h1
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-config.h13
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-csr.h3
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-dbg-tlv.c5
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-drv.c4
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-nvm-parse.c16
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-op-mode.h6
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-trans.c91
-rw-r--r--drivers/net/wireless/intel/iwlwifi/iwl-trans.h6
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/constants.h11
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/d3.c2
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/debugfs.c17
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/ftm-initiator.c85
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/ftm-responder.c27
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/fw.c233
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/mac80211.c59
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/mvm.h12
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/rfi.c6
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/rs-fw.c20
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/rs.c6
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/rxmq.c75
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/scan.c128
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/sta.c10
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/sta.h38
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/time-event.c27
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/tt.c4
-rw-r--r--drivers/net/wireless/intel/iwlwifi/mvm/utils.c18
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/ctxt-info-gen3.c31
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/ctxt-info.c3
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/drv.c110
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/internal.h5
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/rx.c71
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/trans-gen2.c43
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/trans.c29
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/tx-gen2.c7
-rw-r--r--drivers/net/wireless/intel/iwlwifi/pcie/tx.c87
-rw-r--r--drivers/net/wireless/intel/iwlwifi/queue/tx.c41
-rw-r--r--drivers/net/wireless/intel/iwlwifi/queue/tx.h3
-rw-r--r--drivers/net/wireless/intersil/hostap/hostap_cs.c1
-rw-r--r--drivers/net/wireless/intersil/hostap/hostap_pci.c1
-rw-r--r--drivers/net/wireless/intersil/hostap/hostap_plx.c1
-rw-r--r--drivers/net/wireless/intersil/hostap/hostap_proc.c1
-rw-r--r--drivers/net/wireless/intersil/orinoco/orinoco_nortel.c8
-rw-r--r--drivers/net/wireless/intersil/orinoco/orinoco_pci.c8
-rw-r--r--drivers/net/wireless/intersil/orinoco/orinoco_plx.c8
-rw-r--r--drivers/net/wireless/intersil/orinoco/orinoco_tmd.c8
-rw-r--r--drivers/net/wireless/mac80211_hwsim.c24
-rw-r--r--drivers/net/wireless/marvell/libertas/decl.h1
-rw-r--r--drivers/net/wireless/marvell/libertas/mesh.h12
-rw-r--r--drivers/net/wireless/marvell/libertas_tf/libertas_tf.h1
-rw-r--r--drivers/net/wireless/marvell/mwifiex/cfg80211.c11
-rw-r--r--drivers/net/wireless/marvell/mwifiex/scan.c3
-rw-r--r--drivers/net/wireless/marvell/mwl8k.c1
-rw-r--r--drivers/net/wireless/mediatek/mt76/agg-rx.c19
-rw-r--r--drivers/net/wireless/mediatek/mt76/debugfs.c28
-rw-r--r--drivers/net/wireless/mediatek/mt76/dma.c91
-rw-r--r--drivers/net/wireless/mediatek/mt76/dma.h1
-rw-r--r--drivers/net/wireless/mediatek/mt76/eeprom.c240
-rw-r--r--drivers/net/wireless/mediatek/mt76/mac80211.c77
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76.h99
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7603/dma.c4
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7603/init.c3
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7603/mac.c35
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7603/mcu.c5
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7603/mt7603.h2
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7603/pci.c2
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/debugfs.c61
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/dma.c71
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/eeprom.c1
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/init.c42
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/mac.c424
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/mac.h8
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/main.c209
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/mcu.c299
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/mcu.h34
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/mmio.c27
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/mt7615.h31
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/pci.c6
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/pci_init.c17
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/pci_mac.c183
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/regs.h20
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/sdio_txrx.c11
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/soc.c4
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7615/usb_sdio.c7
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76_connac.h59
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76_connac_mac.c23
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.c272
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76_connac_mcu.h81
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76x0/pci.c6
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76x0/usb.c3
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76x02_mac.c4
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76x02_mcu.c5
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c6
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76x02_util.c4
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt76x2/pci.c6
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/Makefile2
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/debugfs.c116
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/dma.c112
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/eeprom.c184
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/eeprom.h51
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/init.c220
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/mac.c268
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/mac.h15
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/main.c132
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/mcu.c437
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/mcu.h16
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/mmio.c152
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/mt7915.h106
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/pci.c37
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/regs.h18
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7915/testmode.c26
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/Makefile4
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/debugfs.c168
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/dma.c242
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/init.c48
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/mac.c525
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/mac.h10
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/main.c236
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/mcu.c214
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/mcu.h60
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/mt7921.h83
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/mt7921_trace.h51
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/pci.c54
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/regs.h58
-rw-r--r--drivers/net/wireless/mediatek/mt76/mt7921/trace.c12
-rw-r--r--drivers/net/wireless/mediatek/mt76/sdio.c3
-rw-r--r--drivers/net/wireless/mediatek/mt76/testmode.c159
-rw-r--r--drivers/net/wireless/mediatek/mt76/testmode.h2
-rw-r--r--drivers/net/wireless/mediatek/mt76/tx.c94
-rw-r--r--drivers/net/wireless/mediatek/mt7601u/eeprom.c2
-rw-r--r--drivers/net/wireless/mediatek/mt7601u/init.c1
-rw-r--r--drivers/net/wireless/microchip/wilc1000/Kconfig1
-rw-r--r--drivers/net/wireless/microchip/wilc1000/netdev.c39
-rw-r--r--drivers/net/wireless/microchip/wilc1000/sdio.c2
-rw-r--r--drivers/net/wireless/microchip/wilc1000/spi.c298
-rw-r--r--drivers/net/wireless/microchip/wilc1000/wlan.c56
-rw-r--r--drivers/net/wireless/microchip/wilc1000/wlan.h7
-rw-r--r--drivers/net/wireless/quantenna/qtnfmac/cfg80211.c27
-rw-r--r--drivers/net/wireless/quantenna/qtnfmac/commands.c67
-rw-r--r--drivers/net/wireless/quantenna/qtnfmac/event.c6
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt2400pci.c1
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt2500pci.c1
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt2500usb.c1
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt2800pci.c1
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt2800usb.c1
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt2x00dev.c6
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt61pci.c2
-rw-r--r--drivers/net/wireless/ralink/rt2x00/rt73usb.c1
-rw-r--r--drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu.h1
-rw-r--r--drivers/net/wireless/realtek/rtl8xxxu/rtl8xxxu_core.c19
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/base.c15
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/core.c38
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/core.h1
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/rtl8188ee/hw.c2
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/rtl8192de/sw.c10
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/rtl8821ae/table.c500
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/usb.c4
-rw-r--r--drivers/net/wireless/realtek/rtlwifi/wifi.h9
-rw-r--r--drivers/net/wireless/realtek/rtw88/coex.c13
-rw-r--r--drivers/net/wireless/realtek/rtw88/coex.h9
-rw-r--r--drivers/net/wireless/realtek/rtw88/debug.c134
-rw-r--r--drivers/net/wireless/realtek/rtw88/debug.h1
-rw-r--r--drivers/net/wireless/realtek/rtw88/fw.c27
-rw-r--r--drivers/net/wireless/realtek/rtw88/fw.h18
-rw-r--r--drivers/net/wireless/realtek/rtw88/hci.h16
-rw-r--r--drivers/net/wireless/realtek/rtw88/mac.c19
-rw-r--r--drivers/net/wireless/realtek/rtw88/mac.h4
-rw-r--r--drivers/net/wireless/realtek/rtw88/mac80211.c2
-rw-r--r--drivers/net/wireless/realtek/rtw88/main.c104
-rw-r--r--drivers/net/wireless/realtek/rtw88/main.h57
-rw-r--r--drivers/net/wireless/realtek/rtw88/pci.c98
-rw-r--r--drivers/net/wireless/realtek/rtw88/pci.h1
-rw-r--r--drivers/net/wireless/realtek/rtw88/phy.c95
-rw-r--r--drivers/net/wireless/realtek/rtw88/phy.h3
-rw-r--r--drivers/net/wireless/realtek/rtw88/reg.h15
-rw-r--r--drivers/net/wireless/realtek/rtw88/rtw8821c.c3
-rw-r--r--drivers/net/wireless/realtek/rtw88/rtw8822c.c892
-rw-r--r--drivers/net/wireless/realtek/rtw88/rtw8822c.h339
-rw-r--r--drivers/net/wireless/realtek/rtw88/rtw8822c_table.c686
-rw-r--r--drivers/net/wireless/rsi/rsi_91x_main.c1
-rw-r--r--drivers/net/wireless/rsi/rsi_91x_ps.c1
-rw-r--r--drivers/net/wireless/rsi/rsi_91x_sdio.c3
-rw-r--r--drivers/net/wireless/rsi/rsi_91x_usb.c1
-rw-r--r--drivers/net/wireless/rsi/rsi_boot_params.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_coex.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_common.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_debugfs.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_hal.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_main.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_mgmt.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_ps.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_sdio.h2
-rw-r--r--drivers/net/wireless/rsi/rsi_usb.h2
-rw-r--r--drivers/net/wireless/st/cw1200/bh.c3
-rw-r--r--drivers/net/wireless/st/cw1200/wsm.h12
-rw-r--r--drivers/net/wireless/ti/wlcore/boot.c13
-rw-r--r--drivers/net/wireless/ti/wlcore/debugfs.h7
-rw-r--r--drivers/net/wireless/virt_wifi.c5
-rw-r--r--drivers/net/wireless/wl3501.h49
-rw-r--r--drivers/net/wireless/wl3501_cs.c54
-rw-r--r--drivers/net/wwan/Kconfig37
-rw-r--r--drivers/net/wwan/Makefile9
-rw-r--r--drivers/net/wwan/mhi_wwan_ctrl.c284
-rw-r--r--drivers/net/wwan/wwan_core.c554
-rw-r--r--drivers/net/xen-netback/netback.c4
-rw-r--r--drivers/net/xen-netback/xenbus.c12
-rw-r--r--drivers/net/xen-netfront.c18
-rw-r--r--drivers/nfc/fdp/fdp.c49
-rw-r--r--drivers/nfc/pn533/i2c.c8
-rw-r--r--drivers/nfc/pn533/pn533.c20
-rw-r--r--drivers/nfc/s3fwrn5/core.c12
-rw-r--r--drivers/nfc/st-nci/spi.c7
-rw-r--r--drivers/nvdimm/btt.c1
-rw-r--r--drivers/nvdimm/bus.c14
-rw-r--r--drivers/nvdimm/pmem.c38
-rw-r--r--drivers/nvdimm/region_devs.c16
-rw-r--r--drivers/nvme/host/Makefile2
-rw-r--r--drivers/nvme/host/core.c1195
-rw-r--r--drivers/nvme/host/fabrics.c61
-rw-r--r--drivers/nvme/host/fabrics.h20
-rw-r--r--drivers/nvme/host/fc.c27
-rw-r--r--drivers/nvme/host/ioctl.c498
-rw-r--r--drivers/nvme/host/lightnvm.c12
-rw-r--r--drivers/nvme/host/multipath.c172
-rw-r--r--drivers/nvme/host/nvme.h86
-rw-r--r--drivers/nvme/host/pci.c34
-rw-r--r--drivers/nvme/host/rdma.c22
-rw-r--r--drivers/nvme/host/tcp.c40
-rw-r--r--drivers/nvme/host/zns.c13
-rw-r--r--drivers/nvme/target/admin-cmd.c25
-rw-r--r--drivers/nvme/target/configfs.c6
-rw-r--r--drivers/nvme/target/core.c50
-rw-r--r--drivers/nvme/target/discovery.c8
-rw-r--r--drivers/nvme/target/fabrics-cmd.c23
-rw-r--r--drivers/nvme/target/fc.c78
-rw-r--r--drivers/nvme/target/io-cmd-bdev.c2
-rw-r--r--drivers/nvme/target/io-cmd-file.c8
-rw-r--r--drivers/nvme/target/loop.c14
-rw-r--r--drivers/nvme/target/nvmet.h14
-rw-r--r--drivers/nvme/target/passthru.c8
-rw-r--r--drivers/nvme/target/rdma.c9
-rw-r--r--drivers/nvme/target/tcp.c81
-rw-r--r--drivers/nvmem/Kconfig10
-rw-r--r--drivers/nvmem/Makefile2
-rw-r--r--drivers/nvmem/brcm_nvram.c78
-rw-r--r--drivers/nvmem/core.c95
-rw-r--r--drivers/nvmem/qcom-spmi-sdam.c2
-rw-r--r--drivers/nvmem/qfprom.c44
-rw-r--r--drivers/nvmem/snvs_lpgpr.c2
-rw-r--r--drivers/of/Makefile6
-rw-r--r--drivers/of/address.c51
-rw-r--r--drivers/of/base.c328
-rw-r--r--drivers/of/device.c7
-rw-r--r--drivers/of/dynamic.c23
-rw-r--r--drivers/of/fdt.c76
-rw-r--r--drivers/of/irq.c14
-rw-r--r--drivers/of/kexec.c458
-rw-r--r--drivers/of/of_net.c88
-rw-r--r--drivers/of/of_private.h2
-rw-r--r--drivers/of/of_reserved_mem.c6
-rw-r--r--drivers/of/overlay.c46
-rw-r--r--drivers/of/platform.c12
-rw-r--r--drivers/of/property.c129
-rw-r--r--drivers/of/unittest-data/Makefile48
-rw-r--r--drivers/of/unittest-data/overlay_base.dts90
-rw-r--r--drivers/of/unittest-data/overlay_common.dtsi91
-rw-r--r--drivers/of/unittest-data/static_base_1.dts4
-rw-r--r--drivers/of/unittest-data/static_base_2.dts4
-rw-r--r--drivers/of/unittest-data/testcases.dts23
-rw-r--r--drivers/of/unittest-data/testcases_common.dtsi19
-rw-r--r--drivers/of/unittest-data/tests-interrupts.dtsi11
-rw-r--r--drivers/of/unittest.c22
-rw-r--r--drivers/opp/core.c170
-rw-r--r--drivers/opp/of.c36
-rw-r--r--drivers/opp/opp.h2
-rw-r--r--drivers/parport/parport_amiga.c1
-rw-r--r--drivers/parport/parport_atari.c1
-rw-r--r--drivers/parport/parport_gsc.c1
-rw-r--r--drivers/parport/parport_ip32.c12
-rw-r--r--drivers/parport/parport_mfc3.c1
-rw-r--r--drivers/parport/parport_sunbpp.c1
-rw-r--r--drivers/pci/ats.c2
-rw-r--r--drivers/pci/controller/Kconfig17
-rw-r--r--drivers/pci/controller/Makefile8
-rw-r--r--drivers/pci/controller/cadence/pci-j721e.c24
-rw-r--r--drivers/pci/controller/cadence/pcie-cadence-host.c3
-rw-r--r--drivers/pci/controller/dwc/Kconfig12
-rw-r--r--drivers/pci/controller/dwc/Makefile10
-rw-r--r--drivers/pci/controller/dwc/pci-dra7xx.c13
-rw-r--r--drivers/pci/controller/dwc/pci-keystone.c14
-rw-r--r--drivers/pci/controller/dwc/pci-layerscape-ep.c2
-rw-r--r--drivers/pci/controller/dwc/pcie-designware-ep.c2
-rw-r--r--drivers/pci/controller/dwc/pcie-designware-host.c4
-rw-r--r--drivers/pci/controller/dwc/pcie-designware.c11
-rw-r--r--drivers/pci/controller/dwc/pcie-designware.h1
-rw-r--r--drivers/pci/controller/dwc/pcie-fu740.c309
-rw-r--r--drivers/pci/controller/dwc/pcie-intel-gw.c5
-rw-r--r--drivers/pci/controller/dwc/pcie-tegra194.c108
-rw-r--r--drivers/pci/controller/mobiveil/Kconfig3
-rw-r--r--drivers/pci/controller/pci-host-common.c1
-rw-r--r--drivers/pci/controller/pci-hyperv.c8
-rw-r--r--drivers/pci/controller/pci-tegra.c349
-rw-r--r--drivers/pci/controller/pci-thunder-ecam.c2
-rw-r--r--drivers/pci/controller/pci-thunder-pem.c13
-rw-r--r--drivers/pci/controller/pci-xgene.c3
-rw-r--r--drivers/pci/controller/pcie-altera-msi.c4
-rw-r--r--drivers/pci/controller/pcie-brcmstb.c20
-rw-r--r--drivers/pci/controller/pcie-iproc-msi.c2
-rw-r--r--drivers/pci/controller/pcie-mediatek-gen3.c1027
-rw-r--r--drivers/pci/controller/pcie-mediatek.c7
-rw-r--r--drivers/pci/controller/pcie-microchip-host.c12
-rw-r--r--drivers/pci/controller/pcie-rcar-host.c355
-rw-r--r--drivers/pci/controller/pcie-xilinx-nwl.c7
-rw-r--r--drivers/pci/controller/pcie-xilinx.c246
-rw-r--r--drivers/pci/controller/vmd.c63
-rw-r--r--drivers/pci/endpoint/functions/pci-epf-ntb.c16
-rw-r--r--drivers/pci/endpoint/functions/pci-epf-test.c22
-rw-r--r--drivers/pci/endpoint/pci-epc-core.c2
-rw-r--r--drivers/pci/endpoint/pci-epf-core.c2
-rw-r--r--drivers/pci/hotplug/acpi_pcihp.c2
-rw-r--r--drivers/pci/hotplug/acpiphp.h3
-rw-r--r--drivers/pci/hotplug/acpiphp_glue.c1
-rw-r--r--drivers/pci/hotplug/cpqphp_nvram.c5
-rw-r--r--drivers/pci/hotplug/rpadlpar_sysfs.c14
-rw-r--r--drivers/pci/hotplug/s390_pci_hpc.c60
-rw-r--r--drivers/pci/hotplug/shpchp_hpc.c5
-rw-r--r--drivers/pci/iov.c102
-rw-r--r--drivers/pci/msi.c45
-rw-r--r--drivers/pci/of.c22
-rw-r--r--drivers/pci/pci-acpi.c2
-rw-r--r--drivers/pci/pci-label.c218
-rw-r--r--drivers/pci/pci-sysfs.c263
-rw-r--r--drivers/pci/pci.c66
-rw-r--r--drivers/pci/pci.h27
-rw-r--r--drivers/pci/pcie/aer.c6
-rw-r--r--drivers/pci/pcie/pme.c2
-rw-r--r--drivers/pci/pcie/rcec.c2
-rw-r--r--drivers/pci/probe.c5
-rw-r--r--drivers/pci/quirks.c29
-rw-r--r--drivers/pci/remove.c2
-rw-r--r--drivers/pci/vpd.c232
-rw-r--r--drivers/pci/xen-pcifront.c6
-rw-r--r--drivers/pcmcia/cistpl.c12
-rw-r--r--drivers/pcmcia/ds.c7
-rw-r--r--drivers/pcmcia/pcmcia_cis.c10
-rw-r--r--drivers/pcmcia/pcmcia_resource.c11
-rw-r--r--drivers/pcmcia/rsrc_nonstatic.c22
-rw-r--r--drivers/perf/arm-cci.c12
-rw-r--r--drivers/perf/arm-ccn.c31
-rw-r--r--drivers/perf/arm-cmn.c22
-rw-r--r--drivers/perf/arm_dmc620_pmu.c3
-rw-r--r--drivers/perf/arm_dsu_pmu.c5
-rw-r--r--drivers/perf/arm_pmu.c30
-rw-r--r--drivers/perf/arm_pmu_platform.c54
-rw-r--r--drivers/perf/arm_smmuv3_pmu.c36
-rw-r--r--drivers/perf/arm_spe_pmu.c3
-rw-r--r--drivers/perf/fsl_imx8_ddr_perf.c7
-rw-r--r--drivers/perf/hisilicon/Makefile3
-rw-r--r--drivers/perf/hisilicon/hisi_uncore_ddrc_pmu.c348
-rw-r--r--drivers/perf/hisilicon/hisi_uncore_hha_pmu.c301
-rw-r--r--drivers/perf/hisilicon/hisi_uncore_l3c_pmu.c355
-rw-r--r--drivers/perf/hisilicon/hisi_uncore_pa_pmu.c500
-rw-r--r--drivers/perf/hisilicon/hisi_uncore_pmu.c79
-rw-r--r--drivers/perf/hisilicon/hisi_uncore_pmu.h20
-rw-r--r--drivers/perf/hisilicon/hisi_uncore_sllc_pmu.c530
-rw-r--r--drivers/perf/qcom_l2_pmu.c2
-rw-r--r--drivers/perf/qcom_l3_pmu.c4
-rw-r--r--drivers/perf/thunderx2_pmu.c4
-rw-r--r--drivers/perf/xgene_pmu.c4
-rw-r--r--drivers/phy/Kconfig1
-rw-r--r--drivers/phy/Makefile1
-rw-r--r--drivers/phy/broadcom/Kconfig2
-rw-r--r--drivers/phy/cadence/Kconfig2
-rw-r--r--drivers/phy/cadence/phy-cadence-sierra.c419
-rw-r--r--drivers/phy/cadence/phy-cadence-torrent.c475
-rw-r--r--drivers/phy/hisilicon/phy-hi6220-usb.c2
-rw-r--r--drivers/phy/hisilicon/phy-hix5hd2-sata.c2
-rw-r--r--drivers/phy/ingenic/phy-ingenic-usb.c4
-rw-r--r--drivers/phy/intel/phy-intel-lgm-combo.c2
-rw-r--r--drivers/phy/marvell/Kconfig12
-rw-r--r--drivers/phy/marvell/Makefile1
-rw-r--r--drivers/phy/marvell/phy-mvebu-cp110-utmi.c384
-rw-r--r--drivers/phy/microchip/Kconfig13
-rw-r--r--drivers/phy/microchip/Makefile6
-rw-r--r--drivers/phy/microchip/sparx5_serdes.c2513
-rw-r--r--drivers/phy/microchip/sparx5_serdes.h136
-rw-r--r--drivers/phy/microchip/sparx5_serdes_regs.h2695
-rw-r--r--drivers/phy/phy-core-mipi-dphy.c2
-rw-r--r--drivers/phy/phy-core.c30
-rw-r--r--drivers/phy/qualcomm/phy-qcom-ipq806x-usb.c4
-rw-r--r--drivers/phy/qualcomm/phy-qcom-qmp.c528
-rw-r--r--drivers/phy/qualcomm/phy-qcom-qmp.h77
-rw-r--r--drivers/phy/qualcomm/phy-qcom-usb-hs.c1
-rw-r--r--drivers/phy/ralink/phy-mt7621-pci.c6
-rw-r--r--drivers/phy/rockchip/phy-rockchip-inno-dsidphy.c8
-rw-r--r--drivers/phy/rockchip/phy-rockchip-typec.c1
-rw-r--r--drivers/phy/st/Kconfig1
-rw-r--r--drivers/phy/st/phy-stm32-usbphyc.c65
-rw-r--r--drivers/phy/ti/phy-j721e-wiz.c449
-rw-r--r--drivers/phy/ti/phy-tusb1210.c27
-rw-r--r--drivers/phy/ti/phy-twl4030-usb.c2
-rw-r--r--drivers/phy/xilinx/phy-zynqmp.c58
-rw-r--r--drivers/pinctrl/Kconfig21
-rw-r--r--drivers/pinctrl/Makefile1
-rw-r--r--drivers/pinctrl/aspeed/pinctrl-aspeed-g5.c17
-rw-r--r--drivers/pinctrl/bcm/Kconfig62
-rw-r--r--drivers/pinctrl/bcm/Makefile7
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm6318.c498
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm63268.c643
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm6328.c404
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm6358.c369
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm6362.c617
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm6368.c523
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm63xx.c109
-rw-r--r--drivers/pinctrl/bcm/pinctrl-bcm63xx.h43
-rw-r--r--drivers/pinctrl/core.c53
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx1.c1
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx25.c3
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx27.c3
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx35.c1
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx50.c3
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx51.c1
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx53.c1
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx6dl.c1
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx6q.c1
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx6sl.c1
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx6sll.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx6sx.c3
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx6ul.c3
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx7d.c3
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx7ulp.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx8dxl.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx8mm.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx8mn.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx8mp.c3
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx8mq.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx8qm.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-imx8qxp.c2
-rw-r--r--drivers/pinctrl/freescale/pinctrl-vf610.c1
-rw-r--r--drivers/pinctrl/intel/pinctrl-intel.c14
-rw-r--r--drivers/pinctrl/intel/pinctrl-lewisburg.c6
-rw-r--r--drivers/pinctrl/mediatek/Kconfig6
-rw-r--r--drivers/pinctrl/mediatek/Makefile1
-rw-r--r--drivers/pinctrl/mediatek/pinctrl-moore.c2
-rw-r--r--drivers/pinctrl/mediatek/pinctrl-mt8195.c850
-rw-r--r--drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.c19
-rw-r--r--drivers/pinctrl/mediatek/pinctrl-mtk-common-v2.h7
-rw-r--r--drivers/pinctrl/mediatek/pinctrl-mtk-mt8195.h1669
-rw-r--r--drivers/pinctrl/mediatek/pinctrl-paris.c2
-rw-r--r--drivers/pinctrl/mvebu/pinctrl-armada-cp110.c4
-rw-r--r--drivers/pinctrl/pinconf-generic.c6
-rw-r--r--drivers/pinctrl/pinconf.c4
-rw-r--r--drivers/pinctrl/pinctrl-at91-pio4.c8
-rw-r--r--drivers/pinctrl/pinctrl-at91.c2
-rw-r--r--drivers/pinctrl/pinctrl-equilibrium.c16
-rw-r--r--drivers/pinctrl/pinctrl-ingenic.c1652
-rw-r--r--drivers/pinctrl/pinctrl-k210.c1
-rw-r--r--drivers/pinctrl/pinctrl-lpc18xx.c4
-rw-r--r--drivers/pinctrl/pinctrl-microchip-sgpio.c2
-rw-r--r--drivers/pinctrl/pinctrl-rockchip.c969
-rw-r--r--drivers/pinctrl/pinctrl-single.c71
-rw-r--r--drivers/pinctrl/pinctrl-zynq.c4
-rw-r--r--drivers/pinctrl/pinctrl-zynqmp.c906
-rw-r--r--drivers/pinctrl/pinmux.c106
-rw-r--r--drivers/pinctrl/pxa/pinctrl-pxa2xx.c4
-rw-r--r--drivers/pinctrl/qcom/Kconfig4
-rw-r--r--drivers/pinctrl/qcom/pinctrl-lpass-lpi.c2
-rw-r--r--drivers/pinctrl/qcom/pinctrl-sc7280.c40
-rw-r--r--drivers/pinctrl/qcom/pinctrl-sc8180x.c123
-rw-r--r--drivers/pinctrl/qcom/pinctrl-sdx55.c2
-rw-r--r--drivers/pinctrl/qcom/pinctrl-sm8350.c21
-rw-r--r--drivers/pinctrl/qcom/pinctrl-spmi-gpio.c7
-rw-r--r--drivers/pinctrl/renesas/core.c20
-rw-r--r--drivers/pinctrl/renesas/core.h8
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a73a4.c48
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a7740.c46
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a7778.c1
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a7791.c387
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a7792.c1
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a77950.c1
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a77951.c31
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a7796.c31
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a77965.c35
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a77970.c1
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a77980.c1
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a77990.c35
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a77995.c1
-rw-r--r--drivers/pinctrl/renesas/pfc-r8a779a0.c1
-rw-r--r--drivers/pinctrl/renesas/pfc-sh73a0.c46
-rw-r--r--drivers/pinctrl/renesas/pinctrl.c109
-rw-r--r--drivers/pinctrl/renesas/sh_pfc.h24
-rw-r--r--drivers/pinctrl/samsung/pinctrl-exynos.c10
-rw-r--r--drivers/pinctrl/stm32/pinctrl-stm32.c18
-rw-r--r--drivers/pinctrl/ti/pinctrl-ti-iodelay.c7
-rw-r--r--drivers/platform/chrome/chromeos_laptop.c100
-rw-r--r--drivers/platform/chrome/cros_ec_lpc_mec.c3
-rw-r--r--drivers/platform/chrome/cros_ec_typec.c47
-rw-r--r--drivers/platform/chrome/cros_usbpd_notify.c3
-rw-r--r--drivers/platform/chrome/wilco_ec/telemetry.c2
-rw-r--r--drivers/platform/mellanox/mlxbf-bootctl.c2
-rw-r--r--drivers/platform/mellanox/mlxreg-hotplug.c4
-rw-r--r--drivers/platform/surface/Kconfig69
-rw-r--r--drivers/platform/surface/Makefile3
-rw-r--r--drivers/platform/surface/aggregator/controller.c16
-rw-r--r--drivers/platform/surface/surface_aggregator_registry.c626
-rw-r--r--drivers/platform/surface/surface_dtx.c1289
-rw-r--r--drivers/platform/surface/surface_platform_profile.c190
-rw-r--r--drivers/platform/surface/surfacepro3_button.c2
-rw-r--r--drivers/platform/x86/Kconfig37
-rw-r--r--drivers/platform/x86/Makefile4
-rw-r--r--drivers/platform/x86/adv_swbutton.c121
-rw-r--r--drivers/platform/x86/asus-laptop.c2
-rw-r--r--drivers/platform/x86/asus-wmi.c5
-rw-r--r--drivers/platform/x86/classmate-laptop.c2
-rw-r--r--drivers/platform/x86/dell/alienware-wmi.c4
-rw-r--r--drivers/platform/x86/dell/dell-smbios-base.c2
-rw-r--r--drivers/platform/x86/dell/dell-smbios-wmi.c3
-rw-r--r--drivers/platform/x86/dell/dell-wmi-descriptor.c5
-rw-r--r--drivers/platform/x86/dell/dell-wmi-sysman/biosattr-interface.c3
-rw-r--r--drivers/platform/x86/dell/dell-wmi-sysman/enum-attributes.c3
-rw-r--r--drivers/platform/x86/dell/dell-wmi-sysman/int-attributes.c3
-rw-r--r--drivers/platform/x86/dell/dell-wmi-sysman/passobj-attributes.c3
-rw-r--r--drivers/platform/x86/dell/dell-wmi-sysman/passwordattr-interface.c3
-rw-r--r--drivers/platform/x86/dell/dell-wmi-sysman/string-attributes.c3
-rw-r--r--drivers/platform/x86/dell/dell-wmi-sysman/sysman.c118
-rw-r--r--drivers/platform/x86/dell/dell-wmi.c3
-rw-r--r--drivers/platform/x86/dell/dell_rbu.c3
-rw-r--r--drivers/platform/x86/gigabyte-wmi.c203
-rw-r--r--drivers/platform/x86/gpd-pocket-fan.c17
-rw-r--r--drivers/platform/x86/hp-wmi.c101
-rw-r--r--drivers/platform/x86/intel-hid.c23
-rw-r--r--drivers/platform/x86/intel-vbtn.c15
-rw-r--r--drivers/platform/x86/intel-wmi-sbl-fw-update.c3
-rw-r--r--drivers/platform/x86/intel-wmi-thunderbolt.c3
-rw-r--r--drivers/platform/x86/intel_cht_int33fe_microb.c6
-rw-r--r--drivers/platform/x86/intel_cht_int33fe_typec.c19
-rw-r--r--drivers/platform/x86/intel_chtdc_ti_pwrbtn.c2
-rw-r--r--drivers/platform/x86/intel_pmc_core.c537
-rw-r--r--drivers/platform/x86/intel_pmc_core.h53
-rw-r--r--drivers/platform/x86/intel_pmt_class.c48
-rw-r--r--drivers/platform/x86/intel_pmt_class.h1
-rw-r--r--drivers/platform/x86/intel_pmt_crashlog.c13
-rw-r--r--drivers/platform/x86/intel_pmt_telemetry.c20
-rw-r--r--drivers/platform/x86/intel_speed_select_if/isst_if_mbox_pci.c33
-rw-r--r--drivers/platform/x86/lg-laptop.c2
-rw-r--r--drivers/platform/x86/panasonic-laptop.c2
-rw-r--r--drivers/platform/x86/pmc_atom.c28
-rw-r--r--drivers/platform/x86/thinkpad_acpi.c362
-rw-r--r--drivers/platform/x86/touchscreen_dmi.c37
-rw-r--r--drivers/platform/x86/wmi-bmof.c3
-rw-r--r--drivers/platform/x86/wmi.c6
-rw-r--r--drivers/platform/x86/xo15-ebook.c6
-rw-r--r--drivers/power/reset/Makefile2
-rw-r--r--drivers/power/reset/at91-reset.c4
-rw-r--r--drivers/power/reset/hisi-reboot.c5
-rw-r--r--drivers/power/reset/ltc2952-poweroff.c8
-rw-r--r--drivers/power/reset/restart-poweroff.c1
-rw-r--r--drivers/power/reset/vexpress-poweroff.c2
-rw-r--r--drivers/power/supply/88pm860x_battery.c4
-rw-r--r--drivers/power/supply/Kconfig40
-rw-r--r--drivers/power/supply/Makefile2
-rw-r--r--drivers/power/supply/ab8500-bm.h733
-rw-r--r--drivers/power/supply/ab8500-chargalg.h51
-rw-r--r--drivers/power/supply/ab8500_bmdata.c3
-rw-r--r--drivers/power/supply/ab8500_btemp.c45
-rw-r--r--drivers/power/supply/ab8500_charger.c27
-rw-r--r--drivers/power/supply/ab8500_fg.c20
-rw-r--r--drivers/power/supply/abx500_chargalg.c22
-rw-r--r--drivers/power/supply/act8945a_charger.c2
-rw-r--r--drivers/power/supply/axp20x_usb_power.c21
-rw-r--r--drivers/power/supply/bq24735-charger.c18
-rw-r--r--drivers/power/supply/bq256xx_charger.c2
-rw-r--r--drivers/power/supply/bq25980_charger.c40
-rw-r--r--drivers/power/supply/bq27xxx_battery.c169
-rw-r--r--drivers/power/supply/bq27xxx_battery_i2c.c2
-rw-r--r--drivers/power/supply/charger-manager.c2
-rw-r--r--drivers/power/supply/cpcap-battery.c2
-rw-r--r--drivers/power/supply/cpcap-charger.c7
-rw-r--r--drivers/power/supply/cw2015_battery.c6
-rw-r--r--drivers/power/supply/ds2781_battery.c8
-rw-r--r--drivers/power/supply/generic-adc-battery.c2
-rw-r--r--drivers/power/supply/lp8788-charger.c2
-rw-r--r--drivers/power/supply/ltc2941-battery-gauge.c20
-rw-r--r--drivers/power/supply/max14577_charger.c2
-rw-r--r--drivers/power/supply/max17040_battery.c12
-rw-r--r--drivers/power/supply/max17042_battery.c8
-rw-r--r--drivers/power/supply/max1721x_battery.c14
-rw-r--r--drivers/power/supply/max8997_charger.c14
-rw-r--r--drivers/power/supply/pm2301_charger.c6
-rw-r--r--drivers/power/supply/power_supply_core.c4
-rw-r--r--drivers/power/supply/s3c_adc_battery.c21
-rw-r--r--drivers/power/supply/sbs-battery.c44
-rw-r--r--drivers/power/supply/sbs-charger.c24
-rw-r--r--drivers/power/supply/sbs-manager.c78
-rw-r--r--drivers/power/supply/smb347-charger.c65
-rw-r--r--drivers/power/supply/surface_battery.c865
-rw-r--r--drivers/power/supply/surface_charger.c282
-rw-r--r--drivers/power/supply/tps65090-charger.c2
-rw-r--r--drivers/power/supply/tps65217_charger.c2
-rw-r--r--drivers/power/supply/z2_battery.c3
-rw-r--r--drivers/powercap/intel_rapl_common.c1
-rw-r--r--drivers/powercap/intel_rapl_msr.c1
-rw-r--r--drivers/pps/clients/pps-gpio.c108
-rw-r--r--drivers/pps/clients/pps-ldisc.c3
-rw-r--r--drivers/ptp/Kconfig2
-rw-r--r--drivers/ptp/Makefile2
-rw-r--r--drivers/ptp/ptp_clockmatrix.c4
-rw-r--r--drivers/ptp/ptp_kvm_arm.c28
-rw-r--r--drivers/ptp/ptp_kvm_common.c (renamed from drivers/ptp/ptp_kvm.c)85
-rw-r--r--drivers/ptp/ptp_kvm_x86.c97
-rw-r--r--drivers/ptp/ptp_pch.c21
-rw-r--r--drivers/ptp/ptp_qoriq.c13
-rw-r--r--drivers/pwm/Kconfig26
-rw-r--r--drivers/pwm/Makefile3
-rw-r--r--drivers/pwm/core.c48
-rw-r--r--drivers/pwm/pwm-ab8500.c54
-rw-r--r--drivers/pwm/pwm-atmel-hlcdc.c3
-rw-r--r--drivers/pwm/pwm-atmel-tcb.c42
-rw-r--r--drivers/pwm/pwm-atmel.c30
-rw-r--r--drivers/pwm/pwm-bcm-iproc.c5
-rw-r--r--drivers/pwm/pwm-bcm-kona.c8
-rw-r--r--drivers/pwm/pwm-bcm2835.c40
-rw-r--r--drivers/pwm/pwm-berlin.c1
-rw-r--r--drivers/pwm/pwm-brcmstb.c1
-rw-r--r--drivers/pwm/pwm-clps711x.c1
-rw-r--r--drivers/pwm/pwm-crc.c1
-rw-r--r--drivers/pwm/pwm-cros-ec.c4
-rw-r--r--drivers/pwm/pwm-dwc.c1
-rw-r--r--drivers/pwm/pwm-ep93xx.c1
-rw-r--r--drivers/pwm/pwm-fsl-ftm.c1
-rw-r--r--drivers/pwm/pwm-hibvt.c1
-rw-r--r--drivers/pwm/pwm-img.c1
-rw-r--r--drivers/pwm/pwm-imx-tpm.c5
-rw-r--r--drivers/pwm/pwm-imx1.c1
-rw-r--r--drivers/pwm/pwm-imx27.c1
-rw-r--r--drivers/pwm/pwm-intel-lgm.c1
-rw-r--r--drivers/pwm/pwm-iqs620a.c1
-rw-r--r--drivers/pwm/pwm-jz4740.c1
-rw-r--r--drivers/pwm/pwm-keembay.c1
-rw-r--r--drivers/pwm/pwm-lp3943.c1
-rw-r--r--drivers/pwm/pwm-lpc18xx-sct.c5
-rw-r--r--drivers/pwm/pwm-lpc32xx.c5
-rw-r--r--drivers/pwm/pwm-lpss.c7
-rw-r--r--drivers/pwm/pwm-mediatek.c7
-rw-r--r--drivers/pwm/pwm-meson.c1
-rw-r--r--drivers/pwm/pwm-mtk-disp.c1
-rw-r--r--drivers/pwm/pwm-mxs.c1
-rw-r--r--drivers/pwm/pwm-ntxec.c184
-rw-r--r--drivers/pwm/pwm-omap-dmtimer.c1
-rw-r--r--drivers/pwm/pwm-pca9685.c303
-rw-r--r--drivers/pwm/pwm-pxa.c1
-rw-r--r--drivers/pwm/pwm-raspberrypi-poe.c206
-rw-r--r--drivers/pwm/pwm-rcar.c1
-rw-r--r--drivers/pwm/pwm-renesas-tpu.c1
-rw-r--r--drivers/pwm/pwm-rockchip.c1
-rw-r--r--drivers/pwm/pwm-samsung.c1
-rw-r--r--drivers/pwm/pwm-sifive.c1
-rw-r--r--drivers/pwm/pwm-sl28cpld.c1
-rw-r--r--drivers/pwm/pwm-spear.c1
-rw-r--r--drivers/pwm/pwm-sprd.c4
-rw-r--r--drivers/pwm/pwm-sti.c7
-rw-r--r--drivers/pwm/pwm-stm32-lp.c1
-rw-r--r--drivers/pwm/pwm-stm32.c1
-rw-r--r--drivers/pwm/pwm-stmpe.c1
-rw-r--r--drivers/pwm/pwm-sun4i.c1
-rw-r--r--drivers/pwm/pwm-tegra.c1
-rw-r--r--drivers/pwm/pwm-tiecap.c1
-rw-r--r--drivers/pwm/pwm-tiehrpwm.c1
-rw-r--r--drivers/pwm/pwm-twl-led.c1
-rw-r--r--drivers/pwm/pwm-twl.c1
-rw-r--r--drivers/pwm/pwm-visconti.c190
-rw-r--r--drivers/pwm/pwm-vt8500.c1
-rw-r--r--drivers/ras/cec.c15
-rw-r--r--drivers/regulator/Kconfig20
-rw-r--r--drivers/regulator/Makefile2
-rw-r--r--drivers/regulator/bd71815-regulator.c652
-rw-r--r--drivers/regulator/bd71828-regulator.c51
-rw-r--r--drivers/regulator/bd718x7-regulator.c60
-rw-r--r--drivers/regulator/bd9571mwv-regulator.c8
-rw-r--r--drivers/regulator/bd9576-regulator.c11
-rw-r--r--drivers/regulator/core.c59
-rw-r--r--drivers/regulator/da9121-regulator.c80
-rw-r--r--drivers/regulator/da9121-regulator.h13
-rw-r--r--drivers/regulator/fan53555.c136
-rw-r--r--drivers/regulator/helpers.c101
-rw-r--r--drivers/regulator/hi6421v600-regulator.c (renamed from drivers/staging/hikey9xx/hi6421v600-regulator.c)0
-rw-r--r--drivers/regulator/mt6315-regulator.c4
-rw-r--r--drivers/regulator/mt6360-regulator.c4
-rw-r--r--drivers/regulator/of_regulator.c6
-rw-r--r--drivers/regulator/pca9450-regulator.c10
-rw-r--r--drivers/regulator/pf8x00-regulator.c1
-rw-r--r--drivers/regulator/qcom-rpmh-regulator.c68
-rw-r--r--drivers/regulator/qcom_spmi-regulator.c36
-rw-r--r--drivers/regulator/rohm-regulator.c23
-rw-r--r--drivers/regulator/rt4831-regulator.c4
-rw-r--r--drivers/regulator/s2mpa01.c4
-rw-r--r--drivers/regulator/s2mps11.c22
-rw-r--r--drivers/regulator/scmi-regulator.c46
-rw-r--r--drivers/remoteproc/Kconfig7
-rw-r--r--drivers/remoteproc/imx_rproc.c322
-rw-r--r--drivers/remoteproc/ingenic_rproc.c2
-rw-r--r--drivers/remoteproc/keystone_remoteproc.c2
-rw-r--r--drivers/remoteproc/mtk_scp.c6
-rw-r--r--drivers/remoteproc/omap_remoteproc.c2
-rw-r--r--drivers/remoteproc/pru_rproc.c67
-rw-r--r--drivers/remoteproc/qcom_pil_info.c2
-rw-r--r--drivers/remoteproc/qcom_q6v5_adsp.c2
-rw-r--r--drivers/remoteproc/qcom_q6v5_mss.c26
-rw-r--r--drivers/remoteproc/qcom_q6v5_pas.c19
-rw-r--r--drivers/remoteproc/qcom_q6v5_wcss.c599
-rw-r--r--drivers/remoteproc/qcom_wcnss.c10
-rw-r--r--drivers/remoteproc/remoteproc_cdev.c21
-rw-r--r--drivers/remoteproc/remoteproc_core.c337
-rw-r--r--drivers/remoteproc/remoteproc_coredump.c8
-rw-r--r--drivers/remoteproc/remoteproc_debugfs.c2
-rw-r--r--drivers/remoteproc/remoteproc_elf_loader.c21
-rw-r--r--drivers/remoteproc/remoteproc_internal.h12
-rw-r--r--drivers/remoteproc/remoteproc_sysfs.c21
-rw-r--r--drivers/remoteproc/st_slim_rproc.c2
-rw-r--r--drivers/remoteproc/stm32_rproc.c205
-rw-r--r--drivers/remoteproc/ti_k3_dsp_remoteproc.c2
-rw-r--r--drivers/remoteproc/ti_k3_r5_remoteproc.c2
-rw-r--r--drivers/remoteproc/wkup_m3_rproc.c2
-rw-r--r--drivers/reset/Kconfig7
-rw-r--r--drivers/reset/core.c215
-rw-r--r--drivers/reset/reset-raspberrypi.c2
-rw-r--r--drivers/reset/reset-scmi.c33
-rw-r--r--drivers/rpmsg/qcom_glink_native.c17
-rw-r--r--drivers/rpmsg/qcom_smd.c16
-rw-r--r--drivers/rpmsg/rpmsg_char.c11
-rw-r--r--drivers/rpmsg/virtio_rpmsg_bus.c62
-rw-r--r--drivers/rtc/Kconfig17
-rw-r--r--drivers/rtc/Makefile1
-rw-r--r--drivers/rtc/interface.c34
-rw-r--r--drivers/rtc/rtc-ab-eoz9.c135
-rw-r--r--drivers/rtc/rtc-bd70528.c104
-rw-r--r--drivers/rtc/rtc-ds1307.c56
-rw-r--r--drivers/rtc/rtc-ds1511.c6
-rw-r--r--drivers/rtc/rtc-fsl-ftm-alarm.c1
-rw-r--r--drivers/rtc/rtc-hid-sensor-time.c4
-rw-r--r--drivers/rtc/rtc-imx-sc.c11
-rw-r--r--drivers/rtc/rtc-imxdi.c4
-rw-r--r--drivers/rtc/rtc-m48t59.c2
-rw-r--r--drivers/rtc/rtc-mxc.c2
-rw-r--r--drivers/rtc/rtc-ntxec.c145
-rw-r--r--drivers/rtc/rtc-omap.c5
-rw-r--r--drivers/rtc/rtc-pcf85063.c7
-rw-r--r--drivers/rtc/rtc-pcf8523.c196
-rw-r--r--drivers/rtc/rtc-pm8xxx.c11
-rw-r--r--drivers/rtc/rtc-rv3028.c4
-rw-r--r--drivers/rtc/rtc-rx6110.c7
-rw-r--r--drivers/rtc/rtc-s5m.c6
-rw-r--r--drivers/rtc/rtc-spear.c6
-rw-r--r--drivers/rtc/rtc-tps65910.c1
-rw-r--r--drivers/rtc/sysfs.c2
-rw-r--r--drivers/s390/block/dasd.c24
-rw-r--r--drivers/s390/block/dasd_devmap.c15
-rw-r--r--drivers/s390/block/dasd_eckd.c1
-rw-r--r--drivers/s390/block/dasd_eckd.h8
-rw-r--r--drivers/s390/block/dasd_fba.c10
-rw-r--r--drivers/s390/block/dasd_genhd.c3
-rw-r--r--drivers/s390/block/dasd_int.h3
-rw-r--r--drivers/s390/char/con3215.c59
-rw-r--r--drivers/s390/char/sclp.c6
-rw-r--r--drivers/s390/char/sclp_con.c28
-rw-r--r--drivers/s390/char/sclp_tty.c18
-rw-r--r--drivers/s390/char/sclp_vt220.c9
-rw-r--r--drivers/s390/char/tty3270.c6
-rw-r--r--drivers/s390/char/zcore.c44
-rw-r--r--drivers/s390/cio/Makefile4
-rw-r--r--drivers/s390/cio/chp.c3
-rw-r--r--drivers/s390/cio/cio_debug.h3
-rw-r--r--drivers/s390/cio/cio_debugfs.c23
-rw-r--r--drivers/s390/cio/cio_inject.c171
-rw-r--r--drivers/s390/cio/cio_inject.h18
-rw-r--r--drivers/s390/cio/css.c6
-rw-r--r--drivers/s390/cio/device.c3
-rw-r--r--drivers/s390/cio/device_fsm.c2
-rw-r--r--drivers/s390/cio/ioasm.c23
-rw-r--r--drivers/s390/cio/qdio.h10
-rw-r--r--drivers/s390/cio/qdio_main.c63
-rw-r--r--drivers/s390/cio/qdio_setup.c49
-rw-r--r--drivers/s390/cio/vfio_ccw_ops.c23
-rw-r--r--drivers/s390/crypto/vfio_ap_ops.c324
-rw-r--r--drivers/s390/crypto/vfio_ap_private.h2
-rw-r--r--drivers/s390/crypto/zcrypt_card.c1
-rw-r--r--drivers/s390/crypto/zcrypt_queue.c1
-rw-r--r--drivers/s390/net/qeth_core.h6
-rw-r--r--drivers/s390/net/qeth_core_main.c232
-rw-r--r--drivers/s390/net/qeth_l3_main.c31
-rw-r--r--drivers/s390/scsi/zfcp_aux.c28
-rw-r--r--drivers/s390/scsi/zfcp_def.h6
-rw-r--r--drivers/s390/scsi/zfcp_diag.c42
-rw-r--r--drivers/s390/scsi/zfcp_diag.h7
-rw-r--r--drivers/s390/scsi/zfcp_ext.h4
-rw-r--r--drivers/s390/scsi/zfcp_fsf.c1
-rw-r--r--drivers/s390/scsi/zfcp_qdio.c75
-rw-r--r--drivers/s390/scsi/zfcp_qdio.h5
-rw-r--r--drivers/s390/scsi/zfcp_sysfs.c14
-rw-r--r--drivers/s390/scsi/zfcp_unit.c4
-rw-r--r--drivers/sbus/char/display7seg.c1
-rw-r--r--drivers/scsi/3w-9xxx.c14
-rw-r--r--drivers/scsi/3w-sas.c13
-rw-r--r--drivers/scsi/3w-xxxx.c6
-rw-r--r--drivers/scsi/53c700.c7
-rw-r--r--drivers/scsi/53c700.h1
-rw-r--r--drivers/scsi/BusLogic.c207
-rw-r--r--drivers/scsi/BusLogic.h11
-rw-r--r--drivers/scsi/FlashPoint.c6
-rw-r--r--drivers/scsi/Kconfig2
-rw-r--r--drivers/scsi/a100u2w.c13
-rw-r--r--drivers/scsi/aacraid/TODO3
-rw-r--r--drivers/scsi/aacraid/aachba.c13
-rw-r--r--drivers/scsi/aacraid/commctrl.c2
-rw-r--r--drivers/scsi/aacraid/commsup.c4
-rw-r--r--drivers/scsi/aacraid/rx.c2
-rw-r--r--drivers/scsi/advansys.c323
-rw-r--r--drivers/scsi/aha1542.c105
-rw-r--r--drivers/scsi/aic94xx/aic94xx.h2
-rw-r--r--drivers/scsi/aic94xx/aic94xx_dump.c186
-rw-r--r--drivers/scsi/aic94xx/aic94xx_hwi.c4
-rw-r--r--drivers/scsi/aic94xx/aic94xx_sds.c2
-rw-r--r--drivers/scsi/atp870u.c7
-rw-r--r--drivers/scsi/be2iscsi/be_iscsi.c2
-rw-r--r--drivers/scsi/be2iscsi/be_main.c6
-rw-r--r--drivers/scsi/be2iscsi/be_mgmt.c2
-rw-r--r--drivers/scsi/bfa/bfa_fc.h4
-rw-r--r--drivers/scsi/bfa/bfa_fcs.h3
-rw-r--r--drivers/scsi/bfa/bfa_fcs_lport.c20
-rw-r--r--drivers/scsi/bfa/bfad_bsg.c2
-rw-r--r--drivers/scsi/bnx2fc/bnx2fc_fcoe.c2
-rw-r--r--drivers/scsi/bnx2fc/bnx2fc_hwi.c2
-rw-r--r--drivers/scsi/bnx2fc/bnx2fc_tgt.c2
-rw-r--r--drivers/scsi/bnx2i/bnx2i_hwi.c85
-rw-r--r--drivers/scsi/bnx2i/bnx2i_sysfs.c2
-rw-r--r--drivers/scsi/ch.c6
-rw-r--r--drivers/scsi/csiostor/csio_hw_t5.c2
-rw-r--r--drivers/scsi/csiostor/csio_scsi.c4
-rw-r--r--drivers/scsi/cxgbi/cxgb3i/cxgb3i.c2
-rw-r--r--drivers/scsi/cxlflash/main.c11
-rw-r--r--drivers/scsi/cxlflash/superpipe.c6
-rw-r--r--drivers/scsi/cxlflash/vlun.c8
-rw-r--r--drivers/scsi/dc395x.c15
-rw-r--r--drivers/scsi/device_handler/scsi_dh_alua.c39
-rw-r--r--drivers/scsi/device_handler/scsi_dh_emc.c4
-rw-r--r--drivers/scsi/device_handler/scsi_dh_rdac.c4
-rw-r--r--drivers/scsi/esas2r/esas2r_log.c7
-rw-r--r--drivers/scsi/esas2r/esas2r_main.c2
-rw-r--r--drivers/scsi/esp_scsi.c4
-rw-r--r--drivers/scsi/fcoe/fcoe.c2
-rw-r--r--drivers/scsi/fcoe/fcoe_ctlr.c4
-rw-r--r--drivers/scsi/fcoe/fcoe_transport.c2
-rw-r--r--drivers/scsi/fnic/fnic_debugfs.c3
-rw-r--r--drivers/scsi/fnic/fnic_fcs.c16
-rw-r--r--drivers/scsi/fnic/fnic_main.c3
-rw-r--r--drivers/scsi/fnic/fnic_scsi.c834
-rw-r--r--drivers/scsi/fnic/fnic_trace.c18
-rw-r--r--drivers/scsi/hisi_sas/hisi_sas.h3
-rw-r--r--drivers/scsi/hisi_sas/hisi_sas_main.c38
-rw-r--r--drivers/scsi/hisi_sas/hisi_sas_v1_hw.c6
-rw-r--r--drivers/scsi/hisi_sas/hisi_sas_v3_hw.c44
-rw-r--r--drivers/scsi/hosts.c7
-rw-r--r--drivers/scsi/hpsa.c1
-rw-r--r--drivers/scsi/hpsa_cmd.h78
-rw-r--r--drivers/scsi/ibmvscsi/ibmvfc.c222
-rw-r--r--drivers/scsi/ibmvscsi/ibmvscsi.c77
-rw-r--r--drivers/scsi/ibmvscsi_tgt/ibmvscsi_tgt.c27
-rw-r--r--drivers/scsi/initio.c18
-rw-r--r--drivers/scsi/ipr.c8
-rw-r--r--drivers/scsi/ips.c20
-rw-r--r--drivers/scsi/ips.h20
-rw-r--r--drivers/scsi/isci/host.c37
-rw-r--r--drivers/scsi/isci/phy.c34
-rw-r--r--drivers/scsi/isci/phy.h1
-rw-r--r--drivers/scsi/isci/port.c62
-rw-r--r--drivers/scsi/isci/port_config.c37
-rw-r--r--drivers/scsi/isci/remote_device.c33
-rw-r--r--drivers/scsi/isci/remote_node_context.c13
-rw-r--r--drivers/scsi/isci/remote_node_table.c64
-rw-r--r--drivers/scsi/isci/request.c60
-rw-r--r--drivers/scsi/isci/task.c3
-rw-r--r--drivers/scsi/jazz_esp.c4
-rw-r--r--drivers/scsi/lasi700.c1
-rw-r--r--drivers/scsi/libfc/fc_fcp.c2
-rw-r--r--drivers/scsi/libfc/fc_lport.c14
-rw-r--r--drivers/scsi/libfc/fc_rport.c2
-rw-r--r--drivers/scsi/libiscsi.c26
-rw-r--r--drivers/scsi/libsas/sas_ata.c83
-rw-r--r--drivers/scsi/libsas/sas_discover.c2
-rw-r--r--drivers/scsi/libsas/sas_expander.c13
-rw-r--r--drivers/scsi/lpfc/lpfc.h3
-rw-r--r--drivers/scsi/lpfc/lpfc_attr.c140
-rw-r--r--drivers/scsi/lpfc/lpfc_bsg.c30
-rw-r--r--drivers/scsi/lpfc/lpfc_crtn.h11
-rw-r--r--drivers/scsi/lpfc/lpfc_ct.c44
-rw-r--r--drivers/scsi/lpfc/lpfc_debugfs.c25
-rw-r--r--drivers/scsi/lpfc/lpfc_disc.h3
-rw-r--r--drivers/scsi/lpfc/lpfc_els.c755
-rw-r--r--drivers/scsi/lpfc/lpfc_hbadisc.c34
-rw-r--r--drivers/scsi/lpfc/lpfc_hw4.h176
-rw-r--r--drivers/scsi/lpfc/lpfc_init.c154
-rw-r--r--drivers/scsi/lpfc/lpfc_mbox.c38
-rw-r--r--drivers/scsi/lpfc/lpfc_nportdisc.c282
-rw-r--r--drivers/scsi/lpfc/lpfc_nvme.c20
-rw-r--r--drivers/scsi/lpfc/lpfc_nvmet.c31
-rw-r--r--drivers/scsi/lpfc/lpfc_scsi.c124
-rw-r--r--drivers/scsi/lpfc/lpfc_sli.c144
-rw-r--r--drivers/scsi/lpfc/lpfc_version.h6
-rw-r--r--drivers/scsi/lpfc/lpfc_vport.c10
-rw-r--r--drivers/scsi/mac53c94.c13
-rw-r--r--drivers/scsi/megaraid/mbox_defs.h2
-rw-r--r--drivers/scsi/megaraid/mega_common.h2
-rw-r--r--drivers/scsi/megaraid/megaraid_mbox.c4
-rw-r--r--drivers/scsi/megaraid/megaraid_mbox.h2
-rw-r--r--drivers/scsi/megaraid/megaraid_mm.c2
-rw-r--r--drivers/scsi/megaraid/megaraid_sas.h5
-rw-r--r--drivers/scsi/megaraid/megaraid_sas_base.c98
-rw-r--r--drivers/scsi/megaraid/megaraid_sas_fusion.c89
-rw-r--r--drivers/scsi/megaraid/megaraid_sas_fusion.h2
-rw-r--r--drivers/scsi/mpt3sas/mpi/mpi2_cnfg.h2
-rw-r--r--drivers/scsi/mpt3sas/mpt3sas_base.c522
-rw-r--r--drivers/scsi/mpt3sas/mpt3sas_base.h5
-rw-r--r--drivers/scsi/mpt3sas/mpt3sas_config.c10
-rw-r--r--drivers/scsi/mpt3sas/mpt3sas_ctl.c45
-rw-r--r--drivers/scsi/mpt3sas/mpt3sas_ctl.h12
-rw-r--r--drivers/scsi/mpt3sas/mpt3sas_scsih.c67
-rw-r--r--drivers/scsi/mpt3sas/mpt3sas_transport.c7
-rw-r--r--drivers/scsi/mvsas/mv_sas.h2
-rw-r--r--drivers/scsi/mvumi.c9
-rw-r--r--drivers/scsi/myrb.c126
-rw-r--r--drivers/scsi/myrs.c119
-rw-r--r--drivers/scsi/nsp32.c31
-rw-r--r--drivers/scsi/pcmcia/nsp_cs.c1
-rw-r--r--drivers/scsi/pm8001/pm8001_ctl.c151
-rw-r--r--drivers/scsi/pm8001/pm8001_ctl.h5
-rw-r--r--drivers/scsi/pm8001/pm8001_hwi.c100
-rw-r--r--drivers/scsi/pm8001/pm8001_hwi.h1
-rw-r--r--drivers/scsi/pm8001/pm8001_init.c19
-rw-r--r--drivers/scsi/pm8001/pm8001_sas.c31
-rw-r--r--drivers/scsi/pm8001/pm8001_sas.h5
-rw-r--r--drivers/scsi/pm8001/pm80xx_hwi.c49
-rw-r--r--drivers/scsi/pm8001/pm80xx_hwi.h1
-rw-r--r--drivers/scsi/pmcraid.c70
-rw-r--r--drivers/scsi/qedf/qedf.h3
-rw-r--r--drivers/scsi/qedf/qedf_dbg.h1
-rw-r--r--drivers/scsi/qedf/qedf_main.c18
-rw-r--r--drivers/scsi/qedi/qedi_main.c5
-rw-r--r--drivers/scsi/qla1280.c22
-rw-r--r--drivers/scsi/qla2xxx/qla_attr.c8
-rw-r--r--drivers/scsi/qla2xxx/qla_bsg.c7
-rw-r--r--drivers/scsi/qla2xxx/qla_dbg.c16
-rw-r--r--drivers/scsi/qla2xxx/qla_dbg.h2
-rw-r--r--drivers/scsi/qla2xxx/qla_def.h15
-rw-r--r--drivers/scsi/qla2xxx/qla_gbl.h3
-rw-r--r--drivers/scsi/qla2xxx/qla_gs.c14
-rw-r--r--drivers/scsi/qla2xxx/qla_init.c118
-rw-r--r--drivers/scsi/qla2xxx/qla_inline.h46
-rw-r--r--drivers/scsi/qla2xxx/qla_iocb.c84
-rw-r--r--drivers/scsi/qla2xxx/qla_isr.c15
-rw-r--r--drivers/scsi/qla2xxx/qla_mbx.c38
-rw-r--r--drivers/scsi/qla2xxx/qla_mr.c16
-rw-r--r--drivers/scsi/qla2xxx/qla_mr.h8
-rw-r--r--drivers/scsi/qla2xxx/qla_nvme.c10
-rw-r--r--drivers/scsi/qla2xxx/qla_nx2.c10
-rw-r--r--drivers/scsi/qla2xxx/qla_os.c224
-rw-r--r--drivers/scsi/qla2xxx/qla_sup.c9
-rw-r--r--drivers/scsi/qla2xxx/qla_target.c24
-rw-r--r--drivers/scsi/qla2xxx/qla_target.h2
-rw-r--r--drivers/scsi/qla2xxx/qla_version.h4
-rw-r--r--drivers/scsi/qla2xxx/tcm_qla2xxx.c23
-rw-r--r--drivers/scsi/qla4xxx/ql4_mbx.c4
-rw-r--r--drivers/scsi/qla4xxx/ql4_nx.c6
-rw-r--r--drivers/scsi/qla4xxx/ql4_os.c33
-rw-r--r--drivers/scsi/scsi.c13
-rw-r--r--drivers/scsi/scsi_debug.c249
-rw-r--r--drivers/scsi/scsi_debugfs.c1
-rw-r--r--drivers/scsi/scsi_dh.c2
-rw-r--r--drivers/scsi/scsi_error.c66
-rw-r--r--drivers/scsi/scsi_lib.c171
-rw-r--r--drivers/scsi/scsi_priv.h7
-rw-r--r--drivers/scsi/scsi_scan.c29
-rw-r--r--drivers/scsi/scsi_sysfs.c8
-rw-r--r--drivers/scsi/scsi_transport_iscsi.c28
-rw-r--r--drivers/scsi/scsi_transport_srp.c2
-rw-r--r--drivers/scsi/scsicam.c1
-rw-r--r--drivers/scsi/sd.c7
-rw-r--r--drivers/scsi/sd_zbc.c21
-rw-r--r--drivers/scsi/sg.c12
-rw-r--r--drivers/scsi/sim710.c14
-rw-r--r--drivers/scsi/smartpqi/smartpqi.h310
-rw-r--r--drivers/scsi/smartpqi/smartpqi_init.c3102
-rw-r--r--drivers/scsi/smartpqi/smartpqi_sas_transport.c39
-rw-r--r--drivers/scsi/smartpqi/smartpqi_sis.c9
-rw-r--r--drivers/scsi/smartpqi/smartpqi_sis.h1
-rw-r--r--drivers/scsi/sni_53c710.c6
-rw-r--r--drivers/scsi/snic/snic_debugfs.c20
-rw-r--r--drivers/scsi/sr_ioctl.c12
-rw-r--r--drivers/scsi/st.c22
-rw-r--r--drivers/scsi/st.h2
-rw-r--r--drivers/scsi/storvsc_drv.c84
-rw-r--r--drivers/scsi/sun3x_esp.c4
-rw-r--r--drivers/scsi/ufs/cdns-pltfrm.c4
-rw-r--r--drivers/scsi/ufs/ufs-debugfs.c90
-rw-r--r--drivers/scsi/ufs/ufs-debugfs.h2
-rw-r--r--drivers/scsi/ufs/ufs-exynos.c3
-rw-r--r--drivers/scsi/ufs/ufs-mediatek.c2
-rw-r--r--drivers/scsi/ufs/ufs-qcom.c21
-rw-r--r--drivers/scsi/ufs/ufs-sysfs.c14
-rw-r--r--drivers/scsi/ufs/ufs.h10
-rw-r--r--drivers/scsi/ufs/ufshcd-pci.c169
-rw-r--r--drivers/scsi/ufs/ufshcd-pltfrm.c8
-rw-r--r--drivers/scsi/ufs/ufshcd.c364
-rw-r--r--drivers/scsi/ufs/ufshcd.h30
-rw-r--r--drivers/scsi/ufs/ufshci.h40
-rw-r--r--drivers/scsi/vmw_pvscsi.c2
-rw-r--r--drivers/scsi/vmw_pvscsi.h2
-rw-r--r--drivers/sh/intc/core.c49
-rw-r--r--drivers/sh/maple/maple.c1
-rw-r--r--drivers/soc/aspeed/aspeed-lpc-ctrl.c20
-rw-r--r--drivers/soc/aspeed/aspeed-lpc-snoop.c27
-rw-r--r--drivers/soc/bcm/bcm63xx/bcm-pmb.c30
-rw-r--r--drivers/soc/bcm/raspberrypi-power.c2
-rw-r--r--drivers/soc/fsl/guts.c2
-rw-r--r--drivers/soc/fsl/qbman/bman.c1
-rw-r--r--drivers/soc/fsl/qbman/bman_portal.c3
-rw-r--r--drivers/soc/fsl/qbman/qman.c2
-rw-r--r--drivers/soc/fsl/qbman/qman_portal.c58
-rw-r--r--drivers/soc/fsl/qe/gpio.c20
-rw-r--r--drivers/soc/fsl/qe/qe.c24
-rw-r--r--drivers/soc/fsl/qe/qe_common.c3
-rw-r--r--drivers/soc/fsl/qe/qe_ic.c4
-rw-r--r--drivers/soc/fsl/qe/qe_io.c36
-rw-r--r--drivers/soc/fsl/qe/ucc_fast.c68
-rw-r--r--drivers/soc/fsl/qe/ucc_slow.c42
-rw-r--r--drivers/soc/fsl/rcpm.c24
-rw-r--r--drivers/soc/imx/soc-imx.c12
-rw-r--r--drivers/soc/litex/litex_soc_ctrl.c1
-rw-r--r--drivers/soc/mediatek/mt8167-mmsys.h35
-rw-r--r--drivers/soc/mediatek/mt8167-pm-domains.h7
-rw-r--r--drivers/soc/mediatek/mt8173-pm-domains.h10
-rw-r--r--drivers/soc/mediatek/mt8183-mmsys.h54
-rw-r--r--drivers/soc/mediatek/mt8183-pm-domains.h15
-rw-r--r--drivers/soc/mediatek/mt8192-pm-domains.h21
-rw-r--r--drivers/soc/mediatek/mtk-mmsys.c314
-rw-r--r--drivers/soc/mediatek/mtk-mmsys.h215
-rw-r--r--drivers/soc/mediatek/mtk-mutex.c52
-rw-r--r--drivers/soc/mediatek/mtk-pm-domains.c11
-rw-r--r--drivers/soc/mediatek/mtk-pm-domains.h2
-rw-r--r--drivers/soc/mediatek/mtk-pmic-wrap.c97
-rw-r--r--drivers/soc/qcom/llcc-qcom.c19
-rw-r--r--drivers/soc/qcom/mdt_loader.c17
-rw-r--r--drivers/soc/qcom/pdr_interface.c2
-rw-r--r--drivers/soc/qcom/qcom-geni-se.c77
-rw-r--r--drivers/soc/qcom/qcom_aoss.c1
-rw-r--r--drivers/soc/qcom/qmi_encdec.c8
-rw-r--r--drivers/soc/qcom/rpmh-rsc.c65
-rw-r--r--drivers/soc/qcom/rpmhpd.c56
-rw-r--r--drivers/soc/qcom/smem.c2
-rw-r--r--drivers/soc/qcom/wcnss_ctrl.c15
-rw-r--r--drivers/soc/renesas/rmobile-sysc.c4
-rw-r--r--drivers/soc/tegra/pmc.c263
-rw-r--r--drivers/soc/tegra/regulators-tegra30.c2
-rw-r--r--drivers/soc/ti/omap_prm.c24
-rw-r--r--drivers/soundwire/Makefile2
-rw-r--r--drivers/soundwire/bus.c100
-rw-r--r--drivers/soundwire/bus.h2
-rw-r--r--drivers/soundwire/bus_type.c15
-rw-r--r--drivers/soundwire/cadence_master.c16
-rw-r--r--drivers/soundwire/dmi-quirks.c96
-rw-r--r--drivers/soundwire/generic_bandwidth_allocation.c15
-rw-r--r--drivers/soundwire/intel.c24
-rw-r--r--drivers/soundwire/intel_init.c9
-rw-r--r--drivers/soundwire/qcom.c652
-rw-r--r--drivers/soundwire/slave.c9
-rw-r--r--drivers/soundwire/stream.c28
-rw-r--r--drivers/spi/spi-cadence-quadspi.c1
-rw-r--r--drivers/spi/spi-geni-qcom.c16
-rw-r--r--drivers/spi/spi-imx.c5
-rw-r--r--drivers/spi/spi-loopback-test.c3
-rw-r--r--drivers/spi/spi-qcom-qspi.c18
-rw-r--r--drivers/staging/Kconfig8
-rw-r--r--drivers/staging/Makefile4
-rw-r--r--drivers/staging/android/uapi/ashmem.h2
-rw-r--r--drivers/staging/axis-fifo/axis-fifo.c1
-rw-r--r--drivers/staging/clocking-wizard/clk-xlnx-clock-wizard.c369
-rw-r--r--drivers/staging/emxx_udc/emxx_udc.c1
-rw-r--r--drivers/staging/fbtft/fbtft-sysfs.c2
-rw-r--r--drivers/staging/fieldbus/anybuss/host.c24
-rw-r--r--drivers/staging/fsl-dpaa2/Kconfig19
-rw-r--r--drivers/staging/fsl-dpaa2/Makefile6
-rw-r--r--drivers/staging/fsl-dpaa2/ethsw/Makefile10
-rw-r--r--drivers/staging/fsl-dpaa2/ethsw/README106
-rw-r--r--drivers/staging/fsl-dpaa2/ethsw/TODO13
-rw-r--r--drivers/staging/fsl-dpaa2/ethsw/dpsw.h594
-rw-r--r--drivers/staging/fsl-dpaa2/ethsw/ethsw.c1839
-rw-r--r--drivers/staging/fsl-dpaa2/ethsw/ethsw.h80
-rw-r--r--drivers/staging/fwserial/fwserial.c25
-rw-r--r--drivers/staging/gasket/Kconfig25
-rw-r--r--drivers/staging/gasket/Makefile10
-rw-r--r--drivers/staging/gasket/TODO22
-rw-r--r--drivers/staging/gasket/apex.h30
-rw-r--r--drivers/staging/gasket/apex_driver.c726
-rw-r--r--drivers/staging/gasket/gasket.h122
-rw-r--r--drivers/staging/gasket/gasket_constants.h44
-rw-r--r--drivers/staging/gasket/gasket_core.c1815
-rw-r--r--drivers/staging/gasket/gasket_core.h638
-rw-r--r--drivers/staging/gasket/gasket_interrupt.c515
-rw-r--r--drivers/staging/gasket/gasket_interrupt.h95
-rw-r--r--drivers/staging/gasket/gasket_ioctl.c388
-rw-r--r--drivers/staging/gasket/gasket_ioctl.h28
-rw-r--r--drivers/staging/gasket/gasket_page_table.c1359
-rw-r--r--drivers/staging/gasket/gasket_page_table.h249
-rw-r--r--drivers/staging/gasket/gasket_sysfs.c398
-rw-r--r--drivers/staging/gasket/gasket_sysfs.h175
-rw-r--r--drivers/staging/gdm724x/gdm_tty.c2
-rw-r--r--drivers/staging/greybus/arche-platform.c10
-rw-r--r--drivers/staging/greybus/audio_module.c4
-rw-r--r--drivers/staging/greybus/audio_topology.c12
-rw-r--r--drivers/staging/greybus/camera.c13
-rw-r--r--drivers/staging/greybus/sdio.c1
-rw-r--r--drivers/staging/greybus/uart.c16
-rw-r--r--drivers/staging/hikey9xx/Kconfig11
-rw-r--r--drivers/staging/hikey9xx/Makefile1
-rw-r--r--drivers/staging/hikey9xx/hi6421-spmi-pmic.c8
-rw-r--r--drivers/staging/hikey9xx/hisilicon,hisi-spmi-controller.yaml18
-rw-r--r--drivers/staging/iio/TODO4
-rw-r--r--drivers/staging/iio/cdc/Kconfig10
-rw-r--r--drivers/staging/iio/cdc/Makefile3
-rw-r--r--drivers/staging/iio/cdc/ad7150.c655
-rw-r--r--drivers/staging/iio/frequency/ad9832.c4
-rw-r--r--drivers/staging/iio/frequency/ad9834.c67
-rw-r--r--drivers/staging/iio/impedance-analyzer/ad5933.c23
-rw-r--r--drivers/staging/kpc2000/kpc2000/pcie.h1
-rw-r--r--drivers/staging/kpc2000/kpc2000_i2c.c6
-rw-r--r--drivers/staging/kpc2000/kpc2000_spi.c2
-rw-r--r--drivers/staging/ks7010/ks_wlan_net.c6
-rw-r--r--drivers/staging/media/atomisp/Kconfig3
-rw-r--r--drivers/staging/media/atomisp/Makefile5
-rw-r--r--drivers/staging/media/atomisp/i2c/atomisp-gc0310.c16
-rw-r--r--drivers/staging/media/atomisp/i2c/atomisp-lm3554.c38
-rw-r--r--drivers/staging/media/atomisp/i2c/atomisp-ov2722.c2
-rw-r--r--drivers/staging/media/atomisp/i2c/gc2235.h2
-rw-r--r--drivers/staging/media/atomisp/pci/atomisp_cmd.c105
-rw-r--r--drivers/staging/media/atomisp/pci/atomisp_cmd.h2
-rw-r--r--drivers/staging/media/atomisp/pci/atomisp_compat.h10
-rw-r--r--drivers/staging/media/atomisp/pci/atomisp_compat_css20.c14
-rw-r--r--drivers/staging/media/atomisp/pci/atomisp_fops.c3
-rw-r--r--drivers/staging/media/atomisp/pci/atomisp_ioctl.c8
-rw-r--r--drivers/staging/media/atomisp/pci/hmm/hmm_bo.c13
-rw-r--r--drivers/staging/media/atomisp/pci/ia_css_firmware.h2
-rw-r--r--drivers/staging/media/atomisp/pci/ia_css_mipi.h17
-rw-r--r--drivers/staging/media/atomisp/pci/isp/kernels/sdis/sdis_1.0/ia_css_sdis.host.c3
-rw-r--r--drivers/staging/media/atomisp/pci/isp/modes/interface/isp_types.h41
-rw-r--r--drivers/staging/media/atomisp/pci/runtime/isp_param/src/isp_param.c3
-rw-r--r--drivers/staging/media/atomisp/pci/runtime/isys/src/virtual_isys.c12
-rw-r--r--drivers/staging/media/atomisp/pci/runtime/queue/src/queue.c44
-rw-r--r--drivers/staging/media/atomisp/pci/runtime/rmgr/src/rmgr_vbuf.c41
-rw-r--r--drivers/staging/media/atomisp/pci/runtime/spctrl/src/spctrl.c7
-rw-r--r--drivers/staging/media/atomisp/pci/runtime/timer/src/timer.c7
-rw-r--r--drivers/staging/media/atomisp/pci/sh_css.c15
-rw-r--r--drivers/staging/media/atomisp/pci/sh_css_firmware.c2
-rw-r--r--drivers/staging/media/atomisp/pci/sh_css_mipi.c11
-rw-r--r--drivers/staging/media/atomisp/pci/sh_css_params.c10
-rw-r--r--drivers/staging/media/hantro/hantro.h3
-rw-r--r--drivers/staging/media/hantro/hantro_drv.c2
-rw-r--r--drivers/staging/media/hantro/hantro_g1_vp8_dec.c55
-rw-r--r--drivers/staging/media/hantro/hantro_hw.h16
-rw-r--r--drivers/staging/media/hantro/hantro_vp8.c10
-rw-r--r--drivers/staging/media/hantro/rk3399_vpu_hw_vp8_dec.c55
-rw-r--r--drivers/staging/media/imx/Kconfig9
-rw-r--r--drivers/staging/media/imx/Makefile6
-rw-r--r--drivers/staging/media/imx/TODO9
-rw-r--r--drivers/staging/media/imx/imx-ic-prp.c4
-rw-r--r--drivers/staging/media/imx/imx-ic-prpencvf.c24
-rw-r--r--drivers/staging/media/imx/imx-media-capture.c687
-rw-r--r--drivers/staging/media/imx/imx-media-csc-scaler.c2
-rw-r--r--drivers/staging/media/imx/imx-media-csi.c33
-rw-r--r--drivers/staging/media/imx/imx-media-dev-common.c4
-rw-r--r--drivers/staging/media/imx/imx-media-utils.c23
-rw-r--r--drivers/staging/media/imx/imx-media-vdic.c7
-rw-r--r--drivers/staging/media/imx/imx-media.h12
-rw-r--r--drivers/staging/media/imx/imx6-mipi-csi2.c6
-rw-r--r--drivers/staging/media/imx/imx7-media-csi.c1014
-rw-r--r--drivers/staging/media/imx/imx7-mipi-csis.c419
-rw-r--r--drivers/staging/media/ipu3/include/intel-ipu3.h2
-rw-r--r--drivers/staging/media/ipu3/ipu3-css-fw.h1
-rw-r--r--drivers/staging/media/ipu3/ipu3-css-pool.h4
-rw-r--r--drivers/staging/media/ipu3/ipu3-v4l2.c36
-rw-r--r--drivers/staging/media/ipu3/ipu3.h5
-rw-r--r--drivers/staging/media/meson/vdec/codec_hevc_common.h4
-rw-r--r--drivers/staging/media/meson/vdec/vdec.c8
-rw-r--r--drivers/staging/media/omap4iss/iss.c11
-rw-r--r--drivers/staging/media/omap4iss/iss_regs.h16
-rw-r--r--drivers/staging/media/omap4iss/iss_video.c3
-rw-r--r--drivers/staging/media/rkvdec/rkvdec.c50
-rw-r--r--drivers/staging/media/rkvdec/rkvdec.h1
-rw-r--r--drivers/staging/media/sunxi/cedrus/cedrus.c2
-rw-r--r--drivers/staging/media/sunxi/cedrus/cedrus.h2
-rw-r--r--drivers/staging/media/sunxi/cedrus/cedrus_dec.c2
-rw-r--r--drivers/staging/media/sunxi/cedrus/cedrus_regs.h17
-rw-r--r--drivers/staging/media/sunxi/cedrus/cedrus_vp8.c113
-rw-r--r--drivers/staging/media/tegra-video/csi.h2
-rw-r--r--drivers/staging/media/tegra-video/vi.c6
-rw-r--r--drivers/staging/media/zoran/zoran_device.c48
-rw-r--r--drivers/staging/most/Kconfig2
-rw-r--r--drivers/staging/most/Makefile1
-rw-r--r--drivers/staging/most/sound/Kconfig14
-rw-r--r--drivers/staging/most/sound/Makefile4
-rw-r--r--drivers/staging/mt7621-dts/gbpc1.dts11
-rw-r--r--drivers/staging/mt7621-dts/mt7621.dtsi82
-rw-r--r--drivers/staging/mt7621-pci/pci-mt7621.c12
-rw-r--r--drivers/staging/netlogic/Kconfig1
-rw-r--r--drivers/staging/nvec/nvec_ps2.c2
-rw-r--r--drivers/staging/octeon-usb/octeon-hcd.c32
-rw-r--r--drivers/staging/octeon/ethernet-spi.c2
-rw-r--r--drivers/staging/octeon/ethernet.c19
-rw-r--r--drivers/staging/qlge/qlge_devlink.c10
-rw-r--r--drivers/staging/qlge/qlge_devlink.h2
-rw-r--r--drivers/staging/qlge/qlge_main.c17
-rw-r--r--drivers/staging/ralink-gdma/ralink-gdma.c5
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_ap.c11
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_cmd.c83
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_debug.c8
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_ieee80211.c6
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_ioctl_set.c8
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_mlme.c2
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_mlme_ext.c12
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_pwrctrl.c5
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_recv.c27
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_security.c78
-rw-r--r--drivers/staging/rtl8188eu/core/rtw_xmit.c7
-rw-r--r--drivers/staging/rtl8188eu/hal/odm_hwconfig.c11
-rw-r--r--drivers/staging/rtl8188eu/hal/rtl8188e_rxdesc.c12
-rw-r--r--drivers/staging/rtl8188eu/hal/rtl8188eu_recv.c2
-rw-r--r--drivers/staging/rtl8188eu/hal/usb_halinit.c5
-rw-r--r--drivers/staging/rtl8188eu/include/drv_types.h2
-rw-r--r--drivers/staging/rtl8188eu/include/odm_hwconfig.h8
-rw-r--r--drivers/staging/rtl8188eu/include/osdep_intf.h2
-rw-r--r--drivers/staging/rtl8188eu/include/osdep_service.h13
-rw-r--r--drivers/staging/rtl8188eu/include/recv_osdep.h2
-rw-r--r--drivers/staging/rtl8188eu/include/rtl8188e_hal.h4
-rw-r--r--drivers/staging/rtl8188eu/include/rtw_cmd.h20
-rw-r--r--drivers/staging/rtl8188eu/include/rtw_efuse.h20
-rw-r--r--drivers/staging/rtl8188eu/include/rtw_mlme_ext.h30
-rw-r--r--drivers/staging/rtl8188eu/include/rtw_pwrctrl.h2
-rw-r--r--drivers/staging/rtl8188eu/include/usb_ops_linux.h32
-rw-r--r--drivers/staging/rtl8188eu/include/wifi.h156
-rw-r--r--drivers/staging/rtl8188eu/include/wlan_bssdef.h2
-rw-r--r--drivers/staging/rtl8188eu/os_dep/ioctl_linux.c94
-rw-r--r--drivers/staging/rtl8188eu/os_dep/os_intfs.c22
-rw-r--r--drivers/staging/rtl8188eu/os_dep/osdep_service.c31
-rw-r--r--drivers/staging/rtl8188eu/os_dep/recv_linux.c3
-rw-r--r--drivers/staging/rtl8188eu/os_dep/rtw_android.c23
-rw-r--r--drivers/staging/rtl8188eu/os_dep/usb_intf.c84
-rw-r--r--drivers/staging/rtl8188eu/os_dep/usb_ops_linux.c171
-rw-r--r--drivers/staging/rtl8188eu/os_dep/xmit_linux.c2
-rw-r--r--drivers/staging/rtl8192e/Kconfig1
-rw-r--r--drivers/staging/rtl8192e/rtl8192e/r8192E_dev.c14
-rw-r--r--drivers/staging/rtl8192e/rtl8192e/rtl_core.c5
-rw-r--r--drivers/staging/rtl8192e/rtl8192e/rtl_dm.c4
-rw-r--r--drivers/staging/rtl8192e/rtl8192e/rtl_wx.c7
-rw-r--r--drivers/staging/rtl8192e/rtl819x_BA.h40
-rw-r--r--drivers/staging/rtl8192e/rtl819x_BAProc.c127
-rw-r--r--drivers/staging/rtl8192e/rtl819x_HT.h23
-rw-r--r--drivers/staging/rtl8192e/rtl819x_HTProc.c81
-rw-r--r--drivers/staging/rtl8192e/rtl819x_TS.h18
-rw-r--r--drivers/staging/rtl8192e/rtl819x_TSProc.c79
-rw-r--r--drivers/staging/rtl8192e/rtllib.h2
-rw-r--r--drivers/staging/rtl8192e/rtllib_crypt_tkip.c2
-rw-r--r--drivers/staging/rtl8192e/rtllib_rx.c172
-rw-r--r--drivers/staging/rtl8192e/rtllib_softmac.c14
-rw-r--r--drivers/staging/rtl8192e/rtllib_tx.c8
-rw-r--r--drivers/staging/rtl8192e/rtllib_wx.c76
-rw-r--r--drivers/staging/rtl8192u/ieee80211/ieee80211_rx.c42
-rw-r--r--drivers/staging/rtl8192u/ieee80211/ieee80211_softmac.c2
-rw-r--r--drivers/staging/rtl8192u/ieee80211/ieee80211_tx.c4
-rw-r--r--drivers/staging/rtl8192u/ieee80211/ieee80211_wx.c6
-rw-r--r--drivers/staging/rtl8192u/ieee80211/rtl819x_BAProc.c4
-rw-r--r--drivers/staging/rtl8192u/r8190_rtl8256.c7
-rw-r--r--drivers/staging/rtl8192u/r8192U.h6
-rw-r--r--drivers/staging/rtl8192u/r8192U_core.c5
-rw-r--r--drivers/staging/rtl8192u/r8192U_wx.c165
-rw-r--r--drivers/staging/rtl8712/drv_types.h2
-rw-r--r--drivers/staging/rtl8712/ieee80211.c18
-rw-r--r--drivers/staging/rtl8712/mlme_osdep.h4
-rw-r--r--drivers/staging/rtl8712/rtl8712_recv.h2
-rw-r--r--drivers/staging/rtl8712/rtl8712_wmac_regdef.h1
-rw-r--r--drivers/staging/rtl8712/rtl8712_xmit.c13
-rw-r--r--drivers/staging/rtl8712/rtl8712_xmit.h5
-rw-r--r--drivers/staging/rtl8712/rtl871x_cmd.c6
-rw-r--r--drivers/staging/rtl8712/rtl871x_cmd.h10
-rw-r--r--drivers/staging/rtl8712/rtl871x_event.h1
-rw-r--r--drivers/staging/rtl8712/rtl871x_ht.h2
-rw-r--r--drivers/staging/rtl8712/rtl871x_io.h1
-rw-r--r--drivers/staging/rtl8712/rtl871x_ioctl_linux.c22
-rw-r--r--drivers/staging/rtl8712/rtl871x_mlme.c28
-rw-r--r--drivers/staging/rtl8712/rtl871x_mlme.h1
-rw-r--r--drivers/staging/rtl8712/rtl871x_mp.c1
-rw-r--r--drivers/staging/rtl8712/rtl871x_mp.h1
-rw-r--r--drivers/staging/rtl8712/rtl871x_mp_ioctl.c3
-rw-r--r--drivers/staging/rtl8712/rtl871x_pwrctrl.h3
-rw-r--r--drivers/staging/rtl8712/rtl871x_recv.c36
-rw-r--r--drivers/staging/rtl8712/rtl871x_recv.h12
-rw-r--r--drivers/staging/rtl8712/rtl871x_security.c24
-rw-r--r--drivers/staging/rtl8712/rtl871x_security.h4
-rw-r--r--drivers/staging/rtl8712/rtl871x_xmit.c8
-rw-r--r--drivers/staging/rtl8712/rtl871x_xmit.h12
-rw-r--r--drivers/staging/rtl8712/sta_info.h1
-rw-r--r--drivers/staging/rtl8712/usb_ops.h6
-rw-r--r--drivers/staging/rtl8712/wifi.h265
-rw-r--r--drivers/staging/rtl8712/wlan_bssdef.h2
-rw-r--r--drivers/staging/rtl8712/xmit_osdep.h8
-rw-r--r--drivers/staging/rtl8723bs/TODO1
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_ap.c336
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_btcoex.c12
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_cmd.c184
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_debug.c61
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_eeprom.c56
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_efuse.c25
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_ieee80211.c156
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_io.c11
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_ioctl_set.c120
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_mlme.c350
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_mlme_ext.c832
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_odm.c60
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_pwrctrl.c316
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_recv.c939
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_security.c283
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_sta_mgt.c35
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_wlan_util.c407
-rw-r--r--drivers/staging/rtl8723bs/core/rtw_xmit.c399
-rw-r--r--drivers/staging/rtl8723bs/hal/Hal8723BReg.h435
-rw-r--r--drivers/staging/rtl8723bs/hal/HalBtc8723b1Ant.c258
-rw-r--r--drivers/staging/rtl8723bs/hal/HalBtc8723b1Ant.h54
-rw-r--r--drivers/staging/rtl8723bs/hal/HalBtc8723b2Ant.c224
-rw-r--r--drivers/staging/rtl8723bs/hal/HalBtc8723b2Ant.h48
-rw-r--r--drivers/staging/rtl8723bs/hal/HalBtcOutSrc.h188
-rw-r--r--drivers/staging/rtl8723bs/hal/HalHWImg8723B_BB.c10
-rw-r--r--drivers/staging/rtl8723bs/hal/HalHWImg8723B_BB.h6
-rw-r--r--drivers/staging/rtl8723bs/hal/HalHWImg8723B_MAC.c6
-rw-r--r--drivers/staging/rtl8723bs/hal/HalHWImg8723B_MAC.h2
-rw-r--r--drivers/staging/rtl8723bs/hal/HalHWImg8723B_RF.c12
-rw-r--r--drivers/staging/rtl8723bs/hal/HalHWImg8723B_RF.h6
-rw-r--r--drivers/staging/rtl8723bs/hal/HalPhyRf.c8
-rw-r--r--drivers/staging/rtl8723bs/hal/HalPhyRf.h25
-rw-r--r--drivers/staging/rtl8723bs/hal/HalPhyRf_8723B.c70
-rw-r--r--drivers/staging/rtl8723bs/hal/HalPhyRf_8723B.h12
-rw-r--r--drivers/staging/rtl8723bs/hal/HalPwrSeqCmd.c59
-rw-r--r--drivers/staging/rtl8723bs/hal/hal_btcoex.c171
-rw-r--r--drivers/staging/rtl8723bs/hal/hal_com.c233
-rw-r--r--drivers/staging/rtl8723bs/hal/hal_com_phycfg.c250
-rw-r--r--drivers/staging/rtl8723bs/hal/hal_intf.c39
-rw-r--r--drivers/staging/rtl8723bs/hal/hal_pwr_seq.c22
-rw-r--r--drivers/staging/rtl8723bs/hal/odm.c1106
-rw-r--r--drivers/staging/rtl8723bs/hal/odm.h412
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_CfoTracking.c28
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_CfoTracking.h4
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_DIG.c66
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_DIG.h32
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_DynamicBBPowerSaving.c8
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_DynamicBBPowerSaving.h4
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_DynamicTxPower.c2
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_EdcaTurboCheck.c6
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_EdcaTurboCheck.h4
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_HWConfig.c35
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_HWConfig.h79
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_NoiseMonitor.c2
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_NoiseMonitor.h4
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_PathDiv.c4
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_RegConfig8723B.c16
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_RegConfig8723B.h16
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_debug.c2
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_debug.h4
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_interface.h11
-rw-r--r--drivers/staging/rtl8723bs/hal/odm_types.h47
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723b_cmd.c1392
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723b_dm.c26
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723b_hal_init.c713
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723b_phycfg.c74
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723b_rf6052.c4
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723b_rxdesc.c3
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723bs_recv.c35
-rw-r--r--drivers/staging/rtl8723bs/hal/rtl8723bs_xmit.c100
-rw-r--r--drivers/staging/rtl8723bs/hal/sdio_halinit.c561
-rw-r--r--drivers/staging/rtl8723bs/hal/sdio_ops.c201
-rw-r--r--drivers/staging/rtl8723bs/include/Hal8192CPhyReg.h8
-rw-r--r--drivers/staging/rtl8723bs/include/HalPwrSeqCmd.h12
-rw-r--r--drivers/staging/rtl8723bs/include/HalVerDef.h62
-rw-r--r--drivers/staging/rtl8723bs/include/autoconf.h10
-rw-r--r--drivers/staging/rtl8723bs/include/basic_types.h6
-rw-r--r--drivers/staging/rtl8723bs/include/drv_conf.h10
-rw-r--r--drivers/staging/rtl8723bs/include/drv_types.h215
-rw-r--r--drivers/staging/rtl8723bs/include/drv_types_sdio.h5
-rw-r--r--drivers/staging/rtl8723bs/include/hal_btcoex.h4
-rw-r--r--drivers/staging/rtl8723bs/include/hal_com.h52
-rw-r--r--drivers/staging/rtl8723bs/include/hal_com_h2c.h185
-rw-r--r--drivers/staging/rtl8723bs/include/hal_com_phycfg.h40
-rw-r--r--drivers/staging/rtl8723bs/include/hal_com_reg.h48
-rw-r--r--drivers/staging/rtl8723bs/include/hal_data.h50
-rw-r--r--drivers/staging/rtl8723bs/include/hal_intf.h72
-rw-r--r--drivers/staging/rtl8723bs/include/hal_phy.h85
-rw-r--r--drivers/staging/rtl8723bs/include/hal_phy_cfg.h6
-rw-r--r--drivers/staging/rtl8723bs/include/hal_pwr_seq.h24
-rw-r--r--drivers/staging/rtl8723bs/include/ieee80211.h202
-rw-r--r--drivers/staging/rtl8723bs/include/ioctl_cfg80211.h6
-rw-r--r--drivers/staging/rtl8723bs/include/osdep_intf.h12
-rw-r--r--drivers/staging/rtl8723bs/include/osdep_service.h39
-rw-r--r--drivers/staging/rtl8723bs/include/osdep_service_linux.h29
-rw-r--r--drivers/staging/rtl8723bs/include/recv_osdep.h8
-rw-r--r--drivers/staging/rtl8723bs/include/rtl8723b_cmd.h8
-rw-r--r--drivers/staging/rtl8723bs/include/rtl8723b_hal.h48
-rw-r--r--drivers/staging/rtl8723bs/include/rtl8723b_recv.h44
-rw-r--r--drivers/staging/rtl8723bs/include/rtl8723b_rf.h2
-rw-r--r--drivers/staging/rtl8723bs/include/rtl8723b_spec.h10
-rw-r--r--drivers/staging/rtl8723bs/include/rtl8723b_xmit.h8
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_cmd.h271
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_debug.h92
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_eeprom.h4
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_efuse.h10
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_event.h11
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_ht.h12
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_io.h113
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_ioctl_set.h16
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_mlme.h142
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_mlme_ext.h90
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_mp.h137
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_odm.h2
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_pwrctrl.h95
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_recv.h66
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_rf.h36
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_security.h58
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_wifi_regd.h12
-rw-r--r--drivers/staging/rtl8723bs/include/rtw_xmit.h62
-rw-r--r--drivers/staging/rtl8723bs/include/sdio_ops.h7
-rw-r--r--drivers/staging/rtl8723bs/include/sta_info.h28
-rw-r--r--drivers/staging/rtl8723bs/include/wifi.h101
-rw-r--r--drivers/staging/rtl8723bs/include/wlan_bssdef.h51
-rw-r--r--drivers/staging/rtl8723bs/include/xmit_osdep.h14
-rw-r--r--drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c557
-rw-r--r--drivers/staging/rtl8723bs/os_dep/ioctl_linux.c752
-rw-r--r--drivers/staging/rtl8723bs/os_dep/mlme_linux.c24
-rw-r--r--drivers/staging/rtl8723bs/os_dep/os_intfs.c568
-rw-r--r--drivers/staging/rtl8723bs/os_dep/osdep_service.c8
-rw-r--r--drivers/staging/rtl8723bs/os_dep/recv_linux.c96
-rw-r--r--drivers/staging/rtl8723bs/os_dep/sdio_intf.c217
-rw-r--r--drivers/staging/rtl8723bs/os_dep/sdio_ops_linux.c97
-rw-r--r--drivers/staging/rtl8723bs/os_dep/wifi_regd.c4
-rw-r--r--drivers/staging/rtl8723bs/os_dep/xmit_linux.c62
-rw-r--r--drivers/staging/rts5208/xd.c2
-rw-r--r--drivers/staging/sm750fb/sm750.h32
-rw-r--r--drivers/staging/unisys/visornic/visornic_main.c34
-rw-r--r--drivers/staging/vc04_services/bcm2835-audio/bcm2835-pcm.c10
-rw-r--r--drivers/staging/vc04_services/interface/TODO6
-rw-r--r--drivers/staging/vc04_services/interface/vchiq_arm/vchiq_arm.c10
-rw-r--r--drivers/staging/vc04_services/interface/vchiq_arm/vchiq_core.c360
-rw-r--r--drivers/staging/vc04_services/interface/vchiq_arm/vchiq_debugfs.h2
-rw-r--r--drivers/staging/vc04_services/vchiq-mmal/mmal-vchiq.c4
-rw-r--r--drivers/staging/vt6655/baseband.c6
-rw-r--r--drivers/staging/vt6655/channel.c42
-rw-r--r--drivers/staging/vt6655/rxtx.h4
-rw-r--r--drivers/staging/vt6656/card.c3
-rw-r--r--drivers/staging/wfx/bh.c1
-rw-r--r--drivers/staging/wfx/bh.h4
-rw-r--r--drivers/staging/wfx/bus.h3
-rw-r--r--drivers/staging/wfx/bus_sdio.c6
-rw-r--r--drivers/staging/wfx/bus_spi.c7
-rw-r--r--drivers/staging/wfx/data_rx.c5
-rw-r--r--drivers/staging/wfx/data_tx.c5
-rw-r--r--drivers/staging/wfx/data_tx.h3
-rw-r--r--drivers/staging/wfx/debug.c6
-rw-r--r--drivers/staging/wfx/fwio.c2
-rw-r--r--drivers/staging/wfx/hif_api_cmd.h4
-rw-r--r--drivers/staging/wfx/hif_api_general.h9
-rw-r--r--drivers/staging/wfx/hif_rx.c6
-rw-r--r--drivers/staging/wfx/hif_tx.c4
-rw-r--r--drivers/staging/wfx/hif_tx_mib.c5
-rw-r--r--drivers/staging/wfx/hwio.c3
-rw-r--r--drivers/staging/wfx/hwio.h2
-rw-r--r--drivers/staging/wfx/key.c2
-rw-r--r--drivers/staging/wfx/key.h2
-rw-r--r--drivers/staging/wfx/main.c14
-rw-r--r--drivers/staging/wfx/main.h3
-rw-r--r--drivers/staging/wfx/queue.c4
-rw-r--r--drivers/staging/wfx/queue.h3
-rw-r--r--drivers/staging/wfx/scan.h2
-rw-r--r--drivers/staging/wfx/sta.c8
-rw-r--r--drivers/staging/wfx/sta.h2
-rw-r--r--drivers/staging/wfx/traces.h3
-rw-r--r--drivers/staging/wfx/wfx.h3
-rw-r--r--drivers/staging/wimax/Documentation/i2400m.rst283
-rw-r--r--drivers/staging/wimax/Documentation/index.rst19
-rw-r--r--drivers/staging/wimax/Documentation/wimax.rst89
-rw-r--r--drivers/staging/wimax/Kconfig47
-rw-r--r--drivers/staging/wimax/Makefile15
-rw-r--r--drivers/staging/wimax/TODO18
-rw-r--r--drivers/staging/wimax/debug-levels.h29
-rw-r--r--drivers/staging/wimax/debugfs.c38
-rw-r--r--drivers/staging/wimax/i2400m/Kconfig37
-rw-r--r--drivers/staging/wimax/i2400m/Makefile23
-rw-r--r--drivers/staging/wimax/i2400m/control.c1434
-rw-r--r--drivers/staging/wimax/i2400m/debug-levels.h32
-rw-r--r--drivers/staging/wimax/i2400m/debugfs.c253
-rw-r--r--drivers/staging/wimax/i2400m/driver.c1003
-rw-r--r--drivers/staging/wimax/i2400m/fw.c1666
-rw-r--r--drivers/staging/wimax/i2400m/i2400m-usb.h275
-rw-r--r--drivers/staging/wimax/i2400m/i2400m.h970
-rw-r--r--drivers/staging/wimax/i2400m/linux-wimax-i2400m.h572
-rw-r--r--drivers/staging/wimax/i2400m/netdev.c603
-rw-r--r--drivers/staging/wimax/i2400m/op-rfkill.c196
-rw-r--r--drivers/staging/wimax/i2400m/rx.c1394
-rw-r--r--drivers/staging/wimax/i2400m/sysfs.c65
-rw-r--r--drivers/staging/wimax/i2400m/tx.c1015
-rw-r--r--drivers/staging/wimax/i2400m/usb-debug-levels.h28
-rw-r--r--drivers/staging/wimax/i2400m/usb-fw.c365
-rw-r--r--drivers/staging/wimax/i2400m/usb-notif.c258
-rw-r--r--drivers/staging/wimax/i2400m/usb-rx.c462
-rw-r--r--drivers/staging/wimax/i2400m/usb-tx.c273
-rw-r--r--drivers/staging/wimax/i2400m/usb.c765
-rw-r--r--drivers/staging/wimax/id-table.c130
-rw-r--r--drivers/staging/wimax/linux-wimax-debug.h491
-rw-r--r--drivers/staging/wimax/linux-wimax.h239
-rw-r--r--drivers/staging/wimax/net-wimax.h503
-rw-r--r--drivers/staging/wimax/op-msg.c391
-rw-r--r--drivers/staging/wimax/op-reset.c108
-rw-r--r--drivers/staging/wimax/op-rfkill.c431
-rw-r--r--drivers/staging/wimax/op-state-get.c52
-rw-r--r--drivers/staging/wimax/stack.c616
-rw-r--r--drivers/staging/wimax/wimax-internal.h85
-rw-r--r--drivers/staging/wlan-ng/p80211conv.h3
-rw-r--r--drivers/staging/wlan-ng/p80211netdev.c2
-rw-r--r--drivers/target/iscsi/iscsi_target.c23
-rw-r--r--drivers/target/iscsi/iscsi_target_configfs.c3
-rw-r--r--drivers/target/iscsi/iscsi_target_nego.c1
-rw-r--r--drivers/target/iscsi/iscsi_target_stat.c1
-rw-r--r--drivers/target/iscsi/iscsi_target_util.c17
-rw-r--r--drivers/target/loopback/tcm_loop.c60
-rw-r--r--drivers/target/loopback/tcm_loop.h1
-rw-r--r--drivers/target/sbp/sbp_target.c8
-rw-r--r--drivers/target/target_core_configfs.c36
-rw-r--r--drivers/target/target_core_device.c12
-rw-r--r--drivers/target/target_core_fabric_configfs.c58
-rw-r--r--drivers/target/target_core_file.c3
-rw-r--r--drivers/target/target_core_iblock.c81
-rw-r--r--drivers/target/target_core_iblock.h10
-rw-r--r--drivers/target/target_core_internal.h2
-rw-r--r--drivers/target/target_core_pr.c42
-rw-r--r--drivers/target/target_core_pscsi.c20
-rw-r--r--drivers/target/target_core_rd.c27
-rw-r--r--drivers/target/target_core_rd.h1
-rw-r--r--drivers/target/target_core_sbc.c4
-rw-r--r--drivers/target/target_core_spc.c6
-rw-r--r--drivers/target/target_core_stat.c3
-rw-r--r--drivers/target/target_core_tmr.c4
-rw-r--r--drivers/target/target_core_transport.c300
-rw-r--r--drivers/target/target_core_user.c444
-rw-r--r--drivers/target/target_core_xcopy.c10
-rw-r--r--drivers/target/tcm_fc/tfc_cmd.c14
-rw-r--r--drivers/target/tcm_fc/tfc_sess.c2
-rw-r--r--drivers/tee/optee/Makefile3
-rw-r--r--drivers/tee/optee/call.c4
-rw-r--r--drivers/tee/optee/core.c11
-rw-r--r--drivers/tee/optee/optee_trace.h67
-rw-r--r--drivers/thermal/amlogic_thermal.c4
-rw-r--r--drivers/thermal/broadcom/bcm2835_thermal.c1
-rw-r--r--drivers/thermal/cpufreq_cooling.c49
-rw-r--r--drivers/thermal/cpuidle_cooling.c37
-rw-r--r--drivers/thermal/devfreq_cooling.c25
-rw-r--r--drivers/thermal/gov_fair_share.c11
-rw-r--r--drivers/thermal/gov_power_allocator.c32
-rw-r--r--drivers/thermal/hisi_thermal.c10
-rw-r--r--drivers/thermal/intel/Kconfig11
-rw-r--r--drivers/thermal/intel/Makefile1
-rw-r--r--drivers/thermal/intel/intel_tcc_cooling.c129
-rw-r--r--drivers/thermal/mtk_thermal.c12
-rw-r--r--drivers/thermal/qcom/qcom-spmi-temp-alarm.c91
-rw-r--r--drivers/thermal/qcom/tsens-8960.c235
-rw-r--r--drivers/thermal/qcom/tsens-v0_1.c98
-rw-r--r--drivers/thermal/qcom/tsens-v1.c4
-rw-r--r--drivers/thermal/qcom/tsens.c165
-rw-r--r--drivers/thermal/qcom/tsens.h6
-rw-r--r--drivers/thermal/rcar_gen3_thermal.c3
-rw-r--r--drivers/thermal/sun8i_thermal.c4
-rw-r--r--drivers/thermal/tegra/soctherm.c15
-rw-r--r--drivers/thermal/thermal_core.c57
-rw-r--r--drivers/thermal/thermal_core.h1
-rw-r--r--drivers/thermal/thermal_helpers.c27
-rw-r--r--drivers/thermal/thermal_mmio.c5
-rw-r--r--drivers/thermal/thermal_of.c7
-rw-r--r--drivers/thermal/thermal_sysfs.c3
-rw-r--r--drivers/thermal/ti-soc-thermal/ti-bandgap.c37
-rw-r--r--drivers/thunderbolt/ctl.c21
-rw-r--r--drivers/thunderbolt/ctl.h8
-rw-r--r--drivers/thunderbolt/debugfs.c37
-rw-r--r--drivers/thunderbolt/dma_test.c35
-rw-r--r--drivers/thunderbolt/domain.c89
-rw-r--r--drivers/thunderbolt/eeprom.c105
-rw-r--r--drivers/thunderbolt/icm.c34
-rw-r--r--drivers/thunderbolt/property.c71
-rw-r--r--drivers/thunderbolt/retimer.c4
-rw-r--r--drivers/thunderbolt/switch.c93
-rw-r--r--drivers/thunderbolt/tb.c56
-rw-r--r--drivers/thunderbolt/tb.h45
-rw-r--r--drivers/thunderbolt/test.c492
-rw-r--r--drivers/thunderbolt/tunnel.c102
-rw-r--r--drivers/thunderbolt/tunnel.h8
-rw-r--r--drivers/thunderbolt/xdomain.c416
-rw-r--r--drivers/tty/Kconfig86
-rw-r--r--drivers/tty/Makefile3
-rw-r--r--drivers/tty/amiserial.c33
-rw-r--r--drivers/tty/cyclades.c4119
-rw-r--r--drivers/tty/hvc/hvc_udbg.c2
-rw-r--r--drivers/tty/hvc/hvcs.c28
-rw-r--r--drivers/tty/ipwireless/tty.c11
-rw-r--r--drivers/tty/isicom.c1699
-rw-r--r--drivers/tty/moxa.c25
-rw-r--r--drivers/tty/mxser.c38
-rw-r--r--drivers/tty/n_gsm.c37
-rw-r--r--drivers/tty/n_hdlc.c2
-rw-r--r--drivers/tty/n_null.c1
-rw-r--r--drivers/tty/n_r3964.c1
-rw-r--r--drivers/tty/n_tty.c3
-rw-r--r--drivers/tty/nozomi.c26
-rw-r--r--drivers/tty/pty.c20
-rw-r--r--drivers/tty/rocket.c3127
-rw-r--r--drivers/tty/rocket.h111
-rw-r--r--drivers/tty/rocket_int.h1214
-rw-r--r--drivers/tty/serial/8250/8250_aspeed_vuart.c107
-rw-r--r--drivers/tty/serial/8250/8250_bcm7271.c1202
-rw-r--r--drivers/tty/serial/8250/8250_exar.c17
-rw-r--r--drivers/tty/serial/8250/8250_fsl.c16
-rw-r--r--drivers/tty/serial/8250/8250_of.c1
-rw-r--r--drivers/tty/serial/8250/8250_omap.c6
-rw-r--r--drivers/tty/serial/8250/8250_port.c30
-rw-r--r--drivers/tty/serial/8250/Kconfig21
-rw-r--r--drivers/tty/serial/8250/Makefile1
-rw-r--r--drivers/tty/serial/8250/serial_cs.c12
-rw-r--r--drivers/tty/serial/Kconfig21
-rw-r--r--drivers/tty/serial/altera_jtaguart.c2
-rw-r--r--drivers/tty/serial/altera_uart.c2
-rw-r--r--drivers/tty/serial/amba-pl010.c2
-rw-r--r--drivers/tty/serial/amba-pl011.c2
-rw-r--r--drivers/tty/serial/apbuart.c2
-rw-r--r--drivers/tty/serial/ar933x_uart.c2
-rw-r--r--drivers/tty/serial/arc_uart.c2
-rw-r--r--drivers/tty/serial/atmel_serial.c18
-rw-r--r--drivers/tty/serial/bcm63xx_uart.c2
-rw-r--r--drivers/tty/serial/icom.c4
-rw-r--r--drivers/tty/serial/imx.c16
-rw-r--r--drivers/tty/serial/jsm/jsm_driver.c1
-rw-r--r--drivers/tty/serial/jsm/jsm_tty.c24
-rw-r--r--drivers/tty/serial/kgdb_nmi.c4
-rw-r--r--drivers/tty/serial/liteuart.c4
-rw-r--r--drivers/tty/serial/lpc32xx_hs.c2
-rw-r--r--drivers/tty/serial/max310x.c31
-rw-r--r--drivers/tty/serial/mcf.c2
-rw-r--r--drivers/tty/serial/meson_uart.c2
-rw-r--r--drivers/tty/serial/mpc52xx_uart.c2
-rw-r--r--drivers/tty/serial/msm_serial.c4
-rw-r--r--drivers/tty/serial/omap-serial.c51
-rw-r--r--drivers/tty/serial/owl-uart.c2
-rw-r--r--drivers/tty/serial/pch_uart.c22
-rw-r--r--drivers/tty/serial/qcom_geni_serial.c36
-rw-r--r--drivers/tty/serial/rda-uart.c2
-rw-r--r--drivers/tty/serial/rp2.c2
-rw-r--r--drivers/tty/serial/sa1100.c2
-rw-r--r--drivers/tty/serial/samsung_tty.c507
-rw-r--r--drivers/tty/serial/sc16is7xx.c2
-rw-r--r--drivers/tty/serial/serial_core.c8
-rw-r--r--drivers/tty/serial/serial_txx9.c4
-rw-r--r--drivers/tty/serial/sh-sci.c15
-rw-r--r--drivers/tty/serial/sifive.c2
-rw-r--r--drivers/tty/serial/stm32-usart.c310
-rw-r--r--drivers/tty/serial/stm32-usart.h17
-rw-r--r--drivers/tty/serial/sunsu.c4
-rw-r--r--drivers/tty/serial/tegra-tcu.c1
-rw-r--r--drivers/tty/serial/timbuart.c2
-rw-r--r--drivers/tty/serial/ucc_uart.c124
-rw-r--r--drivers/tty/serial/vt8500_serial.c2
-rw-r--r--drivers/tty/serial/xilinx_uartps.c3
-rw-r--r--drivers/tty/synclink_gt.c66
-rw-r--r--drivers/tty/sysrq.c35
-rw-r--r--drivers/tty/tty.h114
-rw-r--r--drivers/tty/tty_audit.c1
-rw-r--r--drivers/tty/tty_baudrate.c1
-rw-r--r--drivers/tty/tty_buffer.c2
-rw-r--r--drivers/tty/tty_io.c72
-rw-r--r--drivers/tty/tty_ioctl.c8
-rw-r--r--drivers/tty/tty_jobctrl.c30
-rw-r--r--drivers/tty/tty_ldisc.c42
-rw-r--r--drivers/tty/tty_mutex.c1
-rw-r--r--drivers/tty/tty_port.c1
-rw-r--r--drivers/tty/vcc.c72
-rw-r--r--drivers/tty/vt/.gitignore6
-rw-r--r--drivers/tty/vt/keyboard.c2
-rw-r--r--drivers/tty/vt/vt.c5
-rw-r--r--drivers/tty/vt/vt_ioctl.c57
-rw-r--r--drivers/uio/Kconfig17
-rw-r--r--drivers/uio/Makefile1
-rw-r--r--drivers/uio/uio_dfl.c66
-rw-r--r--drivers/usb/cdns3/cdns3-gadget.c73
-rw-r--r--drivers/usb/cdns3/cdns3-gadget.h3
-rw-r--r--drivers/usb/cdns3/cdns3-imx.c34
-rw-r--r--drivers/usb/cdns3/cdns3-plat.c23
-rw-r--r--drivers/usb/cdns3/cdns3-trace.h5
-rw-r--r--drivers/usb/cdns3/cdnsp-gadget.c21
-rw-r--r--drivers/usb/cdns3/cdnsp-gadget.h1
-rw-r--r--drivers/usb/cdns3/cdnsp-mem.c3
-rw-r--r--drivers/usb/cdns3/cdnsp-ring.c5
-rw-r--r--drivers/usb/cdns3/core.c29
-rw-r--r--drivers/usb/cdns3/drd.c15
-rw-r--r--drivers/usb/cdns3/drd.h2
-rw-r--r--drivers/usb/chipidea/ci_hdrc_tegra.c8
-rw-r--r--drivers/usb/chipidea/host.c2
-rw-r--r--drivers/usb/class/cdc-acm.c156
-rw-r--r--drivers/usb/class/cdc-wdm.c30
-rw-r--r--drivers/usb/class/usblp.c16
-rw-r--r--drivers/usb/common/common.c76
-rw-r--r--drivers/usb/common/debug.c22
-rw-r--r--drivers/usb/core/devices.c21
-rw-r--r--drivers/usb/core/driver.c10
-rw-r--r--drivers/usb/core/endpoint.c35
-rw-r--r--drivers/usb/core/hcd.c6
-rw-r--r--drivers/usb/core/hub.c105
-rw-r--r--drivers/usb/core/hub.h6
-rw-r--r--drivers/usb/core/quirks.c8
-rw-r--r--drivers/usb/core/sysfs.c5
-rw-r--r--drivers/usb/core/usb.c86
-rw-r--r--drivers/usb/dwc2/core.c138
-rw-r--r--drivers/usb/dwc2/core.h51
-rw-r--r--drivers/usb/dwc2/core_intr.c279
-rw-r--r--drivers/usb/dwc2/debugfs.c2
-rw-r--r--drivers/usb/dwc2/gadget.c226
-rw-r--r--drivers/usb/dwc2/hcd.c646
-rw-r--r--drivers/usb/dwc2/hcd_queue.c2
-rw-r--r--drivers/usb/dwc2/hw.h1
-rw-r--r--drivers/usb/dwc2/params.c18
-rw-r--r--drivers/usb/dwc2/platform.c40
-rw-r--r--drivers/usb/dwc3/Kconfig9
-rw-r--r--drivers/usb/dwc3/Makefile1
-rw-r--r--drivers/usb/dwc3/core.c54
-rw-r--r--drivers/usb/dwc3/core.h60
-rw-r--r--drivers/usb/dwc3/debug.h10
-rw-r--r--drivers/usb/dwc3/debugfs.c14
-rw-r--r--drivers/usb/dwc3/dwc3-exynos.c2
-rw-r--r--drivers/usb/dwc3/dwc3-imx8mp.c5
-rw-r--r--drivers/usb/dwc3/dwc3-keystone.c2
-rw-r--r--drivers/usb/dwc3/dwc3-of-simple.c1
-rw-r--r--drivers/usb/dwc3/dwc3-omap.c5
-rw-r--r--drivers/usb/dwc3/dwc3-pci.c9
-rw-r--r--drivers/usb/dwc3/dwc3-qcom.c26
-rw-r--r--drivers/usb/dwc3/dwc3-st.c2
-rw-r--r--drivers/usb/dwc3/dwc3-xilinx.c337
-rw-r--r--drivers/usb/dwc3/gadget.c158
-rw-r--r--drivers/usb/dwc3/gadget.h6
-rw-r--r--drivers/usb/dwc3/io.h2
-rw-r--r--drivers/usb/dwc3/trace.c2
-rw-r--r--drivers/usb/dwc3/trace.h8
-rw-r--r--drivers/usb/gadget/config.c4
-rw-r--r--drivers/usb/gadget/configfs.c14
-rw-r--r--drivers/usb/gadget/function/f_fs.c14
-rw-r--r--drivers/usb/gadget/function/f_mass_storage.c2
-rw-r--r--drivers/usb/gadget/function/f_printer.c2
-rw-r--r--drivers/usb/gadget/function/f_tcm.c36
-rw-r--r--drivers/usb/gadget/function/f_uac1.c273
-rw-r--r--drivers/usb/gadget/function/f_uac2.c265
-rw-r--r--drivers/usb/gadget/function/f_uvc.c8
-rw-r--r--drivers/usb/gadget/function/u_audio.c6
-rw-r--r--drivers/usb/gadget/function/u_ether_configfs.h5
-rw-r--r--drivers/usb/gadget/function/uvc_configfs.c2
-rw-r--r--drivers/usb/gadget/legacy/inode.c3
-rw-r--r--drivers/usb/gadget/legacy/mass_storage.c4
-rw-r--r--drivers/usb/gadget/legacy/multi.c14
-rw-r--r--drivers/usb/gadget/legacy/webcam.c3
-rw-r--r--drivers/usb/gadget/udc/amd5536udc_pci.c10
-rw-r--r--drivers/usb/gadget/udc/aspeed-vhub/core.c3
-rw-r--r--drivers/usb/gadget/udc/aspeed-vhub/epn.c2
-rw-r--r--drivers/usb/gadget/udc/dummy_hcd.c25
-rw-r--r--drivers/usb/gadget/udc/fotg210-udc.c28
-rw-r--r--drivers/usb/gadget/udc/net2272.c3
-rw-r--r--drivers/usb/gadget/udc/pch_udc.c241
-rw-r--r--drivers/usb/gadget/udc/r8a66597-udc.c2
-rw-r--r--drivers/usb/gadget/udc/s3c2410_udc.c28
-rw-r--r--drivers/usb/gadget/udc/snps_udc_plat.c4
-rw-r--r--drivers/usb/gadget/udc/tegra-xudc.c2
-rw-r--r--drivers/usb/host/Kconfig1
-rw-r--r--drivers/usb/host/Makefile6
-rw-r--r--drivers/usb/host/ehci-hcd.c15
-rw-r--r--drivers/usb/host/ehci-hub.c4
-rw-r--r--drivers/usb/host/ehci-pci.c3
-rw-r--r--drivers/usb/host/ehci-platform.c5
-rw-r--r--drivers/usb/host/ehci.h1
-rw-r--r--drivers/usb/host/fotg210-hcd.c16
-rw-r--r--drivers/usb/host/isp116x-hcd.c7
-rw-r--r--drivers/usb/host/isp116x.h1
-rw-r--r--drivers/usb/host/isp1362-hcd.c8
-rw-r--r--drivers/usb/host/isp1362.h1
-rw-r--r--drivers/usb/host/sl811-hcd.c16
-rw-r--r--drivers/usb/host/sl811.h1
-rw-r--r--drivers/usb/host/sl811_cs.c2
-rw-r--r--drivers/usb/host/u132-hcd.c3
-rw-r--r--drivers/usb/host/uhci-hcd.c12
-rw-r--r--drivers/usb/host/uhci-hcd.h4
-rw-r--r--drivers/usb/host/xhci-ext-caps.h5
-rw-r--r--drivers/usb/host/xhci-hub.c320
-rw-r--r--drivers/usb/host/xhci-mem.c14
-rw-r--r--drivers/usb/host/xhci-mtk-sch.c369
-rw-r--r--drivers/usb/host/xhci-mtk.c299
-rw-r--r--drivers/usb/host/xhci-mtk.h60
-rw-r--r--drivers/usb/host/xhci-pci.c21
-rw-r--r--drivers/usb/host/xhci-ring.c67
-rw-r--r--drivers/usb/host/xhci.c130
-rw-r--r--drivers/usb/host/xhci.h9
-rw-r--r--drivers/usb/misc/adutux.c6
-rw-r--r--drivers/usb/misc/ehset.c76
-rw-r--r--drivers/usb/misc/ezusb.c16
-rw-r--r--drivers/usb/misc/ldusb.c1
-rw-r--r--drivers/usb/misc/usbsevseg.c60
-rw-r--r--drivers/usb/mtu3/mtu3_host.c30
-rw-r--r--drivers/usb/mtu3/mtu3_plat.c7
-rw-r--r--drivers/usb/musb/mediatek.c2
-rw-r--r--drivers/usb/musb/musb_core.c14
-rw-r--r--drivers/usb/musb/musb_core.h4
-rw-r--r--drivers/usb/renesas_usbhs/fifo.c4
-rw-r--r--drivers/usb/renesas_usbhs/pipe.c2
-rw-r--r--drivers/usb/roles/class.c2
-rw-r--r--drivers/usb/serial/ark3116.c13
-rw-r--r--drivers/usb/serial/ch341.c1
-rw-r--r--drivers/usb/serial/cp210x.c34
-rw-r--r--drivers/usb/serial/f81232.c12
-rw-r--r--drivers/usb/serial/f81534.c7
-rw-r--r--drivers/usb/serial/ftdi_sio.c35
-rw-r--r--drivers/usb/serial/io_edgeport.c93
-rw-r--r--drivers/usb/serial/io_edgeport.h68
-rw-r--r--drivers/usb/serial/io_ti.c210
-rw-r--r--drivers/usb/serial/io_ti.h38
-rw-r--r--drivers/usb/serial/iuu_phoenix.c4
-rw-r--r--drivers/usb/serial/keyspan.c20
-rw-r--r--drivers/usb/serial/metro-usb.c4
-rw-r--r--drivers/usb/serial/mos7720.c20
-rw-r--r--drivers/usb/serial/mos7840.c23
-rw-r--r--drivers/usb/serial/opticon.c18
-rw-r--r--drivers/usb/serial/option.c2
-rw-r--r--drivers/usb/serial/pl2303.c188
-rw-r--r--drivers/usb/serial/quatech2.c16
-rw-r--r--drivers/usb/serial/ssu100.c16
-rw-r--r--drivers/usb/serial/ti_usb_3410_5052.c213
-rw-r--r--drivers/usb/serial/upd78f0730.c7
-rw-r--r--drivers/usb/serial/usb-serial.c226
-rw-r--r--drivers/usb/serial/usb-wwan.h4
-rw-r--r--drivers/usb/serial/usb_wwan.c45
-rw-r--r--drivers/usb/serial/whiteheat.c17
-rw-r--r--drivers/usb/serial/xr_serial.c757
-rw-r--r--drivers/usb/storage/datafab.c1
-rw-r--r--drivers/usb/storage/transport.c7
-rw-r--r--drivers/usb/storage/unusual_devs.h12
-rw-r--r--drivers/usb/typec/Kconfig15
-rw-r--r--drivers/usb/typec/Makefile4
-rw-r--r--drivers/usb/typec/bus.c2
-rw-r--r--drivers/usb/typec/bus.h19
-rw-r--r--drivers/usb/typec/class.c155
-rw-r--r--drivers/usb/typec/class.h85
-rw-r--r--drivers/usb/typec/mux.c4
-rw-r--r--drivers/usb/typec/mux.h21
-rw-r--r--drivers/usb/typec/port-mapper.c279
-rw-r--r--drivers/usb/typec/stusb160x.c4
-rw-r--r--drivers/usb/typec/tcpm/fusb302.c5
-rw-r--r--drivers/usb/typec/tcpm/tcpci.c17
-rw-r--r--drivers/usb/typec/tcpm/tcpci.h16
-rw-r--r--drivers/usb/typec/tcpm/tcpci_maxim.c2
-rw-r--r--drivers/usb/typec/tcpm/tcpm.c348
-rw-r--r--drivers/usb/typec/tipd/Kconfig12
-rw-r--r--drivers/usb/typec/tipd/Makefile6
-rw-r--r--drivers/usb/typec/tipd/core.c (renamed from drivers/usb/typec/tps6598x.c)69
-rw-r--r--drivers/usb/typec/tipd/tps6598x.h189
-rw-r--r--drivers/usb/typec/tipd/trace.c9
-rw-r--r--drivers/usb/typec/tipd/trace.h283
-rw-r--r--drivers/usb/typec/ucsi/ucsi.c48
-rw-r--r--drivers/usb/typec/ucsi/ucsi.h6
-rw-r--r--drivers/usb/usbip/stub_dev.c49
-rw-r--r--drivers/usb/usbip/usbip_common.h4
-rw-r--r--drivers/usb/usbip/usbip_event.c2
-rw-r--r--drivers/usb/usbip/vhci_hcd.c3
-rw-r--r--drivers/usb/usbip/vhci_sysfs.c63
-rw-r--r--drivers/usb/usbip/vudc_dev.c1
-rw-r--r--drivers/usb/usbip/vudc_sysfs.c56
-rw-r--r--drivers/vdpa/Kconfig15
-rw-r--r--drivers/vdpa/Makefile1
-rw-r--r--drivers/vdpa/ifcvf/ifcvf_base.c24
-rw-r--r--drivers/vdpa/ifcvf/ifcvf_base.h26
-rw-r--r--drivers/vdpa/ifcvf/ifcvf_main.c91
-rw-r--r--drivers/vdpa/mlx5/core/mlx5_vdpa.h4
-rw-r--r--drivers/vdpa/mlx5/core/mr.c13
-rw-r--r--drivers/vdpa/mlx5/core/resources.c3
-rw-r--r--drivers/vdpa/mlx5/net/mlx5_vnet.c125
-rw-r--r--drivers/vdpa/vdpa.c30
-rw-r--r--drivers/vdpa/vdpa_sim/Makefile1
-rw-r--r--drivers/vdpa/vdpa_sim/vdpa_sim.c129
-rw-r--r--drivers/vdpa/vdpa_sim/vdpa_sim.h2
-rw-r--r--drivers/vdpa/vdpa_sim/vdpa_sim_blk.c338
-rw-r--r--drivers/vdpa/vdpa_sim/vdpa_sim_net.c5
-rw-r--r--drivers/vdpa/virtio_pci/Makefile2
-rw-r--r--drivers/vdpa/virtio_pci/vp_vdpa.c484
-rw-r--r--drivers/vfio/Kconfig4
-rw-r--r--drivers/vfio/fsl-mc/vfio_fsl_mc.c127
-rw-r--r--drivers/vfio/fsl-mc/vfio_fsl_mc_private.h1
-rw-r--r--drivers/vfio/mdev/mdev_core.c174
-rw-r--r--drivers/vfio/mdev/mdev_driver.c19
-rw-r--r--drivers/vfio/mdev/mdev_private.h39
-rw-r--r--drivers/vfio/mdev/mdev_sysfs.c59
-rw-r--r--drivers/vfio/mdev/vfio_mdev.c80
-rw-r--r--drivers/vfio/pci/Kconfig6
-rw-r--r--drivers/vfio/pci/Makefile1
-rw-r--r--drivers/vfio/pci/vfio_pci.c278
-rw-r--r--drivers/vfio/pci/vfio_pci_config.c2
-rw-r--r--drivers/vfio/pci/vfio_pci_igd.c53
-rw-r--r--drivers/vfio/pci/vfio_pci_nvlink2.c490
-rw-r--r--drivers/vfio/pci/vfio_pci_private.h15
-rw-r--r--drivers/vfio/platform/Kconfig4
-rw-r--r--drivers/vfio/platform/reset/vfio_platform_calxedaxgmac.c2
-rw-r--r--drivers/vfio/platform/vfio_amba.c8
-rw-r--r--drivers/vfio/platform/vfio_platform.c20
-rw-r--r--drivers/vfio/platform/vfio_platform_common.c56
-rw-r--r--drivers/vfio/platform/vfio_platform_private.h5
-rw-r--r--drivers/vfio/vfio.c210
-rw-r--r--drivers/vfio/vfio_iommu_type1.c97
-rw-r--r--drivers/vhost/scsi.c58
-rw-r--r--drivers/vhost/vdpa.c52
-rw-r--r--drivers/vhost/vhost.c2
-rw-r--r--drivers/vhost/vringh.c69
-rw-r--r--drivers/video/backlight/jornada720_bl.c44
-rw-r--r--drivers/video/backlight/ktd253-backlight.c1
-rw-r--r--drivers/video/backlight/qcom-wled.c41
-rw-r--r--drivers/video/console/vgacon.c61
-rw-r--r--drivers/video/fbdev/amba-clcd.c17
-rw-r--r--drivers/video/fbdev/aty/atyfb.h3
-rw-r--r--drivers/video/fbdev/aty/atyfb_base.c9
-rw-r--r--drivers/video/fbdev/core/fb_defio.c35
-rw-r--r--drivers/video/fbdev/core/fbcmap.c8
-rw-r--r--drivers/video/fbdev/core/fbcon.c5
-rw-r--r--drivers/video/fbdev/core/fbmem.c6
-rw-r--r--drivers/video/fbdev/efifb.c5
-rw-r--r--drivers/video/fbdev/hyperv_fb.c5
-rw-r--r--drivers/video/fbdev/matrox/matroxfb_base.c9
-rw-r--r--drivers/video/fbdev/omap/hwa742.c42
-rw-r--r--drivers/video/fbdev/omap2/omapfb/dss/dsi.c2
-rw-r--r--drivers/video/fbdev/omap2/omapfb/omapfb.h2
-rw-r--r--drivers/video/fbdev/simplefb.c5
-rw-r--r--drivers/video/fbdev/vga16fb.c10
-rw-r--r--drivers/virt/acrn/hsm.c11
-rw-r--r--drivers/virt/acrn/irqfd.c6
-rw-r--r--drivers/virt/acrn/vm.c2
-rw-r--r--drivers/virt/nitro_enclaves/ne_misc_dev.c43
-rw-r--r--drivers/virtio/virtio.c6
-rw-r--r--drivers/virtio/virtio_balloon.c2
-rw-r--r--drivers/virtio/virtio_mmio.c3
-rw-r--r--drivers/virtio/virtio_pci_modern.c27
-rw-r--r--drivers/virtio/virtio_pci_modern_dev.c67
-rw-r--r--drivers/w1/slaves/w1_ds2780.c2
-rw-r--r--drivers/w1/slaves/w1_ds2781.c2
-rw-r--r--drivers/w1/slaves/w1_ds2805.c15
-rw-r--r--drivers/w1/slaves/w1_ds28e17.c16
-rw-r--r--drivers/w1/slaves/w1_therm.c7
-rw-r--r--drivers/watchdog/Kconfig13
-rw-r--r--drivers/watchdog/Makefile1
-rw-r--r--drivers/watchdog/armada_37xx_wdt.c4
-rw-r--r--drivers/watchdog/bd9576_wdt.c291
-rw-r--r--drivers/watchdog/cpu5wdt.c1
-rw-r--r--drivers/watchdog/cpwd.c1
-rw-r--r--drivers/watchdog/npcm_wdt.c1
-rw-r--r--drivers/watchdog/retu_wdt.c22
-rw-r--r--drivers/watchdog/riowd.c1
-rw-r--r--drivers/xen/Kconfig35
-rw-r--r--drivers/xen/Makefile3
-rw-r--r--drivers/xen/events/events_2l.c22
-rw-r--r--drivers/xen/events/events_base.c130
-rw-r--r--drivers/xen/events/events_fifo.c7
-rw-r--r--drivers/xen/events/events_internal.h14
-rw-r--r--drivers/xen/gntdev.c58
-rw-r--r--drivers/xen/pcpu.c35
-rw-r--r--drivers/xen/swiotlb-xen.c187
-rw-r--r--drivers/xen/time.c3
-rw-r--r--drivers/xen/unpopulated-alloc.c4
-rw-r--r--drivers/xen/xen-acpi-cpuhotplug.c446
-rw-r--r--drivers/xen/xen-acpi-memhotplug.c475
-rw-r--r--drivers/xen/xen-pciback/pci_stub.c6
-rw-r--r--drivers/xen/xen-pciback/vpci.c7
-rw-r--r--drivers/xen/xen-scsiback.c21
-rw-r--r--drivers/xen/xen-stub.c90
6341 files changed, 405569 insertions, 161234 deletions
diff --git a/drivers/Kconfig b/drivers/Kconfig
index 62c753a73651..47980c6b1945 100644
--- a/drivers/Kconfig
+++ b/drivers/Kconfig
@@ -149,6 +149,8 @@ source "drivers/xen/Kconfig"
source "drivers/greybus/Kconfig"
+source "drivers/comedi/Kconfig"
+
source "drivers/staging/Kconfig"
source "drivers/platform/Kconfig"
diff --git a/drivers/Makefile b/drivers/Makefile
index 6fba7daba591..5a6d613e868d 100644
--- a/drivers/Makefile
+++ b/drivers/Makefile
@@ -42,6 +42,7 @@ obj-$(CONFIG_DMADEVICES) += dma/
obj-y += soc/
obj-$(CONFIG_VIRTIO) += virtio/
+obj-$(CONFIG_VIRTIO_PCI_LIB) += virtio/
obj-$(CONFIG_VDPA) += vdpa/
obj-$(CONFIG_XEN) += xen/
@@ -148,6 +149,7 @@ obj-$(CONFIG_VHOST_IOTLB) += vhost/
obj-$(CONFIG_VHOST) += vhost/
obj-$(CONFIG_VLYNQ) += vlynq/
obj-$(CONFIG_GREYBUS) += greybus/
+obj-$(CONFIG_COMEDI) += comedi/
obj-$(CONFIG_STAGING) += staging/
obj-y += platform/
diff --git a/drivers/accessibility/speakup/i18n.c b/drivers/accessibility/speakup/i18n.c
index ee240d36f947..46bd50f3c3a4 100644
--- a/drivers/accessibility/speakup/i18n.c
+++ b/drivers/accessibility/speakup/i18n.c
@@ -548,12 +548,10 @@ ssize_t spk_msg_set(enum msg_index_t index, char *text, size_t length)
if ((index < MSG_FIRST_INDEX) || (index >= MSG_LAST_INDEX))
return -EINVAL;
- newstr = kmalloc(length + 1, GFP_KERNEL);
+ newstr = kmemdup_nul(text, length, GFP_KERNEL);
if (!newstr)
return -ENOMEM;
- memcpy(newstr, text, length);
- newstr[length] = '\0';
if (index >= MSG_FORMATTED_START &&
index <= MSG_FORMATTED_END &&
!fmt_validate(speakup_default_msgs[index], newstr)) {
diff --git a/drivers/accessibility/speakup/spk_ttyio.c b/drivers/accessibility/speakup/spk_ttyio.c
index 9af1d4c124d3..2e39fcf492d8 100644
--- a/drivers/accessibility/speakup/spk_ttyio.c
+++ b/drivers/accessibility/speakup/spk_ttyio.c
@@ -104,7 +104,6 @@ static int spk_ttyio_receive_buf2(struct tty_struct *tty,
static struct tty_ldisc_ops spk_ttyio_ldisc_ops = {
.owner = THIS_MODULE,
- .magic = TTY_LDISC_MAGIC,
.name = "speakup_ldisc",
.open = spk_ttyio_ldisc_open,
.close = spk_ttyio_ldisc_close,
diff --git a/drivers/acpi/ac.c b/drivers/acpi/ac.c
index b41180330cc1..b0cb662233f1 100644
--- a/drivers/acpi/ac.c
+++ b/drivers/acpi/ac.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
- * acpi_ac.c - ACPI AC Adapter Driver ($Revision: 27 $)
+ * acpi_ac.c - ACPI AC Adapter Driver (Revision: 27)
*
* Copyright (C) 2001, 2002 Andy Grover <andrew.grover@intel.com>
* Copyright (C) 2001, 2002 Paul Diefenbaugh <paul.s.diefenbaugh@intel.com>
@@ -78,17 +78,14 @@ static struct acpi_driver acpi_ac_driver = {
struct acpi_ac {
struct power_supply *charger;
struct power_supply_desc charger_desc;
- struct acpi_device * device;
+ struct acpi_device *device;
unsigned long long state;
struct notifier_block battery_nb;
};
#define to_acpi_ac(x) power_supply_get_drvdata(x)
-/* --------------------------------------------------------------------------
- AC Adapter Management
- -------------------------------------------------------------------------- */
-
+/* AC Adapter Management */
static int acpi_ac_get_state(struct acpi_ac *ac)
{
acpi_status status = AE_OK;
@@ -109,9 +106,7 @@ static int acpi_ac_get_state(struct acpi_ac *ac)
return 0;
}
-/* --------------------------------------------------------------------------
- sysfs I/F
- -------------------------------------------------------------------------- */
+/* sysfs I/F */
static int get_ac_property(struct power_supply *psy,
enum power_supply_property psp,
union power_supply_propval *val)
@@ -138,10 +133,7 @@ static enum power_supply_property ac_props[] = {
POWER_SUPPLY_PROP_ONLINE,
};
-/* --------------------------------------------------------------------------
- Driver Model
- -------------------------------------------------------------------------- */
-
+/* Driver Model */
static void acpi_ac_notify(struct acpi_device *device, u32 event)
{
struct acpi_ac *ac = acpi_driver_data(device);
@@ -174,8 +166,6 @@ static void acpi_ac_notify(struct acpi_device *device, u32 event)
acpi_notifier_call_chain(device, event, (u32) ac->state);
kobject_uevent(&ac->charger->dev.kobj, KOBJ_CHANGE);
}
-
- return;
}
static int acpi_ac_battery_notify(struct notifier_block *nb,
@@ -187,7 +177,7 @@ static int acpi_ac_battery_notify(struct notifier_block *nb,
/*
* On HP Pavilion dv6-6179er AC status notifications aren't triggered
* when adapter is plugged/unplugged. However, battery status
- * notifcations are triggered when battery starts charging or
+ * notifications are triggered when battery starts charging or
* discharging. Re-reading AC status triggers lost AC notifications,
* if AC status has changed.
*/
@@ -282,9 +272,8 @@ static int acpi_ac_add(struct acpi_device *device)
ac->battery_nb.notifier_call = acpi_ac_battery_notify;
register_acpi_notifier(&ac->battery_nb);
end:
- if (result) {
+ if (result)
kfree(ac);
- }
return result;
}
@@ -293,7 +282,7 @@ end:
static int acpi_ac_resume(struct device *dev)
{
struct acpi_ac *ac;
- unsigned old_state;
+ unsigned int old_state;
if (!dev)
return -EINVAL;
@@ -352,9 +341,8 @@ static int __init acpi_ac_init(void)
}
result = acpi_bus_register_driver(&acpi_ac_driver);
- if (result < 0) {
+ if (result < 0)
return -ENODEV;
- }
return 0;
}
diff --git a/drivers/acpi/acpi_apd.c b/drivers/acpi/acpi_apd.c
index 39359ce0eb2c..0ec5b3f69112 100644
--- a/drivers/acpi/acpi_apd.c
+++ b/drivers/acpi/acpi_apd.c
@@ -176,10 +176,10 @@ static const struct apd_device_desc hip08_spi_desc = {
#endif
-/**
-* Create platform device during acpi scan attach handle.
-* Return value > 0 on success of creating device.
-*/
+/*
+ * Create platform device during acpi scan attach handle.
+ * Return value > 0 on success of creating device.
+ */
static int acpi_apd_create_device(struct acpi_device *adev,
const struct acpi_device_id *id)
{
diff --git a/drivers/acpi/acpi_ipmi.c b/drivers/acpi/acpi_ipmi.c
index 9d6c0fc120d7..bbd00d96b7a8 100644
--- a/drivers/acpi/acpi_ipmi.c
+++ b/drivers/acpi/acpi_ipmi.c
@@ -478,7 +478,6 @@ err_lock:
ipmi_dev_release(ipmi_device);
err_ref:
put_device(smi_data.dev);
- return;
}
static void ipmi_bmc_gone(int iface)
diff --git a/drivers/acpi/acpi_lpss.c b/drivers/acpi/acpi_lpss.c
index be73974ce449..ca742f16a507 100644
--- a/drivers/acpi/acpi_lpss.c
+++ b/drivers/acpi/acpi_lpss.c
@@ -377,6 +377,7 @@ static const struct acpi_device_id acpi_lpss_device_ids[] = {
static int is_memory(struct acpi_resource *res, void *not_used)
{
struct resource r;
+
return !acpi_dev_resource_memory(res, &r);
}
@@ -1200,6 +1201,7 @@ static int acpi_lpss_poweroff_noirq(struct device *dev)
if (pdata->dev_desc->resume_from_noirq) {
/* This is analogous to the acpi_lpss_suspend_noirq() case. */
int ret = acpi_lpss_do_poweroff_late(dev);
+
if (ret)
return ret;
}
diff --git a/drivers/acpi/acpi_memhotplug.c b/drivers/acpi/acpi_memhotplug.c
index b02fd51e5589..8cc195c4c861 100644
--- a/drivers/acpi/acpi_memhotplug.c
+++ b/drivers/acpi/acpi_memhotplug.c
@@ -171,6 +171,7 @@ static int acpi_memory_enable_device(struct acpi_memory_device *mem_device)
acpi_handle handle = mem_device->device->handle;
int result, num_enabled = 0;
struct acpi_memory_info *info;
+ mhp_t mhp_flags = MHP_NONE;
int node;
node = acpi_get_node(handle);
@@ -194,8 +195,10 @@ static int acpi_memory_enable_device(struct acpi_memory_device *mem_device)
if (node < 0)
node = memory_add_physaddr_to_nid(info->start_addr);
+ if (mhp_supports_memmap_on_memory(info->length))
+ mhp_flags |= MHP_MEMMAP_ON_MEMORY;
result = __add_memory(node, info->start_addr, info->length,
- MHP_NONE);
+ mhp_flags);
/*
* If the memory block has been used by the kernel, add_memory()
diff --git a/drivers/acpi/acpi_pad.c b/drivers/acpi/acpi_pad.c
index b84ab722feb4..df4adeb335b2 100644
--- a/drivers/acpi/acpi_pad.c
+++ b/drivers/acpi/acpi_pad.c
@@ -128,6 +128,7 @@ static void round_robin_cpu(unsigned int tsk_index)
static void exit_round_robin(unsigned int tsk_index)
{
struct cpumask *pad_busy_cpus = to_cpumask(pad_busy_cpus_bits);
+
cpumask_clear_cpu(tsk_in_cpu[tsk_index], pad_busy_cpus);
tsk_in_cpu[tsk_index] = -1;
}
@@ -265,6 +266,7 @@ static ssize_t rrtime_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count)
{
unsigned long num;
+
if (kstrtoul(buf, 0, &num))
return -EINVAL;
if (num < 1 || num >= 100)
@@ -286,6 +288,7 @@ static ssize_t idlepct_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count)
{
unsigned long num;
+
if (kstrtoul(buf, 0, &num))
return -EINVAL;
if (num < 1 || num >= 100)
@@ -307,6 +310,7 @@ static ssize_t idlecpus_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count)
{
unsigned long num;
+
if (kstrtoul(buf, 0, &num))
return -EINVAL;
mutex_lock(&isolated_cpus_lock);
diff --git a/drivers/acpi/acpi_processor.c b/drivers/acpi/acpi_processor.c
index 2ee5e05a0d69..2d5bd2a6ddce 100644
--- a/drivers/acpi/acpi_processor.c
+++ b/drivers/acpi/acpi_processor.c
@@ -22,17 +22,10 @@
#include "internal.h"
-#define _COMPONENT ACPI_PROCESSOR_COMPONENT
-
-ACPI_MODULE_NAME("processor");
-
DEFINE_PER_CPU(struct acpi_processor *, processors);
EXPORT_PER_CPU_SYMBOL(processors);
-/* --------------------------------------------------------------------------
- Errata Handling
- -------------------------------------------------------------------------- */
-
+/* Errata Handling */
struct acpi_processor_errata errata __read_mostly;
EXPORT_SYMBOL_GPL(errata);
@@ -51,19 +44,19 @@ static int acpi_processor_errata_piix4(struct pci_dev *dev)
switch (dev->revision) {
case 0:
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found PIIX4 A-step\n"));
+ dev_dbg(&dev->dev, "Found PIIX4 A-step\n");
break;
case 1:
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found PIIX4 B-step\n"));
+ dev_dbg(&dev->dev, "Found PIIX4 B-step\n");
break;
case 2:
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found PIIX4E\n"));
+ dev_dbg(&dev->dev, "Found PIIX4E\n");
break;
case 3:
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found PIIX4M\n"));
+ dev_dbg(&dev->dev, "Found PIIX4M\n");
break;
default:
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found unknown PIIX4\n"));
+ dev_dbg(&dev->dev, "Found unknown PIIX4\n");
break;
}
@@ -129,11 +122,9 @@ static int acpi_processor_errata_piix4(struct pci_dev *dev)
}
if (errata.piix4.bmisx)
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Bus master activity detection (BM-IDE) erratum enabled\n"));
+ dev_dbg(&dev->dev, "Bus master activity detection (BM-IDE) erratum enabled\n");
if (errata.piix4.fdma)
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Type-F DMA livelock erratum (C3 disabled)\n"));
+ dev_dbg(&dev->dev, "Type-F DMA livelock erratum (C3 disabled)\n");
return 0;
}
@@ -157,10 +148,7 @@ static int acpi_processor_errata(void)
return result;
}
-/* --------------------------------------------------------------------------
- Initialization
- -------------------------------------------------------------------------- */
-
+/* Initialization */
#ifdef CONFIG_ACPI_HOTPLUG_CPU
int __weak acpi_map_cpu(acpi_handle handle,
phys_cpuid_t physid, u32 acpi_id, int *pcpu)
@@ -244,11 +232,9 @@ static int acpi_processor_get_info(struct acpi_device *device)
*/
if (acpi_gbl_FADT.pm2_control_block && acpi_gbl_FADT.pm2_control_length) {
pr->flags.bm_control = 1;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Bus mastering arbitration control present\n"));
+ dev_dbg(&device->dev, "Bus mastering arbitration control present\n");
} else
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "No bus mastering arbitration control\n"));
+ dev_dbg(&device->dev, "No bus mastering arbitration control\n");
if (!strcmp(acpi_device_hid(device), ACPI_PROCESSOR_OBJECT_HID)) {
/* Declared with "Processor" statement; match ProcessorID */
@@ -291,7 +277,7 @@ static int acpi_processor_get_info(struct acpi_device *device)
pr->phys_id = acpi_get_phys_id(pr->handle, device_declaration,
pr->acpi_id);
if (invalid_phys_cpuid(pr->phys_id))
- acpi_handle_debug(pr->handle, "failed to get CPU physical ID.\n");
+ dev_dbg(&device->dev, "Failed to get CPU physical ID.\n");
pr->id = acpi_map_cpuid(pr->phys_id, pr->acpi_id);
if (!cpu0_initialized && !acpi_has_cpu_in_madt()) {
@@ -314,6 +300,7 @@ static int acpi_processor_get_info(struct acpi_device *device)
*/
if (invalid_logical_cpuid(pr->id) || !cpu_present(pr->id)) {
int ret = acpi_processor_hotadd_init(pr);
+
if (ret)
return ret;
}
@@ -328,11 +315,10 @@ static int acpi_processor_get_info(struct acpi_device *device)
* CPU+CPU ID.
*/
sprintf(acpi_device_bid(device), "CPU%X", pr->id);
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Processor [%d:%d]\n", pr->id,
- pr->acpi_id));
+ dev_dbg(&device->dev, "Processor [%d:%d]\n", pr->id, pr->acpi_id);
if (!object.processor.pblk_address)
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No PBLK (NULL address)\n"));
+ dev_dbg(&device->dev, "No PBLK (NULL address)\n");
else if (object.processor.pblk_length != 6)
dev_err(&device->dev, "Invalid PBLK length [%d]\n",
object.processor.pblk_length);
@@ -440,10 +426,7 @@ static int acpi_processor_add(struct acpi_device *device,
}
#ifdef CONFIG_ACPI_HOTPLUG_CPU
-/* --------------------------------------------------------------------------
- Removal
- -------------------------------------------------------------------------- */
-
+/* Removal */
static void acpi_processor_remove(struct acpi_device *device)
{
struct acpi_processor *pr;
@@ -901,7 +884,7 @@ int acpi_processor_evaluate_cst(acpi_handle handle, u32 cpu,
info->count = last_index;
- end:
+end:
kfree(buffer.pointer);
return ret;
diff --git a/drivers/acpi/acpi_video.c b/drivers/acpi/acpi_video.c
index 2ea1781290cc..0c884020f74b 100644
--- a/drivers/acpi/acpi_video.c
+++ b/drivers/acpi/acpi_video.c
@@ -1392,7 +1392,7 @@ acpi_video_get_next_level(struct acpi_video_device *device,
break;
}
}
- /* Ajust level_current to closest available level */
+ /* Adjust level_current to closest available level */
level_current += delta;
for (i = ACPI_VIDEO_FIRST_LEVEL; i < device->brightness->count; i++) {
l = device->brightness->levels[i];
@@ -1555,7 +1555,7 @@ acpi_video_bus_get_devices(struct acpi_video_bus *video,
/*
* Win8 requires setting bit2 of _DOS to let firmware know it shouldn't
- * preform any automatic brightness change on receiving a notification.
+ * perform any automatic brightness change on receiving a notification.
*/
static int acpi_video_bus_start_devices(struct acpi_video_bus *video)
{
@@ -2182,6 +2182,30 @@ static bool dmi_is_desktop(void)
return false;
}
+/*
+ * We're seeing a lot of bogus backlight interfaces on newer machines
+ * without a LCD such as desktops, servers and HDMI sticks. Checking the
+ * lcd flag fixes this, enable this by default on any machines which are:
+ * 1. Win8 ready (where we also prefer the native backlight driver, so
+ * normally the acpi_video code should not register there anyways); *and*
+ * 2.1 Report a desktop/server DMI chassis-type, or
+ * 2.2 Are an ACPI-reduced-hardware platform (and thus won't use the EC for
+ backlight control)
+ */
+static bool should_check_lcd_flag(void)
+{
+ if (!acpi_osi_is_win8())
+ return false;
+
+ if (dmi_is_desktop())
+ return true;
+
+ if (acpi_reduced_hardware())
+ return true;
+
+ return false;
+}
+
int acpi_video_register(void)
{
int ret = 0;
@@ -2195,19 +2219,8 @@ int acpi_video_register(void)
goto leave;
}
- /*
- * We're seeing a lot of bogus backlight interfaces on newer machines
- * without a LCD such as desktops, servers and HDMI sticks. Checking
- * the lcd flag fixes this, so enable this on any machines which are
- * win8 ready (where we also prefer the native backlight driver, so
- * normally the acpi_video code should not register there anyways).
- */
- if (only_lcd == -1) {
- if (dmi_is_desktop() && acpi_osi_is_win8())
- only_lcd = true;
- else
- only_lcd = false;
- }
+ if (only_lcd == -1)
+ only_lcd = should_check_lcd_flag();
dmi_check_system(video_dmi_table);
diff --git a/drivers/acpi/acpica/acpredef.h b/drivers/acpi/acpica/acpredef.h
index 15cf904f0751..5951b433c304 100644
--- a/drivers/acpi/acpica/acpredef.h
+++ b/drivers/acpi/acpica/acpredef.h
@@ -328,6 +328,17 @@ const union acpi_predefined_info acpi_gbl_predefined_methods[] = {
{{"_BMS", METHOD_1ARGS(ACPI_TYPE_INTEGER),
METHOD_RETURNS(ACPI_RTYPE_INTEGER)}},
+ {{"_BPC", METHOD_0ARGS,
+ METHOD_RETURNS(ACPI_RTYPE_PACKAGE)}}, /* Fixed-length (4 Int) */
+ PACKAGE_INFO(ACPI_PTYPE1_FIXED, ACPI_RTYPE_INTEGER, 4, 0, 0, 0),
+
+ {{"_BPS", METHOD_0ARGS,
+ METHOD_RETURNS(ACPI_RTYPE_PACKAGE)}}, /* Fixed-length (5 Int) */
+ PACKAGE_INFO(ACPI_PTYPE1_FIXED, ACPI_RTYPE_INTEGER, 5, 0, 0, 0),
+
+ {{"_BPT", METHOD_1ARGS(ACPI_TYPE_PACKAGE),
+ METHOD_RETURNS(ACPI_RTYPE_INTEGER)}},
+
{{"_BQC", METHOD_0ARGS,
METHOD_RETURNS(ACPI_RTYPE_INTEGER)}},
@@ -347,6 +358,10 @@ const union acpi_predefined_info acpi_gbl_predefined_methods[] = {
{{"_CBA", METHOD_0ARGS,
METHOD_RETURNS(ACPI_RTYPE_INTEGER)}}, /* See PCI firmware spec 3.0 */
+ {{"_CBR", METHOD_0ARGS,
+ METHOD_RETURNS(ACPI_RTYPE_PACKAGE)}}, /* Fixed-length (3 Int) */
+ PACKAGE_INFO(ACPI_PTYPE1_FIXED, ACPI_RTYPE_INTEGER, 3, 0, 0, 0),
+
{{"_CCA", METHOD_0ARGS,
METHOD_RETURNS(ACPI_RTYPE_INTEGER)}}, /* ACPI 5.1 */
diff --git a/drivers/acpi/acpica/acresrc.h b/drivers/acpi/acpica/acresrc.h
index 0cb975a3e01d..37c47e185fd4 100644
--- a/drivers/acpi/acpica/acresrc.h
+++ b/drivers/acpi/acpica/acresrc.h
@@ -46,6 +46,7 @@ typedef enum {
ACPI_RSC_1BITFLAG,
ACPI_RSC_2BITFLAG,
ACPI_RSC_3BITFLAG,
+ ACPI_RSC_6BITFLAG,
ACPI_RSC_ADDRESS,
ACPI_RSC_BITMASK,
ACPI_RSC_BITMASK16,
@@ -102,6 +103,7 @@ typedef enum {
ACPI_RSD_1BITFLAG,
ACPI_RSD_2BITFLAG,
ACPI_RSD_3BITFLAG,
+ ACPI_RSD_6BITFLAG,
ACPI_RSD_ADDRESS,
ACPI_RSD_DWORDLIST,
ACPI_RSD_LITERAL,
@@ -295,6 +297,7 @@ extern struct acpi_rsconvert_info acpi_rs_convert_address64[];
extern struct acpi_rsconvert_info acpi_rs_convert_ext_address64[];
extern struct acpi_rsconvert_info acpi_rs_convert_gpio[];
extern struct acpi_rsconvert_info acpi_rs_convert_fixed_dma[];
+extern struct acpi_rsconvert_info acpi_rs_convert_csi2_serial_bus[];
extern struct acpi_rsconvert_info acpi_rs_convert_i2c_serial_bus[];
extern struct acpi_rsconvert_info acpi_rs_convert_spi_serial_bus[];
extern struct acpi_rsconvert_info acpi_rs_convert_uart_serial_bus[];
@@ -349,6 +352,7 @@ extern struct acpi_rsdump_info acpi_rs_dump_gpio[];
extern struct acpi_rsdump_info acpi_rs_dump_pin_function[];
extern struct acpi_rsdump_info acpi_rs_dump_fixed_dma[];
extern struct acpi_rsdump_info acpi_rs_dump_common_serial_bus[];
+extern struct acpi_rsdump_info acpi_rs_dump_csi2_serial_bus[];
extern struct acpi_rsdump_info acpi_rs_dump_i2c_serial_bus[];
extern struct acpi_rsdump_info acpi_rs_dump_spi_serial_bus[];
extern struct acpi_rsdump_info acpi_rs_dump_uart_serial_bus[];
diff --git a/drivers/acpi/acpica/acutils.h b/drivers/acpi/acpica/acutils.h
index be6de7149e67..bccae0d3db75 100644
--- a/drivers/acpi/acpica/acutils.h
+++ b/drivers/acpi/acpica/acutils.h
@@ -28,6 +28,7 @@ extern const char *acpi_gbl_max_decode[];
extern const char *acpi_gbl_mem_decode[];
extern const char *acpi_gbl_min_decode[];
extern const char *acpi_gbl_mtp_decode[];
+extern const char *acpi_gbl_phy_decode[];
extern const char *acpi_gbl_rng_decode[];
extern const char *acpi_gbl_rw_decode[];
extern const char *acpi_gbl_shr_decode[];
diff --git a/drivers/acpi/acpica/amlresrc.h b/drivers/acpi/acpica/amlresrc.h
index a9d91a3c2994..b98123210281 100644
--- a/drivers/acpi/acpica/amlresrc.h
+++ b/drivers/acpi/acpica/amlresrc.h
@@ -40,6 +40,7 @@
#define ACPI_RESTAG_IORESTRICTION "_IOR"
#define ACPI_RESTAG_LENGTH "_LEN"
#define ACPI_RESTAG_LINE "_LIN"
+#define ACPI_RESTAG_LOCALPORT "_PRT"
#define ACPI_RESTAG_MEMATTRIBUTES "_MTP" /* Memory(0), Reserved(1), ACPI(2), NVS(3) */
#define ACPI_RESTAG_MEMTYPE "_MEM" /* non_cache(0), Cacheable(1) Cache+combine(2), Cache+prefetch(3) */
#define ACPI_RESTAG_MAXADDR "_MAX"
@@ -49,6 +50,7 @@
#define ACPI_RESTAG_MODE "_MOD"
#define ACPI_RESTAG_PARITY "_PAR"
#define ACPI_RESTAG_PHASE "_PHA"
+#define ACPI_RESTAG_PHYTYPE "_PHY"
#define ACPI_RESTAG_PIN "_PIN"
#define ACPI_RESTAG_PINCONFIG "_PPI"
#define ACPI_RESTAG_PINCONFIG_TYPE "_TYP"
@@ -316,12 +318,26 @@ struct aml_resource_gpio {
#define AML_RESOURCE_I2C_SERIALBUSTYPE 1
#define AML_RESOURCE_SPI_SERIALBUSTYPE 2
#define AML_RESOURCE_UART_SERIALBUSTYPE 3
-#define AML_RESOURCE_MAX_SERIALBUSTYPE 3
+#define AML_RESOURCE_CSI2_SERIALBUSTYPE 4
+#define AML_RESOURCE_MAX_SERIALBUSTYPE 4
#define AML_RESOURCE_VENDOR_SERIALBUSTYPE 192 /* Vendor defined is 0xC0-0xFF (NOT SUPPORTED) */
struct aml_resource_common_serialbus {
AML_RESOURCE_LARGE_HEADER_COMMON AML_RESOURCE_SERIAL_COMMON};
+struct aml_resource_csi2_serialbus {
+ AML_RESOURCE_LARGE_HEADER_COMMON AML_RESOURCE_SERIAL_COMMON
+ /*
+ * Optional fields follow immediately:
+ * 1) Vendor Data bytes
+ * 2) Resource Source String
+ */
+};
+
+#define AML_RESOURCE_CSI2_REVISION 1 /* ACPI 6.4 */
+#define AML_RESOURCE_CSI2_TYPE_REVISION 1 /* ACPI 6.4 */
+#define AML_RESOURCE_CSI2_MIN_DATA_LEN 0 /* ACPI 6.4 */
+
struct aml_resource_i2c_serialbus {
AML_RESOURCE_LARGE_HEADER_COMMON
AML_RESOURCE_SERIAL_COMMON u32 connection_speed;
@@ -510,6 +526,7 @@ union aml_resource {
struct aml_resource_i2c_serialbus i2c_serial_bus;
struct aml_resource_spi_serialbus spi_serial_bus;
struct aml_resource_uart_serialbus uart_serial_bus;
+ struct aml_resource_csi2_serialbus csi2_serial_bus;
struct aml_resource_common_serialbus common_serial_bus;
struct aml_resource_pin_function pin_function;
struct aml_resource_pin_config pin_config;
diff --git a/drivers/acpi/acpica/nsaccess.c b/drivers/acpi/acpica/nsaccess.c
index 3f045b5953b2..a0c1a665dfc1 100644
--- a/drivers/acpi/acpica/nsaccess.c
+++ b/drivers/acpi/acpica/nsaccess.c
@@ -99,13 +99,12 @@ acpi_status acpi_ns_root_initialize(void)
* just create and link the new node(s) here.
*/
new_node =
- ACPI_ALLOCATE_ZEROED(sizeof(struct acpi_namespace_node));
+ acpi_ns_create_node(*ACPI_CAST_PTR(u32, init_val->name));
if (!new_node) {
status = AE_NO_MEMORY;
goto unlock_and_exit;
}
- ACPI_COPY_NAMESEG(new_node->name.ascii, init_val->name);
new_node->descriptor_type = ACPI_DESC_TYPE_NAMED;
new_node->type = init_val->type;
diff --git a/drivers/acpi/acpica/rscalc.c b/drivers/acpi/acpica/rscalc.c
index fcf129d27baa..90583db459a2 100644
--- a/drivers/acpi/acpica/rscalc.c
+++ b/drivers/acpi/acpica/rscalc.c
@@ -677,10 +677,10 @@ acpi_rs_get_list_length(u8 *aml_buffer,
*size_needed += buffer_size;
ACPI_DEBUG_PRINT((ACPI_DB_RESOURCES,
- "Type %.2X, AmlLength %.2X InternalLength %.2X\n",
+ "Type %.2X, AmlLength %.2X InternalLength %.2X%8X\n",
acpi_ut_get_resource_type(aml_buffer),
acpi_ut_get_descriptor_length(aml_buffer),
- buffer_size));
+ ACPI_FORMAT_UINT64(*size_needed)));
/*
* Point to the next resource within the AML stream using the length
diff --git a/drivers/acpi/acpica/rsdump.c b/drivers/acpi/acpica/rsdump.c
index 6601e71b45e3..611bc71c193f 100644
--- a/drivers/acpi/acpica/rsdump.c
+++ b/drivers/acpi/acpica/rsdump.c
@@ -87,6 +87,9 @@ void acpi_rs_dump_resource_list(struct acpi_resource *resource_list)
("Invalid descriptor type (%X) in resource list\n",
resource_list->type);
return;
+ } else if (!resource_list->type) {
+ ACPI_ERROR((AE_INFO, "Invalid Zero Resource Type"));
+ return;
}
/* Sanity check the length. It must not be zero, or we loop forever */
@@ -258,6 +261,11 @@ acpi_rs_dump_descriptor(void *resource, struct acpi_rsdump_info *table)
table->pointer[*target & 0x07]);
break;
+ case ACPI_RSD_6BITFLAG:
+
+ acpi_rs_out_integer8(name, (ACPI_GET8(target) & 0x3F));
+ break;
+
case ACPI_RSD_SHORTLIST:
/*
* Short byte list (single line output) for DMA and IRQ resources
diff --git a/drivers/acpi/acpica/rsdumpinfo.c b/drivers/acpi/acpica/rsdumpinfo.c
index cafa8134b4c6..b8b37449011b 100644
--- a/drivers/acpi/acpica/rsdumpinfo.c
+++ b/drivers/acpi/acpica/rsdumpinfo.c
@@ -421,6 +421,32 @@ struct acpi_rsdump_info acpi_rs_dump_common_serial_bus[11] = {
ACPI_RS_DUMP_COMMON_SERIAL_BUS
};
+struct acpi_rsdump_info acpi_rs_dump_csi2_serial_bus[11] = {
+ { ACPI_RSD_TITLE, ACPI_RSD_TABLE_SIZE(acpi_rs_dump_csi2_serial_bus),
+ "Camera Serial Bus", NULL },
+ { ACPI_RSD_UINT8, ACPI_RSD_OFFSET(csi2_serial_bus.revision_id),
+ "RevisionId", NULL },
+ { ACPI_RSD_UINT8, ACPI_RSD_OFFSET(csi2_serial_bus.type), "Type",
+ acpi_gbl_sbt_decode },
+ { ACPI_RSD_1BITFLAG, ACPI_RSD_OFFSET(csi2_serial_bus.producer_consumer),
+ "ProducerConsumer", acpi_gbl_consume_decode },
+ { ACPI_RSD_1BITFLAG, ACPI_RSD_OFFSET(csi2_serial_bus.slave_mode),
+ "SlaveMode", acpi_gbl_sm_decode },
+ { ACPI_RSD_2BITFLAG, ACPI_RSD_OFFSET(csi2_serial_bus.phy_type),
+ "PhyType", acpi_gbl_phy_decode },
+ { ACPI_RSD_6BITFLAG,
+ ACPI_RSD_OFFSET(csi2_serial_bus.local_port_instance),
+ "LocalPortInstance", NULL },
+ { ACPI_RSD_UINT8, ACPI_RSD_OFFSET(csi2_serial_bus.type_revision_id),
+ "TypeRevisionId", NULL },
+ { ACPI_RSD_UINT16, ACPI_RSD_OFFSET(csi2_serial_bus.vendor_length),
+ "VendorLength", NULL },
+ { ACPI_RSD_SHORTLISTX, ACPI_RSD_OFFSET(csi2_serial_bus.vendor_data),
+ "VendorData", NULL },
+ { ACPI_RSD_SOURCE, ACPI_RSD_OFFSET(csi2_serial_bus.resource_source),
+ "ResourceSource", NULL },
+};
+
struct acpi_rsdump_info acpi_rs_dump_i2c_serial_bus[14] = {
{ACPI_RSD_TITLE, ACPI_RSD_TABLE_SIZE(acpi_rs_dump_i2c_serial_bus),
"I2C Serial Bus", NULL},
diff --git a/drivers/acpi/acpica/rsinfo.c b/drivers/acpi/acpica/rsinfo.c
index 6e2e596902eb..eaeb7ab58c2a 100644
--- a/drivers/acpi/acpica/rsinfo.c
+++ b/drivers/acpi/acpica/rsinfo.c
@@ -96,13 +96,14 @@ struct acpi_rsconvert_info *acpi_gbl_get_resource_dispatch[] = {
acpi_rs_convert_pin_group_config, /* 0x12, ACPI_RESOURCE_NAME_PIN_GROUP_CONFIG */
};
-/* Subtype table for serial_bus -- I2C, SPI, and UART */
+/* Subtype table for serial_bus -- I2C, SPI, UART, and CSI2 */
struct acpi_rsconvert_info *acpi_gbl_convert_resource_serial_bus_dispatch[] = {
NULL,
acpi_rs_convert_i2c_serial_bus,
acpi_rs_convert_spi_serial_bus,
acpi_rs_convert_uart_serial_bus,
+ acpi_rs_convert_csi2_serial_bus
};
#if defined(ACPI_DEBUG_OUTPUT) || defined(ACPI_DISASSEMBLER) || defined(ACPI_DEBUGGER)
@@ -142,6 +143,7 @@ struct acpi_rsdump_info *acpi_gbl_dump_serial_bus_dispatch[] = {
acpi_rs_dump_i2c_serial_bus, /* AML_RESOURCE_I2C_BUS_TYPE */
acpi_rs_dump_spi_serial_bus, /* AML_RESOURCE_SPI_BUS_TYPE */
acpi_rs_dump_uart_serial_bus, /* AML_RESOURCE_UART_BUS_TYPE */
+ acpi_rs_dump_csi2_serial_bus, /* AML_RESOURCE_CSI2_BUS_TYPE */
};
#endif
@@ -226,6 +228,7 @@ const u8 acpi_gbl_aml_resource_serial_bus_sizes[] = {
sizeof(struct aml_resource_i2c_serialbus),
sizeof(struct aml_resource_spi_serialbus),
sizeof(struct aml_resource_uart_serialbus),
+ sizeof(struct aml_resource_csi2_serialbus),
};
const u8 acpi_gbl_resource_struct_serial_bus_sizes[] = {
@@ -233,4 +236,5 @@ const u8 acpi_gbl_resource_struct_serial_bus_sizes[] = {
ACPI_RS_SIZE(struct acpi_resource_i2c_serialbus),
ACPI_RS_SIZE(struct acpi_resource_spi_serialbus),
ACPI_RS_SIZE(struct acpi_resource_uart_serialbus),
+ ACPI_RS_SIZE(struct acpi_resource_csi2_serialbus),
};
diff --git a/drivers/acpi/acpica/rslist.c b/drivers/acpi/acpica/rslist.c
index 0307675d37be..e46efaa889cd 100644
--- a/drivers/acpi/acpica/rslist.c
+++ b/drivers/acpi/acpica/rslist.c
@@ -59,7 +59,7 @@ acpi_rs_convert_aml_to_resources(u8 * aml,
AML_RESOURCE_MAX_SERIALBUSTYPE) {
conversion_table = NULL;
} else {
- /* This is an I2C, SPI, or UART serial_bus descriptor */
+ /* This is an I2C, SPI, UART, or CSI2 serial_bus descriptor */
conversion_table =
acpi_gbl_convert_resource_serial_bus_dispatch
@@ -89,6 +89,11 @@ acpi_rs_convert_aml_to_resources(u8 * aml,
return_ACPI_STATUS(status);
}
+ if (!resource->length) {
+ ACPI_EXCEPTION((AE_INFO, status,
+ "Zero-length resource returned from RsConvertAmlToResource"));
+ }
+
ACPI_DEBUG_PRINT((ACPI_DB_RESOURCES,
"Type %.2X, AmlLength %.2X InternalLength %.2X\n",
acpi_ut_get_resource_type(aml), length,
@@ -158,7 +163,7 @@ acpi_rs_convert_resources_to_aml(struct acpi_resource *resource,
AML_RESOURCE_MAX_SERIALBUSTYPE) {
conversion_table = NULL;
} else {
- /* This is an I2C, SPI, or UART serial_bus descriptor */
+ /* This is an I2C, SPI, UART or CSI2 serial_bus descriptor */
conversion_table =
acpi_gbl_convert_resource_serial_bus_dispatch
diff --git a/drivers/acpi/acpica/rsmisc.c b/drivers/acpi/acpica/rsmisc.c
index 1763a3dbc9b1..c2dd9aae4745 100644
--- a/drivers/acpi/acpica/rsmisc.c
+++ b/drivers/acpi/acpica/rsmisc.c
@@ -70,6 +70,8 @@ acpi_rs_convert_aml_to_resource(struct acpi_resource *resource,
*/
count = INIT_TABLE_LENGTH(info);
while (count) {
+ target = NULL;
+
/*
* Source is the external AML byte stream buffer,
* destination is the internal resource descriptor
@@ -120,6 +122,14 @@ acpi_rs_convert_aml_to_resource(struct acpi_resource *resource,
((ACPI_GET8(source) >> info->value) & 0x07));
break;
+ case ACPI_RSC_6BITFLAG:
+ /*
+ * Mask and shift the flag bits
+ */
+ ACPI_SET8(destination,
+ ((ACPI_GET8(source) >> info->value) & 0x3F));
+ break;
+
case ACPI_RSC_COUNT:
item_count = ACPI_GET8(source);
@@ -509,6 +519,15 @@ acpi_rs_convert_resource_to_aml(struct acpi_resource *resource,
value));
break;
+ case ACPI_RSC_6BITFLAG:
+ /*
+ * Mask and shift the flag bits
+ */
+ ACPI_SET_BIT(*ACPI_CAST8(destination), (u8)
+ ((ACPI_GET8(source) & 0x3F) << info->
+ value));
+ break;
+
case ACPI_RSC_COUNT:
item_count = ACPI_GET8(source);
diff --git a/drivers/acpi/acpica/rsserial.c b/drivers/acpi/acpica/rsserial.c
index 1b937d88980f..f9267956535c 100644
--- a/drivers/acpi/acpica/rsserial.c
+++ b/drivers/acpi/acpica/rsserial.c
@@ -187,6 +187,81 @@ struct acpi_rsconvert_info acpi_rs_convert_pin_function[13] = {
/*******************************************************************************
*
+ * acpi_rs_convert_csi2_serial_bus
+ *
+ ******************************************************************************/
+
+struct acpi_rsconvert_info acpi_rs_convert_csi2_serial_bus[14] = {
+ { ACPI_RSC_INITGET, ACPI_RESOURCE_TYPE_SERIAL_BUS,
+ ACPI_RS_SIZE(struct acpi_resource_csi2_serialbus),
+ ACPI_RSC_TABLE_SIZE(acpi_rs_convert_csi2_serial_bus) },
+
+ { ACPI_RSC_INITSET, ACPI_RESOURCE_NAME_SERIAL_BUS,
+ sizeof(struct aml_resource_csi2_serialbus),
+ 0 },
+
+ { ACPI_RSC_MOVE8, ACPI_RS_OFFSET(data.common_serial_bus.revision_id),
+ AML_OFFSET(common_serial_bus.revision_id),
+ 1 },
+
+ { ACPI_RSC_MOVE8, ACPI_RS_OFFSET(data.csi2_serial_bus.type),
+ AML_OFFSET(csi2_serial_bus.type),
+ 1 },
+
+ { ACPI_RSC_1BITFLAG,
+ ACPI_RS_OFFSET(data.csi2_serial_bus.producer_consumer),
+ AML_OFFSET(csi2_serial_bus.flags),
+ 1 },
+
+ { ACPI_RSC_1BITFLAG, ACPI_RS_OFFSET(data.csi2_serial_bus.slave_mode),
+ AML_OFFSET(csi2_serial_bus.flags),
+ 0 },
+
+ { ACPI_RSC_2BITFLAG, ACPI_RS_OFFSET(data.csi2_serial_bus.phy_type),
+ AML_OFFSET(csi2_serial_bus.type_specific_flags),
+ 0 },
+
+ { ACPI_RSC_6BITFLAG,
+ ACPI_RS_OFFSET(data.csi2_serial_bus.local_port_instance),
+ AML_OFFSET(csi2_serial_bus.type_specific_flags),
+ 2 },
+
+ { ACPI_RSC_MOVE8, ACPI_RS_OFFSET(data.csi2_serial_bus.type_revision_id),
+ AML_OFFSET(csi2_serial_bus.type_revision_id),
+ 1 },
+
+ /* Vendor data */
+
+ { ACPI_RSC_COUNT_SERIAL_VEN,
+ ACPI_RS_OFFSET(data.csi2_serial_bus.vendor_length),
+ AML_OFFSET(csi2_serial_bus.type_data_length),
+ AML_RESOURCE_CSI2_MIN_DATA_LEN },
+
+ { ACPI_RSC_MOVE_SERIAL_VEN,
+ ACPI_RS_OFFSET(data.csi2_serial_bus.vendor_data),
+ 0,
+ sizeof(struct aml_resource_csi2_serialbus) },
+
+ /* Resource Source */
+
+ { ACPI_RSC_MOVE8,
+ ACPI_RS_OFFSET(data.csi2_serial_bus.resource_source.index),
+ AML_OFFSET(csi2_serial_bus.res_source_index),
+ 1 },
+
+ { ACPI_RSC_COUNT_SERIAL_RES,
+ ACPI_RS_OFFSET(data.csi2_serial_bus.resource_source.string_length),
+ AML_OFFSET(csi2_serial_bus.type_data_length),
+ sizeof(struct aml_resource_csi2_serialbus) },
+
+ { ACPI_RSC_MOVE_SERIAL_RES,
+ ACPI_RS_OFFSET(data.csi2_serial_bus.resource_source.string_ptr),
+ AML_OFFSET(csi2_serial_bus.type_data_length),
+ sizeof(struct aml_resource_csi2_serialbus) },
+};
+
+/*******************************************************************************
+ *
* acpi_rs_convert_i2c_serial_bus
*
******************************************************************************/
diff --git a/drivers/acpi/acpica/utresdecode.c b/drivers/acpi/acpica/utresdecode.c
index 0a9c337346e8..85730fcd7d00 100644
--- a/drivers/acpi/acpica/utresdecode.c
+++ b/drivers/acpi/acpica/utresdecode.c
@@ -82,6 +82,13 @@ const char *acpi_gbl_mtp_decode[] = {
"AddressRangeNVS"
};
+const char *acpi_gbl_phy_decode[] = {
+ "Type C",
+ "Type D",
+ "Unknown Type",
+ "Unknown Type"
+};
+
const char *acpi_gbl_rng_decode[] = {
"InvalidRanges",
"NonISAOnlyRanges",
@@ -161,7 +168,8 @@ const char *acpi_gbl_sbt_decode[] = {
"/* UNKNOWN serial bus type */",
"I2C",
"SPI",
- "UART"
+ "UART",
+ "CSI2"
};
/* I2C serial bus access mode */
diff --git a/drivers/acpi/acpica/utresrc.c b/drivers/acpi/acpica/utresrc.c
index cba5505171da..16f9a7035b39 100644
--- a/drivers/acpi/acpica/utresrc.c
+++ b/drivers/acpi/acpica/utresrc.c
@@ -64,6 +64,7 @@ const u8 acpi_gbl_resource_aml_serial_bus_sizes[] = {
ACPI_AML_SIZE_LARGE(struct aml_resource_i2c_serialbus),
ACPI_AML_SIZE_LARGE(struct aml_resource_spi_serialbus),
ACPI_AML_SIZE_LARGE(struct aml_resource_uart_serialbus),
+ ACPI_AML_SIZE_LARGE(struct aml_resource_csi2_serialbus),
};
/*
diff --git a/drivers/acpi/apei/einj.c b/drivers/acpi/apei/einj.c
index 133156759551..328e8aeece6c 100644
--- a/drivers/acpi/apei/einj.c
+++ b/drivers/acpi/apei/einj.c
@@ -725,7 +725,6 @@ static int __init einj_init(void)
goto err_release;
}
- rc = -ENOMEM;
einj_param = einj_get_parameter_address();
if ((param_extension || acpi5) && einj_param) {
debugfs_create_x32("flags", S_IRUSR | S_IWUSR, einj_debug_dir,
diff --git a/drivers/acpi/apei/erst.c b/drivers/acpi/apei/erst.c
index b9597216d021..242f3c2d5533 100644
--- a/drivers/acpi/apei/erst.c
+++ b/drivers/acpi/apei/erst.c
@@ -54,7 +54,7 @@ EXPORT_SYMBOL_GPL(erst_disable);
static struct acpi_table_erst *erst_tab;
-/* ERST Error Log Address Range atrributes */
+/* ERST Error Log Address Range attributes */
#define ERST_RANGE_RESERVED 0x0001
#define ERST_RANGE_NVRAM 0x0002
#define ERST_RANGE_SLOW 0x0004
diff --git a/drivers/acpi/apei/hest.c b/drivers/acpi/apei/hest.c
index f220bb00e91b..277f00b288d1 100644
--- a/drivers/acpi/apei/hest.c
+++ b/drivers/acpi/apei/hest.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
- * APEI Hardware Error Souce Table support
+ * APEI Hardware Error Source Table support
*
* HEST describes error sources in detail; communicates operational
* parameters (i.e. severity levels, masking bits, and threshold
diff --git a/drivers/acpi/arm64/gtdt.c b/drivers/acpi/arm64/gtdt.c
index f2d0e5915dab..0a0a982f9c28 100644
--- a/drivers/acpi/arm64/gtdt.c
+++ b/drivers/acpi/arm64/gtdt.c
@@ -329,7 +329,7 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd,
int index)
{
struct platform_device *pdev;
- int irq = map_gt_gsi(wd->timer_interrupt, wd->timer_flags);
+ int irq;
/*
* According to SBSA specification the size of refresh and control
@@ -338,7 +338,7 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd,
struct resource res[] = {
DEFINE_RES_MEM(wd->control_frame_address, SZ_4K),
DEFINE_RES_MEM(wd->refresh_frame_address, SZ_4K),
- DEFINE_RES_IRQ(irq),
+ {},
};
int nr_res = ARRAY_SIZE(res);
@@ -348,10 +348,11 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd,
if (!(wd->refresh_frame_address && wd->control_frame_address)) {
pr_err(FW_BUG "failed to get the Watchdog base address.\n");
- acpi_unregister_gsi(wd->timer_interrupt);
return -EINVAL;
}
+ irq = map_gt_gsi(wd->timer_interrupt, wd->timer_flags);
+ res[2] = (struct resource)DEFINE_RES_IRQ(irq);
if (irq <= 0) {
pr_warn("failed to map the Watchdog interrupt.\n");
nr_res--;
@@ -364,7 +365,8 @@ static int __init gtdt_import_sbsa_gwdt(struct acpi_gtdt_watchdog *wd,
*/
pdev = platform_device_register_simple("sbsa-gwdt", index, res, nr_res);
if (IS_ERR(pdev)) {
- acpi_unregister_gsi(wd->timer_interrupt);
+ if (irq > 0)
+ acpi_unregister_gsi(wd->timer_interrupt);
return PTR_ERR(pdev);
}
diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 2494138a6905..3912a1f6058e 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -968,15 +968,16 @@ static int iort_pci_iommu_init(struct pci_dev *pdev, u16 alias, void *data)
static void iort_named_component_init(struct device *dev,
struct acpi_iort_node *node)
{
+ struct property_entry props[2] = {};
struct acpi_iort_named_component *nc;
- struct iommu_fwspec *fwspec = dev_iommu_fwspec_get(dev);
-
- if (!fwspec)
- return;
nc = (struct acpi_iort_named_component *)node->node_data;
- fwspec->num_pasid_bits = FIELD_GET(ACPI_IORT_NC_PASID_BITS,
- nc->node_flags);
+ props[0] = PROPERTY_ENTRY_U32("pasid-num-bits",
+ FIELD_GET(ACPI_IORT_NC_PASID_BITS,
+ nc->node_flags));
+
+ if (device_add_properties(dev, props))
+ dev_warn(dev, "Could not add device properties\n");
}
static int iort_nc_iommu_map(struct device *dev, struct acpi_iort_node *node)
diff --git a/drivers/acpi/battery.c b/drivers/acpi/battery.c
index b822f77afba6..dae91f906cea 100644
--- a/drivers/acpi/battery.c
+++ b/drivers/acpi/battery.c
@@ -74,16 +74,17 @@ enum {
ACPI_BATTERY_XINFO_PRESENT,
ACPI_BATTERY_QUIRK_PERCENTAGE_CAPACITY,
/* On Lenovo Thinkpad models from 2010 and 2011, the power unit
- switches between mWh and mAh depending on whether the system
- is running on battery or not. When mAh is the unit, most
- reported values are incorrect and need to be adjusted by
- 10000/design_voltage. Verified on x201, t410, t410s, and x220.
- Pre-2010 and 2012 models appear to always report in mWh and
- are thus unaffected (tested with t42, t61, t500, x200, x300,
- and x230). Also, in mid-2012 Lenovo issued a BIOS update for
- the 2011 models that fixes the issue (tested on x220 with a
- post-1.29 BIOS), but as of Nov. 2012, no such update is
- available for the 2010 models. */
+ * switches between mWh and mAh depending on whether the system
+ * is running on battery or not. When mAh is the unit, most
+ * reported values are incorrect and need to be adjusted by
+ * 10000/design_voltage. Verified on x201, t410, t410s, and x220.
+ * Pre-2010 and 2012 models appear to always report in mWh and
+ * are thus unaffected (tested with t42, t61, t500, x200, x300,
+ * and x230). Also, in mid-2012 Lenovo issued a BIOS update for
+ * the 2011 models that fixes the issue (tested on x220 with a
+ * post-1.29 BIOS), but as of Nov. 2012, no such update is
+ * available for the 2010 models.
+ */
ACPI_BATTERY_QUIRK_THINKPAD_MAH,
/* for batteries reporting current capacity with design capacity
* on a full charge, but showing degradation in full charge cap.
@@ -371,9 +372,7 @@ static enum power_supply_property energy_battery_full_cap_broken_props[] = {
POWER_SUPPLY_PROP_SERIAL_NUMBER,
};
-/* --------------------------------------------------------------------------
- Battery Management
- -------------------------------------------------------------------------- */
+/* Battery Management */
struct acpi_offsets {
size_t offset; /* offset inside struct acpi_sbs_battery */
u8 mode; /* int or string? */
@@ -431,6 +430,7 @@ static int extract_package(struct acpi_battery *battery,
{
int i;
union acpi_object *element;
+
if (package->type != ACPI_TYPE_PACKAGE)
return -EFAULT;
for (i = 0; i < num; ++i) {
@@ -439,6 +439,7 @@ static int extract_package(struct acpi_battery *battery,
element = &package->package.elements[i];
if (offsets[i].mode) {
u8 *ptr = (u8 *)battery + offsets[i].offset;
+
if (element->type == ACPI_TYPE_STRING ||
element->type == ACPI_TYPE_BUFFER)
strncpy(ptr, element->string.pointer, 32);
@@ -497,10 +498,12 @@ static int extract_battery_info(const int use_bix,
battery->design_capacity_warning *
10000 / battery->design_voltage;
/* Curiously, design_capacity_low, unlike the rest of them,
- is correct. */
+ * is correct.
+ */
/* capacity_granularity_* equal 1 on the systems tested, so
- it's impossible to tell if they would need an adjustment
- or not if their values were higher. */
+ * it's impossible to tell if they would need an adjustment
+ * or not if their values were higher.
+ */
}
if (test_bit(ACPI_BATTERY_QUIRK_DEGRADED_FULL_CHARGE, &battery->flags) &&
battery->capacity_now > battery->full_charge_capacity)
@@ -532,8 +535,8 @@ static int acpi_battery_get_info(struct acpi_battery *battery)
if (ACPI_FAILURE(status)) {
acpi_handle_info(battery->device->handle,
"%s evaluation failed: %s\n",
- use_bix ?"_BIX":"_BIF",
- acpi_format_exception(status));
+ use_bix ? "_BIX":"_BIF",
+ acpi_format_exception(status));
} else {
result = extract_battery_info(use_bix,
battery,
@@ -648,6 +651,7 @@ static ssize_t acpi_battery_alarm_show(struct device *dev,
char *buf)
{
struct acpi_battery *battery = to_acpi_battery(dev_get_drvdata(dev));
+
return sprintf(buf, "%d\n", battery->alarm * 1000);
}
@@ -657,6 +661,7 @@ static ssize_t acpi_battery_alarm_store(struct device *dev,
{
unsigned long x;
struct acpi_battery *battery = to_acpi_battery(dev_get_drvdata(dev));
+
if (sscanf(buf, "%lu\n", &x) == 1)
battery->alarm = x/1000;
if (acpi_battery_present(battery))
@@ -743,7 +748,7 @@ EXPORT_SYMBOL_GPL(battery_hook_register);
* This function gets called right after the battery sysfs
* attributes have been added, so that the drivers that
* define custom sysfs attributes can add their own.
-*/
+ */
static void battery_hook_add_battery(struct acpi_battery *battery)
{
struct acpi_battery_hook *hook_node, *tmp;
@@ -872,10 +877,12 @@ static void find_battery(const struct dmi_header *dm, void *private)
{
struct acpi_battery *battery = (struct acpi_battery *)private;
/* Note: the hardcoded offsets below have been extracted from
- the source code of dmidecode. */
+ * the source code of dmidecode.
+ */
if (dm->type == DMI_ENTRY_PORTABLE_BATTERY && dm->length >= 8) {
const u8 *dmi_data = (const u8 *)(dm + 1);
int dmi_capacity = get_unaligned((const u16 *)(dmi_data + 6));
+
if (dm->length >= 18)
dmi_capacity *= dmi_data[17];
if (battery->design_capacity * battery->design_voltage / 1000
@@ -917,6 +924,7 @@ static void acpi_battery_quirks(struct acpi_battery *battery)
if (battery->power_unit && dmi_name_in_vendors("LENOVO")) {
const char *s;
+
s = dmi_get_system_info(DMI_PRODUCT_VERSION);
if (s && !strncasecmp(s, "ThinkPad", 8)) {
dmi_walk(find_battery, battery);
@@ -1013,10 +1021,7 @@ static void acpi_battery_refresh(struct acpi_battery *battery)
sysfs_add_battery(battery);
}
-/* --------------------------------------------------------------------------
- Driver Interface
- -------------------------------------------------------------------------- */
-
+/* Driver Interface */
static void acpi_battery_notify(struct acpi_device *device, u32 event)
{
struct acpi_battery *battery = acpi_driver_data(device);
@@ -1026,11 +1031,11 @@ static void acpi_battery_notify(struct acpi_device *device, u32 event)
return;
old = battery->bat;
/*
- * On Acer Aspire V5-573G notifications are sometimes triggered too
- * early. For example, when AC is unplugged and notification is
- * triggered, battery state is still reported as "Full", and changes to
- * "Discharging" only after short delay, without any notification.
- */
+ * On Acer Aspire V5-573G notifications are sometimes triggered too
+ * early. For example, when AC is unplugged and notification is
+ * triggered, battery state is still reported as "Full", and changes to
+ * "Discharging" only after short delay, without any notification.
+ */
if (battery_notification_delay_ms > 0)
msleep(battery_notification_delay_ms);
if (event == ACPI_BATTERY_NOTIFY_INFO)
diff --git a/drivers/acpi/button.c b/drivers/acpi/button.c
index 85e5e0328a2e..f25bd336113b 100644
--- a/drivers/acpi/button.c
+++ b/drivers/acpi/button.c
@@ -156,10 +156,7 @@ static unsigned long lid_report_interval __read_mostly = 500;
module_param(lid_report_interval, ulong, 0644);
MODULE_PARM_DESC(lid_report_interval, "Interval (ms) between lid key events");
-/* --------------------------------------------------------------------------
- FS Interface (/proc)
- -------------------------------------------------------------------------- */
-
+/* FS Interface (/proc) */
static struct proc_dir_entry *acpi_button_dir;
static struct proc_dir_entry *acpi_lid_dir;
@@ -348,9 +345,7 @@ static int acpi_button_remove_fs(struct acpi_device *device)
return 0;
}
-/* --------------------------------------------------------------------------
- Driver Interface
- -------------------------------------------------------------------------- */
+/* Driver Interface */
int acpi_lid_open(void)
{
if (!lid_device)
diff --git a/drivers/acpi/cppc_acpi.c b/drivers/acpi/cppc_acpi.c
index 69057fcd2c04..a4d4eebba1da 100644
--- a/drivers/acpi/cppc_acpi.c
+++ b/drivers/acpi/cppc_acpi.c
@@ -33,7 +33,6 @@
#define pr_fmt(fmt) "ACPI CPPC: " fmt
-#include <linux/cpufreq.h>
#include <linux/delay.h>
#include <linux/iopoll.h>
#include <linux/ktime.h>
@@ -101,14 +100,14 @@ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr);
(cpc)->cpc_entry.reg.space_id == \
ACPI_ADR_SPACE_PLATFORM_COMM)
-/* Evalutes to True if reg is a NULL register descriptor */
+/* Evaluates to True if reg is a NULL register descriptor */
#define IS_NULL_REG(reg) ((reg)->space_id == ACPI_ADR_SPACE_SYSTEM_MEMORY && \
(reg)->address == 0 && \
(reg)->bit_width == 0 && \
(reg)->bit_offset == 0 && \
(reg)->access_width == 0)
-/* Evalutes to True if an optional cpc field is supported */
+/* Evaluates to True if an optional cpc field is supported */
#define CPC_SUPPORTED(cpc) ((cpc)->type == ACPI_TYPE_INTEGER ? \
!!(cpc)->cpc_entry.int_value : \
!IS_NULL_REG(&(cpc)->cpc_entry.reg))
@@ -119,23 +118,15 @@ static DEFINE_PER_CPU(struct cpc_desc *, cpc_desc_ptr);
*/
#define NUM_RETRIES 500ULL
-struct cppc_attr {
- struct attribute attr;
- ssize_t (*show)(struct kobject *kobj,
- struct attribute *attr, char *buf);
- ssize_t (*store)(struct kobject *kobj,
- struct attribute *attr, const char *c, ssize_t count);
-};
-
#define define_one_cppc_ro(_name) \
-static struct cppc_attr _name = \
+static struct kobj_attribute _name = \
__ATTR(_name, 0444, show_##_name, NULL)
#define to_cpc_desc(a) container_of(a, struct cpc_desc, kobj)
#define show_cppc_data(access_fn, struct_name, member_name) \
static ssize_t show_##member_name(struct kobject *kobj, \
- struct attribute *attr, char *buf) \
+ struct kobj_attribute *attr, char *buf) \
{ \
struct cpc_desc *cpc_ptr = to_cpc_desc(kobj); \
struct struct_name st_name = {0}; \
@@ -161,7 +152,7 @@ show_cppc_data(cppc_get_perf_ctrs, cppc_perf_fb_ctrs, reference_perf);
show_cppc_data(cppc_get_perf_ctrs, cppc_perf_fb_ctrs, wraparound_time);
static ssize_t show_feedback_ctrs(struct kobject *kobj,
- struct attribute *attr, char *buf)
+ struct kobj_attribute *attr, char *buf)
{
struct cpc_desc *cpc_ptr = to_cpc_desc(kobj);
struct cppc_perf_fb_ctrs fb_ctrs = {0};
@@ -327,6 +318,7 @@ end:
if (unlikely(ret)) {
for_each_possible_cpu(i) {
struct cpc_desc *desc = per_cpu(cpc_desc_ptr, i);
+
if (!desc)
continue;
@@ -778,7 +770,7 @@ int acpi_cppc_processor_probe(struct acpi_processor *pr)
cpc_ptr->cpc_regs[i-2].type = ACPI_TYPE_BUFFER;
memcpy(&cpc_ptr->cpc_regs[i-2].cpc_entry.reg, gas_t, sizeof(*gas_t));
} else {
- pr_debug("Err in entry:%d in CPC table of CPU:%d \n", i, pr->id);
+ pr_debug("Err in entry:%d in CPC table of CPU:%d\n", i, pr->id);
goto out_free;
}
}
@@ -868,7 +860,7 @@ void acpi_cppc_processor_exit(struct acpi_processor *pr)
void __iomem *addr;
int pcc_ss_id = per_cpu(cpu_pcc_subspace_idx, pr->id);
- if (pcc_ss_id >=0 && pcc_data[pcc_ss_id]) {
+ if (pcc_ss_id >= 0 && pcc_data[pcc_ss_id]) {
if (pcc_data[pcc_ss_id]->pcc_channel_acquired) {
pcc_data[pcc_ss_id]->refcount--;
if (!pcc_data[pcc_ss_id]->refcount) {
@@ -955,22 +947,22 @@ static int cpc_read(int cpu, struct cpc_register_resource *reg_res, u64 *val)
val, reg->bit_width);
switch (reg->bit_width) {
- case 8:
- *val = readb_relaxed(vaddr);
- break;
- case 16:
- *val = readw_relaxed(vaddr);
- break;
- case 32:
- *val = readl_relaxed(vaddr);
- break;
- case 64:
- *val = readq_relaxed(vaddr);
- break;
- default:
- pr_debug("Error: Cannot read %u bit width from PCC for ss: %d\n",
- reg->bit_width, pcc_ss_id);
- ret_val = -EFAULT;
+ case 8:
+ *val = readb_relaxed(vaddr);
+ break;
+ case 16:
+ *val = readw_relaxed(vaddr);
+ break;
+ case 32:
+ *val = readl_relaxed(vaddr);
+ break;
+ case 64:
+ *val = readq_relaxed(vaddr);
+ break;
+ default:
+ pr_debug("Error: Cannot read %u bit width from PCC for ss: %d\n",
+ reg->bit_width, pcc_ss_id);
+ ret_val = -EFAULT;
}
return ret_val;
@@ -994,23 +986,23 @@ static int cpc_write(int cpu, struct cpc_register_resource *reg_res, u64 val)
val, reg->bit_width);
switch (reg->bit_width) {
- case 8:
- writeb_relaxed(val, vaddr);
- break;
- case 16:
- writew_relaxed(val, vaddr);
- break;
- case 32:
- writel_relaxed(val, vaddr);
- break;
- case 64:
- writeq_relaxed(val, vaddr);
- break;
- default:
- pr_debug("Error: Cannot write %u bit width to PCC for ss: %d\n",
- reg->bit_width, pcc_ss_id);
- ret_val = -EFAULT;
- break;
+ case 8:
+ writeb_relaxed(val, vaddr);
+ break;
+ case 16:
+ writew_relaxed(val, vaddr);
+ break;
+ case 32:
+ writel_relaxed(val, vaddr);
+ break;
+ case 64:
+ writeq_relaxed(val, vaddr);
+ break;
+ default:
+ pr_debug("Error: Cannot write %u bit width to PCC for ss: %d\n",
+ reg->bit_width, pcc_ss_id);
+ ret_val = -EFAULT;
+ break;
}
return ret_val;
@@ -1330,7 +1322,7 @@ int cppc_set_perf(int cpu, struct cppc_perf_ctrls *perf_ctrls)
* is still with OSPM.
* pending_pcc_write_cmd can also be cleared by a different CPU, if
* there was a pcc CMD_READ waiting on down_write and it steals the lock
- * before the pcc CMD_WRITE is completed. pcc_send_cmd checks for this
+ * before the pcc CMD_WRITE is completed. send_pcc_cmd checks for this
* case during a CMD_READ and if there are pending writes it delivers
* the write command before servicing the read command
*/
@@ -1355,8 +1347,8 @@ EXPORT_SYMBOL_GPL(cppc_set_perf);
/**
* cppc_get_transition_latency - returns frequency transition latency in ns
*
- * ACPI CPPC does not explicitly specifiy how a platform can specify the
- * transition latency for perfromance change requests. The closest we have
+ * ACPI CPPC does not explicitly specify how a platform can specify the
+ * transition latency for performance change requests. The closest we have
* is the timing information from the PCCT tables which provides the info
* on the number and frequency of PCC commands the platform can handle.
*/
diff --git a/drivers/acpi/custom_method.c b/drivers/acpi/custom_method.c
index 7b54dc95d36b..d39a9b474727 100644
--- a/drivers/acpi/custom_method.c
+++ b/drivers/acpi/custom_method.c
@@ -19,7 +19,7 @@ static struct dentry *cm_dentry;
/* /sys/kernel/debug/acpi/custom_method */
-static ssize_t cm_write(struct file *file, const char __user * user_buf,
+static ssize_t cm_write(struct file *file, const char __user *user_buf,
size_t count, loff_t *ppos)
{
static char *buf;
@@ -42,6 +42,8 @@ static ssize_t cm_write(struct file *file, const char __user * user_buf,
sizeof(struct acpi_table_header)))
return -EFAULT;
uncopied_bytes = max_size = table.length;
+ /* make sure the buf is not allocated */
+ kfree(buf);
buf = kzalloc(max_size, GFP_KERNEL);
if (!buf)
return -ENOMEM;
@@ -55,6 +57,7 @@ static ssize_t cm_write(struct file *file, const char __user * user_buf,
(*ppos + count < count) ||
(count > uncopied_bytes)) {
kfree(buf);
+ buf = NULL;
return -EINVAL;
}
@@ -76,7 +79,6 @@ static ssize_t cm_write(struct file *file, const char __user * user_buf,
add_taint(TAINT_OVERRIDDEN_ACPI_TABLE, LOCKDEP_NOW_UNRELIABLE);
}
- kfree(buf);
return count;
}
diff --git a/drivers/acpi/device_pm.c b/drivers/acpi/device_pm.c
index 096153761ebc..d260bc1f3e6e 100644
--- a/drivers/acpi/device_pm.c
+++ b/drivers/acpi/device_pm.c
@@ -966,6 +966,7 @@ EXPORT_SYMBOL_GPL(acpi_dev_resume);
int acpi_subsys_runtime_suspend(struct device *dev)
{
int ret = pm_generic_runtime_suspend(dev);
+
return ret ? ret : acpi_dev_suspend(dev, true);
}
EXPORT_SYMBOL_GPL(acpi_subsys_runtime_suspend);
@@ -980,6 +981,7 @@ EXPORT_SYMBOL_GPL(acpi_subsys_runtime_suspend);
int acpi_subsys_runtime_resume(struct device *dev)
{
int ret = acpi_dev_resume(dev);
+
return ret ? ret : pm_generic_runtime_resume(dev);
}
EXPORT_SYMBOL_GPL(acpi_subsys_runtime_resume);
@@ -1171,6 +1173,7 @@ EXPORT_SYMBOL_GPL(acpi_subsys_freeze);
int acpi_subsys_restore_early(struct device *dev)
{
int ret = acpi_dev_resume(dev);
+
return ret ? ret : pm_generic_restore_early(dev);
}
EXPORT_SYMBOL_GPL(acpi_subsys_restore_early);
@@ -1310,6 +1313,7 @@ int acpi_dev_pm_attach(struct device *dev, bool power_on)
{"PNP0C0B", }, /* Generic ACPI fan */
{"INT3404", }, /* Fan */
{"INTC1044", }, /* Fan for Tiger Lake generation */
+ {"INTC1048", }, /* Fan for Alder Lake generation */
{}
};
struct acpi_device *adev = ACPI_COMPANION(dev);
diff --git a/drivers/acpi/device_sysfs.c b/drivers/acpi/device_sysfs.c
index da4ff2a8b06a..fa2c1c93072c 100644
--- a/drivers/acpi/device_sysfs.c
+++ b/drivers/acpi/device_sysfs.c
@@ -73,6 +73,7 @@ static const struct sysfs_ops acpi_data_node_sysfs_ops = {
static void acpi_data_node_release(struct kobject *kobj)
{
struct acpi_data_node *dn = to_data_node(kobj);
+
complete(&dn->kobj_done);
}
@@ -130,7 +131,7 @@ static void acpi_hide_nondev_subnodes(struct acpi_device_data *data)
* Return: 0: no _HID and no _CID
* -EINVAL: output error
* -ENOMEM: output is truncated
-*/
+ */
static int create_pnp_modalias(struct acpi_device *acpi_dev, char *modalias,
int size)
{
@@ -376,12 +377,12 @@ eject_store(struct device *d, struct device_attribute *attr,
if (ACPI_FAILURE(status) || !acpi_device->flags.ejectable)
return -ENODEV;
- get_device(&acpi_device->dev);
+ acpi_dev_get(acpi_device);
status = acpi_hotplug_schedule(acpi_device, ACPI_OST_EC_OSPM_EJECT);
if (ACPI_SUCCESS(status))
return count;
- put_device(&acpi_device->dev);
+ acpi_dev_put(acpi_device);
acpi_evaluate_ost(acpi_device->handle, ACPI_OST_EC_OSPM_EJECT,
ACPI_OST_SC_NON_SPECIFIC_FAILURE, NULL);
return status == AE_NO_MEMORY ? -ENOMEM : -EAGAIN;
@@ -431,7 +432,8 @@ static DEVICE_ATTR_RO(path);
/* sysfs file that shows description text from the ACPI _STR method */
static ssize_t description_show(struct device *dev,
struct device_attribute *attr,
- char *buf) {
+ char *buf)
+{
struct acpi_device *acpi_dev = to_acpi_device(dev);
int result;
@@ -456,7 +458,8 @@ static DEVICE_ATTR_RO(description);
static ssize_t
sun_show(struct device *dev, struct device_attribute *attr,
- char *buf) {
+ char *buf)
+{
struct acpi_device *acpi_dev = to_acpi_device(dev);
acpi_status status;
unsigned long long sun;
@@ -471,7 +474,8 @@ static DEVICE_ATTR_RO(sun);
static ssize_t
hrv_show(struct device *dev, struct device_attribute *attr,
- char *buf) {
+ char *buf)
+{
struct acpi_device *acpi_dev = to_acpi_device(dev);
acpi_status status;
unsigned long long hrv;
@@ -485,7 +489,8 @@ hrv_show(struct device *dev, struct device_attribute *attr,
static DEVICE_ATTR_RO(hrv);
static ssize_t status_show(struct device *dev, struct device_attribute *attr,
- char *buf) {
+ char *buf)
+{
struct acpi_device *acpi_dev = to_acpi_device(dev);
acpi_status status;
unsigned long long sta;
diff --git a/drivers/acpi/dock.c b/drivers/acpi/dock.c
index 0937ceab052e..7cf92158008f 100644
--- a/drivers/acpi/dock.c
+++ b/drivers/acpi/dock.c
@@ -271,6 +271,7 @@ static void hotplug_dock_devices(struct dock_station *ds, u32 event)
if (!acpi_device_enumerated(adev)) {
int ret = acpi_bus_scan(adev->handle);
+
if (ret)
dev_dbg(&adev->dev, "scan error %d\n", -ret);
}
@@ -502,6 +503,7 @@ static ssize_t flags_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct dock_station *dock_station = dev->platform_data;
+
return snprintf(buf, PAGE_SIZE, "%d\n", dock_station->flags);
}
@@ -523,7 +525,7 @@ static ssize_t undock_store(struct device *dev, struct device_attribute *attr,
begin_undock(dock_station);
ret = handle_eject_request(dock_station, ACPI_NOTIFY_EJECT_REQUEST);
acpi_scan_lock_release();
- return ret ? ret: count;
+ return ret ? ret : count;
}
static DEVICE_ATTR_WO(undock);
@@ -535,10 +537,11 @@ static ssize_t uid_show(struct device *dev,
{
unsigned long long lbuf;
struct dock_station *dock_station = dev->platform_data;
+
acpi_status status = acpi_evaluate_integer(dock_station->handle,
"_UID", NULL, &lbuf);
if (ACPI_FAILURE(status))
- return 0;
+ return 0;
return snprintf(buf, PAGE_SIZE, "%llx\n", lbuf);
}
diff --git a/drivers/acpi/glue.c b/drivers/acpi/glue.c
index 36b24b0658cb..0715e3be99a0 100644
--- a/drivers/acpi/glue.c
+++ b/drivers/acpi/glue.c
@@ -190,7 +190,7 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
if (!acpi_dev)
return -EINVAL;
- get_device(&acpi_dev->dev);
+ acpi_dev_get(acpi_dev);
get_device(dev);
physical_node = kzalloc(sizeof(*physical_node), GFP_KERNEL);
if (!physical_node) {
@@ -217,7 +217,7 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
goto err;
put_device(dev);
- put_device(&acpi_dev->dev);
+ acpi_dev_put(acpi_dev);
return 0;
}
if (pn->node_id == node_id) {
@@ -257,7 +257,7 @@ int acpi_bind_one(struct device *dev, struct acpi_device *acpi_dev)
err:
ACPI_COMPANION_SET(dev, NULL);
put_device(dev);
- put_device(&acpi_dev->dev);
+ acpi_dev_put(acpi_dev);
return retval;
}
EXPORT_SYMBOL_GPL(acpi_bind_one);
@@ -285,7 +285,7 @@ int acpi_unbind_one(struct device *dev)
ACPI_COMPANION_SET(dev, NULL);
/* Drop references taken by acpi_bind_one(). */
put_device(dev);
- put_device(&acpi_dev->dev);
+ acpi_dev_put(acpi_dev);
kfree(entry);
break;
}
diff --git a/drivers/acpi/hed.c b/drivers/acpi/hed.c
index cf148287e2ba..60a2939cde6c 100644
--- a/drivers/acpi/hed.c
+++ b/drivers/acpi/hed.c
@@ -74,7 +74,6 @@ static struct acpi_driver acpi_hed_driver = {
};
module_acpi_driver(acpi_hed_driver);
-ACPI_MODULE_NAME("hed");
MODULE_AUTHOR("Huang Ying");
MODULE_DESCRIPTION("ACPI Hardware Error Device Driver");
MODULE_LICENSE("GPL");
diff --git a/drivers/acpi/internal.h b/drivers/acpi/internal.h
index e6a5d997241c..f973bbe90e5e 100644
--- a/drivers/acpi/internal.h
+++ b/drivers/acpi/internal.h
@@ -9,6 +9,8 @@
#ifndef _ACPI_INTERNAL_H_
#define _ACPI_INTERNAL_H_
+#include <linux/idr.h>
+
#define PREFIX "ACPI: "
int early_acpi_osi_init(void);
@@ -96,17 +98,18 @@ void acpi_scan_table_handler(u32 event, void *table, void *context);
extern struct list_head acpi_bus_id_list;
+#define ACPI_MAX_DEVICE_INSTANCES 4096
+
struct acpi_device_bus_id {
const char *bus_id;
- unsigned int instance_no;
+ struct ida instance_ida;
struct list_head node;
};
int acpi_device_add(struct acpi_device *device,
void (*release)(struct device *));
void acpi_init_device_object(struct acpi_device *device, acpi_handle handle,
- int type, unsigned long long sta,
- struct acpi_device_info *info);
+ int type);
int acpi_device_setup_files(struct acpi_device *dev);
void acpi_device_remove_files(struct acpi_device *dev);
void acpi_device_add_finalize(struct acpi_device *device);
@@ -139,6 +142,7 @@ int acpi_device_sleep_wake(struct acpi_device *dev,
int acpi_power_get_inferred_state(struct acpi_device *device, int *state);
int acpi_power_on_resources(struct acpi_device *device, int state);
int acpi_power_transition(struct acpi_device *device, int state);
+void acpi_turn_off_unused_power_resources(void);
/* --------------------------------------------------------------------------
Device Power Management
diff --git a/drivers/acpi/irq.c b/drivers/acpi/irq.c
index e209081d644b..c68e694fca26 100644
--- a/drivers/acpi/irq.c
+++ b/drivers/acpi/irq.c
@@ -75,8 +75,12 @@ void acpi_unregister_gsi(u32 gsi)
{
struct irq_domain *d = irq_find_matching_fwnode(acpi_gsi_domain_id,
DOMAIN_BUS_ANY);
- int irq = irq_find_mapping(d, gsi);
+ int irq;
+ if (WARN_ON(acpi_irq_model == ACPI_IRQ_MODEL_GIC && gsi < 16))
+ return;
+
+ irq = irq_find_mapping(d, gsi);
irq_dispose_mapping(irq);
}
EXPORT_SYMBOL_GPL(acpi_unregister_gsi);
diff --git a/drivers/acpi/nfit/core.c b/drivers/acpi/nfit/core.c
index 8c5dde628405..23d9a09d7060 100644
--- a/drivers/acpi/nfit/core.c
+++ b/drivers/acpi/nfit/core.c
@@ -686,6 +686,13 @@ int nfit_spa_type(struct acpi_nfit_system_address *spa)
return -1;
}
+static size_t sizeof_spa(struct acpi_nfit_system_address *spa)
+{
+ if (spa->flags & ACPI_NFIT_LOCATION_COOKIE_VALID)
+ return sizeof(*spa);
+ return sizeof(*spa) - 8;
+}
+
static bool add_spa(struct acpi_nfit_desc *acpi_desc,
struct nfit_table_prev *prev,
struct acpi_nfit_system_address *spa)
@@ -693,22 +700,22 @@ static bool add_spa(struct acpi_nfit_desc *acpi_desc,
struct device *dev = acpi_desc->dev;
struct nfit_spa *nfit_spa;
- if (spa->header.length != sizeof(*spa))
+ if (spa->header.length != sizeof_spa(spa))
return false;
list_for_each_entry(nfit_spa, &prev->spas, list) {
- if (memcmp(nfit_spa->spa, spa, sizeof(*spa)) == 0) {
+ if (memcmp(nfit_spa->spa, spa, sizeof_spa(spa)) == 0) {
list_move_tail(&nfit_spa->list, &acpi_desc->spas);
return true;
}
}
- nfit_spa = devm_kzalloc(dev, sizeof(*nfit_spa) + sizeof(*spa),
+ nfit_spa = devm_kzalloc(dev, sizeof(*nfit_spa) + sizeof_spa(spa),
GFP_KERNEL);
if (!nfit_spa)
return false;
INIT_LIST_HEAD(&nfit_spa->list);
- memcpy(nfit_spa->spa, spa, sizeof(*spa));
+ memcpy(nfit_spa->spa, spa, sizeof_spa(spa));
list_add_tail(&nfit_spa->list, &acpi_desc->spas);
dev_dbg(dev, "spa index: %d type: %s\n",
spa->range_index,
@@ -1195,7 +1202,8 @@ static int __nfit_mem_init(struct acpi_nfit_desc *acpi_desc,
return 0;
}
-static int nfit_mem_cmp(void *priv, struct list_head *_a, struct list_head *_b)
+static int nfit_mem_cmp(void *priv, const struct list_head *_a,
+ const struct list_head *_b)
{
struct nfit_mem *a = container_of(_a, typeof(*a), list);
struct nfit_mem *b = container_of(_b, typeof(*b), list);
@@ -3831,7 +3839,7 @@ static __init int nfit_init(void)
int ret;
BUILD_BUG_ON(sizeof(struct acpi_table_nfit) != 40);
- BUILD_BUG_ON(sizeof(struct acpi_nfit_system_address) != 56);
+ BUILD_BUG_ON(sizeof(struct acpi_nfit_system_address) != 64);
BUILD_BUG_ON(sizeof(struct acpi_nfit_memory_map) != 48);
BUILD_BUG_ON(sizeof(struct acpi_nfit_interleave) != 20);
BUILD_BUG_ON(sizeof(struct acpi_nfit_smbios) != 9);
diff --git a/drivers/acpi/numa/hmat.c b/drivers/acpi/numa/hmat.c
index cb73a5d6ea76..c3d783aca196 100644
--- a/drivers/acpi/numa/hmat.c
+++ b/drivers/acpi/numa/hmat.c
@@ -558,7 +558,8 @@ static bool hmat_update_best(u8 type, u32 value, u32 *best)
return updated;
}
-static int initiator_cmp(void *priv, struct list_head *a, struct list_head *b)
+static int initiator_cmp(void *priv, const struct list_head *a,
+ const struct list_head *b)
{
struct memory_initiator *ia;
struct memory_initiator *ib;
@@ -722,7 +723,7 @@ static void hmat_register_target(struct memory_target *target)
/*
* Skip offline nodes. This can happen when memory
* marked EFI_MEMORY_SP, "specific purpose", is applied
- * to all the memory in a promixity domain leading to
+ * to all the memory in a proximity domain leading to
* the node being marked offline / unplugged, or if
* memory-only "hotplug" node is offline.
*/
diff --git a/drivers/acpi/pci_irq.c b/drivers/acpi/pci_irq.c
index 14ee631cb7cf..08e15774fb9f 100644
--- a/drivers/acpi/pci_irq.c
+++ b/drivers/acpi/pci_irq.c
@@ -9,6 +9,7 @@
* Bjorn Helgaas <bjorn.helgaas@hp.com>
*/
+#define pr_fmt(fmt) "ACPI: PCI: " fmt
#include <linux/dmi.h>
#include <linux/kernel.h>
@@ -22,11 +23,6 @@
#include <linux/slab.h>
#include <linux/interrupt.h>
-#define PREFIX "ACPI: "
-
-#define _COMPONENT ACPI_PCI_COMPONENT
-ACPI_MODULE_NAME("pci_irq");
-
struct acpi_prt_entry {
struct acpi_pci_id id;
u8 pin;
@@ -126,7 +122,7 @@ static void do_prt_fixups(struct acpi_prt_entry *entry,
entry->pin == quirk->pin &&
!strcmp(prt->source, quirk->source) &&
strlen(prt->source) >= strlen(quirk->actual_source)) {
- printk(KERN_WARNING PREFIX "firmware reports "
+ pr_warn("Firmware reports "
"%04x:%02x:%02x PCI INT %c connected to %s; "
"changing to %s\n",
entry->id.segment, entry->id.bus,
@@ -191,12 +187,9 @@ static int acpi_pci_irq_check_entry(acpi_handle handle, struct pci_dev *dev,
* the IRQ value, which is hardwired to specific interrupt inputs on
* the interrupt controller.
*/
-
- ACPI_DEBUG_PRINT_RAW((ACPI_DB_INFO,
- " %04x:%02x:%02x[%c] -> %s[%d]\n",
- entry->id.segment, entry->id.bus,
- entry->id.device, pin_name(entry->pin),
- prt->source, entry->index));
+ pr_debug("%04x:%02x:%02x[%c] -> %s[%d]\n",
+ entry->id.segment, entry->id.bus, entry->id.device,
+ pin_name(entry->pin), prt->source, entry->index);
*entry_ptr = entry;
@@ -307,8 +300,7 @@ static struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
#ifdef CONFIG_X86_IO_APIC
acpi_reroute_boot_interrupt(dev, entry);
#endif /* CONFIG_X86_IO_APIC */
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %s[%c] _PRT entry\n",
- pci_name(dev), pin_name(pin)));
+ dev_dbg(&dev->dev, "Found [%c] _PRT entry\n", pin_name(pin));
return entry;
}
@@ -324,9 +316,7 @@ static struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
/* PC card has the same IRQ as its cardbridge */
bridge_pin = bridge->pin;
if (!bridge_pin) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "No interrupt pin configured for device %s\n",
- pci_name(bridge)));
+ dev_dbg(&bridge->dev, "No interrupt pin configured\n");
return NULL;
}
pin = bridge_pin;
@@ -334,10 +324,8 @@ static struct acpi_prt_entry *acpi_pci_irq_lookup(struct pci_dev *dev, int pin)
ret = acpi_pci_irq_find_prt_entry(bridge, pin, &entry);
if (!ret && entry) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Derived GSI for %s INT %c from %s\n",
- pci_name(dev), pin_name(orig_pin),
- pci_name(bridge)));
+ dev_dbg(&dev->dev, "Derived GSI INT %c from %s\n",
+ pin_name(orig_pin), pci_name(bridge));
return entry;
}
@@ -413,9 +401,7 @@ int acpi_pci_irq_enable(struct pci_dev *dev)
pin = dev->pin;
if (!pin) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "No interrupt pin configured for device %s\n",
- pci_name(dev)));
+ dev_dbg(&dev->dev, "No interrupt pin configured\n");
return 0;
}
diff --git a/drivers/acpi/pci_link.c b/drivers/acpi/pci_link.c
index fb4c5632a232..cb7b900d9466 100644
--- a/drivers/acpi/pci_link.c
+++ b/drivers/acpi/pci_link.c
@@ -12,6 +12,8 @@
* for IRQ management (e.g. start()->_SRS).
*/
+#define pr_fmt(fmt) "ACPI: PCI: " fmt
+
#include <linux/syscore_ops.h>
#include <linux/kernel.h>
#include <linux/module.h>
@@ -27,8 +29,6 @@
#include "internal.h"
-#define _COMPONENT ACPI_PCI_COMPONENT
-ACPI_MODULE_NAME("pci_link");
#define ACPI_PCI_LINK_CLASS "pci_irq_routing"
#define ACPI_PCI_LINK_DEVICE_NAME "PCI Interrupt Link"
#define ACPI_PCI_LINK_MAX_POSSIBLE 16
@@ -85,6 +85,7 @@ static acpi_status acpi_pci_link_check_possible(struct acpi_resource *resource,
void *context)
{
struct acpi_pci_link *link = context;
+ acpi_handle handle = link->device->handle;
u32 i;
switch (resource->type) {
@@ -95,17 +96,17 @@ static acpi_status acpi_pci_link_check_possible(struct acpi_resource *resource,
{
struct acpi_resource_irq *p = &resource->data.irq;
if (!p || !p->interrupt_count) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Blank _PRS IRQ resource\n"));
+ acpi_handle_debug(handle,
+ "Blank _PRS IRQ resource\n");
return AE_OK;
}
for (i = 0;
(i < p->interrupt_count
&& i < ACPI_PCI_LINK_MAX_POSSIBLE); i++) {
if (!p->interrupts[i]) {
- printk(KERN_WARNING PREFIX
- "Invalid _PRS IRQ %d\n",
- p->interrupts[i]);
+ acpi_handle_debug(handle,
+ "Invalid _PRS IRQ %d\n",
+ p->interrupts[i]);
continue;
}
link->irq.possible[i] = p->interrupts[i];
@@ -121,17 +122,17 @@ static acpi_status acpi_pci_link_check_possible(struct acpi_resource *resource,
struct acpi_resource_extended_irq *p =
&resource->data.extended_irq;
if (!p || !p->interrupt_count) {
- printk(KERN_WARNING PREFIX
- "Blank _PRS EXT IRQ resource\n");
+ acpi_handle_debug(handle,
+ "Blank _PRS EXT IRQ resource\n");
return AE_OK;
}
for (i = 0;
(i < p->interrupt_count
&& i < ACPI_PCI_LINK_MAX_POSSIBLE); i++) {
if (!p->interrupts[i]) {
- printk(KERN_WARNING PREFIX
- "Invalid _PRS IRQ %d\n",
- p->interrupts[i]);
+ acpi_handle_debug(handle,
+ "Invalid _PRS IRQ %d\n",
+ p->interrupts[i]);
continue;
}
link->irq.possible[i] = p->interrupts[i];
@@ -143,8 +144,8 @@ static acpi_status acpi_pci_link_check_possible(struct acpi_resource *resource,
break;
}
default:
- printk(KERN_ERR PREFIX "_PRS resource type 0x%x isn't an IRQ\n",
- resource->type);
+ acpi_handle_debug(handle, "_PRS resource type 0x%x is not IRQ\n",
+ resource->type);
return AE_OK;
}
@@ -153,18 +154,18 @@ static acpi_status acpi_pci_link_check_possible(struct acpi_resource *resource,
static int acpi_pci_link_get_possible(struct acpi_pci_link *link)
{
+ acpi_handle handle = link->device->handle;
acpi_status status;
- status = acpi_walk_resources(link->device->handle, METHOD_NAME__PRS,
+ status = acpi_walk_resources(handle, METHOD_NAME__PRS,
acpi_pci_link_check_possible, link);
if (ACPI_FAILURE(status)) {
- acpi_handle_debug(link->device->handle, "_PRS not present or invalid");
+ acpi_handle_debug(handle, "_PRS not present or invalid");
return 0;
}
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Found %d possible IRQs\n",
- link->irq.possible_count));
+ acpi_handle_debug(handle, "Found %d possible IRQs\n",
+ link->irq.possible_count);
return 0;
}
@@ -186,8 +187,7 @@ static acpi_status acpi_pci_link_check_current(struct acpi_resource *resource,
* IRQ descriptors may have no IRQ# bits set,
* particularly those those w/ _STA disabled
*/
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Blank _CRS IRQ resource\n"));
+ pr_debug("Blank _CRS IRQ resource\n");
return AE_OK;
}
*irq = p->interrupts[0];
@@ -202,8 +202,7 @@ static acpi_status acpi_pci_link_check_current(struct acpi_resource *resource,
* extended IRQ descriptors must
* return at least 1 IRQ
*/
- printk(KERN_WARNING PREFIX
- "Blank _CRS EXT IRQ resource\n");
+ pr_debug("Blank _CRS EXT IRQ resource\n");
return AE_OK;
}
*irq = p->interrupts[0];
@@ -211,8 +210,8 @@ static acpi_status acpi_pci_link_check_current(struct acpi_resource *resource,
}
break;
default:
- printk(KERN_ERR PREFIX "_CRS resource type 0x%x isn't an IRQ\n",
- resource->type);
+ pr_debug("_CRS resource type 0x%x is not IRQ\n",
+ resource->type);
return AE_OK;
}
@@ -228,8 +227,9 @@ static acpi_status acpi_pci_link_check_current(struct acpi_resource *resource,
*/
static int acpi_pci_link_get_current(struct acpi_pci_link *link)
{
- int result = 0;
+ acpi_handle handle = link->device->handle;
acpi_status status;
+ int result = 0;
int irq = 0;
link->irq.active = 0;
@@ -239,12 +239,12 @@ static int acpi_pci_link_get_current(struct acpi_pci_link *link)
/* Query _STA, set link->device->status */
result = acpi_bus_get_status(link->device);
if (result) {
- printk(KERN_ERR PREFIX "Unable to read status\n");
+ acpi_handle_err(handle, "Unable to read status\n");
goto end;
}
if (!link->device->status.enabled) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Link disabled\n"));
+ acpi_handle_debug(handle, "Link disabled\n");
return 0;
}
}
@@ -253,22 +253,22 @@ static int acpi_pci_link_get_current(struct acpi_pci_link *link)
* Query and parse _CRS to get the current IRQ assignment.
*/
- status = acpi_walk_resources(link->device->handle, METHOD_NAME__CRS,
+ status = acpi_walk_resources(handle, METHOD_NAME__CRS,
acpi_pci_link_check_current, &irq);
if (ACPI_FAILURE(status)) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _CRS"));
+ acpi_evaluation_failure_warn(handle, "_CRS", status);
result = -ENODEV;
goto end;
}
if (acpi_strict && !irq) {
- printk(KERN_ERR PREFIX "_CRS returned 0\n");
+ acpi_handle_err(handle, "_CRS returned 0\n");
result = -ENODEV;
}
link->irq.active = irq;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Link at IRQ %d \n", link->irq.active));
+ acpi_handle_debug(handle, "Link at IRQ %d \n", link->irq.active);
end:
return result;
@@ -276,13 +276,14 @@ static int acpi_pci_link_get_current(struct acpi_pci_link *link)
static int acpi_pci_link_set(struct acpi_pci_link *link, int irq)
{
- int result;
- acpi_status status;
struct {
struct acpi_resource res;
struct acpi_resource end;
} *resource;
struct acpi_buffer buffer = { 0, NULL };
+ acpi_handle handle = link->device->handle;
+ acpi_status status;
+ int result;
if (!irq)
return -EINVAL;
@@ -329,7 +330,8 @@ static int acpi_pci_link_set(struct acpi_pci_link *link, int irq)
/* ignore resource_source, it's optional */
break;
default:
- printk(KERN_ERR PREFIX "Invalid Resource_type %d\n", link->irq.resource_type);
+ acpi_handle_err(handle, "Invalid resource type %d\n",
+ link->irq.resource_type);
result = -EINVAL;
goto end;
@@ -342,7 +344,7 @@ static int acpi_pci_link_set(struct acpi_pci_link *link, int irq)
/* check for total failure */
if (ACPI_FAILURE(status)) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _SRS"));
+ acpi_evaluation_failure_warn(handle, "_SRS", status);
result = -ENODEV;
goto end;
}
@@ -350,15 +352,11 @@ static int acpi_pci_link_set(struct acpi_pci_link *link, int irq)
/* Query _STA, set device->status */
result = acpi_bus_get_status(link->device);
if (result) {
- printk(KERN_ERR PREFIX "Unable to read status\n");
+ acpi_handle_err(handle, "Unable to read status\n");
goto end;
}
- if (!link->device->status.enabled) {
- printk(KERN_WARNING PREFIX
- "%s [%s] disabled and referenced, BIOS bug\n",
- acpi_device_name(link->device),
- acpi_device_bid(link->device));
- }
+ if (!link->device->status.enabled)
+ acpi_handle_warn(handle, "Disabled and referenced, BIOS bug\n");
/* Query _CRS, set link->irq.active */
result = acpi_pci_link_get_current(link);
@@ -375,14 +373,12 @@ static int acpi_pci_link_set(struct acpi_pci_link *link, int irq)
* policy: when _CRS doesn't return what we just _SRS
* assume _SRS worked and override _CRS value.
*/
- printk(KERN_WARNING PREFIX
- "%s [%s] BIOS reported IRQ %d, using IRQ %d\n",
- acpi_device_name(link->device),
- acpi_device_bid(link->device), link->irq.active, irq);
+ acpi_handle_warn(handle, "BIOS reported IRQ %d, using IRQ %d\n",
+ link->irq.active, irq);
link->irq.active = irq;
}
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Set IRQ %d\n", link->irq.active));
+ acpi_handle_debug(handle, "Set IRQ %d\n", link->irq.active);
end:
kfree(resource);
@@ -531,6 +527,7 @@ static int acpi_irq_balance = -1; /* 0: static, 1: balance */
static int acpi_pci_link_allocate(struct acpi_pci_link *link)
{
+ acpi_handle handle = link->device->handle;
int irq;
int i;
@@ -553,8 +550,8 @@ static int acpi_pci_link_allocate(struct acpi_pci_link *link)
*/
if (i == link->irq.possible_count) {
if (acpi_strict)
- printk(KERN_WARNING PREFIX "_CRS %d not found"
- " in _PRS\n", link->irq.active);
+ acpi_handle_warn(handle, "_CRS %d not found in _PRS\n",
+ link->irq.active);
link->irq.active = 0;
}
@@ -578,28 +575,23 @@ static int acpi_pci_link_allocate(struct acpi_pci_link *link)
}
}
if (acpi_irq_get_penalty(irq) >= PIRQ_PENALTY_ISA_ALWAYS) {
- printk(KERN_ERR PREFIX "No IRQ available for %s [%s]. "
- "Try pci=noacpi or acpi=off\n",
- acpi_device_name(link->device),
- acpi_device_bid(link->device));
+ acpi_handle_err(handle,
+ "No IRQ available. Try pci=noacpi or acpi=off\n");
return -ENODEV;
}
/* Attempt to enable the link device at this IRQ. */
if (acpi_pci_link_set(link, irq)) {
- printk(KERN_ERR PREFIX "Unable to set IRQ for %s [%s]. "
- "Try pci=noacpi or acpi=off\n",
- acpi_device_name(link->device),
- acpi_device_bid(link->device));
+ acpi_handle_err(handle,
+ "Unable to set IRQ. Try pci=noacpi or acpi=off\n");
return -ENODEV;
} else {
if (link->irq.active < ACPI_MAX_ISA_IRQS)
acpi_isa_irq_penalty[link->irq.active] +=
PIRQ_PENALTY_PCI_USING;
- pr_info("%s [%s] enabled at IRQ %d\n",
- acpi_device_name(link->device),
- acpi_device_bid(link->device), link->irq.active);
+ acpi_handle_info(handle, "Enabled at IRQ %d\n",
+ link->irq.active);
}
link->irq.initialized = 1;
@@ -620,19 +612,19 @@ int acpi_pci_link_allocate_irq(acpi_handle handle, int index, int *triggering,
result = acpi_bus_get_device(handle, &device);
if (result) {
- printk(KERN_ERR PREFIX "Invalid link device\n");
+ acpi_handle_err(handle, "Invalid link device\n");
return -1;
}
link = acpi_driver_data(device);
if (!link) {
- printk(KERN_ERR PREFIX "Invalid link context\n");
+ acpi_handle_err(handle, "Invalid link context\n");
return -1;
}
/* TBD: Support multiple index (IRQ) entries per Link Device */
if (index) {
- printk(KERN_ERR PREFIX "Invalid index %d\n", index);
+ acpi_handle_err(handle, "Invalid index %d\n", index);
return -1;
}
@@ -644,7 +636,7 @@ int acpi_pci_link_allocate_irq(acpi_handle handle, int index, int *triggering,
if (!link->irq.active) {
mutex_unlock(&acpi_link_lock);
- printk(KERN_ERR PREFIX "Link active IRQ is 0!\n");
+ acpi_handle_err(handle, "Link active IRQ is 0!\n");
return -1;
}
link->refcnt++;
@@ -656,9 +648,7 @@ int acpi_pci_link_allocate_irq(acpi_handle handle, int index, int *triggering,
*polarity = link->irq.polarity;
if (name)
*name = acpi_device_bid(link->device);
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Link %s is referenced\n",
- acpi_device_bid(link->device)));
+ acpi_handle_debug(handle, "Link is referenced\n");
return link->irq.active;
}
@@ -674,20 +664,20 @@ int acpi_pci_link_free_irq(acpi_handle handle)
result = acpi_bus_get_device(handle, &device);
if (result) {
- printk(KERN_ERR PREFIX "Invalid link device\n");
+ acpi_handle_err(handle, "Invalid link device\n");
return -1;
}
link = acpi_driver_data(device);
if (!link) {
- printk(KERN_ERR PREFIX "Invalid link context\n");
+ acpi_handle_err(handle, "Invalid link context\n");
return -1;
}
mutex_lock(&acpi_link_lock);
if (!link->irq.initialized) {
mutex_unlock(&acpi_link_lock);
- printk(KERN_ERR PREFIX "Link isn't initialized\n");
+ acpi_handle_err(handle, "Link isn't initialized\n");
return -1;
}
#ifdef FUTURE_USE
@@ -702,9 +692,7 @@ int acpi_pci_link_free_irq(acpi_handle handle)
*/
link->refcnt--;
#endif
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Link %s is dereferenced\n",
- acpi_device_bid(link->device)));
+ acpi_handle_debug(handle, "Link is dereferenced\n");
if (link->refcnt == 0)
acpi_evaluate_object(link->device->handle, "_DIS", NULL, NULL);
@@ -720,10 +708,10 @@ int acpi_pci_link_free_irq(acpi_handle handle)
static int acpi_pci_link_add(struct acpi_device *device,
const struct acpi_device_id *not_used)
{
- int result;
+ acpi_handle handle = device->handle;
struct acpi_pci_link *link;
+ int result;
int i;
- int found = 0;
link = kzalloc(sizeof(struct acpi_pci_link), GFP_KERNEL);
if (!link)
@@ -742,31 +730,23 @@ static int acpi_pci_link_add(struct acpi_device *device,
/* query and set link->irq.active */
acpi_pci_link_get_current(link);
- printk(KERN_INFO PREFIX "%s [%s] (IRQs", acpi_device_name(device),
- acpi_device_bid(device));
+ pr_info("Interrupt link %s configured for IRQ %d\n",
+ acpi_device_bid(device), link->irq.active);
+
for (i = 0; i < link->irq.possible_count; i++) {
- if (link->irq.active == link->irq.possible[i]) {
- printk(KERN_CONT " *%d", link->irq.possible[i]);
- found = 1;
- } else
- printk(KERN_CONT " %d", link->irq.possible[i]);
+ if (link->irq.active != link->irq.possible[i])
+ acpi_handle_debug(handle, "Possible IRQ %d\n",
+ link->irq.possible[i]);
}
- printk(KERN_CONT ")");
-
- if (!found)
- printk(KERN_CONT " *%d", link->irq.active);
-
if (!link->device->status.enabled)
- printk(KERN_CONT ", disabled.");
-
- printk(KERN_CONT "\n");
+ pr_info("Interrupt link %s disabled\n", acpi_device_bid(device));
list_add_tail(&link->list, &acpi_link_list);
end:
/* disable all links -- to be activated on use */
- acpi_evaluate_object(device->handle, "_DIS", NULL, NULL);
+ acpi_evaluate_object(handle, "_DIS", NULL, NULL);
mutex_unlock(&acpi_link_lock);
if (result)
diff --git a/drivers/acpi/pci_mcfg.c b/drivers/acpi/pci_mcfg.c
index 95f23acd5b80..53cab975f612 100644
--- a/drivers/acpi/pci_mcfg.c
+++ b/drivers/acpi/pci_mcfg.c
@@ -116,6 +116,13 @@ static struct mcfg_fixup mcfg_quirks[] = {
THUNDER_ECAM_QUIRK(2, 12),
THUNDER_ECAM_QUIRK(2, 13),
+ { "NVIDIA", "TEGRA194", 1, 0, MCFG_BUS_ANY, &tegra194_pcie_ops},
+ { "NVIDIA", "TEGRA194", 1, 1, MCFG_BUS_ANY, &tegra194_pcie_ops},
+ { "NVIDIA", "TEGRA194", 1, 2, MCFG_BUS_ANY, &tegra194_pcie_ops},
+ { "NVIDIA", "TEGRA194", 1, 3, MCFG_BUS_ANY, &tegra194_pcie_ops},
+ { "NVIDIA", "TEGRA194", 1, 4, MCFG_BUS_ANY, &tegra194_pcie_ops},
+ { "NVIDIA", "TEGRA194", 1, 5, MCFG_BUS_ANY, &tegra194_pcie_ops},
+
#define XGENE_V1_ECAM_MCFG(rev, seg) \
{"APM ", "XGENE ", rev, seg, MCFG_BUS_ANY, \
&xgene_v1_pcie_ecam_ops }
diff --git a/drivers/acpi/pmic/intel_pmic_chtcrc.c b/drivers/acpi/pmic/intel_pmic_chtcrc.c
index ebf8d3187df1..2900dc3074d2 100644
--- a/drivers/acpi/pmic/intel_pmic_chtcrc.c
+++ b/drivers/acpi/pmic/intel_pmic_chtcrc.c
@@ -15,7 +15,7 @@
/*
* We have no docs for the CHT Crystal Cove PMIC. The Asus Zenfone-2 kernel
* code has 2 Crystal Cove regulator drivers, one calls the PMIC a "Crystal
- * Cove Plus" PMIC and talks about Cherry Trail, so presuambly that one
+ * Cove Plus" PMIC and talks about Cherry Trail, so presumably that one
* could be used to get register info for the regulators if we need to
* implement regulator support in the future.
*
diff --git a/drivers/acpi/power.c b/drivers/acpi/power.c
index 9b608b55d2b2..56102eaaa2da 100644
--- a/drivers/acpi/power.c
+++ b/drivers/acpi/power.c
@@ -925,8 +925,7 @@ int acpi_add_power_resource(acpi_handle handle)
return -ENOMEM;
device = &resource->device;
- acpi_init_device_object(device, handle, ACPI_BUS_TYPE_POWER,
- ACPI_STA_DEFAULT, NULL);
+ acpi_init_device_object(device, handle, ACPI_BUS_TYPE_POWER);
mutex_init(&resource->resource_lock);
INIT_LIST_HEAD(&resource->list_node);
INIT_LIST_HEAD(&resource->dependents);
@@ -935,7 +934,7 @@ int acpi_add_power_resource(acpi_handle handle)
strcpy(acpi_device_class(device), ACPI_POWER_CLASS);
device->power.state = ACPI_STATE_UNKNOWN;
- /* Evalute the object to get the system level and resource order. */
+ /* Evaluate the object to get the system level and resource order. */
status = acpi_evaluate_object(handle, NULL, NULL, &buffer);
if (ACPI_FAILURE(status))
goto err;
@@ -996,6 +995,7 @@ void acpi_resume_power_resources(void)
mutex_unlock(&power_resource_list_lock);
}
+#endif
void acpi_turn_off_unused_power_resources(void)
{
@@ -1004,18 +1004,9 @@ void acpi_turn_off_unused_power_resources(void)
mutex_lock(&power_resource_list_lock);
list_for_each_entry_reverse(resource, &acpi_power_resource_list, list_node) {
- int result, state;
-
mutex_lock(&resource->resource_lock);
- result = acpi_power_get_state(resource->device.handle, &state);
- if (result) {
- mutex_unlock(&resource->resource_lock);
- continue;
- }
-
- if (state == ACPI_POWER_RESOURCE_STATE_ON
- && !resource->ref_count) {
+ if (!resource->ref_count) {
dev_info(&resource->device.dev, "Turning OFF\n");
__acpi_power_off(resource);
}
@@ -1025,4 +1016,3 @@ void acpi_turn_off_unused_power_resources(void)
mutex_unlock(&power_resource_list_lock);
}
-#endif
diff --git a/drivers/acpi/processor_driver.c b/drivers/acpi/processor_driver.c
index 62114a03a51a..77541f939be3 100644
--- a/drivers/acpi/processor_driver.c
+++ b/drivers/acpi/processor_driver.c
@@ -28,9 +28,6 @@
#define ACPI_PROCESSOR_NOTIFY_POWER 0x81
#define ACPI_PROCESSOR_NOTIFY_THROTTLING 0x82
-#define _COMPONENT ACPI_PROCESSOR_COMPONENT
-ACPI_MODULE_NAME("processor_driver");
-
MODULE_AUTHOR("Paul Diefenbaugh");
MODULE_DESCRIPTION("ACPI Processor Driver");
MODULE_LICENSE("GPL");
@@ -87,8 +84,7 @@ static void acpi_processor_notify(acpi_handle handle, u32 event, void *data)
dev_name(&device->dev), event, 0);
break;
default:
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Unsupported event [0x%x]\n", event));
+ acpi_handle_debug(handle, "Unsupported event [0x%x]\n", event);
break;
}
diff --git a/drivers/acpi/processor_idle.c b/drivers/acpi/processor_idle.c
index d93e400940a3..45a019619e4a 100644
--- a/drivers/acpi/processor_idle.c
+++ b/drivers/acpi/processor_idle.c
@@ -29,11 +29,9 @@
*/
#ifdef CONFIG_X86
#include <asm/apic.h>
+#include <asm/cpu.h>
#endif
-#define _COMPONENT ACPI_PROCESSOR_COMPONENT
-ACPI_MODULE_NAME("processor_idle");
-
#define ACPI_IDLE_STATE_START (IS_ENABLED(CONFIG_ARCH_HAS_CPU_RELAX) ? 1 : 0)
static unsigned int max_cstate __read_mostly = ACPI_PROCESSOR_MAX_POWER;
@@ -239,8 +237,8 @@ static int acpi_processor_get_power_info_fadt(struct acpi_processor *pr)
* 100 microseconds.
*/
if (acpi_gbl_FADT.c2_latency > ACPI_PROCESSOR_MAX_C2_LATENCY) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "C2 latency too large [%d]\n", acpi_gbl_FADT.c2_latency));
+ acpi_handle_debug(pr->handle, "C2 latency too large [%d]\n",
+ acpi_gbl_FADT.c2_latency);
/* invalidate C2 */
pr->power.states[ACPI_STATE_C2].address = 0;
}
@@ -250,16 +248,15 @@ static int acpi_processor_get_power_info_fadt(struct acpi_processor *pr)
* 1000 microseconds.
*/
if (acpi_gbl_FADT.c3_latency > ACPI_PROCESSOR_MAX_C3_LATENCY) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "C3 latency too large [%d]\n", acpi_gbl_FADT.c3_latency));
+ acpi_handle_debug(pr->handle, "C3 latency too large [%d]\n",
+ acpi_gbl_FADT.c3_latency);
/* invalidate C3 */
pr->power.states[ACPI_STATE_C3].address = 0;
}
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "lvl2[0x%08x] lvl3[0x%08x]\n",
+ acpi_handle_debug(pr->handle, "lvl2[0x%08x] lvl3[0x%08x]\n",
pr->power.states[ACPI_STATE_C2].address,
- pr->power.states[ACPI_STATE_C3].address));
+ pr->power.states[ACPI_STATE_C3].address);
snprintf(pr->power.states[ACPI_STATE_C2].desc,
ACPI_CX_DESC_LEN, "ACPI P_LVL2 IOPORT 0x%x",
@@ -324,8 +321,8 @@ static void acpi_processor_power_verify_c3(struct acpi_processor *pr,
* devices thus we take the conservative approach.
*/
else if (errata.piix4.fdma) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "C3 not supported on PIIX4 with Type-F DMA\n"));
+ acpi_handle_debug(pr->handle,
+ "C3 not supported on PIIX4 with Type-F DMA\n");
return;
}
@@ -344,13 +341,13 @@ static void acpi_processor_power_verify_c3(struct acpi_processor *pr,
if (!pr->flags.bm_control) {
if (pr->flags.has_cst != 1) {
/* bus mastering control is necessary */
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "C3 support requires BM control\n"));
+ acpi_handle_debug(pr->handle,
+ "C3 support requires BM control\n");
return;
} else {
/* Here we enter C3 without bus mastering */
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "C3 support without BM control\n"));
+ acpi_handle_debug(pr->handle,
+ "C3 support without BM control\n");
}
}
} else {
@@ -359,9 +356,9 @@ static void acpi_processor_power_verify_c3(struct acpi_processor *pr,
* supported on when bm_check is not required.
*/
if (!(acpi_gbl_FADT.flags & ACPI_FADT_WBINVD)) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
+ acpi_handle_debug(pr->handle,
"Cache invalidation should work properly"
- " for C3 to be enabled on SMP systems\n"));
+ " for C3 to be enabled on SMP systems\n");
return;
}
}
@@ -541,6 +538,10 @@ static int acpi_idle_play_dead(struct cpuidle_device *dev, int index)
wait_for_freeze();
} else
return -ENODEV;
+
+#if defined(CONFIG_X86) && defined(CONFIG_HOTPLUG_CPU)
+ cond_wakeup_cpu0();
+#endif
}
/* Never reached */
@@ -784,8 +785,8 @@ static inline void acpi_processor_cstate_first_run_checks(void)
dmi_check_system(processor_power_dmi_table);
max_cstate = acpi_processor_cstate_check(max_cstate);
if (max_cstate < ACPI_C_STATES_MAX)
- pr_notice("ACPI: processor limited to max C-state %d\n",
- max_cstate);
+ pr_notice("processor limited to max C-state %d\n", max_cstate);
+
first_run++;
if (nocst)
@@ -843,7 +844,7 @@ static int acpi_processor_evaluate_lpi(acpi_handle handle,
status = acpi_evaluate_object(handle, "_LPI", NULL, &buffer);
if (ACPI_FAILURE(status)) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No _LPI, giving up\n"));
+ acpi_handle_debug(handle, "No _LPI, giving up\n");
return -ENODEV;
}
diff --git a/drivers/acpi/processor_pdc.c b/drivers/acpi/processor_pdc.c
index 813f1b78c16a..8c3f82c9fff3 100644
--- a/drivers/acpi/processor_pdc.c
+++ b/drivers/acpi/processor_pdc.c
@@ -16,9 +16,6 @@
#include "internal.h"
-#define _COMPONENT ACPI_PROCESSOR_COMPONENT
-ACPI_MODULE_NAME("processor_pdc");
-
static bool __init processor_physically_present(acpi_handle handle)
{
int cpuid, type;
@@ -132,8 +129,8 @@ acpi_processor_eval_pdc(acpi_handle handle, struct acpi_object_list *pdc_in)
status = acpi_evaluate_object(handle, "_PDC", pdc_in, NULL);
if (ACPI_FAILURE(status))
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Could not evaluate _PDC, using legacy perf. control.\n"));
+ acpi_handle_debug(handle,
+ "Could not evaluate _PDC, using legacy perf control\n");
return status;
}
diff --git a/drivers/acpi/processor_perflib.c b/drivers/acpi/processor_perflib.c
index 32f0f554ccae..d088a0089ee9 100644
--- a/drivers/acpi/processor_perflib.c
+++ b/drivers/acpi/processor_perflib.c
@@ -23,8 +23,6 @@
#define PREFIX "ACPI: "
#define ACPI_PROCESSOR_FILE_PERFORMANCE "performance"
-#define _COMPONENT ACPI_PROCESSOR_COMPONENT
-ACPI_MODULE_NAME("processor_perflib");
static DEFINE_MUTEX(performance_mutex);
@@ -65,13 +63,13 @@ static int acpi_processor_get_platform_limit(struct acpi_processor *pr)
* (e.g. 0 = states 0..n; 1 = states 1..n; etc.
*/
status = acpi_evaluate_integer(pr->handle, "_PPC", NULL, &ppc);
-
- if (status != AE_NOT_FOUND)
+ if (status != AE_NOT_FOUND) {
acpi_processor_ppc_in_use = true;
- if (ACPI_FAILURE(status) && status != AE_NOT_FOUND) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _PPC"));
- return -ENODEV;
+ if (ACPI_FAILURE(status)) {
+ acpi_evaluation_failure_warn(pr->handle, "_PPC", status);
+ return -ENODEV;
+ }
}
pr_debug("CPU %d: _PPC is %d - frequency %s limited\n", pr->id,
@@ -98,7 +96,7 @@ static int acpi_processor_get_platform_limit(struct acpi_processor *pr)
* acpi_processor_ppc_ost: Notify firmware the _PPC evaluation status
* @handle: ACPI processor handle
* @status: the status code of _PPC evaluation
- * 0: success. OSPM is now using the performance state specificed.
+ * 0: success. OSPM is now using the performance state specified.
* 1: failure. OSPM has not changed the number of P-states in use
*/
static void acpi_processor_ppc_ost(acpi_handle handle, int status)
@@ -199,7 +197,7 @@ static int acpi_processor_get_performance_control(struct acpi_processor *pr)
status = acpi_evaluate_object(pr->handle, "_PCT", NULL, &buffer);
if (ACPI_FAILURE(status)) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _PCT"));
+ acpi_evaluation_failure_warn(pr->handle, "_PCT", status);
return -ENODEV;
}
@@ -299,7 +297,7 @@ static int acpi_processor_get_performance_states(struct acpi_processor *pr)
status = acpi_evaluate_object(pr->handle, "_PSS", NULL, &buffer);
if (ACPI_FAILURE(status)) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _PSS"));
+ acpi_evaluation_failure_warn(pr->handle, "_PSS", status);
return -ENODEV;
}
@@ -310,8 +308,8 @@ static int acpi_processor_get_performance_states(struct acpi_processor *pr)
goto end;
}
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %d performance states\n",
- pss->package.count));
+ acpi_handle_debug(pr->handle, "Found %d performance states\n",
+ pss->package.count);
pr->performance->state_count = pss->package.count;
pr->performance->states =
@@ -330,12 +328,13 @@ static int acpi_processor_get_performance_states(struct acpi_processor *pr)
state.length = sizeof(struct acpi_processor_px);
state.pointer = px;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Extracting state %d\n", i));
+ acpi_handle_debug(pr->handle, "Extracting state %d\n", i);
status = acpi_extract_package(&(pss->package.elements[i]),
&format, &state);
if (ACPI_FAILURE(status)) {
- ACPI_EXCEPTION((AE_INFO, status, "Invalid _PSS data"));
+ acpi_handle_warn(pr->handle, "Invalid _PSS data: %s\n",
+ acpi_format_exception(status));
result = -EFAULT;
kfree(pr->performance->states);
goto end;
@@ -343,14 +342,14 @@ static int acpi_processor_get_performance_states(struct acpi_processor *pr)
amd_fixup_frequency(px, i);
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
+ acpi_handle_debug(pr->handle,
"State [%d]: core_frequency[%d] power[%d] transition_latency[%d] bus_master_latency[%d] control[0x%x] status[0x%x]\n",
i,
(u32) px->core_frequency,
(u32) px->power,
(u32) px->transition_latency,
(u32) px->bus_master_latency,
- (u32) px->control, (u32) px->status));
+ (u32) px->control, (u32) px->status);
/*
* Check that ACPI's u64 MHz will be valid as u32 KHz in cpufreq
@@ -400,8 +399,8 @@ int acpi_processor_get_performance_info(struct acpi_processor *pr)
return -EINVAL;
if (!acpi_has_method(pr->handle, "_PCT")) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "ACPI-based processor performance control unavailable\n"));
+ acpi_handle_debug(pr->handle,
+ "ACPI-based processor performance control unavailable\n");
return -ENODEV;
}
@@ -442,24 +441,23 @@ int acpi_processor_pstate_control(void)
if (!acpi_gbl_FADT.smi_command || !acpi_gbl_FADT.pstate_control)
return 0;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Writing pstate_control [0x%x] to smi_command [0x%x]\n",
- acpi_gbl_FADT.pstate_control, acpi_gbl_FADT.smi_command));
+ pr_debug("Writing pstate_control [0x%x] to smi_command [0x%x]\n",
+ acpi_gbl_FADT.pstate_control, acpi_gbl_FADT.smi_command);
status = acpi_os_write_port(acpi_gbl_FADT.smi_command,
(u32)acpi_gbl_FADT.pstate_control, 8);
if (ACPI_SUCCESS(status))
return 1;
- ACPI_EXCEPTION((AE_INFO, status,
- "Failed to write pstate_control [0x%x] to smi_command [0x%x]",
- acpi_gbl_FADT.pstate_control, acpi_gbl_FADT.smi_command));
+ pr_warn("Failed to write pstate_control [0x%x] to smi_command [0x%x]: %s\n",
+ acpi_gbl_FADT.pstate_control, acpi_gbl_FADT.smi_command,
+ acpi_format_exception(status));
return -EIO;
}
int acpi_processor_notify_smm(struct module *calling_module)
{
- static int is_done = 0;
+ static int is_done;
int result;
if (!acpi_processor_cpufreq_init)
@@ -485,7 +483,7 @@ int acpi_processor_notify_smm(struct module *calling_module)
result = acpi_processor_pstate_control();
if (!result) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No SMI port or pstate_control\n"));
+ pr_debug("No SMI port or pstate_control\n");
module_put(calling_module);
return 0;
}
diff --git a/drivers/acpi/processor_throttling.c b/drivers/acpi/processor_throttling.c
index b1876534324b..e61b8f038364 100644
--- a/drivers/acpi/processor_throttling.c
+++ b/drivers/acpi/processor_throttling.c
@@ -22,9 +22,6 @@
#define PREFIX "ACPI: "
-#define _COMPONENT ACPI_PROCESSOR_COMPONENT
-ACPI_MODULE_NAME("processor_throttling");
-
/* ignore_tpc:
* 0 -> acpi processor driver doesn't ignore _TPC values
* 1 -> acpi processor driver ignores _TPC values
@@ -196,10 +193,8 @@ err_ret:
*/
void acpi_processor_throttling_init(void)
{
- if (acpi_processor_update_tsd_coord()) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Assume no T-state coordination\n"));
- }
+ if (acpi_processor_update_tsd_coord())
+ pr_debug("Assume no T-state coordination\n");
return;
}
@@ -216,12 +211,13 @@ static int acpi_processor_throttling_notifier(unsigned long event, void *data)
cpu = p_tstate->cpu;
pr = per_cpu(processors, cpu);
if (!pr) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Invalid pr pointer\n"));
+ pr_debug("Invalid pr pointer\n");
return 0;
}
if (!pr->flags.throttling) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Throttling control is "
- "unsupported on CPU %d\n", cpu));
+ acpi_handle_debug(pr->handle,
+ "Throttling control unsupported on CPU %d\n",
+ cpu);
return 0;
}
target_state = p_tstate->target_state;
@@ -245,9 +241,9 @@ static int acpi_processor_throttling_notifier(unsigned long event, void *data)
target_state = p_throttling->state_count - 1;
}
p_tstate->target_state = target_state;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "PreChange Event:"
- "target T-state of CPU %d is T%d\n",
- cpu, target_state));
+ acpi_handle_debug(pr->handle,
+ "PreChange Event: target T-state of CPU %d is T%d\n",
+ cpu, target_state);
break;
case THROTTLING_POSTCHANGE:
/*
@@ -255,9 +251,9 @@ static int acpi_processor_throttling_notifier(unsigned long event, void *data)
* T-state flag of acpi_processor_throttling.
*/
p_throttling->state = target_state;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "PostChange Event:"
- "CPU %d is switched to T%d\n",
- cpu, target_state));
+ acpi_handle_debug(pr->handle,
+ "PostChange Event: CPU %d is switched to T%d\n",
+ cpu, target_state);
break;
default:
printk(KERN_WARNING
@@ -284,9 +280,9 @@ static int acpi_processor_get_platform_limit(struct acpi_processor *pr)
status = acpi_evaluate_integer(pr->handle, "_TPC", NULL, &tpc);
if (ACPI_FAILURE(status)) {
- if (status != AE_NOT_FOUND) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _TPC"));
- }
+ if (status != AE_NOT_FOUND)
+ acpi_evaluation_failure_warn(pr->handle, "_TPC", status);
+
return -ENODEV;
}
@@ -417,9 +413,9 @@ static int acpi_processor_get_throttling_control(struct acpi_processor *pr)
status = acpi_evaluate_object(pr->handle, "_PTC", NULL, &buffer);
if (ACPI_FAILURE(status)) {
- if (status != AE_NOT_FOUND) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _PTC"));
- }
+ if (status != AE_NOT_FOUND)
+ acpi_evaluation_failure_warn(pr->handle, "_PTC", status);
+
return -ENODEV;
}
@@ -502,9 +498,9 @@ static int acpi_processor_get_throttling_states(struct acpi_processor *pr)
status = acpi_evaluate_object(pr->handle, "_TSS", NULL, &buffer);
if (ACPI_FAILURE(status)) {
- if (status != AE_NOT_FOUND) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _TSS"));
- }
+ if (status != AE_NOT_FOUND)
+ acpi_evaluation_failure_warn(pr->handle, "_TSS", status);
+
return -ENODEV;
}
@@ -515,8 +511,8 @@ static int acpi_processor_get_throttling_states(struct acpi_processor *pr)
goto end;
}
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %d throttling states\n",
- tss->package.count));
+ acpi_handle_debug(pr->handle, "Found %d throttling states\n",
+ tss->package.count);
pr->throttling.state_count = tss->package.count;
pr->throttling.states_tss =
@@ -537,12 +533,13 @@ static int acpi_processor_get_throttling_states(struct acpi_processor *pr)
state.length = sizeof(struct acpi_processor_tx_tss);
state.pointer = tx;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Extracting state %d\n", i));
+ acpi_handle_debug(pr->handle, "Extracting state %d\n", i);
status = acpi_extract_package(&(tss->package.elements[i]),
&format, &state);
if (ACPI_FAILURE(status)) {
- ACPI_EXCEPTION((AE_INFO, status, "Invalid _TSS data"));
+ acpi_handle_warn(pr->handle, "Invalid _TSS data: %s\n",
+ acpi_format_exception(status));
result = -EFAULT;
kfree(pr->throttling.states_tss);
goto end;
@@ -582,9 +579,9 @@ static int acpi_processor_get_tsd(struct acpi_processor *pr)
status = acpi_evaluate_object(pr->handle, "_TSD", NULL, &buffer);
if (ACPI_FAILURE(status)) {
- if (status != AE_NOT_FOUND) {
- ACPI_EXCEPTION((AE_INFO, status, "Evaluating _TSD"));
- }
+ if (status != AE_NOT_FOUND)
+ acpi_evaluation_failure_warn(pr->handle, "_TSD", status);
+
return -ENODEV;
}
@@ -698,9 +695,9 @@ static int acpi_processor_get_throttling_fadt(struct acpi_processor *pr)
local_irq_enable();
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
+ acpi_handle_debug(pr->handle,
"Throttling state is T%d (%d%% throttling applied)\n",
- state, pr->throttling.states[state].performance));
+ state, pr->throttling.states[state].performance);
return 0;
}
@@ -875,8 +872,8 @@ static int acpi_processor_get_throttling_ptc(struct acpi_processor *pr)
if (ret >= 0) {
state = acpi_get_throttling_state(pr, value);
if (state == -1) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Invalid throttling state, reset\n"));
+ acpi_handle_debug(pr->handle,
+ "Invalid throttling state, reset\n");
state = 0;
ret = __acpi_processor_set_throttling(pr, state, true,
true);
@@ -921,10 +918,10 @@ static int acpi_processor_get_fadt_info(struct acpi_processor *pr)
int i, step;
if (!pr->throttling.address) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No throttling register\n"));
+ acpi_handle_debug(pr->handle, "No throttling register\n");
return -EINVAL;
} else if (!pr->throttling.duty_width) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "No throttling states\n"));
+ acpi_handle_debug(pr->handle, "No throttling states\n");
return -EINVAL;
}
/* TBD: Support duty_cycle values that span bit 4. */
@@ -1015,10 +1012,10 @@ static int acpi_processor_set_throttling_fadt(struct acpi_processor *pr,
local_irq_enable();
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
+ acpi_handle_debug(pr->handle,
"Throttling state set to T%d (%d%%)\n", state,
(pr->throttling.states[state].performance ? pr->
- throttling.states[state].performance / 10 : 0)));
+ throttling.states[state].performance / 10 : 0));
return 0;
}
@@ -1129,8 +1126,8 @@ static int __acpi_processor_set_throttling(struct acpi_processor *pr,
* error message and continue.
*/
if (!match_pr) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Invalid Pointer for CPU %d\n", i));
+ acpi_handle_debug(pr->handle,
+ "Invalid Pointer for CPU %d\n", i);
continue;
}
/*
@@ -1138,9 +1135,8 @@ static int __acpi_processor_set_throttling(struct acpi_processor *pr,
* we will report the error message and continue.
*/
if (!match_pr->flags.throttling) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Throttling Control is unsupported "
- "on CPU %d\n", i));
+ acpi_handle_debug(pr->handle,
+ "Throttling Control unsupported on CPU %d\n", i);
continue;
}
@@ -1177,11 +1173,11 @@ int acpi_processor_get_throttling_info(struct acpi_processor *pr)
int result = 0;
struct acpi_processor_throttling *pthrottling;
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
+ acpi_handle_debug(pr->handle,
"pblk_address[0x%08x] duty_offset[%d] duty_width[%d]\n",
pr->throttling.address,
pr->throttling.duty_offset,
- pr->throttling.duty_width));
+ pr->throttling.duty_width);
/*
* Evaluate _PTC, _TSS and _TPC
@@ -1221,13 +1217,13 @@ int acpi_processor_get_throttling_info(struct acpi_processor *pr)
* used this part.
*/
if (errata.piix4.throttle) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
- "Throttling not supported on PIIX4 A- or B-step\n"));
+ acpi_handle_debug(pr->handle,
+ "Throttling not supported on PIIX4 A- or B-step\n");
return 0;
}
- ACPI_DEBUG_PRINT((ACPI_DB_INFO, "Found %d throttling states\n",
- pr->throttling.state_count));
+ acpi_handle_debug(pr->handle, "Found %d throttling states\n",
+ pr->throttling.state_count);
pr->flags.throttling = 1;
@@ -1242,9 +1238,9 @@ int acpi_processor_get_throttling_info(struct acpi_processor *pr)
goto end;
if (pr->throttling.state) {
- ACPI_DEBUG_PRINT((ACPI_DB_INFO,
+ acpi_handle_debug(pr->handle,
"Disabling throttling (was T%d)\n",
- pr->throttling.state));
+ pr->throttling.state);
result = acpi_processor_set_throttling(pr, 0, false);
if (result)
goto end;
diff --git a/drivers/acpi/resource.c b/drivers/acpi/resource.c
index 20a7892c6d3f..ee78a210c606 100644
--- a/drivers/acpi/resource.c
+++ b/drivers/acpi/resource.c
@@ -596,7 +596,7 @@ static int __acpi_dev_get_resources(struct acpi_device *adev,
* @preproc_data: Pointer passed to the caller's preprocessing routine.
*
* Evaluate the _CRS method for the given device node and process its output by
- * (1) executing the @preproc() rountine provided by the caller, passing the
+ * (1) executing the @preproc() routine provided by the caller, passing the
* resource pointer and @preproc_data to it as arguments, for each ACPI resource
* returned and (2) converting all of the returned ACPI resources into struct
* resource objects if possible. If the return value of @preproc() in step (1)
diff --git a/drivers/acpi/scan.c b/drivers/acpi/scan.c
index a184529d8fa4..453eff8ec8c3 100644
--- a/drivers/acpi/scan.c
+++ b/drivers/acpi/scan.c
@@ -479,9 +479,8 @@ static void acpi_device_del(struct acpi_device *device)
list_for_each_entry(acpi_device_bus_id, &acpi_bus_id_list, node)
if (!strcmp(acpi_device_bus_id->bus_id,
acpi_device_hid(device))) {
- if (acpi_device_bus_id->instance_no > 0)
- acpi_device_bus_id->instance_no--;
- else {
+ ida_simple_remove(&acpi_device_bus_id->instance_ida, device->pnp.instance_no);
+ if (ida_is_empty(&acpi_device_bus_id->instance_ida)) {
list_del(&acpi_device_bus_id->node);
kfree_const(acpi_device_bus_id->bus_id);
kfree(acpi_device_bus_id);
@@ -531,7 +530,7 @@ static void acpi_device_del_work_fn(struct work_struct *work_not_used)
* used by the device.
*/
acpi_power_transition(adev, ACPI_STATE_D3_COLD);
- put_device(&adev->dev);
+ acpi_dev_put(adev);
}
}
@@ -561,7 +560,7 @@ static void acpi_scan_drop_device(acpi_handle handle, void *context)
* prevents attempts to register device objects identical to those being
* deleted from happening concurrently (such attempts result from
* hotplug events handled via the ACPI hotplug workqueue). It also will
- * run after all of the work items submitted previosuly, which helps
+ * run after all of the work items submitted previously, which helps
* those work items to ensure that they are not accessing stale device
* objects.
*/
@@ -605,8 +604,7 @@ EXPORT_SYMBOL(acpi_bus_get_device);
static void get_acpi_device(void *dev)
{
- if (dev)
- get_device(&((struct acpi_device *)dev)->dev);
+ acpi_dev_get(dev);
}
struct acpi_device *acpi_bus_get_acpi_device(acpi_handle handle)
@@ -616,7 +614,7 @@ struct acpi_device *acpi_bus_get_acpi_device(acpi_handle handle)
void acpi_bus_put_acpi_device(struct acpi_device *adev)
{
- put_device(&adev->dev);
+ acpi_dev_put(adev);
}
static struct acpi_device_bus_id *acpi_device_bus_id_match(const char *dev_id)
@@ -631,6 +629,21 @@ static struct acpi_device_bus_id *acpi_device_bus_id_match(const char *dev_id)
return NULL;
}
+static int acpi_device_set_name(struct acpi_device *device,
+ struct acpi_device_bus_id *acpi_device_bus_id)
+{
+ struct ida *instance_ida = &acpi_device_bus_id->instance_ida;
+ int result;
+
+ result = ida_simple_get(instance_ida, 0, ACPI_MAX_DEVICE_INSTANCES, GFP_KERNEL);
+ if (result < 0)
+ return result;
+
+ device->pnp.instance_no = result;
+ dev_set_name(&device->dev, "%s:%02x", acpi_device_bus_id->bus_id, result);
+ return 0;
+}
+
int acpi_device_add(struct acpi_device *device,
void (*release)(struct device *))
{
@@ -665,7 +678,9 @@ int acpi_device_add(struct acpi_device *device,
acpi_device_bus_id = acpi_device_bus_id_match(acpi_device_hid(device));
if (acpi_device_bus_id) {
- acpi_device_bus_id->instance_no++;
+ result = acpi_device_set_name(device, acpi_device_bus_id);
+ if (result)
+ goto err_unlock;
} else {
acpi_device_bus_id = kzalloc(sizeof(*acpi_device_bus_id),
GFP_KERNEL);
@@ -681,9 +696,17 @@ int acpi_device_add(struct acpi_device *device,
goto err_unlock;
}
+ ida_init(&acpi_device_bus_id->instance_ida);
+
+ result = acpi_device_set_name(device, acpi_device_bus_id);
+ if (result) {
+ kfree_const(acpi_device_bus_id->bus_id);
+ kfree(acpi_device_bus_id);
+ goto err_unlock;
+ }
+
list_add_tail(&acpi_device_bus_id->node, &acpi_bus_id_list);
}
- dev_set_name(&device->dev, "%s:%02x", acpi_device_bus_id->bus_id, acpi_device_bus_id->instance_no);
if (device->parent)
list_add_tail(&device->node, &device->parent->children);
@@ -734,27 +757,25 @@ static bool acpi_info_matches_ids(struct acpi_device_info *info,
const char * const ids[])
{
struct acpi_pnp_device_id_list *cid_list = NULL;
- int i;
+ int i, index;
if (!(info->valid & ACPI_VALID_HID))
return false;
+ index = match_string(ids, -1, info->hardware_id.string);
+ if (index >= 0)
+ return true;
+
if (info->valid & ACPI_VALID_CID)
cid_list = &info->compatible_id_list;
- for (i = 0; ids[i]; i++) {
- int j;
+ if (!cid_list)
+ return false;
- if (!strcmp(info->hardware_id.string, ids[i]))
+ for (i = 0; i < cid_list->count; i++) {
+ index = match_string(ids, -1, cid_list->ids[i].string);
+ if (index >= 0)
return true;
-
- if (!cid_list)
- continue;
-
- for (j = 0; j < cid_list->count; j++) {
- if (!strcmp(cid_list->ids[j].string, ids[i]))
- return true;
- }
}
return false;
@@ -1284,8 +1305,9 @@ static bool acpi_object_is_system_bus(acpi_handle handle)
}
static void acpi_set_pnp_ids(acpi_handle handle, struct acpi_device_pnp *pnp,
- int device_type, struct acpi_device_info *info)
+ int device_type)
{
+ struct acpi_device_info *info = NULL;
struct acpi_pnp_device_id_list *cid_list;
int i;
@@ -1296,6 +1318,7 @@ static void acpi_set_pnp_ids(acpi_handle handle, struct acpi_device_pnp *pnp,
break;
}
+ acpi_get_object_info(handle, &info);
if (!info) {
pr_err(PREFIX "%s: Error reading device info\n",
__func__);
@@ -1321,6 +1344,8 @@ static void acpi_set_pnp_ids(acpi_handle handle, struct acpi_device_pnp *pnp,
if (info->valid & ACPI_VALID_CLS)
acpi_add_id(pnp, info->class_code.string);
+ kfree(info);
+
/*
* Some devices don't reliably have _HIDs & _CIDs, so add
* synthetic HIDs to make sure drivers can find them.
@@ -1626,17 +1651,16 @@ static bool acpi_device_enumeration_by_parent(struct acpi_device *device)
}
void acpi_init_device_object(struct acpi_device *device, acpi_handle handle,
- int type, unsigned long long sta,
- struct acpi_device_info *info)
+ int type)
{
INIT_LIST_HEAD(&device->pnp.ids);
device->device_type = type;
device->handle = handle;
device->parent = acpi_bus_get_parent(handle);
fwnode_init(&device->fwnode, &acpi_device_fwnode_ops);
- acpi_set_device_status(device, sta);
+ acpi_set_device_status(device, ACPI_STA_DEFAULT);
acpi_device_get_busid(device);
- acpi_set_pnp_ids(handle, &device->pnp, type, info);
+ acpi_set_pnp_ids(handle, &device->pnp, type);
acpi_init_properties(device);
acpi_bus_get_flags(device);
device->flags.match_driver = false;
@@ -1647,6 +1671,8 @@ void acpi_init_device_object(struct acpi_device *device, acpi_handle handle,
device_initialize(&device->dev);
dev_set_uevent_suppress(&device->dev, true);
acpi_init_coherency(device);
+ /* Assume there are unmet deps to start with. */
+ device->dep_unmet = 1;
}
void acpi_device_add_finalize(struct acpi_device *device)
@@ -1655,33 +1681,30 @@ void acpi_device_add_finalize(struct acpi_device *device)
kobject_uevent(&device->dev.kobj, KOBJ_ADD);
}
+static void acpi_scan_init_status(struct acpi_device *adev)
+{
+ if (acpi_bus_get_status(adev))
+ acpi_set_device_status(adev, 0);
+}
+
static int acpi_add_single_object(struct acpi_device **child,
- acpi_handle handle, int type,
- unsigned long long sta)
+ acpi_handle handle, int type)
{
- struct acpi_device_info *info = NULL;
struct acpi_device *device;
int result;
- if (handle != ACPI_ROOT_OBJECT && type == ACPI_BUS_TYPE_DEVICE)
- acpi_get_object_info(handle, &info);
-
device = kzalloc(sizeof(struct acpi_device), GFP_KERNEL);
- if (!device) {
- kfree(info);
+ if (!device)
return -ENOMEM;
- }
- acpi_init_device_object(device, handle, type, sta, info);
- kfree(info);
+ acpi_init_device_object(device, handle, type);
/*
- * For ACPI_BUS_TYPE_DEVICE getting the status is delayed till here so
- * that we can call acpi_bus_get_status() and use its quirk handling.
- * Note this must be done before the get power-/wakeup_dev-flags calls.
+ * Getting the status is delayed till here so that we can call
+ * acpi_bus_get_status() and use its quirk handling. Note that
+ * this must be done before the get power-/wakeup_dev-flags calls.
*/
- if (type == ACPI_BUS_TYPE_DEVICE)
- if (acpi_bus_get_status(device) < 0)
- acpi_set_device_status(device, 0);
+ if (type == ACPI_BUS_TYPE_DEVICE || type == ACPI_BUS_TYPE_PROCESSOR)
+ acpi_scan_init_status(device);
acpi_bus_get_power_flags(device);
acpi_bus_get_wakeup_device_flags(device);
@@ -1738,50 +1761,6 @@ static bool acpi_device_should_be_hidden(acpi_handle handle)
return true;
}
-static int acpi_bus_type_and_status(acpi_handle handle, int *type,
- unsigned long long *sta)
-{
- acpi_status status;
- acpi_object_type acpi_type;
-
- status = acpi_get_type(handle, &acpi_type);
- if (ACPI_FAILURE(status))
- return -ENODEV;
-
- switch (acpi_type) {
- case ACPI_TYPE_ANY: /* for ACPI_ROOT_OBJECT */
- case ACPI_TYPE_DEVICE:
- if (acpi_device_should_be_hidden(handle))
- return -ENODEV;
-
- *type = ACPI_BUS_TYPE_DEVICE;
- /*
- * acpi_add_single_object updates this once we've an acpi_device
- * so that acpi_bus_get_status' quirk handling can be used.
- */
- *sta = ACPI_STA_DEFAULT;
- break;
- case ACPI_TYPE_PROCESSOR:
- *type = ACPI_BUS_TYPE_PROCESSOR;
- status = acpi_bus_get_status_handle(handle, sta);
- if (ACPI_FAILURE(status))
- return -ENODEV;
- break;
- case ACPI_TYPE_THERMAL:
- *type = ACPI_BUS_TYPE_THERMAL;
- *sta = ACPI_STA_DEFAULT;
- break;
- case ACPI_TYPE_POWER:
- *type = ACPI_BUS_TYPE_POWER;
- *sta = ACPI_STA_DEFAULT;
- break;
- default:
- return -ENODEV;
- }
-
- return 0;
-}
-
bool acpi_device_is_present(const struct acpi_device *adev)
{
return adev->status.present || adev->status.functional;
@@ -1850,7 +1829,7 @@ static void acpi_scan_init_hotplug(struct acpi_device *adev)
}
}
-static u32 acpi_scan_check_dep(acpi_handle handle)
+static u32 acpi_scan_check_dep(acpi_handle handle, bool check_dep)
{
struct acpi_handle_list dep_devices;
acpi_status status;
@@ -1863,7 +1842,8 @@ static u32 acpi_scan_check_dep(acpi_handle handle)
* 2. ACPI nodes describing USB ports.
* Still, checking for _HID catches more then just these cases ...
*/
- if (!acpi_has_method(handle, "_DEP") || !acpi_has_method(handle, "_HID"))
+ if (!check_dep || !acpi_has_method(handle, "_DEP") ||
+ !acpi_has_method(handle, "_HID"))
return 0;
status = acpi_evaluate_reference(handle, "_DEP", NULL, &dep_devices);
@@ -1910,6 +1890,8 @@ static void acpi_scan_dep_init(struct acpi_device *adev)
{
struct acpi_dep_data *dep;
+ adev->dep_unmet = 0;
+
mutex_lock(&acpi_dep_list_lock);
list_for_each_entry(dep, &acpi_dep_list, node) {
@@ -1926,38 +1908,59 @@ static acpi_status acpi_bus_check_add(acpi_handle handle, bool check_dep,
struct acpi_device **adev_p)
{
struct acpi_device *device = NULL;
- unsigned long long sta;
+ acpi_object_type acpi_type;
int type;
- int result;
acpi_bus_get_device(handle, &device);
if (device)
goto out;
- result = acpi_bus_type_and_status(handle, &type, &sta);
- if (result)
+ if (ACPI_FAILURE(acpi_get_type(handle, &acpi_type)))
return AE_OK;
- if (type == ACPI_BUS_TYPE_POWER) {
- acpi_add_power_resource(handle);
- return AE_OK;
- }
+ switch (acpi_type) {
+ case ACPI_TYPE_DEVICE:
+ if (acpi_device_should_be_hidden(handle))
+ return AE_OK;
- if (type == ACPI_BUS_TYPE_DEVICE && check_dep) {
- u32 count = acpi_scan_check_dep(handle);
- /* Bail out if the number of recorded dependencies is not 0. */
- if (count > 0) {
+ /* Bail out if there are dependencies. */
+ if (acpi_scan_check_dep(handle, check_dep) > 0) {
acpi_bus_scan_second_pass = true;
return AE_CTRL_DEPTH;
}
+
+ fallthrough;
+ case ACPI_TYPE_ANY: /* for ACPI_ROOT_OBJECT */
+ type = ACPI_BUS_TYPE_DEVICE;
+ break;
+
+ case ACPI_TYPE_PROCESSOR:
+ type = ACPI_BUS_TYPE_PROCESSOR;
+ break;
+
+ case ACPI_TYPE_THERMAL:
+ type = ACPI_BUS_TYPE_THERMAL;
+ break;
+
+ case ACPI_TYPE_POWER:
+ acpi_add_power_resource(handle);
+ fallthrough;
+ default:
+ return AE_OK;
}
- acpi_add_single_object(&device, handle, type, sta);
+ acpi_add_single_object(&device, handle, type);
if (!device)
return AE_CTRL_DEPTH;
acpi_scan_init_hotplug(device);
- if (!check_dep)
+ /*
+ * If check_dep is true at this point, the device has no dependencies,
+ * or the creation of the device object would have been postponed above.
+ */
+ if (check_dep)
+ device->dep_unmet = 0;
+ else
acpi_scan_dep_init(device);
out:
@@ -2220,8 +2223,7 @@ int acpi_bus_register_early_device(int type)
struct acpi_device *device = NULL;
int result;
- result = acpi_add_single_object(&device, NULL,
- type, ACPI_STA_DEFAULT);
+ result = acpi_add_single_object(&device, NULL, type);
if (result)
return result;
@@ -2241,8 +2243,7 @@ static int acpi_bus_scan_fixed(void)
struct acpi_device *device = NULL;
result = acpi_add_single_object(&device, NULL,
- ACPI_BUS_TYPE_POWER_BUTTON,
- ACPI_STA_DEFAULT);
+ ACPI_BUS_TYPE_POWER_BUTTON);
if (result)
return result;
@@ -2258,8 +2259,7 @@ static int acpi_bus_scan_fixed(void)
struct acpi_device *device = NULL;
result = acpi_add_single_object(&device, NULL,
- ACPI_BUS_TYPE_SLEEP_BUTTON,
- ACPI_STA_DEFAULT);
+ ACPI_BUS_TYPE_SLEEP_BUTTON);
if (result)
return result;
@@ -2355,11 +2355,13 @@ int __init acpi_scan_init(void)
acpi_detach_data(acpi_root->handle,
acpi_scan_drop_device);
acpi_device_del(acpi_root);
- put_device(&acpi_root->dev);
+ acpi_bus_put_acpi_device(acpi_root);
goto out;
}
}
+ acpi_turn_off_unused_power_resources();
+
acpi_scan_initialized = true;
out:
diff --git a/drivers/acpi/sleep.h b/drivers/acpi/sleep.h
index 1856f76ac83f..7fe41ee489d6 100644
--- a/drivers/acpi/sleep.h
+++ b/drivers/acpi/sleep.h
@@ -8,7 +8,6 @@ extern struct list_head acpi_wakeup_device_list;
extern struct mutex acpi_device_lock;
extern void acpi_resume_power_resources(void);
-extern void acpi_turn_off_unused_power_resources(void);
static inline acpi_status acpi_set_waking_vector(u32 wakeup_address)
{
diff --git a/drivers/acpi/sysfs.c b/drivers/acpi/sysfs.c
index 8baf7644a0d0..d25927195d6d 100644
--- a/drivers/acpi/sysfs.c
+++ b/drivers/acpi/sysfs.c
@@ -12,9 +12,6 @@
#include "internal.h"
-#define _COMPONENT ACPI_SYSTEM_COMPONENT
-ACPI_MODULE_NAME("sysfs");
-
#ifdef CONFIG_ACPI_DEBUG
/*
* ACPI debug sysfs I/F, including:
@@ -51,14 +48,6 @@ static const struct acpi_dlayer acpi_debug_layers[] = {
ACPI_DEBUG_INIT(ACPI_CA_DISASSEMBLER),
ACPI_DEBUG_INIT(ACPI_COMPILER),
ACPI_DEBUG_INIT(ACPI_TOOLS),
-
- ACPI_DEBUG_INIT(ACPI_SBS_COMPONENT),
- ACPI_DEBUG_INIT(ACPI_FAN_COMPONENT),
- ACPI_DEBUG_INIT(ACPI_PCI_COMPONENT),
- ACPI_DEBUG_INIT(ACPI_CONTAINER_COMPONENT),
- ACPI_DEBUG_INIT(ACPI_SYSTEM_COMPONENT),
- ACPI_DEBUG_INIT(ACPI_MEMORY_DEVICE_COMPONENT),
- ACPI_DEBUG_INIT(ACPI_PROCESSOR_COMPONENT),
};
static const struct acpi_dlevel acpi_debug_levels[] = {
@@ -652,8 +641,7 @@ static int get_status(u32 index, acpi_event_status *ret,
if (index < num_gpes) {
status = acpi_get_gpe_device(index, handle);
if (ACPI_FAILURE(status)) {
- ACPI_EXCEPTION((AE_INFO, AE_NOT_FOUND,
- "Invalid GPE 0x%x", index));
+ pr_warn("Invalid GPE 0x%x", index);
return -ENXIO;
}
status = acpi_get_gpe_status(*handle, index, ret);
diff --git a/drivers/acpi/tables.c b/drivers/acpi/tables.c
index e48690a006a4..9d581045acff 100644
--- a/drivers/acpi/tables.c
+++ b/drivers/acpi/tables.c
@@ -780,7 +780,7 @@ acpi_status acpi_os_table_override(struct acpi_table_header *existing_table,
}
/*
- * acpi_table_init()
+ * acpi_locate_initial_tables()
*
* find RSDP, find and checksum SDT/XSDT.
* checksum all tables, print SDT/XSDT
@@ -788,7 +788,7 @@ acpi_status acpi_os_table_override(struct acpi_table_header *existing_table,
* result: sdt_entry[] is initialized
*/
-int __init acpi_table_init(void)
+int __init acpi_locate_initial_tables(void)
{
acpi_status status;
@@ -803,9 +803,45 @@ int __init acpi_table_init(void)
status = acpi_initialize_tables(initial_tables, ACPI_MAX_TABLES, 0);
if (ACPI_FAILURE(status))
return -EINVAL;
- acpi_table_initrd_scan();
+ return 0;
+}
+
+void __init acpi_reserve_initial_tables(void)
+{
+ int i;
+
+ for (i = 0; i < ACPI_MAX_TABLES; i++) {
+ struct acpi_table_desc *table_desc = &initial_tables[i];
+ u64 start = table_desc->address;
+ u64 size = table_desc->length;
+
+ if (!start || !size)
+ break;
+
+ pr_info("Reserving %4s table memory at [mem 0x%llx-0x%llx]\n",
+ table_desc->signature.ascii, start, start + size - 1);
+
+ memblock_reserve(start, size);
+ }
+}
+
+void __init acpi_table_init_complete(void)
+{
+ acpi_table_initrd_scan();
check_multiple_madt();
+}
+
+int __init acpi_table_init(void)
+{
+ int ret;
+
+ ret = acpi_locate_initial_tables();
+ if (ret)
+ return ret;
+
+ acpi_table_init_complete();
+
return 0;
}
diff --git a/drivers/acpi/utils.c b/drivers/acpi/utils.c
index 682edd913b3b..3b54b8fd7396 100644
--- a/drivers/acpi/utils.c
+++ b/drivers/acpi/utils.c
@@ -512,6 +512,20 @@ EXPORT_SYMBOL(__acpi_handle_debug);
#endif
/**
+ * acpi_evaluation_failure_warn - Log evaluation failure warning.
+ * @handle: Parent object handle.
+ * @name: Name of the object whose evaluation has failed.
+ * @status: Status value returned by the failing object evaluation.
+ */
+void acpi_evaluation_failure_warn(acpi_handle handle, const char *name,
+ acpi_status status)
+{
+ acpi_handle_warn(handle, "%s evaluation failed: %s\n", name,
+ acpi_format_exception(status));
+}
+EXPORT_SYMBOL_GPL(acpi_evaluation_failure_warn);
+
+/**
* acpi_has_method: Check whether @handle has a method named @name
* @handle: ACPI device handle
* @name: name of object or method
@@ -797,7 +811,7 @@ static int acpi_dev_match_cb(struct device *dev, const void *data)
* Note that if the device is pluggable, it may since have disappeared.
*
* Note that unlike acpi_dev_found() this function checks the status
- * of the device. So for devices which are present in the dsdt, but
+ * of the device. So for devices which are present in the DSDT, but
* which are disabled (their _STA callback returns 0) this function
* will return false.
*
@@ -824,7 +838,7 @@ EXPORT_SYMBOL(acpi_dev_present);
/**
* acpi_dev_get_next_match_dev - Return the next match of ACPI device
- * @adev: Pointer to the previous acpi_device matching this @hid, @uid and @hrv
+ * @adev: Pointer to the previous ACPI device matching this @hid, @uid and @hrv
* @hid: Hardware ID of the device.
* @uid: Unique ID of the device, pass NULL to not check _UID
* @hrv: Hardware Revision of the device, pass -1 to not check _HRV
@@ -832,7 +846,11 @@ EXPORT_SYMBOL(acpi_dev_present);
* Return the next match of ACPI device if another matching device was present
* at the moment of invocation, or NULL otherwise.
*
- * The caller is responsible to call put_device() on the returned device.
+ * FIXME: The function does not tolerate the sudden disappearance of @adev, e.g.
+ * in the case of a hotplug event. That said, the caller should ensure that
+ * this will never happen.
+ *
+ * The caller is responsible for invoking acpi_dev_put() on the returned device.
*
* See additional information in acpi_dev_present() as well.
*/
@@ -861,7 +879,7 @@ EXPORT_SYMBOL(acpi_dev_get_next_match_dev);
* Return the first match of ACPI device if a matching device was present
* at the moment of invocation, or NULL otherwise.
*
- * The caller is responsible to call put_device() on the returned device.
+ * The caller is responsible for invoking acpi_dev_put() on the returned device.
*
* See additional information in acpi_dev_present() as well.
*/
@@ -872,6 +890,17 @@ acpi_dev_get_first_match_dev(const char *hid, const char *uid, s64 hrv)
}
EXPORT_SYMBOL(acpi_dev_get_first_match_dev);
+/**
+ * acpi_reduced_hardware - Return if this is an ACPI-reduced-hw machine
+ *
+ * Return true when running on an ACPI-reduced-hw machine, false otherwise.
+ */
+bool acpi_reduced_hardware(void)
+{
+ return acpi_gbl_reduced_hardware;
+}
+EXPORT_SYMBOL_GPL(acpi_reduced_hardware);
+
/*
* acpi_backlight= handling, this is done here rather then in video_detect.c
* because __setup cannot be used in modules.
diff --git a/drivers/acpi/video_detect.c b/drivers/acpi/video_detect.c
index 811d298637cb..33474fd96991 100644
--- a/drivers/acpi/video_detect.c
+++ b/drivers/acpi/video_detect.c
@@ -147,6 +147,7 @@ static const struct dmi_system_id video_detect_dmi_table[] = {
},
},
{
+ .callback = video_detect_force_vendor,
.ident = "Sony VPCEH3U1E",
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Sony Corporation"),
@@ -384,6 +385,30 @@ static const struct dmi_system_id video_detect_dmi_table[] = {
DMI_MATCH(DMI_BOARD_NAME, "BA51_MV"),
},
},
+ {
+ .callback = video_detect_force_native,
+ .ident = "ASUSTeK COMPUTER INC. GA401",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "GA401"),
+ },
+ },
+ {
+ .callback = video_detect_force_native,
+ .ident = "ASUSTeK COMPUTER INC. GA502",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "GA502"),
+ },
+ },
+ {
+ .callback = video_detect_force_native,
+ .ident = "ASUSTeK COMPUTER INC. GA503",
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "ASUSTeK COMPUTER INC."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "GA503"),
+ },
+ },
/*
* Desktops which falsely report a backlight and which our heuristics
diff --git a/drivers/android/binder.c b/drivers/android/binder.c
index c119736ca56a..61d34e1dc59c 100644
--- a/drivers/android/binder.c
+++ b/drivers/android/binder.c
@@ -1506,6 +1506,12 @@ static void binder_free_transaction(struct binder_transaction *t)
if (target_proc) {
binder_inner_proc_lock(target_proc);
+ target_proc->outstanding_txns--;
+ if (target_proc->outstanding_txns < 0)
+ pr_warn("%s: Unexpected outstanding_txns %d\n",
+ __func__, target_proc->outstanding_txns);
+ if (!target_proc->outstanding_txns && target_proc->is_frozen)
+ wake_up_interruptible_all(&target_proc->freeze_wait);
if (t->buffer)
t->buffer->transaction = NULL;
binder_inner_proc_unlock(target_proc);
@@ -2331,10 +2337,11 @@ static int binder_fixup_parent(struct binder_transaction *t,
* If the @thread parameter is not NULL, the transaction is always queued
* to the waitlist of that specific thread.
*
- * Return: true if the transactions was successfully queued
- * false if the target process or thread is dead
+ * Return: 0 if the transaction was successfully queued
+ * BR_DEAD_REPLY if the target process or thread is dead
+ * BR_FROZEN_REPLY if the target process or thread is frozen
*/
-static bool binder_proc_transaction(struct binder_transaction *t,
+static int binder_proc_transaction(struct binder_transaction *t,
struct binder_proc *proc,
struct binder_thread *thread)
{
@@ -2353,11 +2360,16 @@ static bool binder_proc_transaction(struct binder_transaction *t,
}
binder_inner_proc_lock(proc);
+ if (proc->is_frozen) {
+ proc->sync_recv |= !oneway;
+ proc->async_recv |= oneway;
+ }
- if (proc->is_dead || (thread && thread->is_dead)) {
+ if ((proc->is_frozen && !oneway) || proc->is_dead ||
+ (thread && thread->is_dead)) {
binder_inner_proc_unlock(proc);
binder_node_unlock(node);
- return false;
+ return proc->is_frozen ? BR_FROZEN_REPLY : BR_DEAD_REPLY;
}
if (!thread && !pending_async)
@@ -2373,10 +2385,11 @@ static bool binder_proc_transaction(struct binder_transaction *t,
if (!pending_async)
binder_wakeup_thread_ilocked(proc, thread, !oneway /* sync */);
+ proc->outstanding_txns++;
binder_inner_proc_unlock(proc);
binder_node_unlock(node);
- return true;
+ return 0;
}
/**
@@ -2700,7 +2713,16 @@ static void binder_transaction(struct binder_proc *proc,
u32 secid;
size_t added_size;
- security_task_getsecid(proc->tsk, &secid);
+ /*
+ * Arguably this should be the task's subjective LSM secid but
+ * we can't reliably access the subjective creds of a task
+ * other than our own so we must use the objective creds, which
+ * are safe to access. The downside is that if a task is
+ * temporarily overriding it's creds it will not be reflected
+ * here; however, it isn't clear that binder would handle that
+ * case well anyway.
+ */
+ security_task_getsecid_obj(proc->tsk, &secid);
ret = security_secid_to_secctx(secid, &secctx, &secctx_sz);
if (ret) {
return_error = BR_FAILED_REPLY;
@@ -3007,19 +3029,25 @@ static void binder_transaction(struct binder_proc *proc,
goto err_bad_object_type;
}
}
- tcomplete->type = BINDER_WORK_TRANSACTION_COMPLETE;
+ if (t->buffer->oneway_spam_suspect)
+ tcomplete->type = BINDER_WORK_TRANSACTION_ONEWAY_SPAM_SUSPECT;
+ else
+ tcomplete->type = BINDER_WORK_TRANSACTION_COMPLETE;
t->work.type = BINDER_WORK_TRANSACTION;
if (reply) {
binder_enqueue_thread_work(thread, tcomplete);
binder_inner_proc_lock(target_proc);
- if (target_thread->is_dead) {
+ if (target_thread->is_dead || target_proc->is_frozen) {
+ return_error = target_thread->is_dead ?
+ BR_DEAD_REPLY : BR_FROZEN_REPLY;
binder_inner_proc_unlock(target_proc);
goto err_dead_proc_or_thread;
}
BUG_ON(t->buffer->async_transaction != 0);
binder_pop_transaction_ilocked(target_thread, in_reply_to);
binder_enqueue_thread_work_ilocked(target_thread, &t->work);
+ target_proc->outstanding_txns++;
binder_inner_proc_unlock(target_proc);
wake_up_interruptible_sync(&target_thread->wait);
binder_free_transaction(in_reply_to);
@@ -3038,7 +3066,9 @@ static void binder_transaction(struct binder_proc *proc,
t->from_parent = thread->transaction_stack;
thread->transaction_stack = t;
binder_inner_proc_unlock(proc);
- if (!binder_proc_transaction(t, target_proc, target_thread)) {
+ return_error = binder_proc_transaction(t,
+ target_proc, target_thread);
+ if (return_error) {
binder_inner_proc_lock(proc);
binder_pop_transaction_ilocked(thread, t);
binder_inner_proc_unlock(proc);
@@ -3048,7 +3078,8 @@ static void binder_transaction(struct binder_proc *proc,
BUG_ON(target_node == NULL);
BUG_ON(t->buffer->async_transaction != 1);
binder_enqueue_thread_work(thread, tcomplete);
- if (!binder_proc_transaction(t, target_proc, NULL))
+ return_error = binder_proc_transaction(t, target_proc, NULL);
+ if (return_error)
goto err_dead_proc_or_thread;
}
if (target_thread)
@@ -3065,7 +3096,6 @@ static void binder_transaction(struct binder_proc *proc,
return;
err_dead_proc_or_thread:
- return_error = BR_DEAD_REPLY;
return_error_line = __LINE__;
binder_dequeue_work(proc, tcomplete);
err_translate_failed:
@@ -3696,7 +3726,7 @@ static int binder_wait_for_work(struct binder_thread *thread,
binder_inner_proc_lock(proc);
list_del_init(&thread->waiting_thread_node);
if (signal_pending(current)) {
- ret = -ERESTARTSYS;
+ ret = -EINTR;
break;
}
}
@@ -3875,9 +3905,14 @@ retry:
binder_stat_br(proc, thread, cmd);
} break;
- case BINDER_WORK_TRANSACTION_COMPLETE: {
+ case BINDER_WORK_TRANSACTION_COMPLETE:
+ case BINDER_WORK_TRANSACTION_ONEWAY_SPAM_SUSPECT: {
+ if (proc->oneway_spam_detection_enabled &&
+ w->type == BINDER_WORK_TRANSACTION_ONEWAY_SPAM_SUSPECT)
+ cmd = BR_ONEWAY_SPAM_SUSPECT;
+ else
+ cmd = BR_TRANSACTION_COMPLETE;
binder_inner_proc_unlock(proc);
- cmd = BR_TRANSACTION_COMPLETE;
kfree(w);
binder_stats_deleted(BINDER_STAT_TRANSACTION_COMPLETE);
if (put_user(cmd, (uint32_t __user *)ptr))
@@ -4298,6 +4333,9 @@ static void binder_free_proc(struct binder_proc *proc)
BUG_ON(!list_empty(&proc->todo));
BUG_ON(!list_empty(&proc->delivered_death));
+ if (proc->outstanding_txns)
+ pr_warn("%s: Unexpected outstanding_txns %d\n",
+ __func__, proc->outstanding_txns);
device = container_of(proc->context, struct binder_device, context);
if (refcount_dec_and_test(&device->ref)) {
kfree(proc->context->name);
@@ -4359,6 +4397,7 @@ static int binder_thread_release(struct binder_proc *proc,
(t->to_thread == thread) ? "in" : "out");
if (t->to_thread == thread) {
+ thread->proc->outstanding_txns--;
t->to_proc = NULL;
t->to_thread = NULL;
if (t->buffer) {
@@ -4609,6 +4648,76 @@ static int binder_ioctl_get_node_debug_info(struct binder_proc *proc,
return 0;
}
+static int binder_ioctl_freeze(struct binder_freeze_info *info,
+ struct binder_proc *target_proc)
+{
+ int ret = 0;
+
+ if (!info->enable) {
+ binder_inner_proc_lock(target_proc);
+ target_proc->sync_recv = false;
+ target_proc->async_recv = false;
+ target_proc->is_frozen = false;
+ binder_inner_proc_unlock(target_proc);
+ return 0;
+ }
+
+ /*
+ * Freezing the target. Prevent new transactions by
+ * setting frozen state. If timeout specified, wait
+ * for transactions to drain.
+ */
+ binder_inner_proc_lock(target_proc);
+ target_proc->sync_recv = false;
+ target_proc->async_recv = false;
+ target_proc->is_frozen = true;
+ binder_inner_proc_unlock(target_proc);
+
+ if (info->timeout_ms > 0)
+ ret = wait_event_interruptible_timeout(
+ target_proc->freeze_wait,
+ (!target_proc->outstanding_txns),
+ msecs_to_jiffies(info->timeout_ms));
+
+ if (!ret && target_proc->outstanding_txns)
+ ret = -EAGAIN;
+
+ if (ret < 0) {
+ binder_inner_proc_lock(target_proc);
+ target_proc->is_frozen = false;
+ binder_inner_proc_unlock(target_proc);
+ }
+
+ return ret;
+}
+
+static int binder_ioctl_get_freezer_info(
+ struct binder_frozen_status_info *info)
+{
+ struct binder_proc *target_proc;
+ bool found = false;
+
+ info->sync_recv = 0;
+ info->async_recv = 0;
+
+ mutex_lock(&binder_procs_lock);
+ hlist_for_each_entry(target_proc, &binder_procs, proc_node) {
+ if (target_proc->pid == info->pid) {
+ found = true;
+ binder_inner_proc_lock(target_proc);
+ info->sync_recv |= target_proc->sync_recv;
+ info->async_recv |= target_proc->async_recv;
+ binder_inner_proc_unlock(target_proc);
+ }
+ }
+ mutex_unlock(&binder_procs_lock);
+
+ if (!found)
+ return -EINVAL;
+
+ return 0;
+}
+
static long binder_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
{
int ret;
@@ -4727,6 +4836,96 @@ static long binder_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
}
break;
}
+ case BINDER_FREEZE: {
+ struct binder_freeze_info info;
+ struct binder_proc **target_procs = NULL, *target_proc;
+ int target_procs_count = 0, i = 0;
+
+ ret = 0;
+
+ if (copy_from_user(&info, ubuf, sizeof(info))) {
+ ret = -EFAULT;
+ goto err;
+ }
+
+ mutex_lock(&binder_procs_lock);
+ hlist_for_each_entry(target_proc, &binder_procs, proc_node) {
+ if (target_proc->pid == info.pid)
+ target_procs_count++;
+ }
+
+ if (target_procs_count == 0) {
+ mutex_unlock(&binder_procs_lock);
+ ret = -EINVAL;
+ goto err;
+ }
+
+ target_procs = kcalloc(target_procs_count,
+ sizeof(struct binder_proc *),
+ GFP_KERNEL);
+
+ if (!target_procs) {
+ mutex_unlock(&binder_procs_lock);
+ ret = -ENOMEM;
+ goto err;
+ }
+
+ hlist_for_each_entry(target_proc, &binder_procs, proc_node) {
+ if (target_proc->pid != info.pid)
+ continue;
+
+ binder_inner_proc_lock(target_proc);
+ target_proc->tmp_ref++;
+ binder_inner_proc_unlock(target_proc);
+
+ target_procs[i++] = target_proc;
+ }
+ mutex_unlock(&binder_procs_lock);
+
+ for (i = 0; i < target_procs_count; i++) {
+ if (ret >= 0)
+ ret = binder_ioctl_freeze(&info,
+ target_procs[i]);
+
+ binder_proc_dec_tmpref(target_procs[i]);
+ }
+
+ kfree(target_procs);
+
+ if (ret < 0)
+ goto err;
+ break;
+ }
+ case BINDER_GET_FROZEN_INFO: {
+ struct binder_frozen_status_info info;
+
+ if (copy_from_user(&info, ubuf, sizeof(info))) {
+ ret = -EFAULT;
+ goto err;
+ }
+
+ ret = binder_ioctl_get_freezer_info(&info);
+ if (ret < 0)
+ goto err;
+
+ if (copy_to_user(ubuf, &info, sizeof(info))) {
+ ret = -EFAULT;
+ goto err;
+ }
+ break;
+ }
+ case BINDER_ENABLE_ONEWAY_SPAM_DETECTION: {
+ uint32_t enable;
+
+ if (copy_from_user(&enable, ubuf, sizeof(enable))) {
+ ret = -EINVAL;
+ goto err;
+ }
+ binder_inner_proc_lock(proc);
+ proc->oneway_spam_detection_enabled = (bool)enable;
+ binder_inner_proc_unlock(proc);
+ break;
+ }
default:
ret = -EINVAL;
goto err;
@@ -4736,7 +4935,7 @@ err:
if (thread)
thread->looper_need_return = false;
wait_event_interruptible(binder_user_error_wait, binder_stop_on_user_error < 2);
- if (ret && ret != -ERESTARTSYS)
+ if (ret && ret != -EINTR)
pr_info("%d:%d ioctl %x %lx returned %d\n", proc->pid, current->pid, cmd, arg, ret);
err_unlocked:
trace_binder_ioctl_done(ret);
@@ -4823,6 +5022,7 @@ static int binder_open(struct inode *nodp, struct file *filp)
get_task_struct(current->group_leader);
proc->tsk = current->group_leader;
INIT_LIST_HEAD(&proc->todo);
+ init_waitqueue_head(&proc->freeze_wait);
proc->default_priority = task_nice(current);
/* binderfs stashes devices in i_private */
if (is_binderfs_device(nodp)) {
@@ -5035,6 +5235,9 @@ static void binder_deferred_release(struct binder_proc *proc)
proc->tmp_ref++;
proc->is_dead = true;
+ proc->is_frozen = false;
+ proc->sync_recv = false;
+ proc->async_recv = false;
threads = 0;
active_transactions = 0;
while ((n = rb_first(&proc->threads))) {
@@ -5385,7 +5588,9 @@ static const char * const binder_return_strings[] = {
"BR_FINISHED",
"BR_DEAD_BINDER",
"BR_CLEAR_DEATH_NOTIFICATION_DONE",
- "BR_FAILED_REPLY"
+ "BR_FAILED_REPLY",
+ "BR_FROZEN_REPLY",
+ "BR_ONEWAY_SPAM_SUSPECT",
};
static const char * const binder_command_strings[] = {
diff --git a/drivers/android/binder_alloc.c b/drivers/android/binder_alloc.c
index 7caf74ad2405..340515f54498 100644
--- a/drivers/android/binder_alloc.c
+++ b/drivers/android/binder_alloc.c
@@ -338,7 +338,7 @@ static inline struct vm_area_struct *binder_alloc_get_vma(
return vma;
}
-static void debug_low_async_space_locked(struct binder_alloc *alloc, int pid)
+static bool debug_low_async_space_locked(struct binder_alloc *alloc, int pid)
{
/*
* Find the amount and size of buffers allocated by the current caller;
@@ -366,13 +366,19 @@ static void debug_low_async_space_locked(struct binder_alloc *alloc, int pid)
/*
* Warn if this pid has more than 50 transactions, or more than 50% of
- * async space (which is 25% of total buffer size).
+ * async space (which is 25% of total buffer size). Oneway spam is only
+ * detected when the threshold is exceeded.
*/
if (num_buffers > 50 || total_alloc_size > alloc->buffer_size / 4) {
binder_alloc_debug(BINDER_DEBUG_USER_ERROR,
"%d: pid %d spamming oneway? %zd buffers allocated for a total size of %zd\n",
alloc->pid, pid, num_buffers, total_alloc_size);
+ if (!alloc->oneway_spam_detected) {
+ alloc->oneway_spam_detected = true;
+ return true;
+ }
}
+ return false;
}
static struct binder_buffer *binder_alloc_new_buf_locked(
@@ -525,6 +531,7 @@ static struct binder_buffer *binder_alloc_new_buf_locked(
buffer->async_transaction = is_async;
buffer->extra_buffers_size = extra_buffers_size;
buffer->pid = pid;
+ buffer->oneway_spam_suspect = false;
if (is_async) {
alloc->free_async_space -= size + sizeof(struct binder_buffer);
binder_alloc_debug(BINDER_DEBUG_BUFFER_ALLOC_ASYNC,
@@ -536,7 +543,9 @@ static struct binder_buffer *binder_alloc_new_buf_locked(
* of async space left (which is less than 10% of total
* buffer size).
*/
- debug_low_async_space_locked(alloc, pid);
+ buffer->oneway_spam_suspect = debug_low_async_space_locked(alloc, pid);
+ } else {
+ alloc->oneway_spam_detected = false;
}
}
return buffer;
diff --git a/drivers/android/binder_alloc.h b/drivers/android/binder_alloc.h
index 6e8e001381af..7dea57a84c79 100644
--- a/drivers/android/binder_alloc.h
+++ b/drivers/android/binder_alloc.h
@@ -26,6 +26,8 @@ struct binder_transaction;
* @clear_on_free: %true if buffer must be zeroed after use
* @allow_user_free: %true if user is allowed to free buffer
* @async_transaction: %true if buffer is in use for an async txn
+ * @oneway_spam_suspect: %true if total async allocate size just exceed
+ * spamming detect threshold
* @debug_id: unique ID for debugging
* @transaction: pointer to associated struct binder_transaction
* @target_node: struct binder_node associated with this buffer
@@ -45,7 +47,8 @@ struct binder_buffer {
unsigned clear_on_free:1;
unsigned allow_user_free:1;
unsigned async_transaction:1;
- unsigned debug_id:28;
+ unsigned oneway_spam_suspect:1;
+ unsigned debug_id:27;
struct binder_transaction *transaction;
@@ -87,6 +90,8 @@ struct binder_lru_page {
* @buffer_size: size of address space specified via mmap
* @pid: pid for associated binder_proc (invariant after init)
* @pages_high: high watermark of offset in @pages
+ * @oneway_spam_detected: %true if oneway spam detection fired, clear that
+ * flag once the async buffer has returned to a healthy state
*
* Bookkeeping structure for per-proc address space management for binder
* buffers. It is normally initialized during binder_init() and binder_mmap()
@@ -107,6 +112,7 @@ struct binder_alloc {
uint32_t buffer_free;
int pid;
size_t pages_high;
+ bool oneway_spam_detected;
};
#ifdef CONFIG_ANDROID_BINDER_IPC_SELFTEST
diff --git a/drivers/android/binder_internal.h b/drivers/android/binder_internal.h
index 6cd79011e35d..810c0b84d3f8 100644
--- a/drivers/android/binder_internal.h
+++ b/drivers/android/binder_internal.h
@@ -155,7 +155,7 @@ enum binder_stat_types {
};
struct binder_stats {
- atomic_t br[_IOC_NR(BR_FAILED_REPLY) + 1];
+ atomic_t br[_IOC_NR(BR_ONEWAY_SPAM_SUSPECT) + 1];
atomic_t bc[_IOC_NR(BC_REPLY_SG) + 1];
atomic_t obj_created[BINDER_STAT_COUNT];
atomic_t obj_deleted[BINDER_STAT_COUNT];
@@ -174,6 +174,7 @@ struct binder_work {
enum binder_work_type {
BINDER_WORK_TRANSACTION = 1,
BINDER_WORK_TRANSACTION_COMPLETE,
+ BINDER_WORK_TRANSACTION_ONEWAY_SPAM_SUSPECT,
BINDER_WORK_RETURN_ERROR,
BINDER_WORK_NODE,
BINDER_WORK_DEAD_BINDER,
@@ -367,9 +368,22 @@ struct binder_ref {
* (protected by binder_deferred_lock)
* @deferred_work: bitmap of deferred work to perform
* (protected by binder_deferred_lock)
+ * @outstanding_txns: number of transactions to be transmitted before
+ * processes in freeze_wait are woken up
+ * (protected by @inner_lock)
* @is_dead: process is dead and awaiting free
* when outstanding transactions are cleaned up
* (protected by @inner_lock)
+ * @is_frozen: process is frozen and unable to service
+ * binder transactions
+ * (protected by @inner_lock)
+ * @sync_recv: process received sync transactions since last frozen
+ * (protected by @inner_lock)
+ * @async_recv: process received async transactions since last frozen
+ * (protected by @inner_lock)
+ * @freeze_wait: waitqueue of processes waiting for all outstanding
+ * transactions to be processed
+ * (protected by @inner_lock)
* @todo: list of work for this process
* (protected by @inner_lock)
* @stats: per-process binder statistics
@@ -396,6 +410,8 @@ struct binder_ref {
* @outer_lock: no nesting under innor or node lock
* Lock order: 1) outer, 2) node, 3) inner
* @binderfs_entry: process-specific binderfs log file
+ * @oneway_spam_detection_enabled: process enabled oneway spam detection
+ * or not
*
* Bookkeeping structure for binder processes
*/
@@ -410,7 +426,12 @@ struct binder_proc {
struct task_struct *tsk;
struct hlist_node deferred_work_node;
int deferred_work;
+ int outstanding_txns;
bool is_dead;
+ bool is_frozen;
+ bool sync_recv;
+ bool async_recv;
+ wait_queue_head_t freeze_wait;
struct list_head todo;
struct binder_stats stats;
@@ -426,6 +447,7 @@ struct binder_proc {
spinlock_t inner_lock;
spinlock_t outer_lock;
struct dentry *binderfs_entry;
+ bool oneway_spam_detection_enabled;
};
/**
diff --git a/drivers/ata/ahci.c b/drivers/ata/ahci.c
index 00ba8e5a1ccc..33192a8f687d 100644
--- a/drivers/ata/ahci.c
+++ b/drivers/ata/ahci.c
@@ -1772,6 +1772,11 @@ static int ahci_init_one(struct pci_dev *pdev, const struct pci_device_id *ent)
hpriv->flags |= AHCI_HFLAG_NO_DEVSLP;
#ifdef CONFIG_ARM64
+ if (pdev->vendor == PCI_VENDOR_ID_HUAWEI &&
+ pdev->device == 0xa235 &&
+ pdev->revision < 0x30)
+ hpriv->flags |= AHCI_HFLAG_NO_SXS;
+
if (pdev->vendor == 0x177d && pdev->device == 0xa01c)
hpriv->irq_handler = ahci_thunderx_irq_handler;
#endif
diff --git a/drivers/ata/ahci.h b/drivers/ata/ahci.h
index 98b8baa47dc5..d1f284f0c83d 100644
--- a/drivers/ata/ahci.h
+++ b/drivers/ata/ahci.h
@@ -242,6 +242,7 @@ enum {
suspend/resume */
AHCI_HFLAG_IGN_NOTSUPP_POWER_ON = (1 << 27), /* ignore -EOPNOTSUPP
from phy_power_on() */
+ AHCI_HFLAG_NO_SXS = (1 << 28), /* SXS not supported */
/* ap->flags bits */
diff --git a/drivers/ata/ahci_brcm.c b/drivers/ata/ahci_brcm.c
index 5b32df5d33ad..6e9c5ade4c2e 100644
--- a/drivers/ata/ahci_brcm.c
+++ b/drivers/ata/ahci_brcm.c
@@ -86,7 +86,8 @@ struct brcm_ahci_priv {
u32 port_mask;
u32 quirks;
enum brcm_ahci_version version;
- struct reset_control *rcdev;
+ struct reset_control *rcdev_rescal;
+ struct reset_control *rcdev_ahci;
};
static inline u32 brcm_sata_readreg(void __iomem *addr)
@@ -352,8 +353,8 @@ static int brcm_ahci_suspend(struct device *dev)
else
ret = 0;
- if (priv->version != BRCM_SATA_BCM7216)
- reset_control_assert(priv->rcdev);
+ reset_control_assert(priv->rcdev_ahci);
+ reset_control_rearm(priv->rcdev_rescal);
return ret;
}
@@ -365,10 +366,10 @@ static int __maybe_unused brcm_ahci_resume(struct device *dev)
struct brcm_ahci_priv *priv = hpriv->plat_data;
int ret = 0;
- if (priv->version == BRCM_SATA_BCM7216)
- ret = reset_control_reset(priv->rcdev);
- else
- ret = reset_control_deassert(priv->rcdev);
+ ret = reset_control_deassert(priv->rcdev_ahci);
+ if (ret)
+ return ret;
+ ret = reset_control_reset(priv->rcdev_rescal);
if (ret)
return ret;
@@ -434,7 +435,6 @@ static int brcm_ahci_probe(struct platform_device *pdev)
{
const struct of_device_id *of_id;
struct device *dev = &pdev->dev;
- const char *reset_name = NULL;
struct brcm_ahci_priv *priv;
struct ahci_host_priv *hpriv;
struct resource *res;
@@ -456,15 +456,15 @@ static int brcm_ahci_probe(struct platform_device *pdev)
if (IS_ERR(priv->top_ctrl))
return PTR_ERR(priv->top_ctrl);
- /* Reset is optional depending on platform and named differently */
- if (priv->version == BRCM_SATA_BCM7216)
- reset_name = "rescal";
- else
- reset_name = "ahci";
-
- priv->rcdev = devm_reset_control_get_optional(&pdev->dev, reset_name);
- if (IS_ERR(priv->rcdev))
- return PTR_ERR(priv->rcdev);
+ if (priv->version == BRCM_SATA_BCM7216) {
+ priv->rcdev_rescal = devm_reset_control_get_optional_shared(
+ &pdev->dev, "rescal");
+ if (IS_ERR(priv->rcdev_rescal))
+ return PTR_ERR(priv->rcdev_rescal);
+ }
+ priv->rcdev_ahci = devm_reset_control_get_optional(&pdev->dev, "ahci");
+ if (IS_ERR(priv->rcdev_ahci))
+ return PTR_ERR(priv->rcdev_ahci);
hpriv = ahci_platform_get_resources(pdev, 0);
if (IS_ERR(hpriv))
@@ -485,10 +485,10 @@ static int brcm_ahci_probe(struct platform_device *pdev)
break;
}
- if (priv->version == BRCM_SATA_BCM7216)
- ret = reset_control_reset(priv->rcdev);
- else
- ret = reset_control_deassert(priv->rcdev);
+ ret = reset_control_reset(priv->rcdev_rescal);
+ if (ret)
+ return ret;
+ ret = reset_control_deassert(priv->rcdev_ahci);
if (ret)
return ret;
@@ -539,8 +539,8 @@ out_disable_regulators:
out_disable_clks:
ahci_platform_disable_clks(hpriv);
out_reset:
- if (priv->version != BRCM_SATA_BCM7216)
- reset_control_assert(priv->rcdev);
+ reset_control_assert(priv->rcdev_ahci);
+ reset_control_rearm(priv->rcdev_rescal);
return ret;
}
diff --git a/drivers/ata/ahci_ceva.c b/drivers/ata/ahci_ceva.c
index b10fd4c8c853..50b56cd0039d 100644
--- a/drivers/ata/ahci_ceva.c
+++ b/drivers/ata/ahci_ceva.c
@@ -12,6 +12,7 @@
#include <linux/module.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <linux/reset.h>
#include "ahci.h"
/* Vendor Specific Register Offsets */
@@ -87,6 +88,7 @@ struct ceva_ahci_priv {
u32 axicc;
bool is_cci_enabled;
int flags;
+ struct reset_control *rst;
};
static unsigned int ceva_ahci_read_id(struct ata_device *dev,
@@ -202,13 +204,46 @@ static int ceva_ahci_probe(struct platform_device *pdev)
cevapriv->ahci_pdev = pdev;
+ cevapriv->rst = devm_reset_control_get_optional_exclusive(&pdev->dev,
+ NULL);
+ if (IS_ERR(cevapriv->rst))
+ dev_err_probe(&pdev->dev, PTR_ERR(cevapriv->rst),
+ "failed to get reset\n");
+
hpriv = ahci_platform_get_resources(pdev, 0);
if (IS_ERR(hpriv))
return PTR_ERR(hpriv);
- rc = ahci_platform_enable_resources(hpriv);
- if (rc)
- return rc;
+ if (!cevapriv->rst) {
+ rc = ahci_platform_enable_resources(hpriv);
+ if (rc)
+ return rc;
+ } else {
+ int i;
+
+ rc = ahci_platform_enable_clks(hpriv);
+ if (rc)
+ return rc;
+ /* Assert the controller reset */
+ reset_control_assert(cevapriv->rst);
+
+ for (i = 0; i < hpriv->nports; i++) {
+ rc = phy_init(hpriv->phys[i]);
+ if (rc)
+ return rc;
+ }
+
+ /* De-assert the controller reset */
+ reset_control_deassert(cevapriv->rst);
+
+ for (i = 0; i < hpriv->nports; i++) {
+ rc = phy_power_on(hpriv->phys[i]);
+ if (rc) {
+ phy_exit(hpriv->phys[i]);
+ return rc;
+ }
+ }
+ }
if (of_property_read_bool(np, "ceva,broken-gen2"))
cevapriv->flags = CEVA_FLAG_BROKEN_GEN2;
diff --git a/drivers/ata/ahci_tegra.c b/drivers/ata/ahci_tegra.c
index cb55ebc1725b..4fb94db1217d 100644
--- a/drivers/ata/ahci_tegra.c
+++ b/drivers/ata/ahci_tegra.c
@@ -59,8 +59,6 @@
#define T_SATA0_CFG_PHY_1_PAD_PLL_IDDQ_EN BIT(22)
#define T_SATA0_NVOOB 0x114
-#define T_SATA0_NVOOB_COMMA_CNT_MASK (0xff << 16)
-#define T_SATA0_NVOOB_COMMA_CNT (0x07 << 16)
#define T_SATA0_NVOOB_SQUELCH_FILTER_MODE_MASK (0x3 << 24)
#define T_SATA0_NVOOB_SQUELCH_FILTER_MODE (0x1 << 24)
#define T_SATA0_NVOOB_SQUELCH_FILTER_LENGTH_MASK (0x3 << 26)
@@ -154,11 +152,18 @@ struct tegra_ahci_ops {
int (*init)(struct ahci_host_priv *hpriv);
};
+struct tegra_ahci_regs {
+ unsigned int nvoob_comma_cnt_mask;
+ unsigned int nvoob_comma_cnt_val;
+};
+
struct tegra_ahci_soc {
const char *const *supply_names;
u32 num_supplies;
bool supports_devslp;
+ bool has_sata_oob_rst;
const struct tegra_ahci_ops *ops;
+ const struct tegra_ahci_regs *regs;
};
struct tegra_ahci_priv {
@@ -240,11 +245,13 @@ static int tegra_ahci_power_on(struct ahci_host_priv *hpriv)
if (ret)
return ret;
- ret = tegra_powergate_sequence_power_up(TEGRA_POWERGATE_SATA,
- tegra->sata_clk,
- tegra->sata_rst);
- if (ret)
- goto disable_regulators;
+ if (!tegra->pdev->dev.pm_domain) {
+ ret = tegra_powergate_sequence_power_up(TEGRA_POWERGATE_SATA,
+ tegra->sata_clk,
+ tegra->sata_rst);
+ if (ret)
+ goto disable_regulators;
+ }
reset_control_assert(tegra->sata_oob_rst);
reset_control_assert(tegra->sata_cold_rst);
@@ -261,7 +268,8 @@ static int tegra_ahci_power_on(struct ahci_host_priv *hpriv)
disable_power:
clk_disable_unprepare(tegra->sata_clk);
- tegra_powergate_power_off(TEGRA_POWERGATE_SATA);
+ if (!tegra->pdev->dev.pm_domain)
+ tegra_powergate_power_off(TEGRA_POWERGATE_SATA);
disable_regulators:
regulator_bulk_disable(tegra->soc->num_supplies, tegra->supplies);
@@ -280,7 +288,8 @@ static void tegra_ahci_power_off(struct ahci_host_priv *hpriv)
reset_control_assert(tegra->sata_cold_rst);
clk_disable_unprepare(tegra->sata_clk);
- tegra_powergate_power_off(TEGRA_POWERGATE_SATA);
+ if (!tegra->pdev->dev.pm_domain)
+ tegra_powergate_power_off(TEGRA_POWERGATE_SATA);
regulator_bulk_disable(tegra->soc->num_supplies, tegra->supplies);
}
@@ -330,10 +339,10 @@ static int tegra_ahci_controller_init(struct ahci_host_priv *hpriv)
writel(val, tegra->sata_regs + SCFG_OFFSET + T_SATA_CFG_PHY_0);
val = readl(tegra->sata_regs + SCFG_OFFSET + T_SATA0_NVOOB);
- val &= ~(T_SATA0_NVOOB_COMMA_CNT_MASK |
+ val &= ~(tegra->soc->regs->nvoob_comma_cnt_mask |
T_SATA0_NVOOB_SQUELCH_FILTER_LENGTH_MASK |
T_SATA0_NVOOB_SQUELCH_FILTER_MODE_MASK);
- val |= (T_SATA0_NVOOB_COMMA_CNT |
+ val |= (tegra->soc->regs->nvoob_comma_cnt_val |
T_SATA0_NVOOB_SQUELCH_FILTER_LENGTH |
T_SATA0_NVOOB_SQUELCH_FILTER_MODE);
writel(val, tegra->sata_regs + SCFG_OFFSET + T_SATA0_NVOOB);
@@ -449,15 +458,35 @@ static const struct tegra_ahci_ops tegra124_ahci_ops = {
.init = tegra124_ahci_init,
};
+static const struct tegra_ahci_regs tegra124_ahci_regs = {
+ .nvoob_comma_cnt_mask = GENMASK(30, 28),
+ .nvoob_comma_cnt_val = (7 << 28),
+};
+
static const struct tegra_ahci_soc tegra124_ahci_soc = {
.supply_names = tegra124_supply_names,
.num_supplies = ARRAY_SIZE(tegra124_supply_names),
.supports_devslp = false,
+ .has_sata_oob_rst = true,
.ops = &tegra124_ahci_ops,
+ .regs = &tegra124_ahci_regs,
};
static const struct tegra_ahci_soc tegra210_ahci_soc = {
.supports_devslp = false,
+ .has_sata_oob_rst = true,
+ .regs = &tegra124_ahci_regs,
+};
+
+static const struct tegra_ahci_regs tegra186_ahci_regs = {
+ .nvoob_comma_cnt_mask = GENMASK(23, 16),
+ .nvoob_comma_cnt_val = (7 << 16),
+};
+
+static const struct tegra_ahci_soc tegra186_ahci_soc = {
+ .supports_devslp = false,
+ .has_sata_oob_rst = false,
+ .regs = &tegra186_ahci_regs,
};
static const struct of_device_id tegra_ahci_of_match[] = {
@@ -469,6 +498,10 @@ static const struct of_device_id tegra_ahci_of_match[] = {
.compatible = "nvidia,tegra210-ahci",
.data = &tegra210_ahci_soc
},
+ {
+ .compatible = "nvidia,tegra186-ahci",
+ .data = &tegra186_ahci_soc
+ },
{}
};
MODULE_DEVICE_TABLE(of, tegra_ahci_of_match);
@@ -518,10 +551,13 @@ static int tegra_ahci_probe(struct platform_device *pdev)
return PTR_ERR(tegra->sata_rst);
}
- tegra->sata_oob_rst = devm_reset_control_get(&pdev->dev, "sata-oob");
- if (IS_ERR(tegra->sata_oob_rst)) {
- dev_err(&pdev->dev, "Failed to get sata-oob reset\n");
- return PTR_ERR(tegra->sata_oob_rst);
+ if (tegra->soc->has_sata_oob_rst) {
+ tegra->sata_oob_rst = devm_reset_control_get(&pdev->dev,
+ "sata-oob");
+ if (IS_ERR(tegra->sata_oob_rst)) {
+ dev_err(&pdev->dev, "Failed to get sata-oob reset\n");
+ return PTR_ERR(tegra->sata_oob_rst);
+ }
}
tegra->sata_cold_rst = devm_reset_control_get(&pdev->dev, "sata-cold");
diff --git a/drivers/ata/ahci_xgene.c b/drivers/ata/ahci_xgene.c
index 16246c843365..dffc432b9d54 100644
--- a/drivers/ata/ahci_xgene.c
+++ b/drivers/ata/ahci_xgene.c
@@ -537,7 +537,7 @@ softreset_retry:
/**
* xgene_ahci_handle_broken_edge_irq - Handle the broken irq.
- * @ata_host: Host that recieved the irq
+ * @host: Host that recieved the irq
* @irq_masked: HOST_IRQ_STAT value
*
* For hardware with broken edge trigger latch
diff --git a/drivers/ata/ata_generic.c b/drivers/ata/ata_generic.c
index 9ff545ce8da3..20a32e4d501d 100644
--- a/drivers/ata/ata_generic.c
+++ b/drivers/ata/ata_generic.c
@@ -151,7 +151,7 @@ static int is_intel_ider(struct pci_dev *dev)
}
/**
- * ata_generic_init - attach generic IDE
+ * ata_generic_init_one - attach generic IDE
* @dev: PCI device found
* @id: match entry
*
diff --git a/drivers/ata/libahci.c b/drivers/ata/libahci.c
index ea5bf5f4cbed..fec2e9754aed 100644
--- a/drivers/ata/libahci.c
+++ b/drivers/ata/libahci.c
@@ -493,6 +493,11 @@ void ahci_save_initial_config(struct device *dev, struct ahci_host_priv *hpriv)
cap |= HOST_CAP_ALPM;
}
+ if ((cap & HOST_CAP_SXS) && (hpriv->flags & AHCI_HFLAG_NO_SXS)) {
+ dev_info(dev, "controller does not support SXS, disabling CAP_SXS\n");
+ cap &= ~HOST_CAP_SXS;
+ }
+
if (hpriv->force_port_map && port_map != hpriv->force_port_map) {
dev_info(dev, "forcing port_map 0x%x -> 0x%x\n",
port_map, hpriv->force_port_map);
diff --git a/drivers/ata/libahci_platform.c b/drivers/ata/libahci_platform.c
index de638dafce21..b2f552088291 100644
--- a/drivers/ata/libahci_platform.c
+++ b/drivers/ata/libahci_platform.c
@@ -582,11 +582,13 @@ int ahci_platform_init_host(struct platform_device *pdev,
int i, irq, n_ports, rc;
irq = platform_get_irq(pdev, 0);
- if (irq <= 0) {
+ if (irq < 0) {
if (irq != -EPROBE_DEFER)
dev_err(dev, "no irq\n");
return irq;
}
+ if (!irq)
+ return -EINVAL;
hpriv->irq = irq;
diff --git a/drivers/ata/libata-acpi.c b/drivers/ata/libata-acpi.c
index 224e3486e9a5..7a7d6642edcc 100644
--- a/drivers/ata/libata-acpi.c
+++ b/drivers/ata/libata-acpi.c
@@ -476,7 +476,7 @@ static int ata_dev_get_GTF(struct ata_device *dev, struct ata_acpi_gtf **gtf)
}
/**
- * ata_acpi_gtm_xfermode - determine xfermode from GTM parameter
+ * ata_acpi_gtm_xfermask - determine xfermode from GTM parameter
* @dev: target device
* @gtm: GTM parameter to use
*
@@ -624,6 +624,7 @@ static int ata_acpi_filter_tf(struct ata_device *dev,
* ata_acpi_run_tf - send taskfile registers to host controller
* @dev: target ATA device
* @gtf: raw ATA taskfile register set (0x1f1 - 0x1f7)
+ * @prev_gtf: previous command
*
* Outputs ATA taskfile to standard ATA host controller.
* Writes the control, feature, nsect, lbal, lbam, and lbah registers.
diff --git a/drivers/ata/libata-eh.c b/drivers/ata/libata-eh.c
index b6f92050e60c..bb3637762985 100644
--- a/drivers/ata/libata-eh.c
+++ b/drivers/ata/libata-eh.c
@@ -1599,7 +1599,7 @@ static unsigned int ata_eh_analyze_tf(struct ata_queued_cmd *qc,
}
if (qc->flags & ATA_QCFLAG_SENSE_VALID) {
- int ret = scsi_check_sense(qc->scsicmd);
+ enum scsi_disposition ret = scsi_check_sense(qc->scsicmd);
/*
* SUCCESS here means that the sense code could be
* evaluated and should be passed to the upper layers
@@ -2613,6 +2613,7 @@ int ata_eh_reset(struct ata_link *link, int classify,
switch (tmp) {
case -EAGAIN:
rc = -EAGAIN;
+ break;
case 0:
break;
default:
diff --git a/drivers/ata/libata-pmp.c b/drivers/ata/libata-pmp.c
index 79f2aeeb482a..ba7be3f38617 100644
--- a/drivers/ata/libata-pmp.c
+++ b/drivers/ata/libata-pmp.c
@@ -62,7 +62,7 @@ static unsigned int sata_pmp_read(struct ata_link *link, int reg, u32 *r_val)
* sata_pmp_write - write PMP register
* @link: link to write PMP register for
* @reg: register to write
- * @r_val: value to write
+ * @val: value to write
*
* Write PMP register.
*
diff --git a/drivers/ata/libata-sata.c b/drivers/ata/libata-sata.c
index c16423e44525..8adeab76dd38 100644
--- a/drivers/ata/libata-sata.c
+++ b/drivers/ata/libata-sata.c
@@ -1067,7 +1067,7 @@ int ata_scsi_change_queue_depth(struct scsi_device *sdev, int queue_depth)
EXPORT_SYMBOL_GPL(ata_scsi_change_queue_depth);
/**
- * port_alloc - Allocate port for a SAS attached SATA device
+ * ata_sas_port_alloc - Allocate port for a SAS attached SATA device
* @host: ATA host container for all SAS ports
* @port_info: Information from low-level host driver
* @shost: SCSI host that the scsi device is attached to
@@ -1127,7 +1127,7 @@ int ata_sas_port_start(struct ata_port *ap)
EXPORT_SYMBOL_GPL(ata_sas_port_start);
/**
- * ata_port_stop - Undo ata_sas_port_start()
+ * ata_sas_port_stop - Undo ata_sas_port_start()
* @ap: Port to shut down
*
* May be used as the port_stop() entry in ata_port_operations.
diff --git a/drivers/ata/libata-scsi.c b/drivers/ata/libata-scsi.c
index 48b8934970f3..fd8b6febbf70 100644
--- a/drivers/ata/libata-scsi.c
+++ b/drivers/ata/libata-scsi.c
@@ -1043,8 +1043,7 @@ int ata_scsi_dev_config(struct scsi_device *sdev, struct ata_device *dev)
blk_queue_max_segments(q, queue_max_segments(q) - 1);
sdev->dma_drain_len = ATAPI_MAX_DRAIN;
- sdev->dma_drain_buf = kmalloc(sdev->dma_drain_len,
- q->bounce_gfp | GFP_KERNEL);
+ sdev->dma_drain_buf = kmalloc(sdev->dma_drain_len, GFP_NOIO);
if (!sdev->dma_drain_buf) {
ata_dev_err(dev, "drain buffer allocation failed\n");
return -ENOMEM;
diff --git a/drivers/ata/libata-transport.c b/drivers/ata/libata-transport.c
index 6a40e3c6cf49..34bb4608bdc6 100644
--- a/drivers/ata/libata-transport.c
+++ b/drivers/ata/libata-transport.c
@@ -250,7 +250,7 @@ static int ata_tport_match(struct attribute_container *cont,
/**
* ata_tport_delete -- remove ATA PORT
- * @port: ATA PORT to remove
+ * @ap: ATA PORT to remove
*
* Removes the specified ATA PORT. Remove the associated link as well.
*/
@@ -376,7 +376,7 @@ static int ata_tlink_match(struct attribute_container *cont,
/**
* ata_tlink_delete -- remove ATA LINK
- * @port: ATA LINK to remove
+ * @link: ATA LINK to remove
*
* Removes the specified ATA LINK. remove associated ATA device(s) as well.
*/
@@ -632,7 +632,7 @@ static void ata_tdev_free(struct ata_device *dev)
/**
* ata_tdev_delete -- remove ATA device
- * @port: ATA PORT to remove
+ * @ata_dev: ATA device to remove
*
* Removes the specified ATA device.
*/
diff --git a/drivers/ata/pata_acpi.c b/drivers/ata/pata_acpi.c
index fa2bfc344a97..ade4c3eee230 100644
--- a/drivers/ata/pata_acpi.c
+++ b/drivers/ata/pata_acpi.c
@@ -28,7 +28,7 @@ struct pata_acpi {
/**
* pacpi_pre_reset - check for 40/80 pin
- * @ap: Port
+ * @link: ATA link
* @deadline: deadline jiffies for the operation
*
* Perform the PATA port setup we need.
@@ -63,8 +63,8 @@ static int pacpi_cable_detect(struct ata_port *ap)
/**
* pacpi_discover_modes - filter non ACPI modes
+ * @ap: ATA port
* @adev: ATA device
- * @mask: proposed modes
*
* Try the modes available and see which ones the ACPI method will
* set up sensibly. From this we get a mask of ACPI modes we can use
@@ -224,7 +224,7 @@ static struct ata_port_operations pacpi_ops = {
/**
* pacpi_init_one - Register ACPI ATA PCI device with kernel services
* @pdev: PCI device to register
- * @ent: Entry in pacpi_pci_tbl matching with @pdev
+ * @id: PCI device ID
*
* Called from kernel PCI layer.
*
diff --git a/drivers/ata/pata_ali.c b/drivers/ata/pata_ali.c
index 0b122f903b8a..557ecf466102 100644
--- a/drivers/ata/pata_ali.c
+++ b/drivers/ata/pata_ali.c
@@ -108,8 +108,8 @@ static int ali_c2_cable_detect(struct ata_port *ap)
/**
* ali_20_filter - filter for earlier ALI DMA
- * @ap: ALi ATA port
- * @adev: attached device
+ * @adev: ATA device
+ * @mask: received mask to manipulate and pass back
*
* Ensure that we do not do DMA on CD devices. We may be able to
* fix that later on. Also ensure we do not do UDMA on WDC drives
@@ -313,7 +313,7 @@ static void ali_lock_sectors(struct ata_device *adev)
/**
* ali_check_atapi_dma - DMA check for most ALi controllers
- * @adev: Device
+ * @qc: Command to complete
*
* Called to decide whether commands should be sent by DMA or PIO
*/
diff --git a/drivers/ata/pata_amd.c b/drivers/ata/pata_amd.c
index 987967f976cb..c8acba162d02 100644
--- a/drivers/ata/pata_amd.c
+++ b/drivers/ata/pata_amd.c
@@ -167,7 +167,6 @@ static int amd_cable_detect(struct ata_port *ap)
/**
* amd_fifo_setup - set the PIO FIFO for ATA/ATAPI
* @ap: ATA interface
- * @adev: ATA device
*
* Set the PCI fifo for this device according to the devices present
* on the bus at this point in time. We need to turn the post write buffer
@@ -320,8 +319,9 @@ static unsigned long nv_mode_filter(struct ata_device *dev,
}
/**
- * nv_probe_init - cable detection
- * @lin: ATA link
+ * nv_pre_reset - cable detection
+ * @link: ATA link
+ * @deadline: deadline jiffies for the operation
*
* Perform cable detection. The BIOS stores this in PCI config
* space for us.
diff --git a/drivers/ata/pata_arasan_cf.c b/drivers/ata/pata_arasan_cf.c
index e9cf31f38450..63f39440a9b4 100644
--- a/drivers/ata/pata_arasan_cf.c
+++ b/drivers/ata/pata_arasan_cf.c
@@ -818,12 +818,19 @@ static int arasan_cf_probe(struct platform_device *pdev)
else
quirk = CF_BROKEN_UDMA; /* as it is on spear1340 */
- /* if irq is 0, support only PIO */
- acdev->irq = platform_get_irq(pdev, 0);
- if (acdev->irq)
+ /*
+ * If there's an error getting IRQ (or we do get IRQ0),
+ * support only PIO
+ */
+ ret = platform_get_irq(pdev, 0);
+ if (ret > 0) {
+ acdev->irq = ret;
irq_handler = arasan_cf_interrupt;
- else
+ } else if (ret == -EPROBE_DEFER) {
+ return ret;
+ } else {
quirk |= CF_BROKEN_MWDMA | CF_BROKEN_UDMA;
+ }
acdev->pbase = res->start;
acdev->vbase = devm_ioremap(&pdev->dev, res->start,
diff --git a/drivers/ata/pata_artop.c b/drivers/ata/pata_artop.c
index 6bd2228bb6ff..ad3c5808aaad 100644
--- a/drivers/ata/pata_artop.c
+++ b/drivers/ata/pata_artop.c
@@ -268,7 +268,7 @@ static void artop6260_set_dmamode (struct ata_port *ap, struct ata_device *adev)
}
/**
- * artop_6210_qc_defer - implement serialization
+ * artop6210_qc_defer - implement serialization
* @qc: command
*
* Issue commands per host on this chip.
@@ -344,7 +344,7 @@ static void atp8xx_fixup(struct pci_dev *pdev)
/**
* artop_init_one - Register ARTOP ATA PCI device with kernel services
* @pdev: PCI device to register
- * @ent: Entry in artop_pci_tbl matching with @pdev
+ * @id: PCI device ID
*
* Called from kernel PCI layer.
*
diff --git a/drivers/ata/pata_atiixp.c b/drivers/ata/pata_atiixp.c
index c68aa3f585f2..d671d33ef287 100644
--- a/drivers/ata/pata_atiixp.c
+++ b/drivers/ata/pata_atiixp.c
@@ -93,6 +93,7 @@ static int atiixp_prereset(struct ata_link *link, unsigned long deadline)
* atiixp_set_pio_timing - set initial PIO mode data
* @ap: ATA interface
* @adev: ATA device
+ * @pio: Requested PIO
*
* Called by both the pio and dma setup functions to set the controller
* timings for PIO transfers. We must load both the mode number and
@@ -227,7 +228,7 @@ static void atiixp_bmdma_start(struct ata_queued_cmd *qc)
}
/**
- * atiixp_dma_stop - DMA stop callback
+ * atiixp_bmdma_stop - DMA stop callback
* @qc: Command in progress
*
* DMA has completed. Clear the UDMA flag as the next operations will
diff --git a/drivers/ata/pata_cs5520.c b/drivers/ata/pata_cs5520.c
index 9052148b306d..d09d432d3c44 100644
--- a/drivers/ata/pata_cs5520.c
+++ b/drivers/ata/pata_cs5520.c
@@ -52,6 +52,7 @@ static const struct pio_clocks cs5520_pio_clocks[]={
* cs5520_set_timings - program PIO timings
* @ap: ATA port
* @adev: ATA device
+ * @pio: PIO ID
*
* Program the PIO mode timings for the controller according to the pio
* clocking table.
@@ -246,6 +247,7 @@ static int cs5520_reinit_one(struct pci_dev *pdev)
/**
* cs5520_pci_device_suspend - device suspend
* @pdev: PCI device
+ * @mesg: PM event message
*
* We have to cut and waste bits from the standard method because
* the 5520 is a bit odd and not just a pure ATA device. As a result
diff --git a/drivers/ata/pata_cs5530.c b/drivers/ata/pata_cs5530.c
index ad75d02b6dac..a1b4aaccaa50 100644
--- a/drivers/ata/pata_cs5530.c
+++ b/drivers/ata/pata_cs5530.c
@@ -271,7 +271,7 @@ fail_put:
/**
* cs5530_init_one - Initialise a CS5530
- * @dev: PCI device
+ * @pdev: PCI device
* @id: Entry in match table
*
* Install a driver for the newly found CS5530 companion chip. Most of
diff --git a/drivers/ata/pata_hpt366.c b/drivers/ata/pata_hpt366.c
index 2574d6fbb1ad..06b7c4a9ec95 100644
--- a/drivers/ata/pata_hpt366.c
+++ b/drivers/ata/pata_hpt366.c
@@ -192,6 +192,7 @@ static int hpt_dma_blacklisted(const struct ata_device *dev, char *modestr,
/**
* hpt366_filter - mode selection filter
* @adev: ATA device
+ * @mask: Current mask to manipulate and pass back
*
* Block UDMA on devices that cause trouble with this controller.
*/
diff --git a/drivers/ata/pata_hpt37x.c b/drivers/ata/pata_hpt37x.c
index fad6c6a87313..f242157bc81b 100644
--- a/drivers/ata/pata_hpt37x.c
+++ b/drivers/ata/pata_hpt37x.c
@@ -275,6 +275,7 @@ static const char * const bad_ata100_5[] = {
/**
* hpt370_filter - mode selection filter
* @adev: ATA device
+ * @mask: mode mask
*
* Block UDMA on devices that cause trouble with this controller.
*/
@@ -293,6 +294,7 @@ static unsigned long hpt370_filter(struct ata_device *adev, unsigned long mask)
/**
* hpt370a_filter - mode selection filter
* @adev: ATA device
+ * @mask: mode mask
*
* Block UDMA on devices that cause trouble with this controller.
*/
@@ -463,7 +465,7 @@ static void hpt370_set_dmamode(struct ata_port *ap, struct ata_device *adev)
}
/**
- * hpt370_bmdma_end - DMA engine stop
+ * hpt370_bmdma_stop - DMA engine stop
* @qc: ATA command
*
* Work around the HPT370 DMA engine.
@@ -557,7 +559,7 @@ static void hpt372_set_dmamode(struct ata_port *ap, struct ata_device *adev)
}
/**
- * hpt37x_bmdma_end - DMA engine stop
+ * hpt37x_bmdma_stop - DMA engine stop
* @qc: ATA command
*
* Clean up after the HPT372 and later DMA engine
diff --git a/drivers/ata/pata_hpt3x2n.c b/drivers/ata/pata_hpt3x2n.c
index 9cb2d50db876..48eef338e050 100644
--- a/drivers/ata/pata_hpt3x2n.c
+++ b/drivers/ata/pata_hpt3x2n.c
@@ -237,7 +237,7 @@ static void hpt3x2n_set_dmamode(struct ata_port *ap, struct ata_device *adev)
}
/**
- * hpt3x2n_bmdma_end - DMA engine stop
+ * hpt3x2n_bmdma_stop - DMA engine stop
* @qc: ATA command
*
* Clean up after the HPT3x2n and later DMA engine
diff --git a/drivers/ata/pata_it821x.c b/drivers/ata/pata_it821x.c
index 9bac79edbc2c..0e2265978a34 100644
--- a/drivers/ata/pata_it821x.c
+++ b/drivers/ata/pata_it821x.c
@@ -334,7 +334,7 @@ static void it821x_passthru_set_dmamode(struct ata_port *ap, struct ata_device *
}
/**
- * it821x_passthru_dma_start - DMA start callback
+ * it821x_passthru_bmdma_start - DMA start callback
* @qc: Command in progress
*
* Usually drivers set the DMA timing at the point the set_dmamode call
@@ -357,7 +357,7 @@ static void it821x_passthru_bmdma_start(struct ata_queued_cmd *qc)
}
/**
- * it821x_passthru_dma_stop - DMA stop callback
+ * it821x_passthru_bmdma_stop - DMA stop callback
* @qc: ATA command
*
* We loaded new timings in dma_start, as a result we need to restore
diff --git a/drivers/ata/pata_ixp4xx_cf.c b/drivers/ata/pata_ixp4xx_cf.c
index d1644a8ef9fa..43215a4c1e54 100644
--- a/drivers/ata/pata_ixp4xx_cf.c
+++ b/drivers/ata/pata_ixp4xx_cf.c
@@ -135,12 +135,12 @@ static void ixp4xx_setup_port(struct ata_port *ap,
static int ixp4xx_pata_probe(struct platform_device *pdev)
{
- unsigned int irq;
struct resource *cs0, *cs1;
struct ata_host *host;
struct ata_port *ap;
struct ixp4xx_pata_data *data = dev_get_platdata(&pdev->dev);
int ret;
+ int irq;
cs0 = platform_get_resource(pdev, IORESOURCE_MEM, 0);
cs1 = platform_get_resource(pdev, IORESOURCE_MEM, 1);
@@ -165,8 +165,12 @@ static int ixp4xx_pata_probe(struct platform_device *pdev)
return -ENOMEM;
irq = platform_get_irq(pdev, 0);
- if (irq)
+ if (irq > 0)
irq_set_irq_type(irq, IRQ_TYPE_EDGE_RISING);
+ else if (irq < 0)
+ return irq;
+ else
+ return -EINVAL;
/* Setup expansion bus chip selects */
*data->cs0_cfg = data->cs0_bits;
diff --git a/drivers/ata/pata_jmicron.c b/drivers/ata/pata_jmicron.c
index c3dedd3e71fb..d1b3ce8958dd 100644
--- a/drivers/ata/pata_jmicron.c
+++ b/drivers/ata/pata_jmicron.c
@@ -120,7 +120,7 @@ static struct ata_port_operations jmicron_ops = {
/**
* jmicron_init_one - Register Jmicron ATA PCI device with kernel services
* @pdev: PCI device to register
- * @ent: Entry in jmicron_pci_tbl matching with @pdev
+ * @id: PCI device ID
*
* Called from kernel PCI layer.
*
diff --git a/drivers/ata/pata_legacy.c b/drivers/ata/pata_legacy.c
index 4fd12b20df23..c3e6592712c4 100644
--- a/drivers/ata/pata_legacy.c
+++ b/drivers/ata/pata_legacy.c
@@ -63,7 +63,66 @@
static int all;
module_param(all, int, 0444);
-MODULE_PARM_DESC(all, "Grab all legacy port devices, even if PCI(0=off, 1=on)");
+MODULE_PARM_DESC(all,
+ "Set to probe unclaimed pri/sec ISA port ranges even if PCI");
+
+static int probe_all;
+module_param(probe_all, int, 0);
+MODULE_PARM_DESC(probe_all,
+ "Set to probe tertiary+ ISA port ranges even if PCI");
+
+static int probe_mask = ~0;
+module_param(probe_mask, int, 0);
+MODULE_PARM_DESC(probe_mask, "Probe mask for legacy ISA PATA ports");
+
+static int autospeed;
+module_param(autospeed, int, 0);
+MODULE_PARM_DESC(autospeed, "Chip present that snoops speed changes");
+
+static int pio_mask = ATA_PIO4;
+module_param(pio_mask, int, 0);
+MODULE_PARM_DESC(pio_mask, "PIO range for autospeed devices");
+
+static int iordy_mask = 0xFFFFFFFF;
+module_param(iordy_mask, int, 0);
+MODULE_PARM_DESC(iordy_mask, "Use IORDY if available");
+
+static int ht6560a;
+module_param(ht6560a, int, 0);
+MODULE_PARM_DESC(ht6560a, "HT 6560A on primary 1, second 2, both 3");
+
+static int ht6560b;
+module_param(ht6560b, int, 0);
+MODULE_PARM_DESC(ht6560b, "HT 6560B on primary 1, secondary 2, both 3");
+
+static int opti82c611a;
+module_param(opti82c611a, int, 0);
+MODULE_PARM_DESC(opti82c611a,
+ "Opti 82c611A on primary 1, secondary 2, both 3");
+
+static int opti82c46x;
+module_param(opti82c46x, int, 0);
+MODULE_PARM_DESC(opti82c46x,
+ "Opti 82c465MV on primary 1, secondary 2, both 3");
+
+#ifdef CONFIG_PATA_QDI_MODULE
+static int qdi = 1;
+#else
+static int qdi;
+#endif
+module_param(qdi, int, 0);
+MODULE_PARM_DESC(qdi, "Set to probe QDI controllers");
+
+#ifdef CONFIG_PATA_WINBOND_VLB_MODULE
+static int winbond = 1;
+#else
+static int winbond;
+#endif
+module_param(winbond, int, 0);
+MODULE_PARM_DESC(winbond,
+ "Set to probe Winbond controllers, "
+ "give I/O port if non standard");
+
enum controller {
BIOS = 0,
@@ -117,30 +176,6 @@ static struct ata_host *legacy_host[NR_HOST];
static int nr_legacy_host;
-static int probe_all; /* Set to check all ISA port ranges */
-static int ht6560a; /* HT 6560A on primary 1, second 2, both 3 */
-static int ht6560b; /* HT 6560A on primary 1, second 2, both 3 */
-static int opti82c611a; /* Opti82c611A on primary 1, sec 2, both 3 */
-static int opti82c46x; /* Opti 82c465MV present(pri/sec autodetect) */
-static int autospeed; /* Chip present which snoops speed changes */
-static int pio_mask = ATA_PIO4; /* PIO range for autospeed devices */
-static int iordy_mask = 0xFFFFFFFF; /* Use iordy if available */
-
-/* Set to probe QDI controllers */
-#ifdef CONFIG_PATA_QDI_MODULE
-static int qdi = 1;
-#else
-static int qdi;
-#endif
-
-#ifdef CONFIG_PATA_WINBOND_VLB_MODULE
-static int winbond = 1; /* Set to probe Winbond controllers,
- give I/O port if non standard */
-#else
-static int winbond; /* Set to probe Winbond controllers,
- give I/O port if non standard */
-#endif
-
/**
* legacy_probe_add - Add interface to probe list
* @port: Controller port
@@ -168,6 +203,8 @@ static int legacy_probe_add(unsigned long port, unsigned int irq,
free = lp;
/* Matching port, or the correct slot for ordering */
if (lp->port == port || legacy_port[i] == port) {
+ if (!(probe_mask & 1 << i))
+ return -1;
free = lp;
break;
}
@@ -588,7 +625,7 @@ static void opti82c46x_set_piomode(struct ata_port *ap, struct ata_device *adev)
}
/**
- * opt82c465mv_qc_issue - command issue
+ * opti82c46x_qc_issue - command issue
* @qc: command pending
*
* Called when the libata layer is about to issue a command. We wrap
@@ -923,7 +960,7 @@ static __init int probe_chip_type(struct legacy_probe *probe)
/**
* legacy_init_one - attach a legacy interface
- * @pl: probe record
+ * @probe: probe record
*
* Register an ISA bus IDE interface. Such interfaces are PIO and we
* assume do not support IRQ sharing.
@@ -1009,8 +1046,8 @@ fail:
/**
* legacy_check_special_cases - ATA special cases
* @p: PCI device to check
- * @master: set this if we find an ATA master
- * @master: set this if we find an ATA secondary
+ * @primary: set this if we find an ATA master
+ * @secondary: set this if we find an ATA secondary
*
* A small number of vendors implemented early PCI ATA interfaces
* on bridge logic without the ATA interface being PCI visible.
@@ -1250,16 +1287,5 @@ MODULE_VERSION(DRV_VERSION);
MODULE_ALIAS("pata_qdi");
MODULE_ALIAS("pata_winbond");
-module_param(probe_all, int, 0);
-module_param(autospeed, int, 0);
-module_param(ht6560a, int, 0);
-module_param(ht6560b, int, 0);
-module_param(opti82c611a, int, 0);
-module_param(opti82c46x, int, 0);
-module_param(qdi, int, 0);
-module_param(winbond, int, 0);
-module_param(pio_mask, int, 0);
-module_param(iordy_mask, int, 0);
-
module_init(legacy_init);
module_exit(legacy_exit);
diff --git a/drivers/ata/pata_marvell.c b/drivers/ata/pata_marvell.c
index b066809ba9a1..361597d14c56 100644
--- a/drivers/ata/pata_marvell.c
+++ b/drivers/ata/pata_marvell.c
@@ -110,7 +110,7 @@ static struct ata_port_operations marvell_ops = {
/**
* marvell_init_one - Register Marvell ATA PCI device with kernel services
* @pdev: PCI device to register
- * @ent: Entry in marvell_pci_tbl matching with @pdev
+ * @id: PCI device ID
*
* Called from kernel PCI layer.
*
diff --git a/drivers/ata/pata_ns87415.c b/drivers/ata/pata_ns87415.c
index 1532b2e3c672..f4949e704356 100644
--- a/drivers/ata/pata_ns87415.c
+++ b/drivers/ata/pata_ns87415.c
@@ -113,7 +113,7 @@ static void ns87415_set_piomode(struct ata_port *ap, struct ata_device *adev)
* ns87415_bmdma_setup - Set up DMA
* @qc: Command block
*
- * Set up for bus masterng DMA. We have to do this ourselves
+ * Set up for bus mastering DMA. We have to do this ourselves
* rather than use the helper due to a chip erratum
*/
@@ -174,7 +174,7 @@ static void ns87415_bmdma_stop(struct ata_queued_cmd *qc)
* ns87415_irq_clear - Clear interrupt
* @ap: Channel to clear
*
- * Erratum: Due to a chip bug regisers 02 and 0A bit 1 and 2 (the
+ * Erratum: Due to a chip bug registers 02 and 0A bit 1 and 2 (the
* error bits) are reset by writing to register 00 or 08.
*/
diff --git a/drivers/ata/pata_opti.c b/drivers/ata/pata_opti.c
index 06a800a3b070..01976c4e4033 100644
--- a/drivers/ata/pata_opti.c
+++ b/drivers/ata/pata_opti.c
@@ -69,7 +69,7 @@ static int opti_pre_reset(struct ata_link *link, unsigned long deadline)
/**
* opti_write_reg - control register setup
* @ap: ATA port
- * @value: value
+ * @val: value
* @reg: control register number
*
* The Opti uses magic 'trapdoor' register accesses to do configuration
diff --git a/drivers/ata/pata_optidma.c b/drivers/ata/pata_optidma.c
index fbcf0af34924..f6278d9de348 100644
--- a/drivers/ata/pata_optidma.c
+++ b/drivers/ata/pata_optidma.c
@@ -287,7 +287,7 @@ static void optiplus_set_dma_mode(struct ata_port *ap, struct ata_device *adev)
}
/**
- * optidma_make_bits - PCI setup helper
+ * optidma_make_bits43 - PCI setup helper
* @adev: ATA device
*
* Turn the ATA device setup into PCI configuration bits
@@ -309,6 +309,7 @@ static u8 optidma_make_bits43(struct ata_device *adev)
/**
* optidma_set_mode - mode setup
* @link: link to set up
+ * @r_failed: out parameter for failed device
*
* Use the standard setup to tune the chipset and then finalise the
* configuration by writing the nibble of extra bits of data into
@@ -354,7 +355,7 @@ static struct ata_port_operations optiplus_port_ops = {
/**
* optiplus_with_udma - Look for UDMA capable setup
- * @pdev; ATA controller
+ * @pdev: ATA controller
*/
static int optiplus_with_udma(struct pci_dev *pdev)
diff --git a/drivers/ata/pata_pdc2027x.c b/drivers/ata/pata_pdc2027x.c
index de834fbb6dfe..effc1a09444d 100644
--- a/drivers/ata/pata_pdc2027x.c
+++ b/drivers/ata/pata_pdc2027x.c
@@ -196,7 +196,7 @@ static inline void __iomem *dev_mmio(struct ata_port *ap, struct ata_device *ade
}
/**
- * pdc2027x_pata_cable_detect - Probe host controller cable detect info
+ * pdc2027x_cable_detect - Probe host controller cable detect info
* @ap: Port for which cable detect info is desired
*
* Read 80c cable indicator from Promise extended register.
@@ -251,7 +251,7 @@ static int pdc2027x_prereset(struct ata_link *link, unsigned long deadline)
}
/**
- * pdc2720x_mode_filter - mode selection filter
+ * pdc2027x_mode_filter - mode selection filter
* @adev: ATA device
* @mask: list of modes proposed
*
@@ -503,11 +503,11 @@ retry:
}
/**
- * adjust_pll - Adjust the PLL input clock in Hz.
+ * pdc_adjust_pll - Adjust the PLL input clock in Hz.
*
- * @pdc_controller: controller specific information
* @host: target ATA host
* @pll_clock: The input of PLL in HZ
+ * @board_idx: board identifier
*/
static void pdc_adjust_pll(struct ata_host *host, long pll_clock, unsigned int board_idx)
{
@@ -590,7 +590,7 @@ static void pdc_adjust_pll(struct ata_host *host, long pll_clock, unsigned int b
}
/**
- * detect_pll_input_clock - Detect the PLL input clock in Hz.
+ * pdc_detect_pll_input_clock - Detect the PLL input clock in Hz.
* @host: target ATA host
* Ex. 16949000 on 33MHz PCI bus for pdc20275.
* Half of the PCI clock.
diff --git a/drivers/ata/pata_pdc202xx_old.c b/drivers/ata/pata_pdc202xx_old.c
index 378ed9ea97e9..0c5cbcd28d0d 100644
--- a/drivers/ata/pata_pdc202xx_old.c
+++ b/drivers/ata/pata_pdc202xx_old.c
@@ -115,7 +115,7 @@ static void pdc202xx_set_piomode(struct ata_port *ap, struct ata_device *adev)
}
/**
- * pdc202xx_configure_dmamode - set DMA mode in chip
+ * pdc202xx_set_dmamode - set DMA mode in chip
* @ap: ATA interface
* @adev: ATA device
*
@@ -214,7 +214,7 @@ static void pdc2026x_bmdma_start(struct ata_queued_cmd *qc)
}
/**
- * pdc2026x_bmdma_end - DMA engine stop
+ * pdc2026x_bmdma_stop - DMA engine stop
* @qc: ATA command
*
* After a DMA completes we need to put the clock back to 33MHz for
diff --git a/drivers/ata/pata_piccolo.c b/drivers/ata/pata_piccolo.c
index 35cb0e263237..389b63b13c70 100644
--- a/drivers/ata/pata_piccolo.c
+++ b/drivers/ata/pata_piccolo.c
@@ -74,7 +74,7 @@ static struct ata_port_operations tosh_port_ops = {
};
/**
- * ata_tosh_init - attach generic IDE
+ * ata_tosh_init_one - attach generic IDE
* @dev: PCI device found
* @id: match entry
*
diff --git a/drivers/ata/pata_platform.c b/drivers/ata/pata_platform.c
index 5aba691f09af..028329428b75 100644
--- a/drivers/ata/pata_platform.c
+++ b/drivers/ata/pata_platform.c
@@ -24,6 +24,8 @@
#define DRV_VERSION "1.2"
static int pio_mask = 1;
+module_param(pio_mask, int, 0);
+MODULE_PARM_DESC(pio_mask, "PIO modes supported, mode 0 only by default");
/*
* Provide our own set_mode() as we don't want to change anything that has
@@ -233,8 +235,6 @@ static struct platform_driver pata_platform_driver = {
module_platform_driver(pata_platform_driver);
-module_param(pio_mask, int, 0);
-
MODULE_AUTHOR("Paul Mundt");
MODULE_DESCRIPTION("low-level driver for platform device ATA");
MODULE_LICENSE("GPL");
diff --git a/drivers/ata/pata_sil680.c b/drivers/ata/pata_sil680.c
index 7ab9aea3b630..43215a664b96 100644
--- a/drivers/ata/pata_sil680.c
+++ b/drivers/ata/pata_sil680.c
@@ -57,6 +57,7 @@ static unsigned long sil680_selreg(struct ata_port *ap, int r)
/**
* sil680_seldev - return register base
* @ap: ATA interface
+ * @adev: ATA device
* @r: config offset
*
* Turn a config register offset into the right address in PCI space
@@ -244,6 +245,7 @@ static struct ata_port_operations sil680_port_ops = {
/**
* sil680_init_chip - chip setup
* @pdev: PCI device
+ * @try_mmio: Indicates to caller whether MMIO should be attempted
*
* Perform all the chip setup which must be done both when the device
* is powered up on boot and when we resume in case we resumed from RAM.
diff --git a/drivers/ata/pata_sis.c b/drivers/ata/pata_sis.c
index d7cd39a9888a..b5b764e18adf 100644
--- a/drivers/ata/pata_sis.c
+++ b/drivers/ata/pata_sis.c
@@ -114,7 +114,6 @@ static int sis_port_base(struct ata_device *adev)
/**
* sis_133_cable_detect - check for 40/80 pin
* @ap: Port
- * @deadline: deadline jiffies for the operation
*
* Perform cable detection for the later UDMA133 capable
* SiS chipset.
@@ -521,6 +520,7 @@ static void sis_133_set_dmamode (struct ata_port *ap, struct ata_device *adev)
/**
* sis_133_mode_filter - mode selection filter
* @adev: ATA device
+ * @mask: received mask to manipulate and pass back
*
* Block UDMA6 on devices that do not support it.
*/
diff --git a/drivers/ata/pata_sl82c105.c b/drivers/ata/pata_sl82c105.c
index ac7ddd87f188..8487470e2e01 100644
--- a/drivers/ata/pata_sl82c105.c
+++ b/drivers/ata/pata_sl82c105.c
@@ -173,7 +173,7 @@ static void sl82c105_bmdma_start(struct ata_queued_cmd *qc)
}
/**
- * sl82c105_bmdma_end - DMA engine stop
+ * sl82c105_bmdma_stop - DMA engine stop
* @qc: ATA command
*
* Reset the DMA engine each use as recommended by the errata
diff --git a/drivers/ata/pata_triflex.c b/drivers/ata/pata_triflex.c
index 2a4d38b98026..8a033598e7e1 100644
--- a/drivers/ata/pata_triflex.c
+++ b/drivers/ata/pata_triflex.c
@@ -129,7 +129,7 @@ static void triflex_set_piomode(struct ata_port *ap, struct ata_device *adev)
}
/**
- * triflex_dma_start - DMA start callback
+ * triflex_bmdma_start - DMA start callback
* @qc: Command in progress
*
* Usually drivers set the DMA timing at the point the set_dmamode call
@@ -146,9 +146,8 @@ static void triflex_bmdma_start(struct ata_queued_cmd *qc)
}
/**
- * triflex_dma_stop - DMA stop callback
- * @ap: ATA interface
- * @adev: ATA device
+ * triflex_bmdma_stop - DMA stop callback
+ * @qc: ATA command
*
* We loaded new timings in dma_start, as a result we need to restore
* the PIO timings in dma_stop so that the next command issue gets the
diff --git a/drivers/ata/pata_via.c b/drivers/ata/pata_via.c
index 38044e679795..475032048984 100644
--- a/drivers/ata/pata_via.c
+++ b/drivers/ata/pata_via.c
@@ -663,7 +663,7 @@ static int via_init_one(struct pci_dev *pdev, const struct pci_device_id *id)
#ifdef CONFIG_PM_SLEEP
/**
* via_reinit_one - reinit after resume
- * @pdev; PCI device
+ * @pdev: PCI device
*
* Called when the VIA PATA device is resumed. We must then
* reconfigure the fifo and other setup we may have altered. In
diff --git a/drivers/ata/sata_dwc_460ex.c b/drivers/ata/sata_dwc_460ex.c
index 9dcef6ac643b..f0ef844428bb 100644
--- a/drivers/ata/sata_dwc_460ex.c
+++ b/drivers/ata/sata_dwc_460ex.c
@@ -543,6 +543,11 @@ static irqreturn_t sata_dwc_isr(int irq, void *dev_instance)
hsdev->sactive_issued |= qcmd_tag_to_mask(tag);
qc = ata_qc_from_tag(ap, tag);
+ if (unlikely(!qc)) {
+ dev_err(ap->dev, "failed to get qc");
+ handled = 1;
+ goto DONE;
+ }
/*
* Start FP DMA for NCQ command. At this point the tag is the
* active tag. It is the tag that matches the command about to
@@ -658,6 +663,11 @@ DRVSTILLBUSY:
tag_mask &= (~0x00000001);
qc = ata_qc_from_tag(ap, tag);
+ if (unlikely(!qc)) {
+ dev_err(ap->dev, "failed to get qc");
+ handled = 1;
+ goto DONE;
+ }
/* To be picked up by completion functions */
qc->ap->link.active_tag = tag;
diff --git a/drivers/ata/sata_mv.c b/drivers/ata/sata_mv.c
index 664ef658a955..c8867c12c0b8 100644
--- a/drivers/ata/sata_mv.c
+++ b/drivers/ata/sata_mv.c
@@ -1146,9 +1146,8 @@ static void mv_set_irq_coalescing(struct ata_host *host,
spin_unlock_irqrestore(&host->lock, flags);
}
-/**
+/*
* mv_start_edma - Enable eDMA engine
- * @base: port base address
* @pp: port private data
*
* Verify the local cache of the eDMA state is accurate with a
@@ -1519,7 +1518,7 @@ static void mv_60x1_errata_sata25(struct ata_port *ap, int want_ncq)
writel(new, hpriv->base + GPIO_PORT_CTL);
}
-/**
+/*
* mv_bmdma_enable - set a magic bit on GEN_IIE to allow bmdma
* @ap: Port being initialized
*
@@ -1918,8 +1917,8 @@ static void mv_bmdma_start(struct ata_queued_cmd *qc)
}
/**
- * mv_bmdma_stop - Stop BMDMA transfer
- * @qc: queued command to stop DMA on.
+ * mv_bmdma_stop_ap - Stop BMDMA transfer
+ * @ap: port to stop
*
* Clears the ATA_DMA_START flag in the bmdma control register
*
@@ -2221,6 +2220,7 @@ static u8 mv_sff_check_status(struct ata_port *ap)
/**
* mv_send_fis - Send a FIS, using the "Vendor-Unique FIS" register
+ * @ap: ATA port to send a FIS
* @fis: fis to be sent
* @nwords: number of 32-bit words in the fis
*/
@@ -3249,7 +3249,7 @@ static void mv6_reset_flash(struct mv_host_priv *hpriv, void __iomem *mmio)
writel(tmp, mmio + GPIO_PORT_CTL);
}
-/**
+/*
* mv6_reset_hc - Perform the 6xxx global soft reset
* @mmio: base address of the HBA
*
@@ -3530,7 +3530,7 @@ static void mv_soc_65n_phy_errata(struct mv_host_priv *hpriv,
writel(reg, port_mmio + PHY_MODE9_GEN1);
}
-/**
+/*
* soc_is_65 - check if the soc is 65 nano device
*
* Detect the type of the SoC, this is done by reading the PHYCFG_OFS
@@ -4097,6 +4097,10 @@ static int mv_platform_probe(struct platform_device *pdev)
n_ports = mv_platform_data->n_ports;
irq = platform_get_irq(pdev, 0);
}
+ if (irq < 0)
+ return irq;
+ if (!irq)
+ return -EINVAL;
host = ata_host_alloc_pinfo(&pdev->dev, ppi, n_ports);
hpriv = devm_kzalloc(&pdev->dev, sizeof(*hpriv), GFP_KERNEL);
diff --git a/drivers/atm/eni.c b/drivers/atm/eni.c
index 316a9947541f..422753d52244 100644
--- a/drivers/atm/eni.c
+++ b/drivers/atm/eni.c
@@ -2054,7 +2054,7 @@ static int eni_send(struct atm_vcc *vcc,struct sk_buff *skb)
}
submitted++;
ATM_SKB(skb)->vcc = vcc;
- tasklet_disable(&ENI_DEV(vcc->dev)->task);
+ tasklet_disable_in_atomic(&ENI_DEV(vcc->dev)->task);
res = do_tx(skb);
tasklet_enable(&ENI_DEV(vcc->dev)->task);
if (res == enq_ok) return 0;
@@ -2260,7 +2260,8 @@ out:
return rc;
err_eni_release:
- eni_do_release(dev);
+ dev->phy = NULL;
+ iounmap(ENI_DEV(dev)->ioaddr);
err_unregister:
atm_dev_deregister(dev);
err_free_consistent:
diff --git a/drivers/atm/firestream.c b/drivers/atm/firestream.c
index 0ddd611b4277..3bc3c314a467 100644
--- a/drivers/atm/firestream.c
+++ b/drivers/atm/firestream.c
@@ -795,6 +795,7 @@ static void process_incoming (struct fs_dev *dev, struct queue *q)
switch (STATUS_CODE (qe)) {
case 0x1:
/* Fall through for streaming mode */
+ fallthrough;
case 0x2:/* Packet received OK.... */
if (atm_vcc) {
skb = pe->skb;
diff --git a/drivers/atm/fore200e.c b/drivers/atm/fore200e.c
index 9a70bee84125..b508df2ecada 100644
--- a/drivers/atm/fore200e.c
+++ b/drivers/atm/fore200e.c
@@ -21,7 +21,6 @@
#include <linux/module.h>
#include <linux/atmdev.h>
#include <linux/sonet.h>
-#include <linux/atm_suni.h>
#include <linux/dma-mapping.h>
#include <linux/delay.h>
#include <linux/firmware.h>
@@ -100,8 +99,6 @@ static LIST_HEAD(fore200e_boards);
MODULE_AUTHOR("Christophe Lizzi - credits to Uwe Dannowski and Heikki Vatiainen");
MODULE_DESCRIPTION("FORE Systems 200E-series ATM driver - version " FORE200E_VERSION);
-MODULE_SUPPORTED_DEVICE("PCA-200E, SBA-200E");
-
static const int fore200e_rx_buf_nbr[ BUFFER_SCHEME_NBR ][ BUFFER_MAGN_NBR ] = {
{ BUFFER_S1_NBR, BUFFER_L1_NBR },
diff --git a/drivers/atm/idt77105.c b/drivers/atm/idt77105.c
index 3c081b6171a8..bfca7b8a6f31 100644
--- a/drivers/atm/idt77105.c
+++ b/drivers/atm/idt77105.c
@@ -262,7 +262,7 @@ static int idt77105_start(struct atm_dev *dev)
{
unsigned long flags;
- if (!(dev->dev_data = kmalloc(sizeof(struct idt77105_priv),GFP_KERNEL)))
+ if (!(dev->phy_data = kmalloc(sizeof(struct idt77105_priv),GFP_KERNEL)))
return -ENOMEM;
PRIV(dev)->dev = dev;
spin_lock_irqsave(&idt77105_priv_lock, flags);
@@ -337,7 +337,7 @@ static int idt77105_stop(struct atm_dev *dev)
else
idt77105_all = walk->next;
dev->phy = NULL;
- dev->dev_data = NULL;
+ dev->phy_data = NULL;
kfree(walk);
break;
}
diff --git a/drivers/atm/idt77252.c b/drivers/atm/idt77252.c
index 0c13cac903de..9e4bd751db79 100644
--- a/drivers/atm/idt77252.c
+++ b/drivers/atm/idt77252.c
@@ -1784,12 +1784,6 @@ set_tct(struct idt77252_dev *card, struct vc_map *vc)
/*****************************************************************************/
static __inline__ int
-idt77252_fbq_level(struct idt77252_dev *card, int queue)
-{
- return (readl(SAR_REG_STAT) >> (16 + (queue << 2))) & 0x0f;
-}
-
-static __inline__ int
idt77252_fbq_full(struct idt77252_dev *card, int queue)
{
return (readl(SAR_REG_STAT) >> (16 + (queue << 2))) == 0x0f;
diff --git a/drivers/atm/iphase.c b/drivers/atm/iphase.c
index eef637fd90b3..933e3ff2ee8d 100644
--- a/drivers/atm/iphase.c
+++ b/drivers/atm/iphase.c
@@ -680,7 +680,7 @@ static void ia_tx_poll (IADEV *iadev) {
skb1 = skb_dequeue(&iavcc->txing_skb);
}
if (!skb1) {
- IF_EVENT(printk("IA: Vci %d - skb not found requed\n",vcc->vci);)
+ IF_EVENT(printk("IA: Vci %d - skb not found requeued\n",vcc->vci);)
ia_enque_head_rtn_q (&iadev->tx_return_q, rtne);
break;
}
diff --git a/drivers/atm/lanai.c b/drivers/atm/lanai.c
index d7277c26e423..32d7aa141d96 100644
--- a/drivers/atm/lanai.c
+++ b/drivers/atm/lanai.c
@@ -2233,6 +2233,7 @@ static int lanai_dev_open(struct atm_dev *atmdev)
conf1_write(lanai);
#endif
iounmap(lanai->base);
+ lanai->base = NULL;
error_pci:
pci_disable_device(lanai->pci);
error:
@@ -2245,6 +2246,8 @@ static int lanai_dev_open(struct atm_dev *atmdev)
static void lanai_dev_close(struct atm_dev *atmdev)
{
struct lanai_dev *lanai = (struct lanai_dev *) atmdev->dev_data;
+ if (lanai->base==NULL)
+ return;
printk(KERN_INFO DEV_LABEL "(itf %d): shutting down interface\n",
lanai->number);
lanai_timed_poll_stop(lanai);
@@ -2552,7 +2555,7 @@ static int lanai_init_one(struct pci_dev *pci,
struct atm_dev *atmdev;
int result;
- lanai = kmalloc(sizeof(*lanai), GFP_KERNEL);
+ lanai = kzalloc(sizeof(*lanai), GFP_KERNEL);
if (lanai == NULL) {
printk(KERN_ERR DEV_LABEL
": couldn't allocate dev_data structure!\n");
diff --git a/drivers/atm/suni.c b/drivers/atm/suni.c
index c920a8c52925..21e5acc766b8 100644
--- a/drivers/atm/suni.c
+++ b/drivers/atm/suni.c
@@ -21,7 +21,6 @@
#include <linux/timer.h>
#include <linux/init.h>
#include <linux/capability.h>
-#include <linux/atm_suni.h>
#include <linux/slab.h>
#include <asm/param.h>
#include <linux/uaccess.h>
diff --git a/drivers/atm/uPD98402.c b/drivers/atm/uPD98402.c
index 7850758b5bb8..239852d85558 100644
--- a/drivers/atm/uPD98402.c
+++ b/drivers/atm/uPD98402.c
@@ -211,7 +211,7 @@ static void uPD98402_int(struct atm_dev *dev)
static int uPD98402_start(struct atm_dev *dev)
{
DPRINTK("phy_start\n");
- if (!(dev->dev_data = kmalloc(sizeof(struct uPD98402_priv),GFP_KERNEL)))
+ if (!(dev->phy_data = kmalloc(sizeof(struct uPD98402_priv),GFP_KERNEL)))
return -ENOMEM;
spin_lock_init(&PRIV(dev)->lock);
memset(&PRIV(dev)->sonet_stats,0,sizeof(struct k_sonet_stats));
diff --git a/drivers/auxdisplay/charlcd.c b/drivers/auxdisplay/charlcd.c
index f43430e9dcee..24fd6f369ebe 100644
--- a/drivers/auxdisplay/charlcd.c
+++ b/drivers/auxdisplay/charlcd.c
@@ -470,12 +470,14 @@ static ssize_t charlcd_write(struct file *file, const char __user *buf,
char c;
for (; count-- > 0; (*ppos)++, tmp++) {
- if (!in_interrupt() && (((count + 1) & 0x1f) == 0))
+ if (((count + 1) & 0x1f) == 0) {
/*
- * let's be a little nice with other processes
- * that need some CPU
+ * charlcd_write() is invoked as a VFS->write() callback
+ * and as such it is always invoked from preemptible
+ * context and may sleep.
*/
- schedule();
+ cond_resched();
+ }
if (get_user(c, tmp))
return -EFAULT;
@@ -537,12 +539,8 @@ static void charlcd_puts(struct charlcd *lcd, const char *s)
int count = strlen(s);
for (; count-- > 0; tmp++) {
- if (!in_interrupt() && (((count + 1) & 0x1f) == 0))
- /*
- * let's be a little nice with other processes
- * that need some CPU
- */
- schedule();
+ if (((count + 1) & 0x1f) == 0)
+ cond_resched();
charlcd_write_char(lcd, *tmp);
}
diff --git a/drivers/auxdisplay/panel.c b/drivers/auxdisplay/panel.c
index ff5755ee5694..eba04c0de7eb 100644
--- a/drivers/auxdisplay/panel.c
+++ b/drivers/auxdisplay/panel.c
@@ -1737,10 +1737,3 @@ module_init(panel_init_module);
module_exit(panel_cleanup_module);
MODULE_AUTHOR("Willy Tarreau");
MODULE_LICENSE("GPL");
-
-/*
- * Local variables:
- * c-indent-level: 4
- * tab-width: 8
- * End:
- */
diff --git a/drivers/base/arch_topology.c b/drivers/base/arch_topology.c
index de8587cc119e..c1179edc0f3b 100644
--- a/drivers/base/arch_topology.c
+++ b/drivers/base/arch_topology.c
@@ -21,17 +21,94 @@
#include <linux/sched.h>
#include <linux/smp.h>
+static DEFINE_PER_CPU(struct scale_freq_data *, sft_data);
+static struct cpumask scale_freq_counters_mask;
+static bool scale_freq_invariant;
+
+static bool supports_scale_freq_counters(const struct cpumask *cpus)
+{
+ return cpumask_subset(cpus, &scale_freq_counters_mask);
+}
+
bool topology_scale_freq_invariant(void)
{
return cpufreq_supports_freq_invariance() ||
- arch_freq_counters_available(cpu_online_mask);
+ supports_scale_freq_counters(cpu_online_mask);
}
-__weak bool arch_freq_counters_available(const struct cpumask *cpus)
+static void update_scale_freq_invariant(bool status)
{
- return false;
+ if (scale_freq_invariant == status)
+ return;
+
+ /*
+ * Task scheduler behavior depends on frequency invariance support,
+ * either cpufreq or counter driven. If the support status changes as
+ * a result of counter initialisation and use, retrigger the build of
+ * scheduling domains to ensure the information is propagated properly.
+ */
+ if (topology_scale_freq_invariant() == status) {
+ scale_freq_invariant = status;
+ rebuild_sched_domains_energy();
+ }
}
-DEFINE_PER_CPU(unsigned long, freq_scale) = SCHED_CAPACITY_SCALE;
+
+void topology_set_scale_freq_source(struct scale_freq_data *data,
+ const struct cpumask *cpus)
+{
+ struct scale_freq_data *sfd;
+ int cpu;
+
+ /*
+ * Avoid calling rebuild_sched_domains() unnecessarily if FIE is
+ * supported by cpufreq.
+ */
+ if (cpumask_empty(&scale_freq_counters_mask))
+ scale_freq_invariant = topology_scale_freq_invariant();
+
+ for_each_cpu(cpu, cpus) {
+ sfd = per_cpu(sft_data, cpu);
+
+ /* Use ARCH provided counters whenever possible */
+ if (!sfd || sfd->source != SCALE_FREQ_SOURCE_ARCH) {
+ per_cpu(sft_data, cpu) = data;
+ cpumask_set_cpu(cpu, &scale_freq_counters_mask);
+ }
+ }
+
+ update_scale_freq_invariant(true);
+}
+EXPORT_SYMBOL_GPL(topology_set_scale_freq_source);
+
+void topology_clear_scale_freq_source(enum scale_freq_source source,
+ const struct cpumask *cpus)
+{
+ struct scale_freq_data *sfd;
+ int cpu;
+
+ for_each_cpu(cpu, cpus) {
+ sfd = per_cpu(sft_data, cpu);
+
+ if (sfd && sfd->source == source) {
+ per_cpu(sft_data, cpu) = NULL;
+ cpumask_clear_cpu(cpu, &scale_freq_counters_mask);
+ }
+ }
+
+ update_scale_freq_invariant(false);
+}
+EXPORT_SYMBOL_GPL(topology_clear_scale_freq_source);
+
+void topology_scale_freq_tick(void)
+{
+ struct scale_freq_data *sfd = *this_cpu_ptr(&sft_data);
+
+ if (sfd)
+ sfd->set_freq_scale();
+}
+
+DEFINE_PER_CPU(unsigned long, arch_freq_scale) = SCHED_CAPACITY_SCALE;
+EXPORT_PER_CPU_SYMBOL_GPL(arch_freq_scale);
void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
unsigned long max_freq)
@@ -47,13 +124,13 @@ void topology_set_freq_scale(const struct cpumask *cpus, unsigned long cur_freq,
* want to update the scale factor with information from CPUFREQ.
* Instead the scale factor will be updated from arch_scale_freq_tick.
*/
- if (arch_freq_counters_available(cpus))
+ if (supports_scale_freq_counters(cpus))
return;
scale = (cur_freq << SCHED_CAPACITY_SHIFT) / max_freq;
for_each_cpu(i, cpus)
- per_cpu(freq_scale, i) = scale;
+ per_cpu(arch_freq_scale, i) = scale;
}
DEFINE_PER_CPU(unsigned long, cpu_scale) = SCHED_CAPACITY_SCALE;
diff --git a/drivers/base/attribute_container.c b/drivers/base/attribute_container.c
index f7bd0f4db13d..9c00d203d61e 100644
--- a/drivers/base/attribute_container.c
+++ b/drivers/base/attribute_container.c
@@ -461,6 +461,10 @@ attribute_container_add_class_device(struct device *classdev)
/**
* attribute_container_add_class_device_adapter - simple adapter for triggers
*
+ * @cont: the container to register.
+ * @dev: the generic device to activate the trigger for
+ * @classdev: the class device to add
+ *
* This function is identical to attribute_container_add_class_device except
* that it is designed to be called from the triggers
*/
diff --git a/drivers/base/auxiliary.c b/drivers/base/auxiliary.c
index d8b314e7d0fd..adc199dfba3c 100644
--- a/drivers/base/auxiliary.c
+++ b/drivers/base/auxiliary.c
@@ -265,8 +265,3 @@ void __init auxiliary_bus_init(void)
{
WARN_ON(bus_register(&auxiliary_bus_type));
}
-
-MODULE_LICENSE("GPL v2");
-MODULE_DESCRIPTION("Auxiliary Bus");
-MODULE_AUTHOR("David Ertman <david.m.ertman@intel.com>");
-MODULE_AUTHOR("Kiran Patil <kiran.patil@intel.com>");
diff --git a/drivers/base/base.h b/drivers/base/base.h
index 52b3d7b75c27..e5f9b7e656c3 100644
--- a/drivers/base/base.h
+++ b/drivers/base/base.h
@@ -185,11 +185,13 @@ extern int device_links_read_lock(void);
extern void device_links_read_unlock(int idx);
extern int device_links_read_lock_held(void);
extern int device_links_check_suppliers(struct device *dev);
+extern void device_links_force_bind(struct device *dev);
extern void device_links_driver_bound(struct device *dev);
extern void device_links_driver_cleanup(struct device *dev);
extern void device_links_no_driver(struct device *dev);
extern bool device_links_busy(struct device *dev);
extern void device_links_unbind_consumers(struct device *dev);
+extern void fw_devlink_drivers_done(void);
/* device pm support */
void device_pm_move_to_tail(struct device *dev);
diff --git a/drivers/base/component.c b/drivers/base/component.c
index dcfbe7251dc4..272ba42392f0 100644
--- a/drivers/base/component.c
+++ b/drivers/base/component.c
@@ -65,7 +65,6 @@ struct master {
const struct component_master_ops *ops;
struct device *dev;
struct component_match *match;
- struct dentry *dentry;
};
struct component {
@@ -125,15 +124,13 @@ core_initcall(component_debug_init);
static void component_master_debugfs_add(struct master *m)
{
- m->dentry = debugfs_create_file(dev_name(m->dev), 0444,
- component_debugfs_dir,
- m, &component_devices_fops);
+ debugfs_create_file(dev_name(m->dev), 0444, component_debugfs_dir, m,
+ &component_devices_fops);
}
static void component_master_debugfs_del(struct master *m)
{
- debugfs_remove(m->dentry);
- m->dentry = NULL;
+ debugfs_remove(debugfs_lookup(dev_name(m->dev), component_debugfs_dir));
}
#else
diff --git a/drivers/base/core.c b/drivers/base/core.c
index f29839382f81..628e33939aca 100644
--- a/drivers/base/core.c
+++ b/drivers/base/core.c
@@ -51,6 +51,7 @@ static LIST_HEAD(deferred_sync);
static unsigned int defer_sync_state_count = 1;
static DEFINE_MUTEX(fwnode_link_lock);
static bool fw_devlink_is_permissive(void);
+static bool fw_devlink_drv_reg_done;
/**
* fwnode_link_add - Create a link between two fwnode_handles.
@@ -149,7 +150,7 @@ void fwnode_links_purge(struct fwnode_handle *fwnode)
fwnode_links_purge_consumers(fwnode);
}
-static void fw_devlink_purge_absent_suppliers(struct fwnode_handle *fwnode)
+void fw_devlink_purge_absent_suppliers(struct fwnode_handle *fwnode)
{
struct fwnode_handle *child;
@@ -163,6 +164,7 @@ static void fw_devlink_purge_absent_suppliers(struct fwnode_handle *fwnode)
fwnode_for_each_available_child_node(fwnode, child)
fw_devlink_purge_absent_suppliers(child);
}
+EXPORT_SYMBOL_GPL(fw_devlink_purge_absent_suppliers);
#ifdef CONFIG_SRCU
static DEFINE_MUTEX(device_links_lock);
@@ -1154,6 +1156,41 @@ static ssize_t waiting_for_supplier_show(struct device *dev,
static DEVICE_ATTR_RO(waiting_for_supplier);
/**
+ * device_links_force_bind - Prepares device to be force bound
+ * @dev: Consumer device.
+ *
+ * device_bind_driver() force binds a device to a driver without calling any
+ * driver probe functions. So the consumer really isn't going to wait for any
+ * supplier before it's bound to the driver. We still want the device link
+ * states to be sensible when this happens.
+ *
+ * In preparation for device_bind_driver(), this function goes through each
+ * supplier device links and checks if the supplier is bound. If it is, then
+ * the device link status is set to CONSUMER_PROBE. Otherwise, the device link
+ * is dropped. Links without the DL_FLAG_MANAGED flag set are ignored.
+ */
+void device_links_force_bind(struct device *dev)
+{
+ struct device_link *link, *ln;
+
+ device_links_write_lock();
+
+ list_for_each_entry_safe(link, ln, &dev->links.suppliers, c_node) {
+ if (!(link->flags & DL_FLAG_MANAGED))
+ continue;
+
+ if (link->status != DL_STATE_AVAILABLE) {
+ device_link_drop_managed(link);
+ continue;
+ }
+ WRITE_ONCE(link->status, DL_STATE_CONSUMER_PROBE);
+ }
+ dev->links.status = DL_DEV_PROBING;
+
+ device_links_write_unlock();
+}
+
+/**
* device_links_driver_bound - Update device links after probing its driver.
* @dev: Device to update the links for.
*
@@ -1503,7 +1540,7 @@ static void device_links_purge(struct device *dev)
#define FW_DEVLINK_FLAGS_RPM (FW_DEVLINK_FLAGS_ON | \
DL_FLAG_PM_RUNTIME)
-static u32 fw_devlink_flags = FW_DEVLINK_FLAGS_PERMISSIVE;
+static u32 fw_devlink_flags = FW_DEVLINK_FLAGS_ON;
static int __init fw_devlink_setup(char *arg)
{
if (!arg)
@@ -1563,6 +1600,52 @@ static void fw_devlink_parse_fwtree(struct fwnode_handle *fwnode)
fw_devlink_parse_fwtree(child);
}
+static void fw_devlink_relax_link(struct device_link *link)
+{
+ if (!(link->flags & DL_FLAG_INFERRED))
+ return;
+
+ if (link->flags == (DL_FLAG_MANAGED | FW_DEVLINK_FLAGS_PERMISSIVE))
+ return;
+
+ pm_runtime_drop_link(link);
+ link->flags = DL_FLAG_MANAGED | FW_DEVLINK_FLAGS_PERMISSIVE;
+ dev_dbg(link->consumer, "Relaxing link with %s\n",
+ dev_name(link->supplier));
+}
+
+static int fw_devlink_no_driver(struct device *dev, void *data)
+{
+ struct device_link *link = to_devlink(dev);
+
+ if (!link->supplier->can_match)
+ fw_devlink_relax_link(link);
+
+ return 0;
+}
+
+void fw_devlink_drivers_done(void)
+{
+ fw_devlink_drv_reg_done = true;
+ device_links_write_lock();
+ class_for_each_device(&devlink_class, NULL, NULL,
+ fw_devlink_no_driver);
+ device_links_write_unlock();
+}
+
+static void fw_devlink_unblock_consumers(struct device *dev)
+{
+ struct device_link *link;
+
+ if (!fw_devlink_flags || fw_devlink_is_permissive())
+ return;
+
+ device_links_write_lock();
+ list_for_each_entry(link, &dev->links.consumers, s_node)
+ fw_devlink_relax_link(link);
+ device_links_write_unlock();
+}
+
/**
* fw_devlink_relax_cycle - Convert cyclic links to SYNC_STATE_ONLY links
* @con: Device to check dependencies for.
@@ -1599,21 +1682,16 @@ static int fw_devlink_relax_cycle(struct device *con, void *sup)
ret = 1;
- if (!(link->flags & DL_FLAG_INFERRED))
- continue;
-
- pm_runtime_drop_link(link);
- link->flags = DL_FLAG_MANAGED | FW_DEVLINK_FLAGS_PERMISSIVE;
- dev_dbg(link->consumer, "Relaxing link with %s\n",
- dev_name(link->supplier));
+ fw_devlink_relax_link(link);
}
return ret;
}
/**
* fw_devlink_create_devlink - Create a device link from a consumer to fwnode
- * @con - Consumer device for the device link
- * @sup_handle - fwnode handle of supplier
+ * @con: consumer device for the device link
+ * @sup_handle: fwnode handle of supplier
+ * @flags: devlink flags
*
* This function will try to create a device link between the consumer device
* @con and the supplier device represented by @sup_handle.
@@ -1709,7 +1787,7 @@ out:
/**
* __fw_devlink_link_to_consumers - Create device links to consumers of a device
- * @dev - Device that needs to be linked to its consumers
+ * @dev: Device that needs to be linked to its consumers
*
* This function looks at all the consumer fwnodes of @dev and creates device
* links between the consumer device and @dev (supplier).
@@ -1779,8 +1857,8 @@ static void __fw_devlink_link_to_consumers(struct device *dev)
/**
* __fw_devlink_link_to_suppliers - Create device links to suppliers of a device
- * @dev - The consumer device that needs to be linked to its suppliers
- * @fwnode - Root of the fwnode tree that is used to create device links
+ * @dev: The consumer device that needs to be linked to its suppliers
+ * @fwnode: Root of the fwnode tree that is used to create device links
*
* This function looks at all the supplier fwnodes of fwnode tree rooted at
* @fwnode and creates device links between @dev (consumer) and all the
@@ -3240,6 +3318,15 @@ int device_add(struct device *dev)
}
bus_probe_device(dev);
+
+ /*
+ * If all driver registration is done and a newly added device doesn't
+ * match with any driver, don't block its consumers from probing in
+ * case the consumer device is able to operate without this supplier.
+ */
+ if (dev->fwnode && fw_devlink_drv_reg_done && !dev->can_match)
+ fw_devlink_unblock_consumers(dev);
+
if (parent)
klist_add_tail(&dev->p->knode_parent,
&parent->p->klist_children);
diff --git a/drivers/base/cpu.c b/drivers/base/cpu.c
index 8f1d6569564c..2b9e41377a07 100644
--- a/drivers/base/cpu.c
+++ b/drivers/base/cpu.c
@@ -409,13 +409,11 @@ __cpu_device_create(struct device *parent, void *drvdata,
const char *fmt, va_list args)
{
struct device *dev = NULL;
- int retval = -ENODEV;
+ int retval = -ENOMEM;
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
- if (!dev) {
- retval = -ENOMEM;
+ if (!dev)
goto error;
- }
device_initialize(dev);
dev->parent = parent;
diff --git a/drivers/base/dd.c b/drivers/base/dd.c
index 9179825ff646..ecd7cf848daf 100644
--- a/drivers/base/dd.c
+++ b/drivers/base/dd.c
@@ -55,7 +55,6 @@ static DEFINE_MUTEX(deferred_probe_mutex);
static LIST_HEAD(deferred_probe_pending_list);
static LIST_HEAD(deferred_probe_active_list);
static atomic_t deferred_trigger_count = ATOMIC_INIT(0);
-static struct dentry *deferred_devices;
static bool initcalls_done;
/* Save the async probe drivers' name from kernel cmdline */
@@ -69,6 +68,12 @@ static char async_probe_drv_names[ASYNC_DRV_NAMES_MAX_LEN];
*/
static bool defer_all_probes;
+static void __device_set_deferred_probe_reason(const struct device *dev, char *reason)
+{
+ kfree(dev->p->deferred_probe_reason);
+ dev->p->deferred_probe_reason = reason;
+}
+
/*
* deferred_probe_work_func() - Retry probing devices in the active list.
*/
@@ -97,6 +102,8 @@ static void deferred_probe_work_func(struct work_struct *work)
get_device(dev);
+ __device_set_deferred_probe_reason(dev, NULL);
+
/*
* Drop the mutex while probing each device; the probe path may
* manipulate the deferred list
@@ -123,6 +130,9 @@ static DECLARE_WORK(deferred_probe_work, deferred_probe_work_func);
void driver_deferred_probe_add(struct device *dev)
{
+ if (!dev->can_match)
+ return;
+
mutex_lock(&deferred_probe_mutex);
if (list_empty(&dev->p->deferred_probe)) {
dev_dbg(dev, "Added to deferred list\n");
@@ -137,8 +147,7 @@ void driver_deferred_probe_del(struct device *dev)
if (!list_empty(&dev->p->deferred_probe)) {
dev_dbg(dev, "Removed from deferred list\n");
list_del_init(&dev->p->deferred_probe);
- kfree(dev->p->deferred_probe_reason);
- dev->p->deferred_probe_reason = NULL;
+ __device_set_deferred_probe_reason(dev, NULL);
}
mutex_unlock(&deferred_probe_mutex);
}
@@ -182,7 +191,7 @@ static void driver_deferred_probe_trigger(void)
* Kick the re-probe thread. It may already be scheduled, but it is
* safe to kick it again.
*/
- schedule_work(&deferred_probe_work);
+ queue_work(system_unbound_wq, &deferred_probe_work);
}
/**
@@ -217,11 +226,12 @@ void device_unblock_probing(void)
void device_set_deferred_probe_reason(const struct device *dev, struct va_format *vaf)
{
const char *drv = dev_driver_string(dev);
+ char *reason;
mutex_lock(&deferred_probe_mutex);
- kfree(dev->p->deferred_probe_reason);
- dev->p->deferred_probe_reason = kasprintf(GFP_KERNEL, "%s: %pV", drv, vaf);
+ reason = kasprintf(GFP_KERNEL, "%s: %pV", drv, vaf);
+ __device_set_deferred_probe_reason(dev, reason);
mutex_unlock(&deferred_probe_mutex);
}
@@ -289,14 +299,18 @@ int driver_deferred_probe_check_state(struct device *dev)
static void deferred_probe_timeout_work_func(struct work_struct *work)
{
- struct device_private *private, *p;
+ struct device_private *p;
+
+ fw_devlink_drivers_done();
driver_deferred_probe_timeout = 0;
driver_deferred_probe_trigger();
flush_work(&deferred_probe_work);
- list_for_each_entry_safe(private, p, &deferred_probe_pending_list, deferred_probe)
- dev_info(private->device, "deferred probe pending\n");
+ mutex_lock(&deferred_probe_mutex);
+ list_for_each_entry(p, &deferred_probe_pending_list, deferred_probe)
+ dev_info(p->device, "deferred probe pending\n");
+ mutex_unlock(&deferred_probe_mutex);
wake_up_all(&probe_timeout_waitqueue);
}
static DECLARE_DELAYED_WORK(deferred_probe_timeout_work, deferred_probe_timeout_work_func);
@@ -310,8 +324,8 @@ static DECLARE_DELAYED_WORK(deferred_probe_timeout_work, deferred_probe_timeout_
*/
static int deferred_probe_initcall(void)
{
- deferred_devices = debugfs_create_file("devices_deferred", 0444, NULL,
- NULL, &deferred_devs_fops);
+ debugfs_create_file("devices_deferred", 0444, NULL, NULL,
+ &deferred_devs_fops);
driver_deferred_probe_enable = true;
driver_deferred_probe_trigger();
@@ -319,6 +333,9 @@ static int deferred_probe_initcall(void)
flush_work(&deferred_probe_work);
initcalls_done = true;
+ if (!IS_ENABLED(CONFIG_MODULES))
+ fw_devlink_drivers_done();
+
/*
* Trigger deferred probe again, this time we won't defer anything
* that is optional
@@ -336,7 +353,7 @@ late_initcall(deferred_probe_initcall);
static void __exit deferred_probe_exit(void)
{
- debugfs_remove_recursive(deferred_devices);
+ debugfs_remove_recursive(debugfs_lookup("devices_deferred", NULL));
}
__exitcall(deferred_probe_exit);
@@ -413,8 +430,11 @@ static int driver_sysfs_add(struct device *dev)
if (ret)
goto rm_dev;
- if (!IS_ENABLED(CONFIG_DEV_COREDUMP) || !dev->driver->coredump ||
- !device_create_file(dev, &dev_attr_coredump))
+ if (!IS_ENABLED(CONFIG_DEV_COREDUMP) || !dev->driver->coredump)
+ return 0;
+
+ ret = device_create_file(dev, &dev_attr_coredump);
+ if (!ret)
return 0;
sysfs_remove_link(&dev->kobj, "driver");
@@ -457,8 +477,10 @@ int device_bind_driver(struct device *dev)
int ret;
ret = driver_sysfs_add(dev);
- if (!ret)
+ if (!ret) {
+ device_links_force_bind(dev);
driver_bound(dev);
+ }
else if (dev->bus)
blocking_notifier_call_chain(&dev->bus->p->bus_notifier,
BUS_NOTIFY_DRIVER_NOT_BOUND, dev);
@@ -726,6 +748,7 @@ static int driver_probe_device(struct device_driver *drv, struct device *dev)
if (!device_is_registered(dev))
return -ENODEV;
+ dev->can_match = true;
pr_debug("bus: '%s': %s: matched device %s with driver %s\n",
drv->bus->name, __func__, dev_name(dev), drv->name);
@@ -829,6 +852,7 @@ static int __device_attach_driver(struct device_driver *drv, void *_data)
return 0;
} else if (ret == -EPROBE_DEFER) {
dev_dbg(dev, "Device match requests probe deferral\n");
+ dev->can_match = true;
driver_deferred_probe_add(dev);
} else if (ret < 0) {
dev_dbg(dev, "Bus failed to match device: %d\n", ret);
@@ -1064,6 +1088,7 @@ static int __driver_attach(struct device *dev, void *data)
return 0;
} else if (ret == -EPROBE_DEFER) {
dev_dbg(dev, "Device match requests probe deferral\n");
+ dev->can_match = true;
driver_deferred_probe_add(dev);
} else if (ret < 0) {
dev_dbg(dev, "Bus failed to match device: %d\n", ret);
diff --git a/drivers/base/devcoredump.c b/drivers/base/devcoredump.c
index 9243468e2c99..8eec0e0ddff7 100644
--- a/drivers/base/devcoredump.c
+++ b/drivers/base/devcoredump.c
@@ -202,7 +202,7 @@ static int devcd_match_failing(struct device *dev, const void *failing)
* NOTE: if two tables allocated with devcd_alloc_sgtable and then chained
* using the sg_chain function then that function should be called only once
* on the chained table
- * @table: pointer to sg_table to free
+ * @data: pointer to sg_table to free
*/
static void devcd_free_sgtable(void *data)
{
@@ -210,7 +210,7 @@ static void devcd_free_sgtable(void *data)
}
/**
- * devcd_read_from_table - copy data from sg_table to a given buffer
+ * devcd_read_from_sgtable - copy data from sg_table to a given buffer
* and return the number of bytes read
* @buffer: the buffer to copy the data to it
* @buf_len: the length of the buffer
@@ -292,13 +292,16 @@ void dev_coredumpm(struct device *dev, struct module *owner,
if (device_add(&devcd->devcd_dev))
goto put_device;
+ /*
+ * These should normally not fail, but there is no problem
+ * continuing without the links, so just warn instead of
+ * failing.
+ */
if (sysfs_create_link(&devcd->devcd_dev.kobj, &dev->kobj,
- "failing_device"))
- /* nothing - symlink will be missing */;
-
- if (sysfs_create_link(&dev->kobj, &devcd->devcd_dev.kobj,
- "devcoredump"))
- /* nothing - symlink will be missing */;
+ "failing_device") ||
+ sysfs_create_link(&dev->kobj, &devcd->devcd_dev.kobj,
+ "devcoredump"))
+ dev_warn(dev, "devcoredump create_link failed\n");
INIT_DELAYED_WORK(&devcd->del_wk, devcd_del);
schedule_delayed_work(&devcd->del_wk, DEVCD_TIMEOUT);
diff --git a/drivers/base/devres.c b/drivers/base/devres.c
index fb9d5289a620..8746f2212781 100644
--- a/drivers/base/devres.c
+++ b/drivers/base/devres.c
@@ -58,8 +58,8 @@ static void devres_log(struct device *dev, struct devres_node *node,
const char *op)
{
if (unlikely(log_devres))
- dev_err(dev, "DEVRES %3s %p %s (%lu bytes)\n",
- op, node, node->name, (unsigned long)node->size);
+ dev_err(dev, "DEVRES %3s %p %s (%zu bytes)\n",
+ op, node, node->name, node->size);
}
#else /* CONFIG_DEBUG_DEVRES */
#define set_node_dbginfo(node, n, s) do {} while (0)
@@ -1228,6 +1228,6 @@ EXPORT_SYMBOL_GPL(__devm_alloc_percpu);
void devm_free_percpu(struct device *dev, void __percpu *pdata)
{
WARN_ON(devres_destroy(dev, devm_percpu_release, devm_percpu_match,
- (void *)pdata));
+ (__force void *)pdata));
}
EXPORT_SYMBOL_GPL(devm_free_percpu);
diff --git a/drivers/base/devtmpfs.c b/drivers/base/devtmpfs.c
index 653c8c6ac7a7..8be352ab4ddb 100644
--- a/drivers/base/devtmpfs.c
+++ b/drivers/base/devtmpfs.c
@@ -371,7 +371,7 @@ int __init devtmpfs_mount(void)
return err;
}
-static DECLARE_COMPLETION(setup_done);
+static __initdata DECLARE_COMPLETION(setup_done);
static int handle(const char *name, umode_t mode, kuid_t uid, kgid_t gid,
struct device *dev)
@@ -405,7 +405,7 @@ static void __noreturn devtmpfs_work_loop(void)
}
}
-static int __init devtmpfs_setup(void *p)
+static noinline int __init devtmpfs_setup(void *p)
{
int err;
@@ -419,7 +419,6 @@ static int __init devtmpfs_setup(void *p)
init_chroot(".");
out:
*(int *)p = err;
- complete(&setup_done);
return err;
}
@@ -432,6 +431,7 @@ static int __ref devtmpfsd(void *p)
{
int err = devtmpfs_setup(p);
+ complete(&setup_done);
if (err)
return err;
devtmpfs_work_loop();
diff --git a/drivers/base/firmware_loader/main.c b/drivers/base/firmware_loader/main.c
index 78355095e00d..4fdb8219cd08 100644
--- a/drivers/base/firmware_loader/main.c
+++ b/drivers/base/firmware_loader/main.c
@@ -15,6 +15,7 @@
#include <linux/kernel_read_file.h>
#include <linux/module.h>
#include <linux/init.h>
+#include <linux/initrd.h>
#include <linux/timer.h>
#include <linux/vmalloc.h>
#include <linux/interrupt.h>
@@ -504,6 +505,7 @@ fw_get_filesystem_firmware(struct device *device, struct fw_priv *fw_priv,
if (!path)
return -ENOMEM;
+ wait_for_initramfs();
for (i = 0; i < ARRAY_SIZE(fw_path); i++) {
size_t file_size = 0;
size_t *file_size_ptr = NULL;
diff --git a/drivers/base/memory.c b/drivers/base/memory.c
index f35298425575..b31b3af5c490 100644
--- a/drivers/base/memory.c
+++ b/drivers/base/memory.c
@@ -169,30 +169,98 @@ int memory_notify(unsigned long val, void *v)
return blocking_notifier_call_chain(&memory_chain, val, v);
}
+static int memory_block_online(struct memory_block *mem)
+{
+ unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
+ unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
+ unsigned long nr_vmemmap_pages = mem->nr_vmemmap_pages;
+ struct zone *zone;
+ int ret;
+
+ zone = zone_for_pfn_range(mem->online_type, mem->nid, start_pfn, nr_pages);
+
+ /*
+ * Although vmemmap pages have a different lifecycle than the pages
+ * they describe (they remain until the memory is unplugged), doing
+ * their initialization and accounting at memory onlining/offlining
+ * stage helps to keep accounting easier to follow - e.g vmemmaps
+ * belong to the same zone as the memory they backed.
+ */
+ if (nr_vmemmap_pages) {
+ ret = mhp_init_memmap_on_memory(start_pfn, nr_vmemmap_pages, zone);
+ if (ret)
+ return ret;
+ }
+
+ ret = online_pages(start_pfn + nr_vmemmap_pages,
+ nr_pages - nr_vmemmap_pages, zone);
+ if (ret) {
+ if (nr_vmemmap_pages)
+ mhp_deinit_memmap_on_memory(start_pfn, nr_vmemmap_pages);
+ return ret;
+ }
+
+ /*
+ * Account once onlining succeeded. If the zone was unpopulated, it is
+ * now already properly populated.
+ */
+ if (nr_vmemmap_pages)
+ adjust_present_page_count(zone, nr_vmemmap_pages);
+
+ return ret;
+}
+
+static int memory_block_offline(struct memory_block *mem)
+{
+ unsigned long start_pfn = section_nr_to_pfn(mem->start_section_nr);
+ unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
+ unsigned long nr_vmemmap_pages = mem->nr_vmemmap_pages;
+ struct zone *zone;
+ int ret;
+
+ zone = page_zone(pfn_to_page(start_pfn));
+
+ /*
+ * Unaccount before offlining, such that unpopulated zone and kthreads
+ * can properly be torn down in offline_pages().
+ */
+ if (nr_vmemmap_pages)
+ adjust_present_page_count(zone, -nr_vmemmap_pages);
+
+ ret = offline_pages(start_pfn + nr_vmemmap_pages,
+ nr_pages - nr_vmemmap_pages);
+ if (ret) {
+ /* offline_pages() failed. Account back. */
+ if (nr_vmemmap_pages)
+ adjust_present_page_count(zone, nr_vmemmap_pages);
+ return ret;
+ }
+
+ if (nr_vmemmap_pages)
+ mhp_deinit_memmap_on_memory(start_pfn, nr_vmemmap_pages);
+
+ return ret;
+}
+
/*
* MEMORY_HOTPLUG depends on SPARSEMEM in mm/Kconfig, so it is
* OK to have direct references to sparsemem variables in here.
*/
static int
-memory_block_action(unsigned long start_section_nr, unsigned long action,
- int online_type, int nid)
+memory_block_action(struct memory_block *mem, unsigned long action)
{
- unsigned long start_pfn;
- unsigned long nr_pages = PAGES_PER_SECTION * sections_per_block;
int ret;
- start_pfn = section_nr_to_pfn(start_section_nr);
-
switch (action) {
case MEM_ONLINE:
- ret = online_pages(start_pfn, nr_pages, online_type, nid);
+ ret = memory_block_online(mem);
break;
case MEM_OFFLINE:
- ret = offline_pages(start_pfn, nr_pages);
+ ret = memory_block_offline(mem);
break;
default:
WARN(1, KERN_WARNING "%s(%ld, %ld) unknown action: "
- "%ld\n", __func__, start_section_nr, action, action);
+ "%ld\n", __func__, mem->start_section_nr, action, action);
ret = -EINVAL;
}
@@ -210,9 +278,7 @@ static int memory_block_change_state(struct memory_block *mem,
if (to_state == MEM_OFFLINE)
mem->state = MEM_GOING_OFFLINE;
- ret = memory_block_action(mem->start_section_nr, to_state,
- mem->online_type, mem->nid);
-
+ ret = memory_block_action(mem, to_state);
mem->state = ret ? from_state_req : to_state;
return ret;
@@ -567,7 +633,8 @@ int register_memory(struct memory_block *memory)
return ret;
}
-static int init_memory_block(unsigned long block_id, unsigned long state)
+static int init_memory_block(unsigned long block_id, unsigned long state,
+ unsigned long nr_vmemmap_pages)
{
struct memory_block *mem;
int ret = 0;
@@ -584,6 +651,7 @@ static int init_memory_block(unsigned long block_id, unsigned long state)
mem->start_section_nr = block_id * sections_per_block;
mem->state = state;
mem->nid = NUMA_NO_NODE;
+ mem->nr_vmemmap_pages = nr_vmemmap_pages;
ret = register_memory(mem);
@@ -603,7 +671,7 @@ static int add_memory_block(unsigned long base_section_nr)
if (section_count == 0)
return 0;
return init_memory_block(memory_block_id(base_section_nr),
- MEM_ONLINE);
+ MEM_ONLINE, 0);
}
static void unregister_memory(struct memory_block *memory)
@@ -625,7 +693,8 @@ static void unregister_memory(struct memory_block *memory)
*
* Called under device_hotplug_lock.
*/
-int create_memory_block_devices(unsigned long start, unsigned long size)
+int create_memory_block_devices(unsigned long start, unsigned long size,
+ unsigned long vmemmap_pages)
{
const unsigned long start_block_id = pfn_to_block_id(PFN_DOWN(start));
unsigned long end_block_id = pfn_to_block_id(PFN_DOWN(start + size));
@@ -638,7 +707,7 @@ int create_memory_block_devices(unsigned long start, unsigned long size)
return -EINVAL;
for (block_id = start_block_id; block_id != end_block_id; block_id++) {
- ret = init_memory_block(block_id, MEM_OFFLINE);
+ ret = init_memory_block(block_id, MEM_OFFLINE, vmemmap_pages);
if (ret)
break;
}
diff --git a/drivers/base/node.c b/drivers/base/node.c
index f449dbb2c746..2c36f61d30bc 100644
--- a/drivers/base/node.c
+++ b/drivers/base/node.c
@@ -268,21 +268,20 @@ static void node_init_cache_dev(struct node *node)
if (!dev)
return;
+ device_initialize(dev);
dev->parent = &node->dev;
dev->release = node_cache_release;
if (dev_set_name(dev, "memory_side_cache"))
- goto free_dev;
+ goto put_device;
- if (device_register(dev))
- goto free_name;
+ if (device_add(dev))
+ goto put_device;
pm_runtime_no_callbacks(dev);
node->cache_dev = dev;
return;
-free_name:
- kfree_const(dev->kobj.name);
-free_dev:
- kfree(dev);
+put_device:
+ put_device(dev);
}
/**
@@ -319,25 +318,24 @@ void node_add_cache(unsigned int nid, struct node_cache_attrs *cache_attrs)
return;
dev = &info->dev;
+ device_initialize(dev);
dev->parent = node->cache_dev;
dev->release = node_cacheinfo_release;
dev->groups = cache_groups;
if (dev_set_name(dev, "index%d", cache_attrs->level))
- goto free_cache;
+ goto put_device;
info->cache_attrs = *cache_attrs;
- if (device_register(dev)) {
+ if (device_add(dev)) {
dev_warn(&node->dev, "failed to add cache level:%d\n",
cache_attrs->level);
- goto free_name;
+ goto put_device;
}
pm_runtime_no_callbacks(dev);
list_add_tail(&info->node, &node->cache_attrs);
return;
-free_name:
- kfree_const(dev->kobj.name);
-free_cache:
- kfree(info);
+put_device:
+ put_device(dev);
}
static void node_remove_caches(struct node *node)
diff --git a/drivers/base/platform-msi.c b/drivers/base/platform-msi.c
index 2c1e2e0c1a59..0b72b134a304 100644
--- a/drivers/base/platform-msi.c
+++ b/drivers/base/platform-msi.c
@@ -316,10 +316,11 @@ void *platform_msi_get_host_data(struct irq_domain *domain)
}
/**
- * platform_msi_create_device_domain - Create a platform-msi domain
+ * __platform_msi_create_device_domain - Create a platform-msi domain
*
* @dev: The device generating the MSIs
* @nvec: The number of MSIs that need to be allocated
+ * @is_tree: flag to indicate tree hierarchy
* @write_msi_msg: Callback to write an interrupt message for @dev
* @ops: The hierarchy domain operations to use
* @host_data: Private data associated to this domain
diff --git a/drivers/base/platform.c b/drivers/base/platform.c
index 6e1f8e0b661c..9cd34def2237 100644
--- a/drivers/base/platform.c
+++ b/drivers/base/platform.c
@@ -192,7 +192,7 @@ int platform_get_irq_optional(struct platform_device *dev, unsigned int num)
#ifdef CONFIG_SPARC
/* sparc does not have irqs represented as IORESOURCE_IRQ resources */
if (!dev || num >= dev->archdata.num_irqs)
- return -ENXIO;
+ goto out_not_found;
ret = dev->archdata.irqs[num];
goto out;
#else
@@ -223,10 +223,8 @@ int platform_get_irq_optional(struct platform_device *dev, unsigned int num)
struct irq_data *irqd;
irqd = irq_get_irq_data(r->start);
- if (!irqd) {
- ret = -ENXIO;
- goto out;
- }
+ if (!irqd)
+ goto out_not_found;
irqd_set_trigger_type(irqd, r->flags & IORESOURCE_BITS);
}
@@ -249,8 +247,9 @@ int platform_get_irq_optional(struct platform_device *dev, unsigned int num)
goto out;
}
- ret = -ENXIO;
#endif
+out_not_found:
+ ret = -ENXIO;
out:
WARN(ret == 0, "0 is an invalid IRQ number\n");
return ret;
diff --git a/drivers/base/power/clock_ops.c b/drivers/base/power/clock_ops.c
index 84d5acb6301b..0251f3e6e61d 100644
--- a/drivers/base/power/clock_ops.c
+++ b/drivers/base/power/clock_ops.c
@@ -140,7 +140,7 @@ static void pm_clk_op_unlock(struct pm_subsys_data *psd, unsigned long *flags)
}
/**
- * pm_clk_enable - Enable a clock, reporting any errors
+ * __pm_clk_enable - Enable a clock, reporting any errors
* @dev: The device for the given clock
* @ce: PM clock entry corresponding to the clock.
*/
diff --git a/drivers/base/power/domain.c b/drivers/base/power/domain.c
index 78c310d3179d..b6a782c31613 100644
--- a/drivers/base/power/domain.c
+++ b/drivers/base/power/domain.c
@@ -1088,34 +1088,6 @@ static void genpd_sync_power_on(struct generic_pm_domain *genpd, bool use_lock,
}
/**
- * resume_needed - Check whether to resume a device before system suspend.
- * @dev: Device to check.
- * @genpd: PM domain the device belongs to.
- *
- * There are two cases in which a device that can wake up the system from sleep
- * states should be resumed by genpd_prepare(): (1) if the device is enabled
- * to wake up the system and it has to remain active for this purpose while the
- * system is in the sleep state and (2) if the device is not enabled to wake up
- * the system from sleep states and it generally doesn't generate wakeup signals
- * by itself (those signals are generated on its behalf by other parts of the
- * system). In the latter case it may be necessary to reconfigure the device's
- * wakeup settings during system suspend, because it may have been set up to
- * signal remote wakeup from the system's working state as needed by runtime PM.
- * Return 'true' in either of the above cases.
- */
-static bool resume_needed(struct device *dev,
- const struct generic_pm_domain *genpd)
-{
- bool active_wakeup;
-
- if (!device_can_wakeup(dev))
- return false;
-
- active_wakeup = genpd_is_active_wakeup(genpd);
- return device_may_wakeup(dev) ? active_wakeup : !active_wakeup;
-}
-
-/**
* genpd_prepare - Start power transition of a device in a PM domain.
* @dev: Device to start the transition of.
*
@@ -1135,14 +1107,6 @@ static int genpd_prepare(struct device *dev)
if (IS_ERR(genpd))
return -EINVAL;
- /*
- * If a wakeup request is pending for the device, it should be woken up
- * at this point and a system wakeup event should be reported if it's
- * set up to wake up the system from sleep states.
- */
- if (resume_needed(dev, genpd))
- pm_runtime_resume(dev);
-
genpd_lock(genpd);
if (genpd->prepared_count++ == 0)
diff --git a/drivers/base/power/runtime.c b/drivers/base/power/runtime.c
index 18b82427d0cb..b570848d23e0 100644
--- a/drivers/base/power/runtime.c
+++ b/drivers/base/power/runtime.c
@@ -305,7 +305,7 @@ static int rpm_get_suppliers(struct device *dev)
return 0;
}
-static void rpm_put_suppliers(struct device *dev)
+static void __rpm_put_suppliers(struct device *dev, bool try_to_suspend)
{
struct device_link *link;
@@ -313,10 +313,30 @@ static void rpm_put_suppliers(struct device *dev)
device_links_read_lock_held()) {
while (refcount_dec_not_one(&link->rpm_active))
- pm_runtime_put(link->supplier);
+ pm_runtime_put_noidle(link->supplier);
+
+ if (try_to_suspend)
+ pm_request_idle(link->supplier);
}
}
+static void rpm_put_suppliers(struct device *dev)
+{
+ __rpm_put_suppliers(dev, true);
+}
+
+static void rpm_suspend_suppliers(struct device *dev)
+{
+ struct device_link *link;
+ int idx = device_links_read_lock();
+
+ list_for_each_entry_rcu(link, &dev->links.suppliers, c_node,
+ device_links_read_lock_held())
+ pm_request_idle(link->supplier);
+
+ device_links_read_unlock(idx);
+}
+
/**
* __rpm_callback - Run a given runtime PM callback for a given device.
* @cb: Runtime PM callback to run.
@@ -325,27 +345,29 @@ static void rpm_put_suppliers(struct device *dev)
static int __rpm_callback(int (*cb)(struct device *), struct device *dev)
__releases(&dev->power.lock) __acquires(&dev->power.lock)
{
- bool use_links = dev->power.links_count > 0;
- bool get = false;
int retval, idx;
- bool put;
+ bool use_links = dev->power.links_count > 0;
if (dev->power.irq_safe) {
spin_unlock(&dev->power.lock);
- } else if (!use_links) {
- spin_unlock_irq(&dev->power.lock);
} else {
- get = dev->power.runtime_status == RPM_RESUMING;
-
spin_unlock_irq(&dev->power.lock);
- /* Resume suppliers if necessary. */
- if (get) {
+ /*
+ * Resume suppliers if necessary.
+ *
+ * The device's runtime PM status cannot change until this
+ * routine returns, so it is safe to read the status outside of
+ * the lock.
+ */
+ if (use_links && dev->power.runtime_status == RPM_RESUMING) {
idx = device_links_read_lock();
retval = rpm_get_suppliers(dev);
- if (retval)
+ if (retval) {
+ rpm_put_suppliers(dev);
goto fail;
+ }
device_links_read_unlock(idx);
}
@@ -355,36 +377,24 @@ static int __rpm_callback(int (*cb)(struct device *), struct device *dev)
if (dev->power.irq_safe) {
spin_lock(&dev->power.lock);
- return retval;
- }
-
- spin_lock_irq(&dev->power.lock);
-
- if (!use_links)
- return retval;
-
- /*
- * If the device is suspending and the callback has returned success,
- * drop the usage counters of the suppliers that have been reference
- * counted on its resume.
- *
- * Do that if the resume fails too.
- */
- put = dev->power.runtime_status == RPM_SUSPENDING && !retval;
- if (put)
- __update_runtime_status(dev, RPM_SUSPENDED);
- else
- put = get && retval;
-
- if (put) {
- spin_unlock_irq(&dev->power.lock);
+ } else {
+ /*
+ * If the device is suspending and the callback has returned
+ * success, drop the usage counters of the suppliers that have
+ * been reference counted on its resume.
+ *
+ * Do that if resume fails too.
+ */
+ if (use_links
+ && ((dev->power.runtime_status == RPM_SUSPENDING && !retval)
+ || (dev->power.runtime_status == RPM_RESUMING && retval))) {
+ idx = device_links_read_lock();
- idx = device_links_read_lock();
+ __rpm_put_suppliers(dev, false);
fail:
- rpm_put_suppliers(dev);
-
- device_links_read_unlock(idx);
+ device_links_read_unlock(idx);
+ }
spin_lock_irq(&dev->power.lock);
}
@@ -654,8 +664,11 @@ static int rpm_suspend(struct device *dev, int rpmflags)
goto out;
}
+ if (dev->power.irq_safe)
+ goto out;
+
/* Maybe the parent is now able to suspend. */
- if (parent && !parent->power.ignore_children && !dev->power.irq_safe) {
+ if (parent && !parent->power.ignore_children) {
spin_unlock(&dev->power.lock);
spin_lock(&parent->power.lock);
@@ -664,6 +677,14 @@ static int rpm_suspend(struct device *dev, int rpmflags)
spin_lock(&dev->power.lock);
}
+ /* Maybe the suppliers are now able to suspend. */
+ if (dev->power.links_count > 0) {
+ spin_unlock_irq(&dev->power.lock);
+
+ rpm_suspend_suppliers(dev);
+
+ spin_lock_irq(&dev->power.lock);
+ }
out:
trace_rpm_return_int_rcuidle(dev, _THIS_IP_, retval);
@@ -930,7 +951,7 @@ static void pm_runtime_work(struct work_struct *work)
/**
* pm_suspend_timer_fn - Timer function for pm_schedule_suspend().
- * @data: Device pointer passed by pm_schedule_suspend().
+ * @timer: hrtimer used by pm_schedule_suspend().
*
* Check if the time is right and queue a suspend request.
*/
@@ -1616,6 +1637,7 @@ void pm_runtime_init(struct device *dev)
dev->power.request_pending = false;
dev->power.request = RPM_REQ_NONE;
dev->power.deferred_resume = false;
+ dev->power.needs_force_resume = 0;
INIT_WORK(&dev->power.work, pm_runtime_work);
dev->power.timer_expires = 0;
@@ -1669,8 +1691,8 @@ void pm_runtime_get_suppliers(struct device *dev)
device_links_read_lock_held())
if (link->flags & DL_FLAG_PM_RUNTIME) {
link->supplier_preactivated = true;
- refcount_inc(&link->rpm_active);
pm_runtime_get_sync(link->supplier);
+ refcount_inc(&link->rpm_active);
}
device_links_read_unlock(idx);
@@ -1683,6 +1705,8 @@ void pm_runtime_get_suppliers(struct device *dev)
void pm_runtime_put_suppliers(struct device *dev)
{
struct device_link *link;
+ unsigned long flags;
+ bool put;
int idx;
idx = device_links_read_lock();
@@ -1691,7 +1715,11 @@ void pm_runtime_put_suppliers(struct device *dev)
device_links_read_lock_held())
if (link->supplier_preactivated) {
link->supplier_preactivated = false;
- if (refcount_dec_not_one(&link->rpm_active))
+ spin_lock_irqsave(&dev->power.lock, flags);
+ put = pm_runtime_status_suspended(dev) &&
+ refcount_dec_not_one(&link->rpm_active);
+ spin_unlock_irqrestore(&dev->power.lock, flags);
+ if (put)
pm_runtime_put(link->supplier);
}
@@ -1777,10 +1805,12 @@ int pm_runtime_force_suspend(struct device *dev)
* its parent, but set its status to RPM_SUSPENDED anyway in case this
* function will be called again for it in the meantime.
*/
- if (pm_runtime_need_not_resume(dev))
+ if (pm_runtime_need_not_resume(dev)) {
pm_runtime_set_suspended(dev);
- else
+ } else {
__update_runtime_status(dev, RPM_SUSPENDED);
+ dev->power.needs_force_resume = 1;
+ }
return 0;
@@ -1807,7 +1837,7 @@ int pm_runtime_force_resume(struct device *dev)
int (*callback)(struct device *);
int ret = 0;
- if (!pm_runtime_status_suspended(dev) || pm_runtime_need_not_resume(dev))
+ if (!pm_runtime_status_suspended(dev) || !dev->power.needs_force_resume)
goto out;
/*
@@ -1826,6 +1856,7 @@ int pm_runtime_force_resume(struct device *dev)
pm_runtime_mark_last_busy(dev);
out:
+ dev->power.needs_force_resume = 0;
pm_runtime_enable(dev);
return ret;
}
diff --git a/drivers/base/power/wakeup.c b/drivers/base/power/wakeup.c
index 92073ac68473..f0b37c188514 100644
--- a/drivers/base/power/wakeup.c
+++ b/drivers/base/power/wakeup.c
@@ -400,9 +400,9 @@ void device_wakeup_detach_irq(struct device *dev)
}
/**
- * device_wakeup_arm_wake_irqs(void)
+ * device_wakeup_arm_wake_irqs -
*
- * Itereates over the list of device wakeirqs to arm them.
+ * Iterates over the list of device wakeirqs to arm them.
*/
void device_wakeup_arm_wake_irqs(void)
{
@@ -416,9 +416,9 @@ void device_wakeup_arm_wake_irqs(void)
}
/**
- * device_wakeup_disarm_wake_irqs(void)
+ * device_wakeup_disarm_wake_irqs -
*
- * Itereates over the list of device wakeirqs to disarm them.
+ * Iterates over the list of device wakeirqs to disarm them.
*/
void device_wakeup_disarm_wake_irqs(void)
{
@@ -532,6 +532,7 @@ EXPORT_SYMBOL_GPL(device_init_wakeup);
/**
* device_set_wakeup_enable - Enable or disable a device to wake up the system.
* @dev: Device to handle.
+ * @enable: enable/disable flag
*/
int device_set_wakeup_enable(struct device *dev, bool enable)
{
@@ -581,7 +582,7 @@ static bool wakeup_source_not_registered(struct wakeup_source *ws)
*/
/**
- * wakup_source_activate - Mark given wakeup source as active.
+ * wakeup_source_activate - Mark given wakeup source as active.
* @ws: Wakeup source to handle.
*
* Update the @ws' statistics and, if @ws has just been activated, notify the PM
@@ -686,7 +687,7 @@ static inline void update_prevent_sleep_time(struct wakeup_source *ws,
#endif
/**
- * wakup_source_deactivate - Mark given wakeup source as inactive.
+ * wakeup_source_deactivate - Mark given wakeup source as inactive.
* @ws: Wakeup source to handle.
*
* Update the @ws' statistics and notify the PM core that the wakeup source has
@@ -785,7 +786,7 @@ EXPORT_SYMBOL_GPL(pm_relax);
/**
* pm_wakeup_timer_fn - Delayed finalization of a wakeup event.
- * @data: Address of the wakeup source object associated with the event source.
+ * @t: timer list
*
* Call wakeup_source_deactivate() for the wakeup source whose address is stored
* in @data if it is currently active and its timer has not been canceled and
@@ -1021,7 +1022,7 @@ bool pm_save_wakeup_count(unsigned int count)
#ifdef CONFIG_PM_AUTOSLEEP
/**
* pm_wakep_autosleep_enabled - Modify autosleep_enabled for all wakeup sources.
- * @enabled: Whether to set or to clear the autosleep_enabled flags.
+ * @set: Whether to set or to clear the autosleep_enabled flags.
*/
void pm_wakep_autosleep_enabled(bool set)
{
diff --git a/drivers/base/power/wakeup_stats.c b/drivers/base/power/wakeup_stats.c
index d638259b829a..924fac493c4f 100644
--- a/drivers/base/power/wakeup_stats.c
+++ b/drivers/base/power/wakeup_stats.c
@@ -137,7 +137,7 @@ static struct device *wakeup_source_device_create(struct device *parent,
struct wakeup_source *ws)
{
struct device *dev = NULL;
- int retval = -ENODEV;
+ int retval;
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
if (!dev) {
@@ -154,7 +154,7 @@ static struct device *wakeup_source_device_create(struct device *parent,
dev_set_drvdata(dev, ws);
device_set_pm_not_required(dev);
- retval = kobject_set_name(&dev->kobj, "wakeup%d", ws->id);
+ retval = dev_set_name(dev, "wakeup%d", ws->id);
if (retval)
goto error;
diff --git a/drivers/base/regmap/regmap-debugfs.c b/drivers/base/regmap/regmap-debugfs.c
index ff2ee87987c7..211a335a608d 100644
--- a/drivers/base/regmap/regmap-debugfs.c
+++ b/drivers/base/regmap/regmap-debugfs.c
@@ -660,6 +660,7 @@ void regmap_debugfs_exit(struct regmap *map)
regmap_debugfs_free_dump_cache(map);
mutex_unlock(&map->cache_lock);
kfree(map->debugfs_name);
+ map->debugfs_name = NULL;
} else {
struct regmap_debugfs_node *node, *tmp;
diff --git a/drivers/base/regmap/regmap-irq.c b/drivers/base/regmap/regmap-irq.c
index 19db764ffa4a..760296a4b606 100644
--- a/drivers/base/regmap/regmap-irq.c
+++ b/drivers/base/regmap/regmap-irq.c
@@ -38,6 +38,7 @@ struct regmap_irq_chip_data {
unsigned int *wake_buf;
unsigned int *type_buf;
unsigned int *type_buf_def;
+ unsigned int **virt_buf;
unsigned int irq_reg_stride;
unsigned int type_reg_stride;
@@ -45,6 +46,27 @@ struct regmap_irq_chip_data {
bool clear_status:1;
};
+static int sub_irq_reg(struct regmap_irq_chip_data *data,
+ unsigned int base_reg, int i)
+{
+ const struct regmap_irq_chip *chip = data->chip;
+ struct regmap *map = data->map;
+ struct regmap_irq_sub_irq_map *subreg;
+ unsigned int offset;
+ int reg = 0;
+
+ if (!chip->sub_reg_offsets || !chip->not_fixed_stride) {
+ /* Assume linear mapping */
+ reg = base_reg + (i * map->reg_stride * data->irq_reg_stride);
+ } else {
+ subreg = &chip->sub_reg_offsets[i];
+ offset = subreg->offset[0];
+ reg = base_reg + offset;
+ }
+
+ return reg;
+}
+
static inline const
struct regmap_irq *irq_to_regmap_irq(struct regmap_irq_chip_data *data,
int irq)
@@ -73,7 +95,7 @@ static void regmap_irq_sync_unlock(struct irq_data *data)
{
struct regmap_irq_chip_data *d = irq_data_get_irq_chip_data(data);
struct regmap *map = d->map;
- int i, ret;
+ int i, j, ret;
u32 reg;
u32 unmask_offset;
u32 val;
@@ -87,8 +109,7 @@ static void regmap_irq_sync_unlock(struct irq_data *data)
if (d->clear_status) {
for (i = 0; i < d->chip->num_regs; i++) {
- reg = d->chip->status_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->status_base, i);
ret = regmap_read(map, reg, &val);
if (ret)
@@ -108,8 +129,7 @@ static void regmap_irq_sync_unlock(struct irq_data *data)
if (!d->chip->mask_base)
continue;
- reg = d->chip->mask_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->mask_base, i);
if (d->chip->mask_invert) {
ret = regmap_irq_update_bits(d, reg,
d->mask_buf_def[i], ~d->mask_buf[i]);
@@ -136,8 +156,7 @@ static void regmap_irq_sync_unlock(struct irq_data *data)
dev_err(d->map->dev, "Failed to sync masks in %x\n",
reg);
- reg = d->chip->wake_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->wake_base, i);
if (d->wake_buf) {
if (d->chip->wake_invert)
ret = regmap_irq_update_bits(d, reg,
@@ -161,8 +180,8 @@ static void regmap_irq_sync_unlock(struct irq_data *data)
* it'll be ignored in irq handler, then may introduce irq storm
*/
if (d->mask_buf[i] && (d->chip->ack_base || d->chip->use_ack)) {
- reg = d->chip->ack_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->ack_base, i);
+
/* some chips ack by write 0 */
if (d->chip->ack_invert)
ret = regmap_write(map, reg, ~d->mask_buf[i]);
@@ -187,8 +206,7 @@ static void regmap_irq_sync_unlock(struct irq_data *data)
for (i = 0; i < d->chip->num_type_reg; i++) {
if (!d->type_buf_def[i])
continue;
- reg = d->chip->type_base +
- (i * map->reg_stride * d->type_reg_stride);
+ reg = sub_irq_reg(d, d->chip->type_base, i);
if (d->chip->type_invert)
ret = regmap_irq_update_bits(d, reg,
d->type_buf_def[i], ~d->type_buf[i]);
@@ -201,6 +219,20 @@ static void regmap_irq_sync_unlock(struct irq_data *data)
}
}
+ if (d->chip->num_virt_regs) {
+ for (i = 0; i < d->chip->num_virt_regs; i++) {
+ for (j = 0; j < d->chip->num_regs; j++) {
+ reg = sub_irq_reg(d, d->chip->virt_reg_base[i],
+ j);
+ ret = regmap_write(map, reg, d->virt_buf[i][j]);
+ if (ret != 0)
+ dev_err(d->map->dev,
+ "Failed to write virt 0x%x: %d\n",
+ reg, ret);
+ }
+ }
+ }
+
if (d->chip->runtime_pm)
pm_runtime_put(map->dev);
@@ -301,6 +333,11 @@ static int regmap_irq_set_type(struct irq_data *data, unsigned int type)
default:
return -EINVAL;
}
+
+ if (d->chip->set_type_virt)
+ return d->chip->set_type_virt(d->virt_buf, type, data->hwirq,
+ reg);
+
return 0;
}
@@ -352,8 +389,15 @@ static inline int read_sub_irq_data(struct regmap_irq_chip_data *data,
for (i = 0; i < subreg->num_regs; i++) {
unsigned int offset = subreg->offset[i];
- ret = regmap_read(map, chip->status_base + offset,
- &data->status_buf[offset]);
+ if (chip->not_fixed_stride)
+ ret = regmap_read(map,
+ chip->status_base + offset,
+ &data->status_buf[b]);
+ else
+ ret = regmap_read(map,
+ chip->status_base + offset,
+ &data->status_buf[offset]);
+
if (ret)
break;
}
@@ -474,10 +518,9 @@ static irqreturn_t regmap_irq_thread(int irq, void *d)
} else {
for (i = 0; i < data->chip->num_regs; i++) {
- ret = regmap_read(map, chip->status_base +
- (i * map->reg_stride
- * data->irq_reg_stride),
- &data->status_buf[i]);
+ unsigned int reg = sub_irq_reg(data,
+ data->chip->status_base, i);
+ ret = regmap_read(map, reg, &data->status_buf[i]);
if (ret != 0) {
dev_err(map->dev,
@@ -499,8 +542,8 @@ static irqreturn_t regmap_irq_thread(int irq, void *d)
data->status_buf[i] &= ~data->mask_buf[i];
if (data->status_buf[i] && (chip->ack_base || chip->use_ack)) {
- reg = chip->ack_base +
- (i * map->reg_stride * data->irq_reg_stride);
+ reg = sub_irq_reg(data, data->chip->ack_base, i);
+
if (chip->ack_invert)
ret = regmap_write(map, reg,
~data->status_buf[i]);
@@ -605,6 +648,12 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode,
return -EINVAL;
}
+ if (chip->not_fixed_stride) {
+ for (i = 0; i < chip->num_regs; i++)
+ if (chip->sub_reg_offsets[i].num_regs != 1)
+ return -EINVAL;
+ }
+
if (irq_base) {
irq_base = irq_alloc_descs(irq_base, 0, chip->num_irqs, 0);
if (irq_base < 0) {
@@ -662,6 +711,24 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode,
goto err_alloc;
}
+ if (chip->num_virt_regs) {
+ /*
+ * Create virt_buf[chip->num_extra_config_regs][chip->num_regs]
+ */
+ d->virt_buf = kcalloc(chip->num_virt_regs, sizeof(*d->virt_buf),
+ GFP_KERNEL);
+ if (!d->virt_buf)
+ goto err_alloc;
+
+ for (i = 0; i < chip->num_virt_regs; i++) {
+ d->virt_buf[i] = kcalloc(chip->num_regs,
+ sizeof(unsigned int),
+ GFP_KERNEL);
+ if (!d->virt_buf[i])
+ goto err_alloc;
+ }
+ }
+
d->irq_chip = regmap_irq_chip;
d->irq_chip.name = chip->name;
d->irq = irq;
@@ -700,8 +767,8 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode,
if (!chip->mask_base)
continue;
- reg = chip->mask_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->mask_base, i);
+
if (chip->mask_invert)
ret = regmap_irq_update_bits(d, reg,
d->mask_buf[i], ~d->mask_buf[i]);
@@ -725,8 +792,7 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode,
continue;
/* Ack masked but set interrupts */
- reg = chip->status_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->status_base, i);
ret = regmap_read(map, reg, &d->status_buf[i]);
if (ret != 0) {
dev_err(map->dev, "Failed to read IRQ status: %d\n",
@@ -735,8 +801,7 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode,
}
if (d->status_buf[i] && (chip->ack_base || chip->use_ack)) {
- reg = chip->ack_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->ack_base, i);
if (chip->ack_invert)
ret = regmap_write(map, reg,
~(d->status_buf[i] & d->mask_buf[i]));
@@ -765,8 +830,7 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode,
if (d->wake_buf) {
for (i = 0; i < chip->num_regs; i++) {
d->wake_buf[i] = d->mask_buf_def[i];
- reg = chip->wake_base +
- (i * map->reg_stride * d->irq_reg_stride);
+ reg = sub_irq_reg(d, d->chip->wake_base, i);
if (chip->wake_invert)
ret = regmap_irq_update_bits(d, reg,
@@ -786,8 +850,7 @@ int regmap_add_irq_chip_fwnode(struct fwnode_handle *fwnode,
if (chip->num_type_reg && !chip->type_in_mask) {
for (i = 0; i < chip->num_type_reg; ++i) {
- reg = chip->type_base +
- (i * map->reg_stride * d->type_reg_stride);
+ reg = sub_irq_reg(d, d->chip->type_base, i);
ret = regmap_read(map, reg, &d->type_buf_def[i]);
@@ -838,6 +901,11 @@ err_alloc:
kfree(d->mask_buf);
kfree(d->status_buf);
kfree(d->status_reg_buf);
+ if (d->virt_buf) {
+ for (i = 0; i < chip->num_virt_regs; i++)
+ kfree(d->virt_buf[i]);
+ kfree(d->virt_buf);
+ }
kfree(d);
return ret;
}
diff --git a/drivers/base/swnode.c b/drivers/base/swnode.c
index 37179a8b1ceb..3cc11b813f28 100644
--- a/drivers/base/swnode.c
+++ b/drivers/base/swnode.c
@@ -12,10 +12,10 @@
#include <linux/slab.h>
struct swnode {
- int id;
struct kobject kobj;
struct fwnode_handle fwnode;
const struct software_node *node;
+ int id;
/* hierarchy */
struct ida child_ids;
@@ -720,19 +720,30 @@ software_node_find_by_name(const struct software_node *parent, const char *name)
}
EXPORT_SYMBOL_GPL(software_node_find_by_name);
-static int
-software_node_register_properties(struct software_node *node,
- const struct property_entry *properties)
+static struct software_node *software_node_alloc(const struct property_entry *properties)
{
struct property_entry *props;
+ struct software_node *node;
props = property_entries_dup(properties);
if (IS_ERR(props))
- return PTR_ERR(props);
+ return ERR_CAST(props);
+
+ node = kzalloc(sizeof(*node), GFP_KERNEL);
+ if (!node) {
+ property_entries_free(props);
+ return ERR_PTR(-ENOMEM);
+ }
node->properties = props;
- return 0;
+ return node;
+}
+
+static void software_node_free(const struct software_node *node)
+{
+ property_entries_free(node->properties);
+ kfree(node);
}
static void software_node_release(struct kobject *kobj)
@@ -746,10 +757,9 @@ static void software_node_release(struct kobject *kobj)
ida_simple_remove(&swnode_root_ids, swnode->id);
}
- if (swnode->allocated) {
- property_entries_free(swnode->node->properties);
- kfree(swnode->node);
- }
+ if (swnode->allocated)
+ software_node_free(swnode->node);
+
ida_destroy(&swnode->child_ids);
kfree(swnode);
}
@@ -767,22 +777,19 @@ swnode_register(const struct software_node *node, struct swnode *parent,
int ret;
swnode = kzalloc(sizeof(*swnode), GFP_KERNEL);
- if (!swnode) {
- ret = -ENOMEM;
- goto out_err;
- }
+ if (!swnode)
+ return ERR_PTR(-ENOMEM);
ret = ida_simple_get(parent ? &parent->child_ids : &swnode_root_ids,
0, 0, GFP_KERNEL);
if (ret < 0) {
kfree(swnode);
- goto out_err;
+ return ERR_PTR(ret);
}
swnode->id = ret;
swnode->node = node;
swnode->parent = parent;
- swnode->allocated = allocated;
swnode->kobj.kset = swnode_kset;
fwnode_init(&swnode->fwnode, &software_node_ops);
@@ -803,16 +810,17 @@ swnode_register(const struct software_node *node, struct swnode *parent,
return ERR_PTR(ret);
}
+ /*
+ * Assign the flag only in the successful case, so
+ * the above kobject_put() won't mess up with properties.
+ */
+ swnode->allocated = allocated;
+
if (parent)
list_add_tail(&swnode->entry, &parent->children);
kobject_uevent(&swnode->kobj, KOBJ_ADD);
return &swnode->fwnode;
-
-out_err:
- if (allocated)
- property_entries_free(node->properties);
- return ERR_PTR(ret);
}
/**
@@ -880,7 +888,11 @@ EXPORT_SYMBOL_GPL(software_node_unregister_nodes);
* software_node_register_node_group - Register a group of software nodes
* @node_group: NULL terminated array of software node pointers to be registered
*
- * Register multiple software nodes at once.
+ * Register multiple software nodes at once. If any node in the array
+ * has its .parent pointer set (which can only be to another software_node),
+ * then its parent **must** have been registered before it is; either outside
+ * of this function or by ordering the array such that parent comes before
+ * child.
*/
int software_node_register_node_group(const struct software_node **node_group)
{
@@ -906,10 +918,14 @@ EXPORT_SYMBOL_GPL(software_node_register_node_group);
* software_node_unregister_node_group - Unregister a group of software nodes
* @node_group: NULL terminated array of software node pointers to be unregistered
*
- * Unregister multiple software nodes at once. The array will be unwound in
- * reverse order (i.e. last entry first) and thus if any members of the array are
- * children of another member then the children must appear later in the list such
- * that they are unregistered first.
+ * Unregister multiple software nodes at once. If parent pointers are set up
+ * in any of the software nodes then the array **must** be ordered such that
+ * parents come before their children.
+ *
+ * NOTE: If you are uncertain whether the array is ordered such that
+ * parents will be unregistered before their children, it is wiser to
+ * remove the nodes individually, in the correct order (child before
+ * parent).
*/
void software_node_unregister_node_group(
const struct software_node **node_group)
@@ -938,6 +954,9 @@ int software_node_register(const struct software_node *node)
if (software_node_to_swnode(node))
return -EEXIST;
+ if (node->parent && !parent)
+ return -EINVAL;
+
return PTR_ERR_OR_ZERO(swnode_register(node, parent, 0));
}
EXPORT_SYMBOL_GPL(software_node_register);
@@ -960,31 +979,28 @@ struct fwnode_handle *
fwnode_create_software_node(const struct property_entry *properties,
const struct fwnode_handle *parent)
{
+ struct fwnode_handle *fwnode;
struct software_node *node;
- struct swnode *p = NULL;
- int ret;
+ struct swnode *p;
- if (parent) {
- if (IS_ERR(parent))
- return ERR_CAST(parent);
- if (!is_software_node(parent))
- return ERR_PTR(-EINVAL);
- p = to_swnode(parent);
- }
+ if (IS_ERR(parent))
+ return ERR_CAST(parent);
- node = kzalloc(sizeof(*node), GFP_KERNEL);
- if (!node)
- return ERR_PTR(-ENOMEM);
+ p = to_swnode(parent);
+ if (parent && !p)
+ return ERR_PTR(-EINVAL);
- ret = software_node_register_properties(node, properties);
- if (ret) {
- kfree(node);
- return ERR_PTR(ret);
- }
+ node = software_node_alloc(properties);
+ if (IS_ERR(node))
+ return ERR_CAST(node);
node->parent = p ? p->node : NULL;
- return swnode_register(node, p, 1);
+ fwnode = swnode_register(node, p, 1);
+ if (IS_ERR(fwnode))
+ software_node_free(node);
+
+ return fwnode;
}
EXPORT_SYMBOL_GPL(fwnode_create_software_node);
@@ -1002,25 +1018,34 @@ EXPORT_SYMBOL_GPL(fwnode_remove_software_node);
/**
* device_add_software_node - Assign software node to a device
* @dev: The device the software node is meant for.
- * @swnode: The software node.
+ * @node: The software node.
*
- * This function will register @swnode and make it the secondary firmware node
- * pointer of @dev. If @dev has no primary node, then @swnode will become the primary
- * node.
+ * This function will make @node the secondary firmware node pointer of @dev. If
+ * @dev has no primary node, then @node will become the primary node. The
+ * function will register @node automatically if it wasn't already registered.
*/
-int device_add_software_node(struct device *dev, const struct software_node *swnode)
+int device_add_software_node(struct device *dev, const struct software_node *node)
{
+ struct swnode *swnode;
int ret;
/* Only one software node per device. */
if (dev_to_swnode(dev))
return -EBUSY;
- ret = software_node_register(swnode);
- if (ret)
- return ret;
+ swnode = software_node_to_swnode(node);
+ if (swnode) {
+ kobject_get(&swnode->kobj);
+ } else {
+ ret = software_node_register(node);
+ if (ret)
+ return ret;
+
+ swnode = software_node_to_swnode(node);
+ }
- set_secondary_fwnode(dev, software_node_fwnode(swnode));
+ set_secondary_fwnode(dev, &swnode->fwnode);
+ software_node_notify(dev, KOBJ_ADD);
return 0;
}
@@ -1094,8 +1119,8 @@ int software_node_notify(struct device *dev, unsigned long action)
switch (action) {
case KOBJ_ADD:
- ret = sysfs_create_link(&dev->kobj, &swnode->kobj,
- "software_node");
+ ret = sysfs_create_link_nowarn(&dev->kobj, &swnode->kobj,
+ "software_node");
if (ret)
break;
diff --git a/drivers/base/test/Kconfig b/drivers/base/test/Kconfig
index ba225eb1b761..2f3fa31a948e 100644
--- a/drivers/base/test/Kconfig
+++ b/drivers/base/test/Kconfig
@@ -8,7 +8,7 @@ config TEST_ASYNC_DRIVER_PROBE
The module name will be test_async_driver_probe.ko
If unsure say N.
-config KUNIT_DRIVER_PE_TEST
+config DRIVER_PE_KUNIT_TEST
bool "KUnit Tests for property entry API" if !KUNIT_ALL_TESTS
depends on KUNIT=y
default KUNIT_ALL_TESTS
diff --git a/drivers/base/test/Makefile b/drivers/base/test/Makefile
index 2f15fae8625f..64b2f3d744d5 100644
--- a/drivers/base/test/Makefile
+++ b/drivers/base/test/Makefile
@@ -1,5 +1,5 @@
# SPDX-License-Identifier: GPL-2.0
obj-$(CONFIG_TEST_ASYNC_DRIVER_PROBE) += test_async_driver_probe.o
-obj-$(CONFIG_KUNIT_DRIVER_PE_TEST) += property-entry-test.o
+obj-$(CONFIG_DRIVER_PE_KUNIT_TEST) += property-entry-test.o
CFLAGS_REMOVE_property-entry-test.o += -fplugin-arg-structleak_plugin-byref -fplugin-arg-structleak_plugin-byref-all
diff --git a/drivers/base/test/property-entry-test.c b/drivers/base/test/property-entry-test.c
index abe03315180f..1106fedcceed 100644
--- a/drivers/base/test/property-entry-test.c
+++ b/drivers/base/test/property-entry-test.c
@@ -27,6 +27,9 @@ static void pe_test_uints(struct kunit *test)
node = fwnode_create_software_node(entries, NULL);
KUNIT_ASSERT_NOT_ERR_OR_NULL(test, node);
+ error = fwnode_property_count_u8(node, "prop-u8");
+ KUNIT_EXPECT_EQ(test, error, 1);
+
error = fwnode_property_read_u8(node, "prop-u8", &val_u8);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u8, 8);
@@ -48,6 +51,9 @@ static void pe_test_uints(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u16, 16);
+ error = fwnode_property_count_u16(node, "prop-u16");
+ KUNIT_EXPECT_EQ(test, error, 1);
+
error = fwnode_property_read_u16_array(node, "prop-u16", array_u16, 1);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)array_u16[0], 16);
@@ -65,6 +71,9 @@ static void pe_test_uints(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u32, 32);
+ error = fwnode_property_count_u32(node, "prop-u32");
+ KUNIT_EXPECT_EQ(test, error, 1);
+
error = fwnode_property_read_u32_array(node, "prop-u32", array_u32, 1);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)array_u32[0], 32);
@@ -82,6 +91,9 @@ static void pe_test_uints(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u64, 64);
+ error = fwnode_property_count_u64(node, "prop-u64");
+ KUNIT_EXPECT_EQ(test, error, 1);
+
error = fwnode_property_read_u64_array(node, "prop-u64", array_u64, 1);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)array_u64[0], 64);
@@ -95,15 +107,19 @@ static void pe_test_uints(struct kunit *test)
error = fwnode_property_read_u64_array(node, "no-prop-u64", array_u64, 1);
KUNIT_EXPECT_NE(test, error, 0);
+ /* Count 64-bit values as 16-bit */
+ error = fwnode_property_count_u16(node, "prop-u64");
+ KUNIT_EXPECT_EQ(test, error, 4);
+
fwnode_remove_software_node(node);
}
static void pe_test_uint_arrays(struct kunit *test)
{
- static const u8 a_u8[16] = { 8, 9 };
- static const u16 a_u16[16] = { 16, 17 };
- static const u32 a_u32[16] = { 32, 33 };
- static const u64 a_u64[16] = { 64, 65 };
+ static const u8 a_u8[10] = { 8, 9 };
+ static const u16 a_u16[10] = { 16, 17 };
+ static const u32 a_u32[10] = { 32, 33 };
+ static const u64 a_u64[10] = { 64, 65 };
static const struct property_entry entries[] = {
PROPERTY_ENTRY_U8_ARRAY("prop-u8", a_u8),
PROPERTY_ENTRY_U16_ARRAY("prop-u16", a_u16),
@@ -126,6 +142,9 @@ static void pe_test_uint_arrays(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u8, 8);
+ error = fwnode_property_count_u8(node, "prop-u8");
+ KUNIT_EXPECT_EQ(test, error, 10);
+
error = fwnode_property_read_u8_array(node, "prop-u8", array_u8, 1);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)array_u8[0], 8);
@@ -148,6 +167,9 @@ static void pe_test_uint_arrays(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u16, 16);
+ error = fwnode_property_count_u16(node, "prop-u16");
+ KUNIT_EXPECT_EQ(test, error, 10);
+
error = fwnode_property_read_u16_array(node, "prop-u16", array_u16, 1);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)array_u16[0], 16);
@@ -170,6 +192,9 @@ static void pe_test_uint_arrays(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u32, 32);
+ error = fwnode_property_count_u32(node, "prop-u32");
+ KUNIT_EXPECT_EQ(test, error, 10);
+
error = fwnode_property_read_u32_array(node, "prop-u32", array_u32, 1);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)array_u32[0], 32);
@@ -192,6 +217,9 @@ static void pe_test_uint_arrays(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)val_u64, 64);
+ error = fwnode_property_count_u64(node, "prop-u64");
+ KUNIT_EXPECT_EQ(test, error, 10);
+
error = fwnode_property_read_u64_array(node, "prop-u64", array_u64, 1);
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_EQ(test, (int)array_u64[0], 64);
@@ -210,6 +238,14 @@ static void pe_test_uint_arrays(struct kunit *test)
error = fwnode_property_read_u64_array(node, "no-prop-u64", array_u64, 1);
KUNIT_EXPECT_NE(test, error, 0);
+ /* Count 64-bit values as 16-bit */
+ error = fwnode_property_count_u16(node, "prop-u64");
+ KUNIT_EXPECT_EQ(test, error, 40);
+
+ /* Other way around */
+ error = fwnode_property_count_u64(node, "prop-u16");
+ KUNIT_EXPECT_EQ(test, error, 2);
+
fwnode_remove_software_node(node);
}
@@ -239,6 +275,9 @@ static void pe_test_strings(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_STREQ(test, str, "single");
+ error = fwnode_property_string_array_count(node, "str");
+ KUNIT_EXPECT_EQ(test, error, 1);
+
error = fwnode_property_read_string_array(node, "str", strs, 1);
KUNIT_EXPECT_EQ(test, error, 1);
KUNIT_EXPECT_STREQ(test, strs[0], "single");
@@ -258,6 +297,9 @@ static void pe_test_strings(struct kunit *test)
KUNIT_EXPECT_EQ(test, error, 0);
KUNIT_EXPECT_STREQ(test, str, "");
+ error = fwnode_property_string_array_count(node, "strs");
+ KUNIT_EXPECT_EQ(test, error, 2);
+
error = fwnode_property_read_string_array(node, "strs", strs, 3);
KUNIT_EXPECT_EQ(test, error, 2);
KUNIT_EXPECT_STREQ(test, strs[0], "string-a");
@@ -370,15 +412,8 @@ static void pe_test_reference(struct kunit *test)
};
static const struct software_node_ref_args refs[] = {
- {
- .node = &nodes[0],
- .nargs = 0,
- },
- {
- .node = &nodes[1],
- .nargs = 2,
- .args = { 3, 4 },
- },
+ SOFTWARE_NODE_REFERENCE(&nodes[0]),
+ SOFTWARE_NODE_REFERENCE(&nodes[1], 3, 4),
};
const struct property_entry entries[] = {
diff --git a/drivers/bcma/driver_mips.c b/drivers/bcma/driver_mips.c
index 87760aa60446..12aca34e8db0 100644
--- a/drivers/bcma/driver_mips.c
+++ b/drivers/bcma/driver_mips.c
@@ -52,13 +52,6 @@ static inline u32 mips_read32(struct bcma_drv_mips *mcore,
return bcma_read32(mcore->core, offset);
}
-static inline void mips_write32(struct bcma_drv_mips *mcore,
- u16 offset,
- u32 value)
-{
- bcma_write32(mcore->core, offset, value);
-}
-
static u32 bcma_core_mips_irqflag(struct bcma_device *dev)
{
u32 flag;
diff --git a/drivers/block/Kconfig b/drivers/block/Kconfig
index fd236158f32d..63056cfd4b62 100644
--- a/drivers/block/Kconfig
+++ b/drivers/block/Kconfig
@@ -50,7 +50,7 @@ config MAC_FLOPPY
config BLK_DEV_SWIM
tristate "Support for SWIM Macintosh floppy"
- depends on M68K && MAC
+ depends on M68K && MAC && !HIGHMEM
help
You should select this option if you want floppy support
and you don't have a II, IIfx, Q900, Q950 or AV series.
@@ -121,23 +121,6 @@ source "drivers/block/mtip32xx/Kconfig"
source "drivers/block/zram/Kconfig"
-config BLK_DEV_UMEM
- tristate "Micro Memory MM5415 Battery Backed RAM support"
- depends on PCI
- help
- Saying Y here will include support for the MM5415 family of
- battery backed (Non-volatile) RAM cards.
- <http://www.umem.com/>
-
- The cards appear as block devices that can be partitioned into
- as many as 15 partitions.
-
- To compile this driver as a module, choose M here: the
- module will be called umem.
-
- The umem driver has not yet been allocated a MAJOR number, so
- one is chosen dynamically.
-
config BLK_DEV_UBD
bool "Virtual block device"
depends on UML
@@ -378,12 +361,6 @@ config SUNVDC
source "drivers/s390/block/Kconfig"
-config XILINX_SYSACE
- tristate "Xilinx SystemACE support"
- depends on 4xx || MICROBLAZE
- help
- Include support for the Xilinx SystemACE CompactFlash interface
-
config XEN_BLKDEV_FRONTEND
tristate "Xen virtual block device support"
depends on XEN
diff --git a/drivers/block/Makefile b/drivers/block/Makefile
index e3e3f1c79a82..bc68817ef496 100644
--- a/drivers/block/Makefile
+++ b/drivers/block/Makefile
@@ -20,11 +20,9 @@ obj-$(CONFIG_AMIGA_Z2RAM) += z2ram.o
obj-$(CONFIG_N64CART) += n64cart.o
obj-$(CONFIG_BLK_DEV_RAM) += brd.o
obj-$(CONFIG_BLK_DEV_LOOP) += loop.o
-obj-$(CONFIG_XILINX_SYSACE) += xsysace.o
obj-$(CONFIG_CDROM_PKTCDVD) += pktcdvd.o
obj-$(CONFIG_SUNVDC) += sunvdc.o
-obj-$(CONFIG_BLK_DEV_UMEM) += umem.o
obj-$(CONFIG_BLK_DEV_NBD) += nbd.o
obj-$(CONFIG_BLK_DEV_CRYPTOLOOP) += cryptoloop.o
obj-$(CONFIG_VIRTIO_BLK) += virtio_blk.o
diff --git a/drivers/block/ataflop.c b/drivers/block/ataflop.c
index 104b713f4055..d601e49f80e0 100644
--- a/drivers/block/ataflop.c
+++ b/drivers/block/ataflop.c
@@ -729,8 +729,12 @@ static int do_format(int drive, int type, struct atari_format_descr *desc)
unsigned long flags;
int ret;
- if (type)
+ if (type) {
type--;
+ if (type >= NUM_DISK_MINORS ||
+ minor2disktype[type].drive_types > DriveType)
+ return -EINVAL;
+ }
q = unit[drive].disk[type]->queue;
blk_mq_freeze_queue(q);
@@ -742,11 +746,6 @@ static int do_format(int drive, int type, struct atari_format_descr *desc)
local_irq_restore(flags);
if (type) {
- if (type >= NUM_DISK_MINORS ||
- minor2disktype[type].drive_types > DriveType) {
- ret = -EINVAL;
- goto out;
- }
type = minor2disktype[type].index;
UDT = &atari_disk_type[type];
}
@@ -2002,7 +2001,10 @@ static void ataflop_probe(dev_t dev)
int drive = MINOR(dev) & 3;
int type = MINOR(dev) >> 2;
- if (drive >= FD_MAX_UNITS || type > NUM_DISK_MINORS)
+ if (type)
+ type--;
+
+ if (drive >= FD_MAX_UNITS || type >= NUM_DISK_MINORS)
return;
mutex_lock(&ataflop_probe_lock);
if (!unit[drive].disk[type]) {
diff --git a/drivers/block/brd.c b/drivers/block/brd.c
index 18bf99906662..7562cf30b14e 100644
--- a/drivers/block/brd.c
+++ b/drivers/block/brd.c
@@ -18,10 +18,12 @@
#include <linux/bio.h>
#include <linux/highmem.h>
#include <linux/mutex.h>
+#include <linux/pagemap.h>
#include <linux/radix-tree.h>
#include <linux/fs.h>
#include <linux/slab.h>
#include <linux/backing-dev.h>
+#include <linux/debugfs.h>
#include <linux/uaccess.h>
@@ -48,6 +50,7 @@ struct brd_device {
*/
spinlock_t brd_lock;
struct radix_tree_root brd_pages;
+ u64 brd_nr_pages;
};
/*
@@ -116,6 +119,8 @@ static struct page *brd_insert_page(struct brd_device *brd, sector_t sector)
page = radix_tree_lookup(&brd->brd_pages, idx);
BUG_ON(!page);
BUG_ON(page->index != idx);
+ } else {
+ brd->brd_nr_pages++;
}
spin_unlock(&brd->brd_lock);
@@ -365,11 +370,13 @@ __setup("ramdisk_size=", ramdisk_size);
*/
static LIST_HEAD(brd_devices);
static DEFINE_MUTEX(brd_devices_mutex);
+static struct dentry *brd_debugfs_dir;
static struct brd_device *brd_alloc(int i)
{
struct brd_device *brd;
struct gendisk *disk;
+ char buf[DISK_NAME_LEN];
brd = kzalloc(sizeof(*brd), GFP_KERNEL);
if (!brd)
@@ -382,6 +389,11 @@ static struct brd_device *brd_alloc(int i)
if (!brd->brd_queue)
goto out_free_dev;
+ snprintf(buf, DISK_NAME_LEN, "ram%d", i);
+ if (!IS_ERR_OR_NULL(brd_debugfs_dir))
+ debugfs_create_u64(buf, 0444, brd_debugfs_dir,
+ &brd->brd_nr_pages);
+
/* This is so fdisk will align partitions on 4k, because of
* direct_access API needing 4k alignment, returning a PFN
* (This is only a problem on very small devices <= 4M,
@@ -397,7 +409,7 @@ static struct brd_device *brd_alloc(int i)
disk->fops = &brd_fops;
disk->private_data = brd;
disk->flags = GENHD_FL_EXT_DEVT;
- sprintf(disk->disk_name, "ram%d", i);
+ strlcpy(disk->disk_name, buf, DISK_NAME_LEN);
set_capacity(disk, rd_size * 2);
/* Tell the block layer that this is not a rotational device */
@@ -495,6 +507,8 @@ static int __init brd_init(void)
brd_check_and_reset_par();
+ brd_debugfs_dir = debugfs_create_dir("ramdisk_pages", NULL);
+
mutex_lock(&brd_devices_mutex);
for (i = 0; i < rd_nr; i++) {
brd = brd_alloc(i);
@@ -519,6 +533,8 @@ static int __init brd_init(void)
return 0;
out_free:
+ debugfs_remove_recursive(brd_debugfs_dir);
+
list_for_each_entry_safe(brd, next, &brd_devices, brd_list) {
list_del(&brd->brd_list);
brd_free(brd);
@@ -534,6 +550,8 @@ static void __exit brd_exit(void)
{
struct brd_device *brd, *next;
+ debugfs_remove_recursive(brd_debugfs_dir);
+
list_for_each_entry_safe(brd, next, &brd_devices, brd_list)
brd_del_one(brd);
diff --git a/drivers/block/drbd/drbd_int.h b/drivers/block/drbd/drbd_int.h
index 7d9cc433b758..5d9181382ce1 100644
--- a/drivers/block/drbd/drbd_int.h
+++ b/drivers/block/drbd/drbd_int.h
@@ -1324,7 +1324,7 @@ struct bm_extent {
* A followup commit may allow even bigger BIO sizes,
* once we thought that through. */
#define DRBD_MAX_BIO_SIZE (1U << 20)
-#if DRBD_MAX_BIO_SIZE > (BIO_MAX_PAGES << PAGE_SHIFT)
+#if DRBD_MAX_BIO_SIZE > (BIO_MAX_VECS << PAGE_SHIFT)
#error Architecture not supported: DRBD_MAX_BIO_SIZE > BIO_MAX_SIZE
#endif
#define DRBD_MAX_BIO_SIZE_SAFE (1U << 12) /* Works always = 4k */
diff --git a/drivers/block/drbd/drbd_interval.c b/drivers/block/drbd/drbd_interval.c
index 651bd0236a99..f07b4378388b 100644
--- a/drivers/block/drbd/drbd_interval.c
+++ b/drivers/block/drbd/drbd_interval.c
@@ -3,7 +3,7 @@
#include <linux/rbtree_augmented.h>
#include "drbd_interval.h"
-/**
+/*
* interval_end - return end of @node
*/
static inline
@@ -18,7 +18,7 @@ sector_t interval_end(struct rb_node *node)
RB_DECLARE_CALLBACKS_MAX(static, augment_callbacks,
struct drbd_interval, rb, sector_t, end, NODE_END);
-/**
+/*
* drbd_insert_interval - insert a new interval into a tree
*/
bool
@@ -56,6 +56,7 @@ drbd_insert_interval(struct rb_root *root, struct drbd_interval *this)
/**
* drbd_contains_interval - check if a tree contains a given interval
+ * @root: red black tree root
* @sector: start sector of @interval
* @interval: may not be a valid pointer
*
@@ -88,7 +89,7 @@ drbd_contains_interval(struct rb_root *root, sector_t sector,
return false;
}
-/**
+/*
* drbd_remove_interval - remove an interval from a tree
*/
void
@@ -99,6 +100,7 @@ drbd_remove_interval(struct rb_root *root, struct drbd_interval *this)
/**
* drbd_find_overlap - search for an interval overlapping with [sector, sector + size)
+ * @root: red black tree root
* @sector: start sector
* @size: size, aligned to 512 bytes
*
diff --git a/drivers/block/drbd/drbd_main.c b/drivers/block/drbd/drbd_main.c
index 25cd8a2f729d..de463773b530 100644
--- a/drivers/block/drbd/drbd_main.c
+++ b/drivers/block/drbd/drbd_main.c
@@ -125,7 +125,7 @@ struct bio_set drbd_io_bio_set;
member of struct page.
*/
struct page *drbd_pp_pool;
-spinlock_t drbd_pp_lock;
+DEFINE_SPINLOCK(drbd_pp_lock);
int drbd_pp_vacant;
wait_queue_head_t drbd_pp_wait;
@@ -268,7 +268,7 @@ void tl_restart(struct drbd_connection *connection, enum drbd_req_event what)
/**
* tl_clear() - Clears all requests and &struct drbd_tl_epoch objects out of the TL
- * @device: DRBD device.
+ * @connection: DRBD connection.
*
* This is called after the connection to the peer was lost. The storage covered
* by the requests on the transfer gets marked as our of sync. Called from the
@@ -479,7 +479,7 @@ int conn_lowest_minor(struct drbd_connection *connection)
}
#ifdef CONFIG_SMP
-/**
+/*
* drbd_calc_cpu_mask() - Generate CPU masks, spread over all CPUs
*
* Forces all threads of a resource onto the same CPU. This is beneficial for
@@ -518,7 +518,6 @@ static void drbd_calc_cpu_mask(cpumask_var_t *cpu_mask)
/**
* drbd_thread_current_set_cpu() - modifies the cpu mask of the _current_ thread
- * @device: DRBD device.
* @thi: drbd_thread object
*
* call in the "main loop" of _all_ threads, no need for any mutex, current won't die
@@ -538,7 +537,7 @@ void drbd_thread_current_set_cpu(struct drbd_thread *thi)
#define drbd_calc_cpu_mask(A) ({})
#endif
-/**
+/*
* drbd_header_size - size of a packet header
*
* The header size is a multiple of 8, so any payload following the header is
@@ -1193,7 +1192,7 @@ static int fill_bitmap_rle_bits(struct drbd_device *device,
return len;
}
-/**
+/*
* send_bitmap_rle_or_plain
*
* Return 0 when done, 1 when another iteration is needed, and a negative error
@@ -1324,11 +1323,11 @@ void drbd_send_b_ack(struct drbd_connection *connection, u32 barrier_nr, u32 set
/**
* _drbd_send_ack() - Sends an ack packet
- * @device: DRBD device.
- * @cmd: Packet command code.
- * @sector: sector, needs to be in big endian byte order
- * @blksize: size in byte, needs to be in big endian byte order
- * @block_id: Id, big endian byte order
+ * @peer_device: DRBD peer device.
+ * @cmd: Packet command code.
+ * @sector: sector, needs to be in big endian byte order
+ * @blksize: size in byte, needs to be in big endian byte order
+ * @block_id: Id, big endian byte order
*/
static int _drbd_send_ack(struct drbd_peer_device *peer_device, enum drbd_packet cmd,
u64 sector, u32 blksize, u64 block_id)
@@ -1370,9 +1369,9 @@ void drbd_send_ack_rp(struct drbd_peer_device *peer_device, enum drbd_packet cmd
/**
* drbd_send_ack() - Sends an ack packet
- * @device: DRBD device
- * @cmd: packet command code
- * @peer_req: peer request
+ * @peer_device: DRBD peer device
+ * @cmd: packet command code
+ * @peer_req: peer request
*/
int drbd_send_ack(struct drbd_peer_device *peer_device, enum drbd_packet cmd,
struct drbd_peer_request *peer_req)
@@ -1882,7 +1881,7 @@ int drbd_send(struct drbd_connection *connection, struct socket *sock,
return sent;
}
-/**
+/*
* drbd_send_all - Send an entire buffer
*
* Returns 0 upon success and a negative error value otherwise.
@@ -2161,9 +2160,6 @@ static int drbd_create_mempools(void)
if (ret)
goto Enomem;
- /* drbd's page pool */
- spin_lock_init(&drbd_pp_lock);
-
for (i = 0; i < number; i++) {
page = alloc_page(GFP_HIGHUSER);
if (!page)
@@ -3509,6 +3505,7 @@ static int w_bitmap_io(struct drbd_work *w, int unused)
* @io_fn: IO callback to be called when bitmap IO is possible
* @done: callback to be called after the bitmap IO was performed
* @why: Descriptive text of the reason for doing the IO
+ * @flags: Bitmap flags
*
* While IO on the bitmap happens we freeze application IO thus we ensure
* that drbd_set_out_of_sync() can not be called. This function MAY ONLY be
@@ -3554,6 +3551,7 @@ void drbd_queue_bitmap_io(struct drbd_device *device,
* @device: DRBD device.
* @io_fn: IO callback to be called when bitmap IO is possible
* @why: Descriptive text of the reason for doing the IO
+ * @flags: Bitmap flags
*
* freezes application IO while that the actual IO operations runs. This
* functions MAY NOT be called from worker context.
@@ -3657,7 +3655,6 @@ const char *cmdname(enum drbd_packet cmd)
[P_RS_CANCEL] = "RSCancel",
[P_CONN_ST_CHG_REQ] = "conn_st_chg_req",
[P_CONN_ST_CHG_REPLY] = "conn_st_chg_reply",
- [P_RETRY_WRITE] = "retry_write",
[P_PROTOCOL_UPDATE] = "protocol_update",
[P_RS_THIN_REQ] = "rs_thin_req",
[P_RS_DEALLOCATED] = "rs_deallocated",
diff --git a/drivers/block/drbd/drbd_nl.c b/drivers/block/drbd/drbd_nl.c
index bf7de4c7b96c..e7d0e637e632 100644
--- a/drivers/block/drbd/drbd_nl.c
+++ b/drivers/block/drbd/drbd_nl.c
@@ -790,9 +790,11 @@ int drbd_adm_set_role(struct sk_buff *skb, struct genl_info *info)
mutex_lock(&adm_ctx.resource->adm_mutex);
if (info->genlhdr->cmd == DRBD_ADM_PRIMARY)
- retcode = drbd_set_role(adm_ctx.device, R_PRIMARY, parms.assume_uptodate);
+ retcode = (enum drbd_ret_code)drbd_set_role(adm_ctx.device,
+ R_PRIMARY, parms.assume_uptodate);
else
- retcode = drbd_set_role(adm_ctx.device, R_SECONDARY, 0);
+ retcode = (enum drbd_ret_code)drbd_set_role(adm_ctx.device,
+ R_SECONDARY, 0);
mutex_unlock(&adm_ctx.resource->adm_mutex);
genl_lock();
@@ -916,7 +918,7 @@ void drbd_resume_io(struct drbd_device *device)
wake_up(&device->misc_wait);
}
-/**
+/*
* drbd_determine_dev_size() - Sets the right device size obeying all constraints
* @device: DRBD device.
*
@@ -1134,7 +1136,7 @@ drbd_new_dev_size(struct drbd_device *device, struct drbd_backing_dev *bdev,
return size;
}
-/**
+/*
* drbd_check_al_size() - Ensures that the AL is of the right size
* @device: DRBD device.
*
@@ -1962,7 +1964,7 @@ int drbd_adm_attach(struct sk_buff *skb, struct genl_info *info)
drbd_flush_workqueue(&connection->sender_work);
rv = _drbd_request_state(device, NS(disk, D_ATTACHING), CS_VERBOSE);
- retcode = rv; /* FIXME: Type mismatch. */
+ retcode = (enum drbd_ret_code)rv;
drbd_resume_io(device);
if (rv < SS_SUCCESS)
goto fail;
@@ -2687,7 +2689,8 @@ int drbd_adm_connect(struct sk_buff *skb, struct genl_info *info)
}
rcu_read_unlock();
- retcode = conn_request_state(connection, NS(conn, C_UNCONNECTED), CS_VERBOSE);
+ retcode = (enum drbd_ret_code)conn_request_state(connection,
+ NS(conn, C_UNCONNECTED), CS_VERBOSE);
conn_reconfig_done(connection);
mutex_unlock(&adm_ctx.resource->adm_mutex);
@@ -2800,7 +2803,7 @@ int drbd_adm_disconnect(struct sk_buff *skb, struct genl_info *info)
mutex_lock(&adm_ctx.resource->adm_mutex);
rv = conn_try_disconnect(connection, parms.force_disconnect);
if (rv < SS_SUCCESS)
- retcode = rv; /* FIXME: Type mismatch. */
+ retcode = (enum drbd_ret_code)rv;
else
retcode = NO_ERROR;
mutex_unlock(&adm_ctx.resource->adm_mutex);
diff --git a/drivers/block/drbd/drbd_receiver.c b/drivers/block/drbd/drbd_receiver.c
index c3f09a122f20..69284ebba786 100644
--- a/drivers/block/drbd/drbd_receiver.c
+++ b/drivers/block/drbd/drbd_receiver.c
@@ -242,9 +242,9 @@ static void conn_reclaim_net_peer_reqs(struct drbd_connection *connection)
/**
* drbd_alloc_pages() - Returns @number pages, retries forever (or until signalled)
- * @device: DRBD device.
- * @number: number of pages requested
- * @retry: whether to retry, if not enough pages are available right now
+ * @peer_device: DRBD device.
+ * @number: number of pages requested
+ * @retry: whether to retry, if not enough pages are available right now
*
* Tries to allocate number pages, first from our own page pool, then from
* the kernel.
@@ -1352,7 +1352,7 @@ static void drbd_flush(struct drbd_connection *connection)
/**
* drbd_may_finish_epoch() - Applies an epoch_event to the epoch's state, eventually finishes it.
- * @device: DRBD device.
+ * @connection: DRBD connection.
* @epoch: Epoch object.
* @ev: Epoch event.
*/
@@ -1441,9 +1441,8 @@ max_allowed_wo(struct drbd_backing_dev *bdev, enum write_ordering_e wo)
return wo;
}
-/**
+/*
* drbd_bump_write_ordering() - Fall back to an other write ordering method
- * @connection: DRBD connection.
* @wo: Write ordering method to try.
*/
void drbd_bump_write_ordering(struct drbd_resource *resource, struct drbd_backing_dev *bdev,
@@ -1619,11 +1618,10 @@ static void drbd_issue_peer_wsame(struct drbd_device *device,
}
-/**
+/*
* drbd_submit_peer_request()
* @device: DRBD device.
* @peer_req: peer request
- * @rw: flag field, see bio->bi_opf
*
* May spread the pages to multiple bios,
* depending on bio_add_page restrictions.
@@ -3048,7 +3046,7 @@ out_free_e:
return -EIO;
}
-/**
+/*
* drbd_asb_recover_0p - Recover after split-brain with no remaining primaries
*/
static int drbd_asb_recover_0p(struct drbd_peer_device *peer_device) __must_hold(local)
@@ -3131,7 +3129,7 @@ static int drbd_asb_recover_0p(struct drbd_peer_device *peer_device) __must_hold
return rv;
}
-/**
+/*
* drbd_asb_recover_1p - Recover after split-brain with one remaining primary
*/
static int drbd_asb_recover_1p(struct drbd_peer_device *peer_device) __must_hold(local)
@@ -3188,7 +3186,7 @@ static int drbd_asb_recover_1p(struct drbd_peer_device *peer_device) __must_hold
return rv;
}
-/**
+/*
* drbd_asb_recover_2p - Recover after split-brain with two remaining primaries
*/
static int drbd_asb_recover_2p(struct drbd_peer_device *peer_device) __must_hold(local)
@@ -4672,7 +4670,7 @@ static int receive_sync_uuid(struct drbd_connection *connection, struct packet_i
return 0;
}
-/**
+/*
* receive_bitmap_plain
*
* Return 0 when done, 1 when another iteration is needed, and a negative error
@@ -4724,7 +4722,7 @@ static int dcbp_get_pad_bits(struct p_compressed_bm *p)
return (p->encoding >> 4) & 0x7;
}
-/**
+/*
* recv_bm_rle_bits
*
* Return 0 when done, 1 when another iteration is needed, and a negative error
@@ -4793,7 +4791,7 @@ recv_bm_rle_bits(struct drbd_peer_device *peer_device,
return (s != c->bm_bits);
}
-/**
+/*
* decode_bitmap_c
*
* Return 0 when done, 1 when another iteration is needed, and a negative error
@@ -5865,6 +5863,7 @@ static int got_NegRSDReply(struct drbd_connection *connection, struct packet_inf
switch (pi->cmd) {
case P_NEG_RS_DREPLY:
drbd_rs_failed_io(device, sector, size);
+ break;
case P_RS_CANCEL:
break;
default:
diff --git a/drivers/block/drbd/drbd_req.c b/drivers/block/drbd/drbd_req.c
index 9398c2c2cb2d..13beb98a7c5a 100644
--- a/drivers/block/drbd/drbd_req.c
+++ b/drivers/block/drbd/drbd_req.c
@@ -753,6 +753,7 @@ int __req_mod(struct drbd_request *req, enum drbd_req_event what,
case WRITE_ACKED_BY_PEER_AND_SIS:
req->rq_state |= RQ_NET_SIS;
+ fallthrough;
case WRITE_ACKED_BY_PEER:
/* Normal operation protocol C: successfully written on peer.
* During resync, even in protocol != C,
diff --git a/drivers/block/drbd/drbd_state.c b/drivers/block/drbd/drbd_state.c
index 0067d328f0b5..b8a27818ab3f 100644
--- a/drivers/block/drbd/drbd_state.c
+++ b/drivers/block/drbd/drbd_state.c
@@ -904,9 +904,9 @@ out:
* is_valid_soft_transition() - Returns an SS_ error code if the state transition is not possible
* This function limits state transitions that may be declined by DRBD. I.e.
* user requests (aka soft transitions).
- * @device: DRBD device.
- * @ns: new state.
* @os: old state.
+ * @ns: new state.
+ * @connection: DRBD connection.
*/
static enum drbd_state_rv
is_valid_soft_transition(union drbd_state os, union drbd_state ns, struct drbd_connection *connection)
@@ -1044,7 +1044,7 @@ static void print_sanitize_warnings(struct drbd_device *device, enum sanitize_st
* @device: DRBD device.
* @os: old state.
* @ns: new state.
- * @warn_sync_abort:
+ * @warn: placeholder for returned state warning.
*
* When we loose connection, we have to set the state of the peers disk (pdsk)
* to D_UNKNOWN. This rule and many more along those lines are in this function.
@@ -1696,6 +1696,7 @@ static bool lost_contact_to_peer_data(enum drbd_disk_state os, enum drbd_disk_st
* @os: old state.
* @ns: new state.
* @flags: Flags
+ * @state_change: state change to broadcast
*/
static void after_state_ch(struct drbd_device *device, union drbd_state os,
union drbd_state ns, enum chg_state_flags flags,
diff --git a/drivers/block/floppy.c b/drivers/block/floppy.c
index 0b71292d9d5a..8a9d22207c59 100644
--- a/drivers/block/floppy.c
+++ b/drivers/block/floppy.c
@@ -145,8 +145,6 @@
* Better audit of register_blkdev.
*/
-#undef FLOPPY_SILENT_DCL_CLEAR
-
#define REALLY_SLOW_IO
#define DEBUGT 2
@@ -2399,11 +2397,10 @@ static void rw_interrupt(void)
probing = 0;
}
- if (CT(raw_cmd->cmd[COMMAND]) != FD_READ ||
- raw_cmd->kernel_data == bio_data(current_req->bio)) {
+ if (CT(raw_cmd->cmd[COMMAND]) != FD_READ) {
/* transfer directly from buffer */
cont->done(1);
- } else if (CT(raw_cmd->cmd[COMMAND]) == FD_READ) {
+ } else {
buffer_track = raw_cmd->track;
buffer_drive = current_drive;
INFBOUND(buffer_max, nr_sectors + fsector_t);
@@ -2411,27 +2408,6 @@ static void rw_interrupt(void)
cont->redo();
}
-/* Compute maximal contiguous buffer size. */
-static int buffer_chain_size(void)
-{
- struct bio_vec bv;
- int size;
- struct req_iterator iter;
- char *base;
-
- base = bio_data(current_req->bio);
- size = 0;
-
- rq_for_each_segment(bv, current_req, iter) {
- if (page_address(bv.bv_page) + bv.bv_offset != base + size)
- break;
-
- size += bv.bv_len;
- }
-
- return size >> 9;
-}
-
/* Compute the maximal transfer size */
static int transfer_size(int ssize, int max_sector, int max_size)
{
@@ -2453,7 +2429,6 @@ static void copy_buffer(int ssize, int max_sector, int max_sector_2)
{
int remaining; /* number of transferred 512-byte sectors */
struct bio_vec bv;
- char *buffer;
char *dma_buffer;
int size;
struct req_iterator iter;
@@ -2492,8 +2467,6 @@ static void copy_buffer(int ssize, int max_sector, int max_sector_2)
size = bv.bv_len;
SUPBOUND(size, remaining);
-
- buffer = page_address(bv.bv_page) + bv.bv_offset;
if (dma_buffer + size >
floppy_track_buffer + (max_buffer_sectors << 10) ||
dma_buffer < floppy_track_buffer) {
@@ -2509,13 +2482,13 @@ static void copy_buffer(int ssize, int max_sector, int max_sector_2)
pr_info("write\n");
break;
}
- if (((unsigned long)buffer) % 512)
- DPRINT("%p buffer not aligned\n", buffer);
if (CT(raw_cmd->cmd[COMMAND]) == FD_READ)
- memcpy(buffer, dma_buffer, size);
+ memcpy_to_page(bv.bv_page, bv.bv_offset, dma_buffer,
+ size);
else
- memcpy(dma_buffer, buffer, size);
+ memcpy_from_page(dma_buffer, bv.bv_page, bv.bv_offset,
+ size);
remaining -= size;
dma_buffer += size;
@@ -2690,54 +2663,6 @@ static int make_raw_rw_request(void)
raw_cmd->flags &= ~FD_RAW_WRITE;
raw_cmd->flags |= FD_RAW_READ;
raw_cmd->cmd[COMMAND] = FM_MODE(_floppy, FD_READ);
- } else if ((unsigned long)bio_data(current_req->bio) < MAX_DMA_ADDRESS) {
- unsigned long dma_limit;
- int direct, indirect;
-
- indirect =
- transfer_size(ssize, max_sector,
- max_buffer_sectors * 2) - fsector_t;
-
- /*
- * Do NOT use minimum() here---MAX_DMA_ADDRESS is 64 bits wide
- * on a 64 bit machine!
- */
- max_size = buffer_chain_size();
- dma_limit = (MAX_DMA_ADDRESS -
- ((unsigned long)bio_data(current_req->bio))) >> 9;
- if ((unsigned long)max_size > dma_limit)
- max_size = dma_limit;
- /* 64 kb boundaries */
- if (CROSS_64KB(bio_data(current_req->bio), max_size << 9))
- max_size = (K_64 -
- ((unsigned long)bio_data(current_req->bio)) %
- K_64) >> 9;
- direct = transfer_size(ssize, max_sector, max_size) - fsector_t;
- /*
- * We try to read tracks, but if we get too many errors, we
- * go back to reading just one sector at a time.
- *
- * This means we should be able to read a sector even if there
- * are other bad sectors on this track.
- */
- if (!direct ||
- (indirect * 2 > direct * 3 &&
- *errors < drive_params[current_drive].max_errors.read_track &&
- ((!probing ||
- (drive_params[current_drive].read_track & (1 << drive_state[current_drive].probed_format)))))) {
- max_size = blk_rq_sectors(current_req);
- } else {
- raw_cmd->kernel_data = bio_data(current_req->bio);
- raw_cmd->length = current_count_sectors << 9;
- if (raw_cmd->length == 0) {
- DPRINT("%s: zero dma transfer attempted\n", __func__);
- DPRINT("indirect=%d direct=%d fsector_t=%d\n",
- indirect, direct, fsector_t);
- return 0;
- }
- virtualdmabug_workaround();
- return 2;
- }
}
if (CT(raw_cmd->cmd[COMMAND]) == FD_READ)
@@ -2781,19 +2706,17 @@ static int make_raw_rw_request(void)
raw_cmd->length = ((raw_cmd->length - 1) | (ssize - 1)) + 1;
raw_cmd->length <<= 9;
if ((raw_cmd->length < current_count_sectors << 9) ||
- (raw_cmd->kernel_data != bio_data(current_req->bio) &&
- CT(raw_cmd->cmd[COMMAND]) == FD_WRITE &&
+ (CT(raw_cmd->cmd[COMMAND]) == FD_WRITE &&
(aligned_sector_t + (raw_cmd->length >> 9) > buffer_max ||
aligned_sector_t < buffer_min)) ||
raw_cmd->length % (128 << raw_cmd->cmd[SIZECODE]) ||
raw_cmd->length <= 0 || current_count_sectors <= 0) {
DPRINT("fractionary current count b=%lx s=%lx\n",
raw_cmd->length, current_count_sectors);
- if (raw_cmd->kernel_data != bio_data(current_req->bio))
- pr_info("addr=%d, length=%ld\n",
- (int)((raw_cmd->kernel_data -
- floppy_track_buffer) >> 9),
- current_count_sectors);
+ pr_info("addr=%d, length=%ld\n",
+ (int)((raw_cmd->kernel_data -
+ floppy_track_buffer) >> 9),
+ current_count_sectors);
pr_info("st=%d ast=%d mse=%d msi=%d\n",
fsector_t, aligned_sector_t, max_sector, max_size);
pr_info("ssize=%x SIZECODE=%d\n", ssize, raw_cmd->cmd[SIZECODE]);
@@ -2807,31 +2730,21 @@ static int make_raw_rw_request(void)
return 0;
}
- if (raw_cmd->kernel_data != bio_data(current_req->bio)) {
- if (raw_cmd->kernel_data < floppy_track_buffer ||
- current_count_sectors < 0 ||
- raw_cmd->length < 0 ||
- raw_cmd->kernel_data + raw_cmd->length >
- floppy_track_buffer + (max_buffer_sectors << 10)) {
- DPRINT("buffer overrun in schedule dma\n");
- pr_info("fsector_t=%d buffer_min=%d current_count=%ld\n",
- fsector_t, buffer_min, raw_cmd->length >> 9);
- pr_info("current_count_sectors=%ld\n",
- current_count_sectors);
- if (CT(raw_cmd->cmd[COMMAND]) == FD_READ)
- pr_info("read\n");
- if (CT(raw_cmd->cmd[COMMAND]) == FD_WRITE)
- pr_info("write\n");
- return 0;
- }
- } else if (raw_cmd->length > blk_rq_bytes(current_req) ||
- current_count_sectors > blk_rq_sectors(current_req)) {
- DPRINT("buffer overrun in direct transfer\n");
+ if (raw_cmd->kernel_data < floppy_track_buffer ||
+ current_count_sectors < 0 ||
+ raw_cmd->length < 0 ||
+ raw_cmd->kernel_data + raw_cmd->length >
+ floppy_track_buffer + (max_buffer_sectors << 10)) {
+ DPRINT("buffer overrun in schedule dma\n");
+ pr_info("fsector_t=%d buffer_min=%d current_count=%ld\n",
+ fsector_t, buffer_min, raw_cmd->length >> 9);
+ pr_info("current_count_sectors=%ld\n",
+ current_count_sectors);
+ if (CT(raw_cmd->cmd[COMMAND]) == FD_READ)
+ pr_info("read\n");
+ if (CT(raw_cmd->cmd[COMMAND]) == FD_WRITE)
+ pr_info("write\n");
return 0;
- } else if (raw_cmd->length < current_count_sectors << 9) {
- DPRINT("more sectors than bytes\n");
- pr_info("bytes=%ld\n", raw_cmd->length >> 9);
- pr_info("sectors=%ld\n", current_count_sectors);
}
if (raw_cmd->length == 0) {
DPRINT("zero dma transfer attempted from make_raw_request\n");
@@ -3073,8 +2986,6 @@ static const char *drive_name(int type, int drive)
/* raw commands */
static void raw_cmd_done(int flag)
{
- int i;
-
if (!flag) {
raw_cmd->flags |= FD_RAW_FAILURE;
raw_cmd->flags |= FD_RAW_HARDFAILURE;
@@ -3082,8 +2993,7 @@ static void raw_cmd_done(int flag)
raw_cmd->reply_count = inr;
if (raw_cmd->reply_count > FD_RAW_REPLY_SIZE)
raw_cmd->reply_count = 0;
- for (i = 0; i < raw_cmd->reply_count; i++)
- raw_cmd->reply[i] = reply_buffer[i];
+ memcpy(raw_cmd->reply, reply_buffer, raw_cmd->reply_count);
if (raw_cmd->flags & (FD_RAW_READ | FD_RAW_WRITE)) {
unsigned long flags;
@@ -3175,7 +3085,6 @@ static int raw_cmd_copyin(int cmd, void __user *param,
{
struct floppy_raw_cmd *ptr;
int ret;
- int i;
*rcmd = NULL;
@@ -3194,8 +3103,7 @@ loop:
if (ptr->cmd_count > FD_RAW_CMD_FULLSIZE)
return -EINVAL;
- for (i = 0; i < FD_RAW_REPLY_SIZE; i++)
- ptr->reply[i] = 0;
+ memset(ptr->reply, 0, FD_RAW_REPLY_SIZE);
ptr->resultcode = 0;
if (ptr->flags & (FD_RAW_READ | FD_RAW_WRITE)) {
@@ -4317,7 +4225,7 @@ static char __init get_fdc_version(int fdc)
r = result(fdc);
if (r <= 0x00)
return FDC_NONE; /* No FDC present ??? */
- if ((r == 1) && (reply_buffer[0] == 0x80)) {
+ if ((r == 1) && (reply_buffer[ST0] == 0x80)) {
pr_info("FDC %d is an 8272A\n", fdc);
return FDC_8272A; /* 8272a/765 don't know DUMPREGS */
}
@@ -4342,12 +4250,12 @@ static char __init get_fdc_version(int fdc)
output_byte(fdc, FD_UNLOCK);
r = result(fdc);
- if ((r == 1) && (reply_buffer[0] == 0x80)) {
+ if ((r == 1) && (reply_buffer[ST0] == 0x80)) {
pr_info("FDC %d is a pre-1991 82077\n", fdc);
return FDC_82077_ORIG; /* Pre-1991 82077, doesn't know
* LOCK/UNLOCK */
}
- if ((r != 1) || (reply_buffer[0] != 0x00)) {
+ if ((r != 1) || (reply_buffer[ST0] != 0x00)) {
pr_info("FDC %d init: UNLOCK: unexpected return of %d bytes.\n",
fdc, r);
return FDC_UNKNOWN;
@@ -4359,11 +4267,11 @@ static char __init get_fdc_version(int fdc)
fdc, r);
return FDC_UNKNOWN;
}
- if (reply_buffer[0] == 0x80) {
+ if (reply_buffer[ST0] == 0x80) {
pr_info("FDC %d is a post-1991 82077\n", fdc);
return FDC_82077; /* Revised 82077AA passes all the tests */
}
- switch (reply_buffer[0] >> 5) {
+ switch (reply_buffer[ST0] >> 5) {
case 0x0:
/* Either a 82078-1 or a 82078SL running at 5Volt */
pr_info("FDC %d is an 82078.\n", fdc);
@@ -4379,7 +4287,7 @@ static char __init get_fdc_version(int fdc)
return FDC_87306;
default:
pr_info("FDC %d init: 82078 variant with unknown PARTID=%d.\n",
- fdc, reply_buffer[0] >> 5);
+ fdc, reply_buffer[ST0] >> 5);
return FDC_82078_UNKN;
}
} /* get_fdc_version */
@@ -4597,7 +4505,6 @@ static int floppy_alloc_disk(unsigned int drive, unsigned int type)
return err;
}
- blk_queue_bounce_limit(disk->queue, BLK_BOUNCE_HIGH);
blk_queue_max_hw_sectors(disk->queue, 64);
disk->major = FLOPPY_MAJOR;
disk->first_minor = TOMINOR(drive) | (type << 2);
@@ -5091,7 +4998,6 @@ module_param(floppy, charp, 0);
module_param(FLOPPY_IRQ, int, 0);
module_param(FLOPPY_DMA, int, 0);
MODULE_AUTHOR("Alain L. Knaff");
-MODULE_SUPPORTED_DEVICE("fd");
MODULE_LICENSE("GPL");
/* This doesn't actually get used other than for module information */
diff --git a/drivers/block/loop.c b/drivers/block/loop.c
index a370cde3ddd4..d58d68f3c7cd 100644
--- a/drivers/block/loop.c
+++ b/drivers/block/loop.c
@@ -53,6 +53,7 @@
#include <linux/moduleparam.h>
#include <linux/sched.h>
#include <linux/fs.h>
+#include <linux/pagemap.h>
#include <linux/file.h>
#include <linux/stat.h>
#include <linux/errno.h>
diff --git a/drivers/block/mtip32xx/mtip32xx.c b/drivers/block/mtip32xx/mtip32xx.c
index 3be0dbc674bd..589cb0f1e030 100644
--- a/drivers/block/mtip32xx/mtip32xx.c
+++ b/drivers/block/mtip32xx/mtip32xx.c
@@ -95,9 +95,9 @@
/* Device instance number, incremented each time a device is probed. */
static int instance;
-static struct list_head online_list;
-static struct list_head removing_list;
-static spinlock_t dev_lock;
+static LIST_HEAD(online_list);
+static LIST_HEAD(removing_list);
+static DEFINE_SPINLOCK(dev_lock);
/*
* Global variable used to hold the major block device number
@@ -1213,7 +1213,7 @@ static int mtip_standby_immediate(struct mtip_port *port)
{
int rv;
struct host_to_dev_fis fis;
- unsigned long start;
+ unsigned long __maybe_unused start;
unsigned int timeout;
/* Build the FIS. */
@@ -4363,11 +4363,6 @@ static int __init mtip_init(void)
pr_info(MTIP_DRV_NAME " Version " MTIP_DRV_VERSION "\n");
- spin_lock_init(&dev_lock);
-
- INIT_LIST_HEAD(&online_list);
- INIT_LIST_HEAD(&removing_list);
-
/* Allocate a major block device number to use with this driver. */
error = register_blkdev(0, MTIP_DRV_NAME);
if (error <= 0) {
diff --git a/drivers/block/nbd.c b/drivers/block/nbd.c
index 4ff71b579cfc..45d2c28c8fc8 100644
--- a/drivers/block/nbd.c
+++ b/drivers/block/nbd.c
@@ -1980,7 +1980,8 @@ static void nbd_disconnect_and_put(struct nbd_device *nbd)
* config ref and try to destroy the workqueue from inside the work
* queue.
*/
- flush_workqueue(nbd->recv_workq);
+ if (nbd->recv_workq)
+ flush_workqueue(nbd->recv_workq);
if (test_and_clear_bit(NBD_RT_HAS_CONFIG_REF,
&nbd->config->runtime_flags))
nbd_config_put(nbd);
@@ -2014,12 +2015,11 @@ static int nbd_genl_disconnect(struct sk_buff *skb, struct genl_info *info)
return -EINVAL;
}
mutex_unlock(&nbd_index_mutex);
- if (!refcount_inc_not_zero(&nbd->config_refs)) {
- nbd_put(nbd);
- return 0;
- }
+ if (!refcount_inc_not_zero(&nbd->config_refs))
+ goto put_nbd;
nbd_disconnect_and_put(nbd);
nbd_config_put(nbd);
+put_nbd:
nbd_put(nbd);
return 0;
}
diff --git a/drivers/block/null_blk/main.c b/drivers/block/null_blk/main.c
index d6c821d48090..5f006d9e1472 100644
--- a/drivers/block/null_blk/main.c
+++ b/drivers/block/null_blk/main.c
@@ -84,6 +84,10 @@ enum {
NULL_Q_MQ = 2,
};
+static bool g_virt_boundary = false;
+module_param_named(virt_boundary, g_virt_boundary, bool, 0444);
+MODULE_PARM_DESC(virt_boundary, "Require a virtual boundary for the device. Default: False");
+
static int g_no_sched;
module_param_named(no_sched, g_no_sched, int, 0444);
MODULE_PARM_DESC(no_sched, "No io scheduler");
@@ -366,6 +370,7 @@ NULLB_DEVICE_ATTR(zone_capacity, ulong, NULL);
NULLB_DEVICE_ATTR(zone_nr_conv, uint, NULL);
NULLB_DEVICE_ATTR(zone_max_open, uint, NULL);
NULLB_DEVICE_ATTR(zone_max_active, uint, NULL);
+NULLB_DEVICE_ATTR(virt_boundary, bool, NULL);
static ssize_t nullb_device_power_show(struct config_item *item, char *page)
{
@@ -486,6 +491,7 @@ static struct configfs_attribute *nullb_device_attrs[] = {
&nullb_device_attr_zone_nr_conv,
&nullb_device_attr_zone_max_open,
&nullb_device_attr_zone_max_active,
+ &nullb_device_attr_virt_boundary,
NULL,
};
@@ -539,7 +545,7 @@ nullb_group_drop_item(struct config_group *group, struct config_item *item)
static ssize_t memb_group_features_show(struct config_item *item, char *page)
{
return snprintf(page, PAGE_SIZE,
- "memory_backed,discard,bandwidth,cache,badblocks,zoned,zone_size,zone_capacity,zone_nr_conv,zone_max_open,zone_max_active,blocksize,max_sectors\n");
+ "memory_backed,discard,bandwidth,cache,badblocks,zoned,zone_size,zone_capacity,zone_nr_conv,zone_max_open,zone_max_active,blocksize,max_sectors,virt_boundary\n");
}
CONFIGFS_ATTR_RO(memb_group_, features);
@@ -605,6 +611,7 @@ static struct nullb_device *null_alloc_dev(void)
dev->zone_nr_conv = g_zone_nr_conv;
dev->zone_max_open = g_zone_max_open;
dev->zone_max_active = g_zone_max_active;
+ dev->virt_boundary = g_virt_boundary;
return dev;
}
@@ -1369,10 +1376,13 @@ static blk_status_t null_handle_cmd(struct nullb_cmd *cmd, sector_t sector,
}
if (dev->zoned)
- cmd->error = null_process_zoned_cmd(cmd, op,
- sector, nr_sectors);
+ sts = null_process_zoned_cmd(cmd, op, sector, nr_sectors);
else
- cmd->error = null_process_cmd(cmd, op, sector, nr_sectors);
+ sts = null_process_cmd(cmd, op, sector, nr_sectors);
+
+ /* Do not overwrite errors (e.g. timeout errors) */
+ if (cmd->error == BLK_STS_OK)
+ cmd->error = sts;
out:
nullb_complete_cmd(cmd);
@@ -1451,8 +1461,20 @@ static bool should_requeue_request(struct request *rq)
static enum blk_eh_timer_return null_timeout_rq(struct request *rq, bool res)
{
+ struct nullb_cmd *cmd = blk_mq_rq_to_pdu(rq);
+
pr_info("rq %p timed out\n", rq);
- blk_mq_complete_request(rq);
+
+ /*
+ * If the device is marked as blocking (i.e. memory backed or zoned
+ * device), the submission path may be blocked waiting for resources
+ * and cause real timeouts. For these real timeouts, the submission
+ * path will complete the request using blk_mq_complete_request().
+ * Only fake timeouts need to execute blk_mq_complete_request() here.
+ */
+ cmd->error = BLK_STS_TIMEOUT;
+ if (cmd->fake_timeout)
+ blk_mq_complete_request(rq);
return BLK_EH_DONE;
}
@@ -1473,6 +1495,7 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx,
cmd->rq = bd->rq;
cmd->error = BLK_STS_OK;
cmd->nq = nq;
+ cmd->fake_timeout = should_timeout_request(bd->rq);
blk_mq_start_request(bd->rq);
@@ -1489,7 +1512,7 @@ static blk_status_t null_queue_rq(struct blk_mq_hw_ctx *hctx,
return BLK_STS_OK;
}
}
- if (should_timeout_request(bd->rq))
+ if (cmd->fake_timeout)
return BLK_STS_OK;
return null_handle_cmd(cmd, sector, nr_sectors, req_op(bd->rq));
@@ -1880,6 +1903,9 @@ static int null_add_dev(struct nullb_device *dev)
BLK_DEF_MAX_SECTORS);
blk_queue_max_hw_sectors(nullb->q, dev->max_sectors);
+ if (dev->virt_boundary)
+ blk_queue_virt_boundary(nullb->q, PAGE_SIZE - 1);
+
null_config_discard(nullb);
sprintf(nullb->disk_name, "nullb%d", nullb->index);
diff --git a/drivers/block/null_blk/null_blk.h b/drivers/block/null_blk/null_blk.h
index 83504f3cc9d6..64bef125d1df 100644
--- a/drivers/block/null_blk/null_blk.h
+++ b/drivers/block/null_blk/null_blk.h
@@ -22,6 +22,7 @@ struct nullb_cmd {
blk_status_t error;
struct nullb_queue *nq;
struct hrtimer timer;
+ bool fake_timeout;
};
struct nullb_queue {
@@ -96,6 +97,7 @@ struct nullb_device {
bool memory_backed; /* if data is stored in memory */
bool discard; /* if support discard */
bool zoned; /* if device is zoned */
+ bool virt_boundary; /* virtual boundary on/off for the device */
};
struct nullb {
diff --git a/drivers/block/null_blk/zoned.c b/drivers/block/null_blk/zoned.c
index bfcab1c782b5..dae54dd1aeac 100644
--- a/drivers/block/null_blk/zoned.c
+++ b/drivers/block/null_blk/zoned.c
@@ -180,6 +180,7 @@ int null_register_zoned_dev(struct nullb *nullb)
void null_free_zoned_dev(struct nullb_device *dev)
{
kvfree(dev->zones);
+ dev->zones = NULL;
}
int null_report_zones(struct gendisk *disk, sector_t sector,
diff --git a/drivers/block/paride/pd.c b/drivers/block/paride/pd.c
index 897acda20ac8..828a45ffe0e7 100644
--- a/drivers/block/paride/pd.c
+++ b/drivers/block/paride/pd.c
@@ -859,16 +859,6 @@ static unsigned int pd_check_events(struct gendisk *p, unsigned int clearing)
return r ? DISK_EVENT_MEDIA_CHANGE : 0;
}
-static int pd_revalidate(struct gendisk *p)
-{
- struct pd_unit *disk = p->private_data;
- if (pd_special_command(disk, pd_identify) == 0)
- set_capacity(p, disk->capacity);
- else
- set_capacity(p, 0);
- return 0;
-}
-
static const struct block_device_operations pd_fops = {
.owner = THIS_MODULE,
.open = pd_open,
@@ -877,7 +867,6 @@ static const struct block_device_operations pd_fops = {
.compat_ioctl = pd_ioctl,
.getgeo = pd_getgeo,
.check_events = pd_check_events,
- .revalidate_disk= pd_revalidate
};
/* probing */
diff --git a/drivers/block/pktcdvd.c b/drivers/block/pktcdvd.c
index fc4b0f1aa86d..bd3556585122 100644
--- a/drivers/block/pktcdvd.c
+++ b/drivers/block/pktcdvd.c
@@ -1199,6 +1199,42 @@ try_next_bio:
return 1;
}
+/**
+ * bio_list_copy_data - copy contents of data buffers from one chain of bios to
+ * another
+ * @src: source bio list
+ * @dst: destination bio list
+ *
+ * Stops when it reaches the end of either the @src list or @dst list - that is,
+ * copies min(src->bi_size, dst->bi_size) bytes (or the equivalent for lists of
+ * bios).
+ */
+static void bio_list_copy_data(struct bio *dst, struct bio *src)
+{
+ struct bvec_iter src_iter = src->bi_iter;
+ struct bvec_iter dst_iter = dst->bi_iter;
+
+ while (1) {
+ if (!src_iter.bi_size) {
+ src = src->bi_next;
+ if (!src)
+ break;
+
+ src_iter = src->bi_iter;
+ }
+
+ if (!dst_iter.bi_size) {
+ dst = dst->bi_next;
+ if (!dst)
+ break;
+
+ dst_iter = dst->bi_iter;
+ }
+
+ bio_copy_data_iter(dst, &dst_iter, src, &src_iter);
+ }
+}
+
/*
* Assemble a bio to write one packet and queue the bio for processing
* by the underlying block device.
diff --git a/drivers/block/rnbd/rnbd-clt-sysfs.c b/drivers/block/rnbd/rnbd-clt-sysfs.c
index d4aa6bfc9555..324afdd63a96 100644
--- a/drivers/block/rnbd/rnbd-clt-sysfs.c
+++ b/drivers/block/rnbd/rnbd-clt-sysfs.c
@@ -34,6 +34,7 @@ enum {
RNBD_OPT_DEV_PATH = 1 << 2,
RNBD_OPT_ACCESS_MODE = 1 << 3,
RNBD_OPT_SESSNAME = 1 << 6,
+ RNBD_OPT_NR_POLL_QUEUES = 1 << 7,
};
static const unsigned int rnbd_opt_mandatory[] = {
@@ -42,12 +43,13 @@ static const unsigned int rnbd_opt_mandatory[] = {
};
static const match_table_t rnbd_opt_tokens = {
- {RNBD_OPT_PATH, "path=%s" },
- {RNBD_OPT_DEV_PATH, "device_path=%s"},
- {RNBD_OPT_DEST_PORT, "dest_port=%d" },
- {RNBD_OPT_ACCESS_MODE, "access_mode=%s"},
- {RNBD_OPT_SESSNAME, "sessname=%s" },
- {RNBD_OPT_ERR, NULL },
+ {RNBD_OPT_PATH, "path=%s" },
+ {RNBD_OPT_DEV_PATH, "device_path=%s" },
+ {RNBD_OPT_DEST_PORT, "dest_port=%d" },
+ {RNBD_OPT_ACCESS_MODE, "access_mode=%s" },
+ {RNBD_OPT_SESSNAME, "sessname=%s" },
+ {RNBD_OPT_NR_POLL_QUEUES, "nr_poll_queues=%d" },
+ {RNBD_OPT_ERR, NULL },
};
struct rnbd_map_options {
@@ -57,6 +59,7 @@ struct rnbd_map_options {
char *pathname;
u16 *dest_port;
enum rnbd_access_mode *access_mode;
+ u32 *nr_poll_queues;
};
static int rnbd_clt_parse_map_options(const char *buf, size_t max_path_cnt,
@@ -68,7 +71,7 @@ static int rnbd_clt_parse_map_options(const char *buf, size_t max_path_cnt,
int opt_mask = 0;
int token;
int ret = -EINVAL;
- int i, dest_port;
+ int i, dest_port, nr_poll_queues;
int p_cnt = 0;
options = kstrdup(buf, GFP_KERNEL);
@@ -96,7 +99,7 @@ static int rnbd_clt_parse_map_options(const char *buf, size_t max_path_cnt,
kfree(p);
goto out;
}
- strlcpy(opt->sessname, p, NAME_MAX);
+ strscpy(opt->sessname, p, NAME_MAX);
kfree(p);
break;
@@ -139,7 +142,7 @@ static int rnbd_clt_parse_map_options(const char *buf, size_t max_path_cnt,
kfree(p);
goto out;
}
- strlcpy(opt->pathname, p, NAME_MAX);
+ strscpy(opt->pathname, p, NAME_MAX);
kfree(p);
break;
@@ -178,6 +181,19 @@ static int rnbd_clt_parse_map_options(const char *buf, size_t max_path_cnt,
kfree(p);
break;
+ case RNBD_OPT_NR_POLL_QUEUES:
+ if (match_int(args, &nr_poll_queues) || nr_poll_queues < -1 ||
+ nr_poll_queues > (int)nr_cpu_ids) {
+ pr_err("bad nr_poll_queues parameter '%d'\n",
+ nr_poll_queues);
+ ret = -EINVAL;
+ goto out;
+ }
+ if (nr_poll_queues == -1)
+ nr_poll_queues = nr_cpu_ids;
+ *opt->nr_poll_queues = nr_poll_queues;
+ break;
+
default:
pr_err("map_device: Unknown parameter or missing value '%s'\n",
p);
@@ -227,6 +243,19 @@ static ssize_t state_show(struct kobject *kobj,
static struct kobj_attribute rnbd_clt_state_attr = __ATTR_RO(state);
+static ssize_t nr_poll_queues_show(struct kobject *kobj,
+ struct kobj_attribute *attr, char *page)
+{
+ struct rnbd_clt_dev *dev;
+
+ dev = container_of(kobj, struct rnbd_clt_dev, kobj);
+
+ return sysfs_emit(page, "%d\n", dev->nr_poll_queues);
+}
+
+static struct kobj_attribute rnbd_clt_nr_poll_queues =
+ __ATTR_RO(nr_poll_queues);
+
static ssize_t mapping_path_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
@@ -421,6 +450,7 @@ static struct attribute *rnbd_dev_attrs[] = {
&rnbd_clt_state_attr.attr,
&rnbd_clt_session_attr.attr,
&rnbd_clt_access_mode.attr,
+ &rnbd_clt_nr_poll_queues.attr,
NULL,
};
@@ -432,10 +462,14 @@ void rnbd_clt_remove_dev_symlink(struct rnbd_clt_dev *dev)
* i.e. rnbd_clt_unmap_dev_store() leading to a sysfs warning because
* of sysfs link already was removed already.
*/
- if (dev->blk_symlink_name && try_module_get(THIS_MODULE)) {
- sysfs_remove_link(rnbd_devs_kobj, dev->blk_symlink_name);
+ if (dev->blk_symlink_name) {
+ if (try_module_get(THIS_MODULE)) {
+ sysfs_remove_link(rnbd_devs_kobj, dev->blk_symlink_name);
+ module_put(THIS_MODULE);
+ }
+ /* It should be freed always. */
kfree(dev->blk_symlink_name);
- module_put(THIS_MODULE);
+ dev->blk_symlink_name = NULL;
}
}
@@ -456,6 +490,7 @@ static int rnbd_clt_add_dev_kobj(struct rnbd_clt_dev *dev)
ret);
kobject_put(&dev->kobj);
}
+ kobject_uevent(gd_kobj, KOBJ_ONLINE);
return ret;
}
@@ -465,7 +500,7 @@ static ssize_t rnbd_clt_map_device_show(struct kobject *kobj,
char *page)
{
return scnprintf(page, PAGE_SIZE,
- "Usage: echo \"[dest_port=server port number] sessname=<name of the rtrs session> path=<[srcaddr@]dstaddr> [path=<[srcaddr@]dstaddr>] device_path=<full path on remote side> [access_mode=<ro|rw|migration>]\" > %s\n\naddr ::= [ ip:<ipv4> | ip:<ipv6> | gid:<gid> ]\n",
+ "Usage: echo \"[dest_port=server port number] sessname=<name of the rtrs session> path=<[srcaddr@]dstaddr> [path=<[srcaddr@]dstaddr>] device_path=<full path on remote side> [access_mode=<ro|rw|migration>] [nr_poll_queues=<number of queues>]\" > %s\n\naddr ::= [ ip:<ipv4> | ip:<ipv6> | gid:<gid> ]\n",
attr->attr.name);
}
@@ -475,15 +510,11 @@ static int rnbd_clt_get_path_name(struct rnbd_clt_dev *dev, char *buf,
int ret;
char pathname[NAME_MAX], *s;
- strlcpy(pathname, dev->pathname, sizeof(pathname));
+ strscpy(pathname, dev->pathname, sizeof(pathname));
while ((s = strchr(pathname, '/')))
s[0] = '!';
- ret = snprintf(buf, len, "%s", pathname);
- if (ret >= len)
- return -ENAMETOOLONG;
-
- ret = snprintf(buf, len, "%s@%s", buf, dev->sess->sessname);
+ ret = snprintf(buf, len, "%s@%s", pathname, dev->sess->sessname);
if (ret >= len)
return -ENAMETOOLONG;
@@ -537,6 +568,7 @@ static ssize_t rnbd_clt_map_device_store(struct kobject *kobj,
char sessname[NAME_MAX];
enum rnbd_access_mode access_mode = RNBD_ACCESS_RW;
u16 port_nr = RTRS_PORT;
+ u32 nr_poll_queues = 0;
struct sockaddr_storage *addrs;
struct rtrs_addr paths[6];
@@ -548,6 +580,7 @@ static ssize_t rnbd_clt_map_device_store(struct kobject *kobj,
opt.pathname = pathname;
opt.dest_port = &port_nr;
opt.access_mode = &access_mode;
+ opt.nr_poll_queues = &nr_poll_queues;
addrs = kcalloc(ARRAY_SIZE(paths) * 2, sizeof(*addrs), GFP_KERNEL);
if (!addrs)
return -ENOMEM;
@@ -561,12 +594,13 @@ static ssize_t rnbd_clt_map_device_store(struct kobject *kobj,
if (ret)
goto out;
- pr_info("Mapping device %s on session %s, (access_mode: %s)\n",
+ pr_info("Mapping device %s on session %s, (access_mode: %s, nr_poll_queues: %d)\n",
pathname, sessname,
- rnbd_access_mode_str(access_mode));
+ rnbd_access_mode_str(access_mode),
+ nr_poll_queues);
dev = rnbd_clt_map_device(sessname, paths, path_cnt, port_nr, pathname,
- access_mode);
+ access_mode, nr_poll_queues);
if (IS_ERR(dev)) {
ret = PTR_ERR(dev);
goto out;
@@ -639,13 +673,9 @@ cls_destroy:
return err;
}
-void rnbd_clt_destroy_default_group(void)
-{
- sysfs_remove_group(&rnbd_dev->kobj, &default_attr_group);
-}
-
void rnbd_clt_destroy_sysfs_files(void)
{
+ sysfs_remove_group(&rnbd_dev->kobj, &default_attr_group);
kobject_del(rnbd_devs_kobj);
kobject_put(rnbd_devs_kobj);
device_destroy(rnbd_dev_class, MKDEV(0, 0));
diff --git a/drivers/block/rnbd/rnbd-clt.c b/drivers/block/rnbd/rnbd-clt.c
index 45a470076652..c604a402cd5c 100644
--- a/drivers/block/rnbd/rnbd-clt.c
+++ b/drivers/block/rnbd/rnbd-clt.c
@@ -88,7 +88,7 @@ static int rnbd_clt_set_dev_attr(struct rnbd_clt_dev *dev,
dev->discard_alignment = le32_to_cpu(rsp->discard_alignment);
dev->secure_discard = le16_to_cpu(rsp->secure_discard);
dev->rotational = rsp->rotational;
- dev->wc = !!(rsp->cache_policy & RNBD_WRITEBACK);
+ dev->wc = !!(rsp->cache_policy & RNBD_WRITEBACK);
dev->fua = !!(rsp->cache_policy & RNBD_FUA);
dev->max_hw_sectors = sess->max_io_size / SECTOR_SIZE;
@@ -110,6 +110,7 @@ static int rnbd_clt_change_capacity(struct rnbd_clt_dev *dev,
static int process_msg_open_rsp(struct rnbd_clt_dev *dev,
struct rnbd_msg_open_rsp *rsp)
{
+ struct kobject *gd_kobj;
int err = 0;
mutex_lock(&dev->lock);
@@ -128,6 +129,8 @@ static int process_msg_open_rsp(struct rnbd_clt_dev *dev,
*/
if (dev->nsectors != nsectors)
rnbd_clt_change_capacity(dev, nsectors);
+ gd_kobj = &disk_to_dev(dev->gd)->kobj;
+ kobject_uevent(gd_kobj, KOBJ_ONLINE);
rnbd_clt_info(dev, "Device online, device remapped successfully\n");
}
err = rnbd_clt_set_dev_attr(dev, rsp);
@@ -238,7 +241,7 @@ static bool rnbd_rerun_if_needed(struct rnbd_clt_session *sess)
cpu_q = rnbd_get_cpu_qlist(sess, nxt_cpu(cpu_q->cpu))) {
if (!spin_trylock_irqsave(&cpu_q->requeue_lock, flags))
continue;
- if (unlikely(!test_bit(cpu_q->cpu, sess->cpu_queues_bm)))
+ if (!test_bit(cpu_q->cpu, sess->cpu_queues_bm))
goto unlock;
q = list_first_entry_or_null(&cpu_q->requeue_list,
typeof(*q), requeue_list);
@@ -312,14 +315,12 @@ static void rnbd_rerun_all_if_idle(struct rnbd_clt_session *sess)
static struct rtrs_permit *rnbd_get_permit(struct rnbd_clt_session *sess,
enum rtrs_clt_con_type con_type,
- int wait)
+ enum wait_type wait)
{
struct rtrs_permit *permit;
- permit = rtrs_clt_get_permit(sess->rtrs, con_type,
- wait ? RTRS_PERMIT_WAIT :
- RTRS_PERMIT_NOWAIT);
- if (likely(permit))
+ permit = rtrs_clt_get_permit(sess->rtrs, con_type, wait);
+ if (permit)
/* We have a subtle rare case here, when all permits can be
* consumed before busy counter increased. This is safe,
* because loser will get NULL as a permit, observe 0 busy
@@ -344,20 +345,17 @@ static void rnbd_put_permit(struct rnbd_clt_session *sess,
static struct rnbd_iu *rnbd_get_iu(struct rnbd_clt_session *sess,
enum rtrs_clt_con_type con_type,
- int wait)
+ enum wait_type wait)
{
struct rnbd_iu *iu;
struct rtrs_permit *permit;
iu = kzalloc(sizeof(*iu), GFP_KERNEL);
- if (!iu) {
+ if (!iu)
return NULL;
- }
- permit = rnbd_get_permit(sess, con_type,
- wait ? RTRS_PERMIT_WAIT :
- RTRS_PERMIT_NOWAIT);
- if (unlikely(!permit)) {
+ permit = rnbd_get_permit(sess, con_type, wait);
+ if (!permit) {
kfree(iu);
return NULL;
}
@@ -435,16 +433,11 @@ static void msg_conf(void *priv, int errno)
schedule_work(&iu->work);
}
-enum wait_type {
- NO_WAIT = 0,
- WAIT = 1
-};
-
static int send_usr_msg(struct rtrs_clt *rtrs, int dir,
struct rnbd_iu *iu, struct kvec *vec,
size_t len, struct scatterlist *sg, unsigned int sg_len,
void (*conf)(struct work_struct *work),
- int *errno, enum wait_type wait)
+ int *errno, int wait)
{
int err;
struct rtrs_clt_req_ops req_ops;
@@ -476,7 +469,8 @@ static void msg_close_conf(struct work_struct *work)
rnbd_clt_put_dev(dev);
}
-static int send_msg_close(struct rnbd_clt_dev *dev, u32 device_id, bool wait)
+static int send_msg_close(struct rnbd_clt_dev *dev, u32 device_id,
+ enum wait_type wait)
{
struct rnbd_clt_session *sess = dev->sess;
struct rnbd_msg_close msg;
@@ -530,7 +524,7 @@ static void msg_open_conf(struct work_struct *work)
* If server thinks its fine, but we fail to process
* then be nice and send a close to server.
*/
- (void)send_msg_close(dev, device_id, NO_WAIT);
+ send_msg_close(dev, device_id, RTRS_PERMIT_NOWAIT);
}
}
kfree(rsp);
@@ -554,7 +548,7 @@ static void msg_sess_info_conf(struct work_struct *work)
rnbd_clt_put_sess(sess);
}
-static int send_msg_open(struct rnbd_clt_dev *dev, bool wait)
+static int send_msg_open(struct rnbd_clt_dev *dev, enum wait_type wait)
{
struct rnbd_clt_session *sess = dev->sess;
struct rnbd_msg_open_rsp *rsp;
@@ -583,7 +577,7 @@ static int send_msg_open(struct rnbd_clt_dev *dev, bool wait)
msg.hdr.type = cpu_to_le16(RNBD_MSG_OPEN);
msg.access_mode = dev->access_mode;
- strlcpy(msg.dev_name, dev->pathname, sizeof(msg.dev_name));
+ strscpy(msg.dev_name, dev->pathname, sizeof(msg.dev_name));
WARN_ON(!rnbd_clt_get_dev(dev));
err = send_usr_msg(sess->rtrs, READ, iu,
@@ -601,7 +595,7 @@ static int send_msg_open(struct rnbd_clt_dev *dev, bool wait)
return err;
}
-static int send_msg_sess_info(struct rnbd_clt_session *sess, bool wait)
+static int send_msg_sess_info(struct rnbd_clt_session *sess, enum wait_type wait)
{
struct rnbd_msg_sess_info_rsp *rsp;
struct rnbd_msg_sess_info msg;
@@ -657,14 +651,18 @@ put_iu:
static void set_dev_states_to_disconnected(struct rnbd_clt_session *sess)
{
struct rnbd_clt_dev *dev;
+ struct kobject *gd_kobj;
mutex_lock(&sess->lock);
list_for_each_entry(dev, &sess->devs_list, list) {
rnbd_clt_err(dev, "Device disconnected.\n");
mutex_lock(&dev->lock);
- if (dev->dev_state == DEV_STATE_MAPPED)
+ if (dev->dev_state == DEV_STATE_MAPPED) {
dev->dev_state = DEV_STATE_MAPPED_DISCONNECTED;
+ gd_kobj = &disk_to_dev(dev->gd)->kobj;
+ kobject_uevent(gd_kobj, KOBJ_OFFLINE);
+ }
mutex_unlock(&dev->lock);
}
mutex_unlock(&sess->lock);
@@ -687,13 +685,17 @@ static void remap_devs(struct rnbd_clt_session *sess)
* be asynchronous.
*/
- err = send_msg_sess_info(sess, NO_WAIT);
+ err = send_msg_sess_info(sess, RTRS_PERMIT_NOWAIT);
if (err) {
pr_err("send_msg_sess_info(\"%s\"): %d\n", sess->sessname, err);
return;
}
- rtrs_clt_query(sess->rtrs, &attrs);
+ err = rtrs_clt_query(sess->rtrs, &attrs);
+ if (err) {
+ pr_err("rtrs_clt_query(\"%s\"): %d\n", sess->sessname, err);
+ return;
+ }
mutex_lock(&sess->lock);
sess->max_io_size = attrs.max_io_size;
@@ -711,7 +713,7 @@ static void remap_devs(struct rnbd_clt_session *sess)
continue;
rnbd_clt_info(dev, "session reconnected, remapping device\n");
- err = send_msg_open(dev, NO_WAIT);
+ err = send_msg_open(dev, RTRS_PERMIT_NOWAIT);
if (err) {
rnbd_clt_err(dev, "send_msg_open(): %d\n", err);
break;
@@ -801,12 +803,12 @@ static struct rnbd_clt_session *alloc_sess(const char *sessname)
sess = kzalloc_node(sizeof(*sess), GFP_KERNEL, NUMA_NO_NODE);
if (!sess)
return ERR_PTR(-ENOMEM);
- strlcpy(sess->sessname, sessname, sizeof(sess->sessname));
+ strscpy(sess->sessname, sessname, sizeof(sess->sessname));
atomic_set(&sess->busy, 0);
mutex_init(&sess->lock);
INIT_LIST_HEAD(&sess->devs_list);
INIT_LIST_HEAD(&sess->list);
- bitmap_zero(sess->cpu_queues_bm, NR_CPUS);
+ bitmap_zero(sess->cpu_queues_bm, num_possible_cpus());
init_waitqueue_head(&sess->rtrs_waitq);
refcount_set(&sess->refcount, 1);
@@ -918,6 +920,7 @@ again:
return NULL;
}
+/* caller is responsible for initializing 'first' to false */
static struct
rnbd_clt_session *find_or_create_sess(const char *sessname, bool *first)
{
@@ -933,8 +936,7 @@ rnbd_clt_session *find_or_create_sess(const char *sessname, bool *first)
}
list_add(&sess->list, &sess_list);
*first = true;
- } else
- *first = false;
+ }
mutex_unlock(&sess_lock);
return sess;
@@ -1048,7 +1050,7 @@ static int rnbd_client_xfer_request(struct rnbd_clt_dev *dev,
};
err = rtrs_clt_request(rq_data_dir(rq), &req_ops, rtrs, permit,
&vec, 1, size, iu->sgt.sgl, sg_cnt);
- if (unlikely(err)) {
+ if (err) {
rnbd_clt_err_rl(dev, "RTRS failed to transfer IO, err: %d\n",
err);
return err;
@@ -1079,7 +1081,7 @@ static bool rnbd_clt_dev_add_to_requeue(struct rnbd_clt_dev *dev,
cpu_q = get_cpu_ptr(sess->cpu_queues);
spin_lock_irqsave(&cpu_q->requeue_lock, flags);
- if (likely(!test_and_set_bit_lock(0, &q->in_list))) {
+ if (!test_and_set_bit_lock(0, &q->in_list)) {
if (WARN_ON(!list_empty(&q->requeue_list)))
goto unlock;
@@ -1091,7 +1093,7 @@ static bool rnbd_clt_dev_add_to_requeue(struct rnbd_clt_dev *dev,
*/
smp_mb__before_atomic();
}
- if (likely(atomic_read(&sess->busy))) {
+ if (atomic_read(&sess->busy)) {
list_add_tail(&q->requeue_list, &cpu_q->requeue_list);
} else {
/* Very unlikely, but possible: busy counter was
@@ -1119,7 +1121,7 @@ static void rnbd_clt_dev_kick_mq_queue(struct rnbd_clt_dev *dev,
if (delay != RNBD_DELAY_IFBUSY)
blk_mq_delay_run_hw_queue(hctx, delay);
- else if (unlikely(!rnbd_clt_dev_add_to_requeue(dev, q)))
+ else if (!rnbd_clt_dev_add_to_requeue(dev, q))
/*
* If session is not busy we have to restart
* the queue ourselves.
@@ -1136,12 +1138,12 @@ static blk_status_t rnbd_queue_rq(struct blk_mq_hw_ctx *hctx,
int err;
blk_status_t ret = BLK_STS_IOERR;
- if (unlikely(dev->dev_state != DEV_STATE_MAPPED))
+ if (dev->dev_state != DEV_STATE_MAPPED)
return BLK_STS_IOERR;
iu->permit = rnbd_get_permit(dev->sess, RTRS_IO_CON,
RTRS_PERMIT_NOWAIT);
- if (unlikely(!iu->permit)) {
+ if (!iu->permit) {
rnbd_clt_dev_kick_mq_queue(dev, hctx, RNBD_DELAY_IFBUSY);
return BLK_STS_RESOURCE;
}
@@ -1149,7 +1151,8 @@ static blk_status_t rnbd_queue_rq(struct blk_mq_hw_ctx *hctx,
iu->sgt.sgl = iu->first_sgl;
err = sg_alloc_table_chained(&iu->sgt,
/* Even-if the request has no segment,
- * sglist must have one entry at least */
+ * sglist must have one entry at least.
+ */
blk_rq_nr_phys_segments(rq) ? : 1,
iu->sgt.sgl,
RNBD_INLINE_SG_CNT);
@@ -1162,9 +1165,9 @@ static blk_status_t rnbd_queue_rq(struct blk_mq_hw_ctx *hctx,
blk_mq_start_request(rq);
err = rnbd_client_xfer_request(dev, rq, iu);
- if (likely(err == 0))
+ if (err == 0)
return BLK_STS_OK;
- if (unlikely(err == -EAGAIN || err == -ENOMEM)) {
+ if (err == -EAGAIN || err == -ENOMEM) {
rnbd_clt_dev_kick_mq_queue(dev, hctx, 10/*ms*/);
ret = BLK_STS_RESOURCE;
}
@@ -1173,9 +1176,54 @@ static blk_status_t rnbd_queue_rq(struct blk_mq_hw_ctx *hctx,
return ret;
}
+static int rnbd_rdma_poll(struct blk_mq_hw_ctx *hctx)
+{
+ struct rnbd_queue *q = hctx->driver_data;
+ struct rnbd_clt_dev *dev = q->dev;
+ int cnt;
+
+ cnt = rtrs_clt_rdma_cq_direct(dev->sess->rtrs, hctx->queue_num);
+ return cnt;
+}
+
+static int rnbd_rdma_map_queues(struct blk_mq_tag_set *set)
+{
+ struct rnbd_clt_session *sess = set->driver_data;
+
+ /* shared read/write queues */
+ set->map[HCTX_TYPE_DEFAULT].nr_queues = num_online_cpus();
+ set->map[HCTX_TYPE_DEFAULT].queue_offset = 0;
+ set->map[HCTX_TYPE_READ].nr_queues = num_online_cpus();
+ set->map[HCTX_TYPE_READ].queue_offset = 0;
+ blk_mq_map_queues(&set->map[HCTX_TYPE_DEFAULT]);
+ blk_mq_map_queues(&set->map[HCTX_TYPE_READ]);
+
+ if (sess->nr_poll_queues) {
+ /* dedicated queue for poll */
+ set->map[HCTX_TYPE_POLL].nr_queues = sess->nr_poll_queues;
+ set->map[HCTX_TYPE_POLL].queue_offset = set->map[HCTX_TYPE_READ].queue_offset +
+ set->map[HCTX_TYPE_READ].nr_queues;
+ blk_mq_map_queues(&set->map[HCTX_TYPE_POLL]);
+ pr_info("[session=%s] mapped %d/%d/%d default/read/poll queues.\n",
+ sess->sessname,
+ set->map[HCTX_TYPE_DEFAULT].nr_queues,
+ set->map[HCTX_TYPE_READ].nr_queues,
+ set->map[HCTX_TYPE_POLL].nr_queues);
+ } else {
+ pr_info("[session=%s] mapped %d/%d default/read queues.\n",
+ sess->sessname,
+ set->map[HCTX_TYPE_DEFAULT].nr_queues,
+ set->map[HCTX_TYPE_READ].nr_queues);
+ }
+
+ return 0;
+}
+
static struct blk_mq_ops rnbd_mq_ops = {
.queue_rq = rnbd_queue_rq,
.complete = rnbd_softirq_done_fn,
+ .map_queues = rnbd_rdma_map_queues,
+ .poll = rnbd_rdma_poll,
};
static int setup_mq_tags(struct rnbd_clt_session *sess)
@@ -1189,7 +1237,15 @@ static int setup_mq_tags(struct rnbd_clt_session *sess)
tag_set->flags = BLK_MQ_F_SHOULD_MERGE |
BLK_MQ_F_TAG_QUEUE_SHARED;
tag_set->cmd_size = sizeof(struct rnbd_iu) + RNBD_RDMA_SGL_SIZE;
- tag_set->nr_hw_queues = num_online_cpus();
+
+ /* for HCTX_TYPE_DEFAULT, HCTX_TYPE_READ, HCTX_TYPE_POLL */
+ tag_set->nr_maps = sess->nr_poll_queues ? HCTX_MAX_TYPES : 2;
+ /*
+ * HCTX_TYPE_DEFAULT and HCTX_TYPE_READ share one set of queues
+ * others are for HCTX_TYPE_POLL
+ */
+ tag_set->nr_hw_queues = num_online_cpus() + sess->nr_poll_queues;
+ tag_set->driver_data = sess;
return blk_mq_alloc_tag_set(tag_set);
}
@@ -1197,18 +1253,27 @@ static int setup_mq_tags(struct rnbd_clt_session *sess)
static struct rnbd_clt_session *
find_and_get_or_create_sess(const char *sessname,
const struct rtrs_addr *paths,
- size_t path_cnt, u16 port_nr)
+ size_t path_cnt, u16 port_nr, u32 nr_poll_queues)
{
struct rnbd_clt_session *sess;
struct rtrs_attrs attrs;
int err;
- bool first;
+ bool first = false;
struct rtrs_clt_ops rtrs_ops;
sess = find_or_create_sess(sessname, &first);
if (sess == ERR_PTR(-ENOMEM))
return ERR_PTR(-ENOMEM);
- else if (!first)
+ else if ((nr_poll_queues && !first) || (!nr_poll_queues && sess->nr_poll_queues)) {
+ /*
+ * A device MUST have its own session to use the polling-mode.
+ * It must fail to map new device with the same session.
+ */
+ err = -EINVAL;
+ goto put_sess;
+ }
+
+ if (!first)
return sess;
if (!path_cnt) {
@@ -1228,21 +1293,25 @@ find_and_get_or_create_sess(const char *sessname,
paths, path_cnt, port_nr,
0, /* Do not use pdu of rtrs */
RECONNECT_DELAY, BMAX_SEGMENTS,
- BLK_MAX_SEGMENT_SIZE,
- MAX_RECONNECTS);
+ MAX_RECONNECTS, nr_poll_queues);
if (IS_ERR(sess->rtrs)) {
err = PTR_ERR(sess->rtrs);
goto wake_up_and_put;
}
- rtrs_clt_query(sess->rtrs, &attrs);
+
+ err = rtrs_clt_query(sess->rtrs, &attrs);
+ if (err)
+ goto close_rtrs;
+
sess->max_io_size = attrs.max_io_size;
sess->queue_depth = attrs.queue_depth;
+ sess->nr_poll_queues = nr_poll_queues;
err = setup_mq_tags(sess);
if (err)
goto close_rtrs;
- err = send_msg_sess_info(sess, WAIT);
+ err = send_msg_sess_info(sess, RTRS_PERMIT_WAIT);
if (err)
goto close_rtrs;
@@ -1352,12 +1421,12 @@ static void rnbd_clt_setup_gen_disk(struct rnbd_clt_dev *dev, int idx)
if (!dev->rotational)
blk_queue_flag_set(QUEUE_FLAG_NONROT, dev->queue);
+ add_disk(dev->gd);
}
-static int rnbd_client_setup_device(struct rnbd_clt_session *sess,
- struct rnbd_clt_dev *dev, int idx)
+static int rnbd_client_setup_device(struct rnbd_clt_dev *dev)
{
- int err;
+ int err, idx = dev->clt_device_id;
dev->size = dev->nsectors * dev->logical_block_size;
@@ -1380,7 +1449,8 @@ static int rnbd_client_setup_device(struct rnbd_clt_session *sess,
static struct rnbd_clt_dev *init_dev(struct rnbd_clt_session *sess,
enum rnbd_access_mode access_mode,
- const char *pathname)
+ const char *pathname,
+ u32 nr_poll_queues)
{
struct rnbd_clt_dev *dev;
int ret;
@@ -1389,7 +1459,12 @@ static struct rnbd_clt_dev *init_dev(struct rnbd_clt_session *sess,
if (!dev)
return ERR_PTR(-ENOMEM);
- dev->hw_queues = kcalloc(nr_cpu_ids, sizeof(*dev->hw_queues),
+ /*
+ * nr_cpu_ids: the number of softirq queues
+ * nr_poll_queues: the number of polling queues
+ */
+ dev->hw_queues = kcalloc(nr_cpu_ids + nr_poll_queues,
+ sizeof(*dev->hw_queues),
GFP_KERNEL);
if (!dev->hw_queues) {
ret = -ENOMEM;
@@ -1415,6 +1490,7 @@ static struct rnbd_clt_dev *init_dev(struct rnbd_clt_session *sess,
dev->clt_device_id = ret;
dev->sess = sess;
dev->access_mode = access_mode;
+ dev->nr_poll_queues = nr_poll_queues;
mutex_init(&dev->lock);
refcount_set(&dev->refcount, 1);
dev->dev_state = DEV_STATE_INIT;
@@ -1471,14 +1547,13 @@ static bool exists_devpath(const char *pathname, const char *sessname)
return found;
}
-static bool insert_dev_if_not_exists_devpath(const char *pathname,
- struct rnbd_clt_session *sess,
- struct rnbd_clt_dev *dev)
+static bool insert_dev_if_not_exists_devpath(struct rnbd_clt_dev *dev)
{
bool found;
+ struct rnbd_clt_session *sess = dev->sess;
mutex_lock(&sess_lock);
- found = __exists_dev(pathname, sess->sessname);
+ found = __exists_dev(dev->pathname, sess->sessname);
if (!found) {
mutex_lock(&sess->lock);
list_add_tail(&dev->list, &sess->devs_list);
@@ -1502,31 +1577,32 @@ struct rnbd_clt_dev *rnbd_clt_map_device(const char *sessname,
struct rtrs_addr *paths,
size_t path_cnt, u16 port_nr,
const char *pathname,
- enum rnbd_access_mode access_mode)
+ enum rnbd_access_mode access_mode,
+ u32 nr_poll_queues)
{
struct rnbd_clt_session *sess;
struct rnbd_clt_dev *dev;
int ret;
- if (unlikely(exists_devpath(pathname, sessname)))
+ if (exists_devpath(pathname, sessname))
return ERR_PTR(-EEXIST);
- sess = find_and_get_or_create_sess(sessname, paths, path_cnt, port_nr);
+ sess = find_and_get_or_create_sess(sessname, paths, path_cnt, port_nr, nr_poll_queues);
if (IS_ERR(sess))
return ERR_CAST(sess);
- dev = init_dev(sess, access_mode, pathname);
+ dev = init_dev(sess, access_mode, pathname, nr_poll_queues);
if (IS_ERR(dev)) {
pr_err("map_device: failed to map device '%s' from session %s, can't initialize device, err: %ld\n",
pathname, sess->sessname, PTR_ERR(dev));
ret = PTR_ERR(dev);
goto put_sess;
}
- if (insert_dev_if_not_exists_devpath(pathname, sess, dev)) {
+ if (insert_dev_if_not_exists_devpath(dev)) {
ret = -EEXIST;
goto put_dev;
}
- ret = send_msg_open(dev, WAIT);
+ ret = send_msg_open(dev, RTRS_PERMIT_WAIT);
if (ret) {
rnbd_clt_err(dev,
"map_device: failed, can't open remote device, err: %d\n",
@@ -1536,7 +1612,7 @@ struct rnbd_clt_dev *rnbd_clt_map_device(const char *sessname,
mutex_lock(&dev->lock);
pr_debug("Opened remote device: session=%s, path='%s'\n",
sess->sessname, pathname);
- ret = rnbd_client_setup_device(sess, dev, dev->clt_device_id);
+ ret = rnbd_client_setup_device(dev);
if (ret) {
rnbd_clt_err(dev,
"map_device: Failed to configure device, err: %d\n",
@@ -1555,14 +1631,12 @@ struct rnbd_clt_dev *rnbd_clt_map_device(const char *sessname,
dev->max_hw_sectors, dev->rotational, dev->wc, dev->fua);
mutex_unlock(&dev->lock);
-
- add_disk(dev->gd);
rnbd_clt_put_sess(sess);
return dev;
send_close:
- send_msg_close(dev, dev->device_id, WAIT);
+ send_msg_close(dev, dev->device_id, RTRS_PERMIT_WAIT);
del_dev:
delete_dev(dev);
put_dev:
@@ -1622,7 +1696,7 @@ int rnbd_clt_unmap_device(struct rnbd_clt_dev *dev, bool force,
destroy_sysfs(dev, sysfs_self);
destroy_gen_disk(dev);
if (was_mapped && sess->rtrs)
- send_msg_close(dev, dev->device_id, WAIT);
+ send_msg_close(dev, dev->device_id, RTRS_PERMIT_WAIT);
rnbd_clt_info(dev, "Device is unmapped\n");
@@ -1656,7 +1730,7 @@ int rnbd_clt_remap_device(struct rnbd_clt_dev *dev)
mutex_unlock(&dev->lock);
if (!err) {
rnbd_clt_info(dev, "Remapping device.\n");
- err = send_msg_open(dev, WAIT);
+ err = send_msg_open(dev, RTRS_PERMIT_WAIT);
if (err)
rnbd_clt_err(dev, "remap_device: %d\n", err);
}
@@ -1678,7 +1752,6 @@ static void rnbd_destroy_sessions(void)
struct rnbd_clt_dev *dev, *tn;
/* Firstly forbid access through sysfs interface */
- rnbd_clt_destroy_default_group();
rnbd_clt_destroy_sysfs_files();
/*
diff --git a/drivers/block/rnbd/rnbd-clt.h b/drivers/block/rnbd/rnbd-clt.h
index 537d499dad3b..b5322c5aaac0 100644
--- a/drivers/block/rnbd/rnbd-clt.h
+++ b/drivers/block/rnbd/rnbd-clt.h
@@ -87,9 +87,10 @@ struct rnbd_clt_session {
DECLARE_BITMAP(cpu_queues_bm, NR_CPUS);
int __percpu *cpu_rr; /* per-cpu var for CPU round-robin */
atomic_t busy;
- int queue_depth;
+ size_t queue_depth;
u32 max_io_size;
struct blk_mq_tag_set tag_set;
+ u32 nr_poll_queues;
struct mutex lock; /* protects state and devs_list */
struct list_head devs_list; /* list of struct rnbd_clt_dev */
refcount_t refcount;
@@ -118,6 +119,7 @@ struct rnbd_clt_dev {
enum rnbd_clt_dev_state dev_state;
char *pathname;
enum rnbd_access_mode access_mode;
+ u32 nr_poll_queues;
bool read_only;
bool rotational;
bool wc;
@@ -147,7 +149,8 @@ struct rnbd_clt_dev *rnbd_clt_map_device(const char *sessname,
struct rtrs_addr *paths,
size_t path_cnt, u16 port_nr,
const char *pathname,
- enum rnbd_access_mode access_mode);
+ enum rnbd_access_mode access_mode,
+ u32 nr_poll_queues);
int rnbd_clt_unmap_device(struct rnbd_clt_dev *dev, bool force,
const struct attribute *sysfs_self);
@@ -159,7 +162,6 @@ int rnbd_clt_resize_disk(struct rnbd_clt_dev *dev, size_t newsize);
int rnbd_clt_create_sysfs_files(void);
void rnbd_clt_destroy_sysfs_files(void);
-void rnbd_clt_destroy_default_group(void);
void rnbd_clt_remove_dev_symlink(struct rnbd_clt_dev *dev);
diff --git a/drivers/block/rnbd/rnbd-srv-sysfs.c b/drivers/block/rnbd/rnbd-srv-sysfs.c
index 05ffe488ddc6..acf5fced11ef 100644
--- a/drivers/block/rnbd/rnbd-srv-sysfs.c
+++ b/drivers/block/rnbd/rnbd-srv-sysfs.c
@@ -147,10 +147,7 @@ static ssize_t rnbd_srv_dev_session_force_close_store(struct kobject *kobj,
}
rnbd_srv_info(sess_dev, "force close requested\n");
-
- /* first remove sysfs itself to avoid deadlock */
- sysfs_remove_file_self(&sess_dev->kobj, &attr->attr);
- rnbd_srv_sess_dev_force_close(sess_dev);
+ rnbd_srv_sess_dev_force_close(sess_dev, attr);
return count;
}
diff --git a/drivers/block/rnbd/rnbd-srv.c b/drivers/block/rnbd/rnbd-srv.c
index a6a68d44f517..aafecfe97055 100644
--- a/drivers/block/rnbd/rnbd-srv.c
+++ b/drivers/block/rnbd/rnbd-srv.c
@@ -104,7 +104,7 @@ rnbd_get_sess_dev(int dev_id, struct rnbd_srv_session *srv_sess)
rcu_read_lock();
sess_dev = xa_load(&srv_sess->index_idr, dev_id);
- if (likely(sess_dev))
+ if (sess_dev)
ret = kref_get_unless_zero(&sess_dev->kref);
rcu_read_unlock();
@@ -114,8 +114,7 @@ rnbd_get_sess_dev(int dev_id, struct rnbd_srv_session *srv_sess)
return sess_dev;
}
-static int process_rdma(struct rtrs_srv *sess,
- struct rnbd_srv_session *srv_sess,
+static int process_rdma(struct rnbd_srv_session *srv_sess,
struct rtrs_srv_op *id, void *data, u32 datalen,
const void *usr, size_t usrlen)
{
@@ -178,8 +177,10 @@ err:
return err;
}
-static void destroy_device(struct rnbd_srv_dev *dev)
+static void destroy_device(struct kref *kref)
{
+ struct rnbd_srv_dev *dev = container_of(kref, struct rnbd_srv_dev, kref);
+
WARN_ONCE(!list_empty(&dev->sess_dev_list),
"Device %s is being destroyed but still in use!\n",
dev->id);
@@ -198,18 +199,9 @@ static void destroy_device(struct rnbd_srv_dev *dev)
kfree(dev);
}
-static void destroy_device_cb(struct kref *kref)
-{
- struct rnbd_srv_dev *dev;
-
- dev = container_of(kref, struct rnbd_srv_dev, kref);
-
- destroy_device(dev);
-}
-
static void rnbd_put_srv_dev(struct rnbd_srv_dev *dev)
{
- kref_put(&dev->kref, destroy_device_cb);
+ kref_put(&dev->kref, destroy_device);
}
void rnbd_destroy_sess_dev(struct rnbd_srv_sess_dev *sess_dev, bool keep_id)
@@ -306,7 +298,7 @@ static int create_sess(struct rtrs_srv *rtrs)
mutex_unlock(&sess_lock);
srv_sess->rtrs = rtrs;
- strlcpy(srv_sess->sessname, sessname, sizeof(srv_sess->sessname));
+ strscpy(srv_sess->sessname, sessname, sizeof(srv_sess->sessname));
rtrs_srv_set_sess_priv(rtrs, srv_sess);
@@ -336,18 +328,22 @@ static int rnbd_srv_link_ev(struct rtrs_srv *rtrs,
}
}
-void rnbd_srv_sess_dev_force_close(struct rnbd_srv_sess_dev *sess_dev)
+void rnbd_srv_sess_dev_force_close(struct rnbd_srv_sess_dev *sess_dev,
+ struct kobj_attribute *attr)
{
struct rnbd_srv_session *sess = sess_dev->sess;
sess_dev->keep_id = true;
- mutex_lock(&sess->lock);
+ /* It is already started to close by client's close message. */
+ if (!mutex_trylock(&sess->lock))
+ return;
+ /* first remove sysfs itself to avoid deadlock */
+ sysfs_remove_file_self(&sess_dev->kobj, &attr->attr);
rnbd_srv_destroy_dev_session_sysfs(sess_dev);
mutex_unlock(&sess->lock);
}
-static int process_msg_close(struct rtrs_srv *rtrs,
- struct rnbd_srv_session *srv_sess,
+static int process_msg_close(struct rnbd_srv_session *srv_sess,
void *data, size_t datalen, const void *usr,
size_t usrlen)
{
@@ -366,20 +362,18 @@ static int process_msg_close(struct rtrs_srv *rtrs,
return 0;
}
-static int process_msg_open(struct rtrs_srv *rtrs,
- struct rnbd_srv_session *srv_sess,
+static int process_msg_open(struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen);
-static int process_msg_sess_info(struct rtrs_srv *rtrs,
- struct rnbd_srv_session *srv_sess,
+static int process_msg_sess_info(struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen);
-static int rnbd_srv_rdma_ev(struct rtrs_srv *rtrs, void *priv,
- struct rtrs_srv_op *id, int dir,
- void *data, size_t datalen, const void *usr,
- size_t usrlen)
+static int rnbd_srv_rdma_ev(void *priv,
+ struct rtrs_srv_op *id, int dir,
+ void *data, size_t datalen, const void *usr,
+ size_t usrlen)
{
struct rnbd_srv_session *srv_sess = priv;
const struct rnbd_msg_hdr *hdr = usr;
@@ -393,19 +387,16 @@ static int rnbd_srv_rdma_ev(struct rtrs_srv *rtrs, void *priv,
switch (type) {
case RNBD_MSG_IO:
- return process_rdma(rtrs, srv_sess, id, data, datalen, usr,
- usrlen);
+ return process_rdma(srv_sess, id, data, datalen, usr, usrlen);
case RNBD_MSG_CLOSE:
- ret = process_msg_close(rtrs, srv_sess, data, datalen,
- usr, usrlen);
+ ret = process_msg_close(srv_sess, data, datalen, usr, usrlen);
break;
case RNBD_MSG_OPEN:
- ret = process_msg_open(rtrs, srv_sess, usr, usrlen,
- data, datalen);
+ ret = process_msg_open(srv_sess, usr, usrlen, data, datalen);
break;
case RNBD_MSG_SESS_INFO:
- ret = process_msg_sess_info(rtrs, srv_sess, usr, usrlen,
- data, datalen);
+ ret = process_msg_sess_info(srv_sess, usr, usrlen, data,
+ datalen);
break;
default:
pr_warn("Received unexpected message type %d with dir %d from session %s\n",
@@ -446,7 +437,7 @@ static struct rnbd_srv_dev *rnbd_srv_init_srv_dev(const char *id)
if (!dev)
return ERR_PTR(-ENOMEM);
- strlcpy(dev->id, id, sizeof(dev->id));
+ strscpy(dev->id, id, sizeof(dev->id));
kref_init(&dev->kref);
INIT_LIST_HEAD(&dev->sess_dev_list);
mutex_init(&dev->lock);
@@ -598,7 +589,7 @@ rnbd_srv_create_set_sess_dev(struct rnbd_srv_session *srv_sess,
kref_init(&sdev->kref);
- strlcpy(sdev->pathname, open_msg->dev_name, sizeof(sdev->pathname));
+ strscpy(sdev->pathname, open_msg->dev_name, sizeof(sdev->pathname));
sdev->rnbd_dev = rnbd_dev;
sdev->sess = srv_sess;
@@ -658,8 +649,7 @@ static char *rnbd_srv_get_full_path(struct rnbd_srv_session *srv_sess,
return full_path;
}
-static int process_msg_sess_info(struct rtrs_srv *rtrs,
- struct rnbd_srv_session *srv_sess,
+static int process_msg_sess_info(struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen)
{
@@ -700,8 +690,7 @@ find_srv_sess_dev(struct rnbd_srv_session *srv_sess, const char *dev_name)
return NULL;
}
-static int process_msg_open(struct rtrs_srv *rtrs,
- struct rnbd_srv_session *srv_sess,
+static int process_msg_open(struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen)
{
diff --git a/drivers/block/rnbd/rnbd-srv.h b/drivers/block/rnbd/rnbd-srv.h
index b157371c25ed..98ddc31eb408 100644
--- a/drivers/block/rnbd/rnbd-srv.h
+++ b/drivers/block/rnbd/rnbd-srv.h
@@ -64,7 +64,8 @@ struct rnbd_srv_sess_dev {
enum rnbd_access_mode access_mode;
};
-void rnbd_srv_sess_dev_force_close(struct rnbd_srv_sess_dev *sess_dev);
+void rnbd_srv_sess_dev_force_close(struct rnbd_srv_sess_dev *sess_dev,
+ struct kobj_attribute *attr);
/* rnbd-srv-sysfs.c */
int rnbd_srv_create_dev_sysfs(struct rnbd_srv_dev *dev,
diff --git a/drivers/block/rsxx/core.c b/drivers/block/rsxx/core.c
index 5ac1881396af..83636714b8d7 100644
--- a/drivers/block/rsxx/core.c
+++ b/drivers/block/rsxx/core.c
@@ -392,7 +392,7 @@ static irqreturn_t rsxx_isr(int irq, void *pdata)
}
/*----------------- Card Event Handler -------------------*/
-static const char * const rsxx_card_state_to_str(unsigned int state)
+static const char *rsxx_card_state_to_str(unsigned int state)
{
static const char * const state_strings[] = {
"Unknown", "Shutdown", "Starting", "Formatting",
@@ -871,6 +871,7 @@ static int rsxx_pci_probe(struct pci_dev *dev,
card->event_wq = create_singlethread_workqueue(DRIVER_NAME"_event");
if (!card->event_wq) {
dev_err(CARD_TO_DEV(card), "Failed card event setup.\n");
+ st = -ENOMEM;
goto failed_event_handler;
}
diff --git a/drivers/block/swim.c b/drivers/block/swim.c
index cc6a0bc6c005..2917b21f48ff 100644
--- a/drivers/block/swim.c
+++ b/drivers/block/swim.c
@@ -816,8 +816,6 @@ static int swim_floppy_init(struct swim_priv *swd)
}
swd->unit[drive].disk->queue = q;
- blk_queue_bounce_limit(swd->unit[drive].disk->queue,
- BLK_BOUNCE_HIGH);
swd->unit[drive].disk->queue->queuedata = &swd->unit[drive];
swd->unit[drive].swd = swd;
}
diff --git a/drivers/block/swim3.c b/drivers/block/swim3.c
index c2d922d125e2..a515d0c1d2cb 100644
--- a/drivers/block/swim3.c
+++ b/drivers/block/swim3.c
@@ -234,7 +234,6 @@ static unsigned short write_postamble[] = {
};
static void seek_track(struct floppy_state *fs, int n);
-static void init_dma(struct dbdma_cmd *cp, int cmd, void *buf, int count);
static void act(struct floppy_state *fs);
static void scan_timeout(struct timer_list *t);
static void seek_timeout(struct timer_list *t);
@@ -404,12 +403,28 @@ static inline void seek_track(struct floppy_state *fs, int n)
fs->settle_time = 0;
}
+/*
+ * XXX: this is a horrible hack, but at least allows ppc32 to get
+ * out of defining virt_to_bus, and this driver out of using the
+ * deprecated block layer bounce buffering for highmem addresses
+ * for no good reason.
+ */
+static unsigned long swim3_phys_to_bus(phys_addr_t paddr)
+{
+ return paddr + PCI_DRAM_OFFSET;
+}
+
+static phys_addr_t swim3_bio_phys(struct bio *bio)
+{
+ return page_to_phys(bio_page(bio)) + bio_offset(bio);
+}
+
static inline void init_dma(struct dbdma_cmd *cp, int cmd,
- void *buf, int count)
+ phys_addr_t paddr, int count)
{
cp->req_count = cpu_to_le16(count);
cp->command = cpu_to_le16(cmd);
- cp->phy_addr = cpu_to_le32(virt_to_bus(buf));
+ cp->phy_addr = cpu_to_le32(swim3_phys_to_bus(paddr));
cp->xfer_status = 0;
}
@@ -441,16 +456,18 @@ static inline void setup_transfer(struct floppy_state *fs)
out_8(&sw->sector, fs->req_sector);
out_8(&sw->nsect, n);
out_8(&sw->gap3, 0);
- out_le32(&dr->cmdptr, virt_to_bus(cp));
+ out_le32(&dr->cmdptr, swim3_phys_to_bus(virt_to_phys(cp)));
if (rq_data_dir(req) == WRITE) {
/* Set up 3 dma commands: write preamble, data, postamble */
- init_dma(cp, OUTPUT_MORE, write_preamble, sizeof(write_preamble));
+ init_dma(cp, OUTPUT_MORE, virt_to_phys(write_preamble),
+ sizeof(write_preamble));
++cp;
- init_dma(cp, OUTPUT_MORE, bio_data(req->bio), 512);
+ init_dma(cp, OUTPUT_MORE, swim3_bio_phys(req->bio), 512);
++cp;
- init_dma(cp, OUTPUT_LAST, write_postamble, sizeof(write_postamble));
+ init_dma(cp, OUTPUT_LAST, virt_to_phys(write_postamble),
+ sizeof(write_postamble));
} else {
- init_dma(cp, INPUT_LAST, bio_data(req->bio), n * 512);
+ init_dma(cp, INPUT_LAST, swim3_bio_phys(req->bio), n * 512);
}
++cp;
out_le16(&cp->command, DBDMA_STOP);
@@ -1201,7 +1218,6 @@ static int swim3_attach(struct macio_dev *mdev,
disk->queue = NULL;
goto out_put_disk;
}
- blk_queue_bounce_limit(disk->queue, BLK_BOUNCE_HIGH);
disk->queue->queuedata = fs;
rc = swim3_add_device(mdev, floppy_count);
diff --git a/drivers/block/umem.c b/drivers/block/umem.c
deleted file mode 100644
index 982732dbe82e..000000000000
--- a/drivers/block/umem.c
+++ /dev/null
@@ -1,1127 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * mm.c - Micro Memory(tm) PCI memory board block device driver - v2.3
- *
- * (C) 2001 San Mehat <nettwerk@valinux.com>
- * (C) 2001 Johannes Erdfelt <jerdfelt@valinux.com>
- * (C) 2001 NeilBrown <neilb@cse.unsw.edu.au>
- *
- * This driver for the Micro Memory PCI Memory Module with Battery Backup
- * is Copyright Micro Memory Inc 2001-2002. All rights reserved.
- *
- * This driver provides a standard block device interface for Micro Memory(tm)
- * PCI based RAM boards.
- * 10/05/01: Phap Nguyen - Rebuilt the driver
- * 10/22/01: Phap Nguyen - v2.1 Added disk partitioning
- * 29oct2001:NeilBrown - Use make_request_fn instead of request_fn
- * - use stand disk partitioning (so fdisk works).
- * 08nov2001:NeilBrown - change driver name from "mm" to "umem"
- * - incorporate into main kernel
- * 08apr2002:NeilBrown - Move some of interrupt handle to tasklet
- * - use spin_lock_bh instead of _irq
- * - Never block on make_request. queue
- * bh's instead.
- * - unregister umem from devfs at mod unload
- * - Change version to 2.3
- * 07Nov2001:Phap Nguyen - Select pci read command: 06, 12, 15 (Decimal)
- * 07Jan2002: P. Nguyen - Used PCI Memory Write & Invalidate for DMA
- * 15May2002:NeilBrown - convert to bio for 2.5
- * 17May2002:NeilBrown - remove init_mem initialisation. Instead detect
- * - a sequence of writes that cover the card, and
- * - set initialised bit then.
- */
-
-#undef DEBUG /* #define DEBUG if you want debugging info (pr_debug) */
-#include <linux/fs.h>
-#include <linux/bio.h>
-#include <linux/kernel.h>
-#include <linux/mm.h>
-#include <linux/mman.h>
-#include <linux/gfp.h>
-#include <linux/ioctl.h>
-#include <linux/module.h>
-#include <linux/init.h>
-#include <linux/interrupt.h>
-#include <linux/timer.h>
-#include <linux/pci.h>
-#include <linux/dma-mapping.h>
-
-#include <linux/fcntl.h> /* O_ACCMODE */
-#include <linux/hdreg.h> /* HDIO_GETGEO */
-
-#include "umem.h"
-
-#include <linux/uaccess.h>
-#include <asm/io.h>
-
-#define MM_MAXCARDS 4
-#define MM_RAHEAD 2 /* two sectors */
-#define MM_BLKSIZE 1024 /* 1k blocks */
-#define MM_HARDSECT 512 /* 512-byte hardware sectors */
-#define MM_SHIFT 6 /* max 64 partitions on 4 cards */
-
-/*
- * Version Information
- */
-
-#define DRIVER_NAME "umem"
-#define DRIVER_VERSION "v2.3"
-#define DRIVER_AUTHOR "San Mehat, Johannes Erdfelt, NeilBrown"
-#define DRIVER_DESC "Micro Memory(tm) PCI memory board block driver"
-
-static int debug;
-/* #define HW_TRACE(x) writeb(x,cards[0].csr_remap + MEMCTRLSTATUS_MAGIC) */
-#define HW_TRACE(x)
-
-#define DEBUG_LED_ON_TRANSFER 0x01
-#define DEBUG_BATTERY_POLLING 0x02
-
-module_param(debug, int, 0644);
-MODULE_PARM_DESC(debug, "Debug bitmask");
-
-static int pci_read_cmd = 0x0C; /* Read Multiple */
-module_param(pci_read_cmd, int, 0);
-MODULE_PARM_DESC(pci_read_cmd, "PCI read command");
-
-static int pci_write_cmd = 0x0F; /* Write and Invalidate */
-module_param(pci_write_cmd, int, 0);
-MODULE_PARM_DESC(pci_write_cmd, "PCI write command");
-
-static int pci_cmds;
-
-static int major_nr;
-
-#include <linux/blkdev.h>
-#include <linux/blkpg.h>
-
-struct cardinfo {
- struct pci_dev *dev;
-
- unsigned char __iomem *csr_remap;
- unsigned int mm_size; /* size in kbytes */
-
- unsigned int init_size; /* initial segment, in sectors,
- * that we know to
- * have been written
- */
- struct bio *bio, *currentbio, **biotail;
- struct bvec_iter current_iter;
-
- struct request_queue *queue;
-
- struct mm_page {
- dma_addr_t page_dma;
- struct mm_dma_desc *desc;
- int cnt, headcnt;
- struct bio *bio, **biotail;
- struct bvec_iter iter;
- } mm_pages[2];
-#define DESC_PER_PAGE ((PAGE_SIZE*2)/sizeof(struct mm_dma_desc))
-
- int Active, Ready;
-
- struct tasklet_struct tasklet;
- unsigned int dma_status;
-
- struct {
- int good;
- int warned;
- unsigned long last_change;
- } battery[2];
-
- spinlock_t lock;
- int check_batteries;
-
- int flags;
-};
-
-static struct cardinfo cards[MM_MAXCARDS];
-static struct timer_list battery_timer;
-
-static int num_cards;
-
-static struct gendisk *mm_gendisk[MM_MAXCARDS];
-
-static void check_batteries(struct cardinfo *card);
-
-static int get_userbit(struct cardinfo *card, int bit)
-{
- unsigned char led;
-
- led = readb(card->csr_remap + MEMCTRLCMD_LEDCTRL);
- return led & bit;
-}
-
-static int set_userbit(struct cardinfo *card, int bit, unsigned char state)
-{
- unsigned char led;
-
- led = readb(card->csr_remap + MEMCTRLCMD_LEDCTRL);
- if (state)
- led |= bit;
- else
- led &= ~bit;
- writeb(led, card->csr_remap + MEMCTRLCMD_LEDCTRL);
-
- return 0;
-}
-
-/*
- * NOTE: For the power LED, use the LED_POWER_* macros since they differ
- */
-static void set_led(struct cardinfo *card, int shift, unsigned char state)
-{
- unsigned char led;
-
- led = readb(card->csr_remap + MEMCTRLCMD_LEDCTRL);
- if (state == LED_FLIP)
- led ^= (1<<shift);
- else {
- led &= ~(0x03 << shift);
- led |= (state << shift);
- }
- writeb(led, card->csr_remap + MEMCTRLCMD_LEDCTRL);
-
-}
-
-#ifdef MM_DIAG
-static void dump_regs(struct cardinfo *card)
-{
- unsigned char *p;
- int i, i1;
-
- p = card->csr_remap;
- for (i = 0; i < 8; i++) {
- printk(KERN_DEBUG "%p ", p);
-
- for (i1 = 0; i1 < 16; i1++)
- printk("%02x ", *p++);
-
- printk("\n");
- }
-}
-#endif
-
-static void dump_dmastat(struct cardinfo *card, unsigned int dmastat)
-{
- dev_printk(KERN_DEBUG, &card->dev->dev, "DMAstat - ");
- if (dmastat & DMASCR_ANY_ERR)
- printk(KERN_CONT "ANY_ERR ");
- if (dmastat & DMASCR_MBE_ERR)
- printk(KERN_CONT "MBE_ERR ");
- if (dmastat & DMASCR_PARITY_ERR_REP)
- printk(KERN_CONT "PARITY_ERR_REP ");
- if (dmastat & DMASCR_PARITY_ERR_DET)
- printk(KERN_CONT "PARITY_ERR_DET ");
- if (dmastat & DMASCR_SYSTEM_ERR_SIG)
- printk(KERN_CONT "SYSTEM_ERR_SIG ");
- if (dmastat & DMASCR_TARGET_ABT)
- printk(KERN_CONT "TARGET_ABT ");
- if (dmastat & DMASCR_MASTER_ABT)
- printk(KERN_CONT "MASTER_ABT ");
- if (dmastat & DMASCR_CHAIN_COMPLETE)
- printk(KERN_CONT "CHAIN_COMPLETE ");
- if (dmastat & DMASCR_DMA_COMPLETE)
- printk(KERN_CONT "DMA_COMPLETE ");
- printk("\n");
-}
-
-/*
- * Theory of request handling
- *
- * Each bio is assigned to one mm_dma_desc - which may not be enough FIXME
- * We have two pages of mm_dma_desc, holding about 64 descriptors
- * each. These are allocated at init time.
- * One page is "Ready" and is either full, or can have request added.
- * The other page might be "Active", which DMA is happening on it.
- *
- * Whenever IO on the active page completes, the Ready page is activated
- * and the ex-Active page is clean out and made Ready.
- * Otherwise the Ready page is only activated when it becomes full.
- *
- * If a request arrives while both pages a full, it is queued, and b_rdev is
- * overloaded to record whether it was a read or a write.
- *
- * The interrupt handler only polls the device to clear the interrupt.
- * The processing of the result is done in a tasklet.
- */
-
-static void mm_start_io(struct cardinfo *card)
-{
- /* we have the lock, we know there is
- * no IO active, and we know that card->Active
- * is set
- */
- struct mm_dma_desc *desc;
- struct mm_page *page;
- int offset;
-
- /* make the last descriptor end the chain */
- page = &card->mm_pages[card->Active];
- pr_debug("start_io: %d %d->%d\n",
- card->Active, page->headcnt, page->cnt - 1);
- desc = &page->desc[page->cnt-1];
-
- desc->control_bits |= cpu_to_le32(DMASCR_CHAIN_COMP_EN);
- desc->control_bits &= ~cpu_to_le32(DMASCR_CHAIN_EN);
- desc->sem_control_bits = desc->control_bits;
-
-
- if (debug & DEBUG_LED_ON_TRANSFER)
- set_led(card, LED_REMOVE, LED_ON);
-
- desc = &page->desc[page->headcnt];
- writel(0, card->csr_remap + DMA_PCI_ADDR);
- writel(0, card->csr_remap + DMA_PCI_ADDR + 4);
-
- writel(0, card->csr_remap + DMA_LOCAL_ADDR);
- writel(0, card->csr_remap + DMA_LOCAL_ADDR + 4);
-
- writel(0, card->csr_remap + DMA_TRANSFER_SIZE);
- writel(0, card->csr_remap + DMA_TRANSFER_SIZE + 4);
-
- writel(0, card->csr_remap + DMA_SEMAPHORE_ADDR);
- writel(0, card->csr_remap + DMA_SEMAPHORE_ADDR + 4);
-
- offset = ((char *)desc) - ((char *)page->desc);
- writel(cpu_to_le32((page->page_dma+offset) & 0xffffffff),
- card->csr_remap + DMA_DESCRIPTOR_ADDR);
- /* Force the value to u64 before shifting otherwise >> 32 is undefined C
- * and on some ports will do nothing ! */
- writel(cpu_to_le32(((u64)page->page_dma)>>32),
- card->csr_remap + DMA_DESCRIPTOR_ADDR + 4);
-
- /* Go, go, go */
- writel(cpu_to_le32(DMASCR_GO | DMASCR_CHAIN_EN | pci_cmds),
- card->csr_remap + DMA_STATUS_CTRL);
-}
-
-static int add_bio(struct cardinfo *card);
-
-static void activate(struct cardinfo *card)
-{
- /* if No page is Active, and Ready is
- * not empty, then switch Ready page
- * to active and start IO.
- * Then add any bh's that are available to Ready
- */
-
- do {
- while (add_bio(card))
- ;
-
- if (card->Active == -1 &&
- card->mm_pages[card->Ready].cnt > 0) {
- card->Active = card->Ready;
- card->Ready = 1-card->Ready;
- mm_start_io(card);
- }
-
- } while (card->Active == -1 && add_bio(card));
-}
-
-static inline void reset_page(struct mm_page *page)
-{
- page->cnt = 0;
- page->headcnt = 0;
- page->bio = NULL;
- page->biotail = &page->bio;
-}
-
-/*
- * If there is room on Ready page, take
- * one bh off list and add it.
- * return 1 if there was room, else 0.
- */
-static int add_bio(struct cardinfo *card)
-{
- struct mm_page *p;
- struct mm_dma_desc *desc;
- dma_addr_t dma_handle;
- int offset;
- struct bio *bio;
- struct bio_vec vec;
-
- bio = card->currentbio;
- if (!bio && card->bio) {
- card->currentbio = card->bio;
- card->current_iter = card->bio->bi_iter;
- card->bio = card->bio->bi_next;
- if (card->bio == NULL)
- card->biotail = &card->bio;
- card->currentbio->bi_next = NULL;
- return 1;
- }
- if (!bio)
- return 0;
-
- if (card->mm_pages[card->Ready].cnt >= DESC_PER_PAGE)
- return 0;
-
- vec = bio_iter_iovec(bio, card->current_iter);
-
- dma_handle = dma_map_page(&card->dev->dev,
- vec.bv_page,
- vec.bv_offset,
- vec.bv_len,
- bio_op(bio) == REQ_OP_READ ?
- DMA_FROM_DEVICE : DMA_TO_DEVICE);
-
- p = &card->mm_pages[card->Ready];
- desc = &p->desc[p->cnt];
- p->cnt++;
- if (p->bio == NULL)
- p->iter = card->current_iter;
- if ((p->biotail) != &bio->bi_next) {
- *(p->biotail) = bio;
- p->biotail = &(bio->bi_next);
- bio->bi_next = NULL;
- }
-
- desc->data_dma_handle = dma_handle;
-
- desc->pci_addr = cpu_to_le64((u64)desc->data_dma_handle);
- desc->local_addr = cpu_to_le64(card->current_iter.bi_sector << 9);
- desc->transfer_size = cpu_to_le32(vec.bv_len);
- offset = (((char *)&desc->sem_control_bits) - ((char *)p->desc));
- desc->sem_addr = cpu_to_le64((u64)(p->page_dma+offset));
- desc->zero1 = desc->zero2 = 0;
- offset = (((char *)(desc+1)) - ((char *)p->desc));
- desc->next_desc_addr = cpu_to_le64(p->page_dma+offset);
- desc->control_bits = cpu_to_le32(DMASCR_GO|DMASCR_ERR_INT_EN|
- DMASCR_PARITY_INT_EN|
- DMASCR_CHAIN_EN |
- DMASCR_SEM_EN |
- pci_cmds);
- if (bio_op(bio) == REQ_OP_WRITE)
- desc->control_bits |= cpu_to_le32(DMASCR_TRANSFER_READ);
- desc->sem_control_bits = desc->control_bits;
-
-
- bio_advance_iter(bio, &card->current_iter, vec.bv_len);
- if (!card->current_iter.bi_size)
- card->currentbio = NULL;
-
- return 1;
-}
-
-static void process_page(unsigned long data)
-{
- /* check if any of the requests in the page are DMA_COMPLETE,
- * and deal with them appropriately.
- * If we find a descriptor without DMA_COMPLETE in the semaphore, then
- * dma must have hit an error on that descriptor, so use dma_status
- * instead and assume that all following descriptors must be re-tried.
- */
- struct mm_page *page;
- struct bio *return_bio = NULL;
- struct cardinfo *card = (struct cardinfo *)data;
- unsigned int dma_status = card->dma_status;
-
- spin_lock(&card->lock);
- if (card->Active < 0)
- goto out_unlock;
- page = &card->mm_pages[card->Active];
-
- while (page->headcnt < page->cnt) {
- struct bio *bio = page->bio;
- struct mm_dma_desc *desc = &page->desc[page->headcnt];
- int control = le32_to_cpu(desc->sem_control_bits);
- int last = 0;
- struct bio_vec vec;
-
- if (!(control & DMASCR_DMA_COMPLETE)) {
- control = dma_status;
- last = 1;
- }
-
- page->headcnt++;
- vec = bio_iter_iovec(bio, page->iter);
- bio_advance_iter(bio, &page->iter, vec.bv_len);
-
- if (!page->iter.bi_size) {
- page->bio = bio->bi_next;
- if (page->bio)
- page->iter = page->bio->bi_iter;
- }
-
- dma_unmap_page(&card->dev->dev, desc->data_dma_handle,
- vec.bv_len,
- (control & DMASCR_TRANSFER_READ) ?
- DMA_TO_DEVICE : DMA_FROM_DEVICE);
- if (control & DMASCR_HARD_ERROR) {
- /* error */
- bio->bi_status = BLK_STS_IOERR;
- dev_printk(KERN_WARNING, &card->dev->dev,
- "I/O error on sector %d/%d\n",
- le32_to_cpu(desc->local_addr)>>9,
- le32_to_cpu(desc->transfer_size));
- dump_dmastat(card, control);
- } else if (op_is_write(bio_op(bio)) &&
- le32_to_cpu(desc->local_addr) >> 9 ==
- card->init_size) {
- card->init_size += le32_to_cpu(desc->transfer_size) >> 9;
- if (card->init_size >> 1 >= card->mm_size) {
- dev_printk(KERN_INFO, &card->dev->dev,
- "memory now initialised\n");
- set_userbit(card, MEMORY_INITIALIZED, 1);
- }
- }
- if (bio != page->bio) {
- bio->bi_next = return_bio;
- return_bio = bio;
- }
-
- if (last)
- break;
- }
-
- if (debug & DEBUG_LED_ON_TRANSFER)
- set_led(card, LED_REMOVE, LED_OFF);
-
- if (card->check_batteries) {
- card->check_batteries = 0;
- check_batteries(card);
- }
- if (page->headcnt >= page->cnt) {
- reset_page(page);
- card->Active = -1;
- activate(card);
- } else {
- /* haven't finished with this one yet */
- pr_debug("do some more\n");
- mm_start_io(card);
- }
- out_unlock:
- spin_unlock(&card->lock);
-
- while (return_bio) {
- struct bio *bio = return_bio;
-
- return_bio = bio->bi_next;
- bio->bi_next = NULL;
- bio_endio(bio);
- }
-}
-
-static void mm_unplug(struct blk_plug_cb *cb, bool from_schedule)
-{
- struct cardinfo *card = cb->data;
-
- spin_lock_irq(&card->lock);
- activate(card);
- spin_unlock_irq(&card->lock);
- kfree(cb);
-}
-
-static int mm_check_plugged(struct cardinfo *card)
-{
- return !!blk_check_plugged(mm_unplug, card, sizeof(struct blk_plug_cb));
-}
-
-static blk_qc_t mm_submit_bio(struct bio *bio)
-{
- struct cardinfo *card = bio->bi_bdev->bd_disk->private_data;
-
- pr_debug("mm_make_request %llu %u\n",
- (unsigned long long)bio->bi_iter.bi_sector,
- bio->bi_iter.bi_size);
-
- blk_queue_split(&bio);
-
- spin_lock_irq(&card->lock);
- *card->biotail = bio;
- bio->bi_next = NULL;
- card->biotail = &bio->bi_next;
- if (op_is_sync(bio->bi_opf) || !mm_check_plugged(card))
- activate(card);
- spin_unlock_irq(&card->lock);
-
- return BLK_QC_T_NONE;
-}
-
-static irqreturn_t mm_interrupt(int irq, void *__card)
-{
- struct cardinfo *card = (struct cardinfo *) __card;
- unsigned int dma_status;
- unsigned short cfg_status;
-
-HW_TRACE(0x30);
-
- dma_status = le32_to_cpu(readl(card->csr_remap + DMA_STATUS_CTRL));
-
- if (!(dma_status & (DMASCR_ERROR_MASK | DMASCR_CHAIN_COMPLETE))) {
- /* interrupt wasn't for me ... */
- return IRQ_NONE;
- }
-
- /* clear COMPLETION interrupts */
- if (card->flags & UM_FLAG_NO_BYTE_STATUS)
- writel(cpu_to_le32(DMASCR_DMA_COMPLETE|DMASCR_CHAIN_COMPLETE),
- card->csr_remap + DMA_STATUS_CTRL);
- else
- writeb((DMASCR_DMA_COMPLETE|DMASCR_CHAIN_COMPLETE) >> 16,
- card->csr_remap + DMA_STATUS_CTRL + 2);
-
- /* log errors and clear interrupt status */
- if (dma_status & DMASCR_ANY_ERR) {
- unsigned int data_log1, data_log2;
- unsigned int addr_log1, addr_log2;
- unsigned char stat, count, syndrome, check;
-
- stat = readb(card->csr_remap + MEMCTRLCMD_ERRSTATUS);
-
- data_log1 = le32_to_cpu(readl(card->csr_remap +
- ERROR_DATA_LOG));
- data_log2 = le32_to_cpu(readl(card->csr_remap +
- ERROR_DATA_LOG + 4));
- addr_log1 = le32_to_cpu(readl(card->csr_remap +
- ERROR_ADDR_LOG));
- addr_log2 = readb(card->csr_remap + ERROR_ADDR_LOG + 4);
-
- count = readb(card->csr_remap + ERROR_COUNT);
- syndrome = readb(card->csr_remap + ERROR_SYNDROME);
- check = readb(card->csr_remap + ERROR_CHECK);
-
- dump_dmastat(card, dma_status);
-
- if (stat & 0x01)
- dev_printk(KERN_ERR, &card->dev->dev,
- "Memory access error detected (err count %d)\n",
- count);
- if (stat & 0x02)
- dev_printk(KERN_ERR, &card->dev->dev,
- "Multi-bit EDC error\n");
-
- dev_printk(KERN_ERR, &card->dev->dev,
- "Fault Address 0x%02x%08x, Fault Data 0x%08x%08x\n",
- addr_log2, addr_log1, data_log2, data_log1);
- dev_printk(KERN_ERR, &card->dev->dev,
- "Fault Check 0x%02x, Fault Syndrome 0x%02x\n",
- check, syndrome);
-
- writeb(0, card->csr_remap + ERROR_COUNT);
- }
-
- if (dma_status & DMASCR_PARITY_ERR_REP) {
- dev_printk(KERN_ERR, &card->dev->dev,
- "PARITY ERROR REPORTED\n");
- pci_read_config_word(card->dev, PCI_STATUS, &cfg_status);
- pci_write_config_word(card->dev, PCI_STATUS, cfg_status);
- }
-
- if (dma_status & DMASCR_PARITY_ERR_DET) {
- dev_printk(KERN_ERR, &card->dev->dev,
- "PARITY ERROR DETECTED\n");
- pci_read_config_word(card->dev, PCI_STATUS, &cfg_status);
- pci_write_config_word(card->dev, PCI_STATUS, cfg_status);
- }
-
- if (dma_status & DMASCR_SYSTEM_ERR_SIG) {
- dev_printk(KERN_ERR, &card->dev->dev, "SYSTEM ERROR\n");
- pci_read_config_word(card->dev, PCI_STATUS, &cfg_status);
- pci_write_config_word(card->dev, PCI_STATUS, cfg_status);
- }
-
- if (dma_status & DMASCR_TARGET_ABT) {
- dev_printk(KERN_ERR, &card->dev->dev, "TARGET ABORT\n");
- pci_read_config_word(card->dev, PCI_STATUS, &cfg_status);
- pci_write_config_word(card->dev, PCI_STATUS, cfg_status);
- }
-
- if (dma_status & DMASCR_MASTER_ABT) {
- dev_printk(KERN_ERR, &card->dev->dev, "MASTER ABORT\n");
- pci_read_config_word(card->dev, PCI_STATUS, &cfg_status);
- pci_write_config_word(card->dev, PCI_STATUS, cfg_status);
- }
-
- /* and process the DMA descriptors */
- card->dma_status = dma_status;
- tasklet_schedule(&card->tasklet);
-
-HW_TRACE(0x36);
-
- return IRQ_HANDLED;
-}
-
-/*
- * If both batteries are good, no LED
- * If either battery has been warned, solid LED
- * If both batteries are bad, flash the LED quickly
- * If either battery is bad, flash the LED semi quickly
- */
-static void set_fault_to_battery_status(struct cardinfo *card)
-{
- if (card->battery[0].good && card->battery[1].good)
- set_led(card, LED_FAULT, LED_OFF);
- else if (card->battery[0].warned || card->battery[1].warned)
- set_led(card, LED_FAULT, LED_ON);
- else if (!card->battery[0].good && !card->battery[1].good)
- set_led(card, LED_FAULT, LED_FLASH_7_0);
- else
- set_led(card, LED_FAULT, LED_FLASH_3_5);
-}
-
-static void init_battery_timer(void);
-
-static int check_battery(struct cardinfo *card, int battery, int status)
-{
- if (status != card->battery[battery].good) {
- card->battery[battery].good = !card->battery[battery].good;
- card->battery[battery].last_change = jiffies;
-
- if (card->battery[battery].good) {
- dev_printk(KERN_ERR, &card->dev->dev,
- "Battery %d now good\n", battery + 1);
- card->battery[battery].warned = 0;
- } else
- dev_printk(KERN_ERR, &card->dev->dev,
- "Battery %d now FAILED\n", battery + 1);
-
- return 1;
- } else if (!card->battery[battery].good &&
- !card->battery[battery].warned &&
- time_after_eq(jiffies, card->battery[battery].last_change +
- (HZ * 60 * 60 * 5))) {
- dev_printk(KERN_ERR, &card->dev->dev,
- "Battery %d still FAILED after 5 hours\n", battery + 1);
- card->battery[battery].warned = 1;
-
- return 1;
- }
-
- return 0;
-}
-
-static void check_batteries(struct cardinfo *card)
-{
- /* NOTE: this must *never* be called while the card
- * is doing (bus-to-card) DMA, or you will need the
- * reset switch
- */
- unsigned char status;
- int ret1, ret2;
-
- status = readb(card->csr_remap + MEMCTRLSTATUS_BATTERY);
- if (debug & DEBUG_BATTERY_POLLING)
- dev_printk(KERN_DEBUG, &card->dev->dev,
- "checking battery status, 1 = %s, 2 = %s\n",
- (status & BATTERY_1_FAILURE) ? "FAILURE" : "OK",
- (status & BATTERY_2_FAILURE) ? "FAILURE" : "OK");
-
- ret1 = check_battery(card, 0, !(status & BATTERY_1_FAILURE));
- ret2 = check_battery(card, 1, !(status & BATTERY_2_FAILURE));
-
- if (ret1 || ret2)
- set_fault_to_battery_status(card);
-}
-
-static void check_all_batteries(struct timer_list *unused)
-{
- int i;
-
- for (i = 0; i < num_cards; i++)
- if (!(cards[i].flags & UM_FLAG_NO_BATT)) {
- struct cardinfo *card = &cards[i];
- spin_lock_bh(&card->lock);
- if (card->Active >= 0)
- card->check_batteries = 1;
- else
- check_batteries(card);
- spin_unlock_bh(&card->lock);
- }
-
- init_battery_timer();
-}
-
-static void init_battery_timer(void)
-{
- timer_setup(&battery_timer, check_all_batteries, 0);
- battery_timer.expires = jiffies + (HZ * 60);
- add_timer(&battery_timer);
-}
-
-static void del_battery_timer(void)
-{
- del_timer(&battery_timer);
-}
-
-/*
- * Note no locks taken out here. In a worst case scenario, we could drop
- * a chunk of system memory. But that should never happen, since validation
- * happens at open or mount time, when locks are held.
- *
- * That's crap, since doing that while some partitions are opened
- * or mounted will give you really nasty results.
- */
-static int mm_revalidate(struct gendisk *disk)
-{
- struct cardinfo *card = disk->private_data;
- set_capacity(disk, card->mm_size << 1);
- return 0;
-}
-
-static int mm_getgeo(struct block_device *bdev, struct hd_geometry *geo)
-{
- struct cardinfo *card = bdev->bd_disk->private_data;
- int size = card->mm_size * (1024 / MM_HARDSECT);
-
- /*
- * get geometry: we have to fake one... trim the size to a
- * multiple of 2048 (1M): tell we have 32 sectors, 64 heads,
- * whatever cylinders.
- */
- geo->heads = 64;
- geo->sectors = 32;
- geo->cylinders = size / (geo->heads * geo->sectors);
- return 0;
-}
-
-static const struct block_device_operations mm_fops = {
- .owner = THIS_MODULE,
- .submit_bio = mm_submit_bio,
- .getgeo = mm_getgeo,
- .revalidate_disk = mm_revalidate,
-};
-
-static int mm_pci_probe(struct pci_dev *dev, const struct pci_device_id *id)
-{
- int ret;
- struct cardinfo *card = &cards[num_cards];
- unsigned char mem_present;
- unsigned char batt_status;
- unsigned int saved_bar, data;
- unsigned long csr_base;
- unsigned long csr_len;
- int magic_number;
- static int printed_version;
-
- if (!printed_version++)
- printk(KERN_INFO DRIVER_VERSION " : " DRIVER_DESC "\n");
-
- ret = pci_enable_device(dev);
- if (ret)
- return ret;
-
- pci_write_config_byte(dev, PCI_LATENCY_TIMER, 0xF8);
- pci_set_master(dev);
-
- card->dev = dev;
-
- csr_base = pci_resource_start(dev, 0);
- csr_len = pci_resource_len(dev, 0);
- if (!csr_base || !csr_len)
- return -ENODEV;
-
- dev_printk(KERN_INFO, &dev->dev,
- "Micro Memory(tm) controller found (PCI Mem Module (Battery Backup))\n");
-
- if (dma_set_mask(&dev->dev, DMA_BIT_MASK(64)) &&
- dma_set_mask(&dev->dev, DMA_BIT_MASK(32))) {
- dev_printk(KERN_WARNING, &dev->dev, "NO suitable DMA found\n");
- return -ENOMEM;
- }
-
- ret = pci_request_regions(dev, DRIVER_NAME);
- if (ret) {
- dev_printk(KERN_ERR, &card->dev->dev,
- "Unable to request memory region\n");
- goto failed_req_csr;
- }
-
- card->csr_remap = ioremap(csr_base, csr_len);
- if (!card->csr_remap) {
- dev_printk(KERN_ERR, &card->dev->dev,
- "Unable to remap memory region\n");
- ret = -ENOMEM;
-
- goto failed_remap_csr;
- }
-
- dev_printk(KERN_INFO, &card->dev->dev,
- "CSR 0x%08lx -> 0x%p (0x%lx)\n",
- csr_base, card->csr_remap, csr_len);
-
- switch (card->dev->device) {
- case 0x5415:
- card->flags |= UM_FLAG_NO_BYTE_STATUS | UM_FLAG_NO_BATTREG;
- magic_number = 0x59;
- break;
-
- case 0x5425:
- card->flags |= UM_FLAG_NO_BYTE_STATUS;
- magic_number = 0x5C;
- break;
-
- case 0x6155:
- card->flags |= UM_FLAG_NO_BYTE_STATUS |
- UM_FLAG_NO_BATTREG | UM_FLAG_NO_BATT;
- magic_number = 0x99;
- break;
-
- default:
- magic_number = 0x100;
- break;
- }
-
- if (readb(card->csr_remap + MEMCTRLSTATUS_MAGIC) != magic_number) {
- dev_printk(KERN_ERR, &card->dev->dev, "Magic number invalid\n");
- ret = -ENOMEM;
- goto failed_magic;
- }
-
- card->mm_pages[0].desc = dma_alloc_coherent(&card->dev->dev,
- PAGE_SIZE * 2, &card->mm_pages[0].page_dma, GFP_KERNEL);
- card->mm_pages[1].desc = dma_alloc_coherent(&card->dev->dev,
- PAGE_SIZE * 2, &card->mm_pages[1].page_dma, GFP_KERNEL);
- if (card->mm_pages[0].desc == NULL ||
- card->mm_pages[1].desc == NULL) {
- dev_printk(KERN_ERR, &card->dev->dev, "alloc failed\n");
- goto failed_alloc;
- }
- reset_page(&card->mm_pages[0]);
- reset_page(&card->mm_pages[1]);
- card->Ready = 0; /* page 0 is ready */
- card->Active = -1; /* no page is active */
- card->bio = NULL;
- card->biotail = &card->bio;
- spin_lock_init(&card->lock);
-
- card->queue = blk_alloc_queue(NUMA_NO_NODE);
- if (!card->queue)
- goto failed_alloc;
-
- tasklet_init(&card->tasklet, process_page, (unsigned long)card);
-
- card->check_batteries = 0;
-
- mem_present = readb(card->csr_remap + MEMCTRLSTATUS_MEMORY);
- switch (mem_present) {
- case MEM_128_MB:
- card->mm_size = 1024 * 128;
- break;
- case MEM_256_MB:
- card->mm_size = 1024 * 256;
- break;
- case MEM_512_MB:
- card->mm_size = 1024 * 512;
- break;
- case MEM_1_GB:
- card->mm_size = 1024 * 1024;
- break;
- case MEM_2_GB:
- card->mm_size = 1024 * 2048;
- break;
- default:
- card->mm_size = 0;
- break;
- }
-
- /* Clear the LED's we control */
- set_led(card, LED_REMOVE, LED_OFF);
- set_led(card, LED_FAULT, LED_OFF);
-
- batt_status = readb(card->csr_remap + MEMCTRLSTATUS_BATTERY);
-
- card->battery[0].good = !(batt_status & BATTERY_1_FAILURE);
- card->battery[1].good = !(batt_status & BATTERY_2_FAILURE);
- card->battery[0].last_change = card->battery[1].last_change = jiffies;
-
- if (card->flags & UM_FLAG_NO_BATT)
- dev_printk(KERN_INFO, &card->dev->dev,
- "Size %d KB\n", card->mm_size);
- else {
- dev_printk(KERN_INFO, &card->dev->dev,
- "Size %d KB, Battery 1 %s (%s), Battery 2 %s (%s)\n",
- card->mm_size,
- batt_status & BATTERY_1_DISABLED ? "Disabled" : "Enabled",
- card->battery[0].good ? "OK" : "FAILURE",
- batt_status & BATTERY_2_DISABLED ? "Disabled" : "Enabled",
- card->battery[1].good ? "OK" : "FAILURE");
-
- set_fault_to_battery_status(card);
- }
-
- pci_read_config_dword(dev, PCI_BASE_ADDRESS_1, &saved_bar);
- data = 0xffffffff;
- pci_write_config_dword(dev, PCI_BASE_ADDRESS_1, data);
- pci_read_config_dword(dev, PCI_BASE_ADDRESS_1, &data);
- pci_write_config_dword(dev, PCI_BASE_ADDRESS_1, saved_bar);
- data &= 0xfffffff0;
- data = ~data;
- data += 1;
-
- if (request_irq(dev->irq, mm_interrupt, IRQF_SHARED, DRIVER_NAME,
- card)) {
- dev_printk(KERN_ERR, &card->dev->dev,
- "Unable to allocate IRQ\n");
- ret = -ENODEV;
- goto failed_req_irq;
- }
-
- dev_printk(KERN_INFO, &card->dev->dev,
- "Window size %d bytes, IRQ %d\n", data, dev->irq);
-
- pci_set_drvdata(dev, card);
-
- if (pci_write_cmd != 0x0F) /* If not Memory Write & Invalidate */
- pci_write_cmd = 0x07; /* then Memory Write command */
-
- if (pci_write_cmd & 0x08) { /* use Memory Write and Invalidate */
- unsigned short cfg_command;
- pci_read_config_word(dev, PCI_COMMAND, &cfg_command);
- cfg_command |= 0x10; /* Memory Write & Invalidate Enable */
- pci_write_config_word(dev, PCI_COMMAND, cfg_command);
- }
- pci_cmds = (pci_read_cmd << 28) | (pci_write_cmd << 24);
-
- num_cards++;
-
- if (!get_userbit(card, MEMORY_INITIALIZED)) {
- dev_printk(KERN_INFO, &card->dev->dev,
- "memory NOT initialized. Consider over-writing whole device.\n");
- card->init_size = 0;
- } else {
- dev_printk(KERN_INFO, &card->dev->dev,
- "memory already initialized\n");
- card->init_size = card->mm_size;
- }
-
- /* Enable ECC */
- writeb(EDC_STORE_CORRECT, card->csr_remap + MEMCTRLCMD_ERRCTRL);
-
- return 0;
-
- failed_req_irq:
- failed_alloc:
- if (card->mm_pages[0].desc)
- dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
- card->mm_pages[0].desc,
- card->mm_pages[0].page_dma);
- if (card->mm_pages[1].desc)
- dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
- card->mm_pages[1].desc,
- card->mm_pages[1].page_dma);
- failed_magic:
- iounmap(card->csr_remap);
- failed_remap_csr:
- pci_release_regions(dev);
- failed_req_csr:
-
- return ret;
-}
-
-static void mm_pci_remove(struct pci_dev *dev)
-{
- struct cardinfo *card = pci_get_drvdata(dev);
-
- tasklet_kill(&card->tasklet);
- free_irq(dev->irq, card);
- iounmap(card->csr_remap);
-
- if (card->mm_pages[0].desc)
- dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
- card->mm_pages[0].desc,
- card->mm_pages[0].page_dma);
- if (card->mm_pages[1].desc)
- dma_free_coherent(&card->dev->dev, PAGE_SIZE * 2,
- card->mm_pages[1].desc,
- card->mm_pages[1].page_dma);
- blk_cleanup_queue(card->queue);
-
- pci_release_regions(dev);
- pci_disable_device(dev);
-}
-
-static const struct pci_device_id mm_pci_ids[] = {
- {PCI_DEVICE(PCI_VENDOR_ID_MICRO_MEMORY, PCI_DEVICE_ID_MICRO_MEMORY_5415CN)},
- {PCI_DEVICE(PCI_VENDOR_ID_MICRO_MEMORY, PCI_DEVICE_ID_MICRO_MEMORY_5425CN)},
- {PCI_DEVICE(PCI_VENDOR_ID_MICRO_MEMORY, PCI_DEVICE_ID_MICRO_MEMORY_6155)},
- {
- .vendor = 0x8086,
- .device = 0xB555,
- .subvendor = 0x1332,
- .subdevice = 0x5460,
- .class = 0x050000,
- .class_mask = 0,
- }, { /* end: all zeroes */ }
-};
-
-MODULE_DEVICE_TABLE(pci, mm_pci_ids);
-
-static struct pci_driver mm_pci_driver = {
- .name = DRIVER_NAME,
- .id_table = mm_pci_ids,
- .probe = mm_pci_probe,
- .remove = mm_pci_remove,
-};
-
-static int __init mm_init(void)
-{
- int retval, i;
- int err;
-
- retval = pci_register_driver(&mm_pci_driver);
- if (retval)
- return -ENOMEM;
-
- err = major_nr = register_blkdev(0, DRIVER_NAME);
- if (err < 0) {
- pci_unregister_driver(&mm_pci_driver);
- return -EIO;
- }
-
- for (i = 0; i < num_cards; i++) {
- mm_gendisk[i] = alloc_disk(1 << MM_SHIFT);
- if (!mm_gendisk[i])
- goto out;
- }
-
- for (i = 0; i < num_cards; i++) {
- struct gendisk *disk = mm_gendisk[i];
- sprintf(disk->disk_name, "umem%c", 'a'+i);
- spin_lock_init(&cards[i].lock);
- disk->major = major_nr;
- disk->first_minor = i << MM_SHIFT;
- disk->fops = &mm_fops;
- disk->private_data = &cards[i];
- disk->queue = cards[i].queue;
- set_capacity(disk, cards[i].mm_size << 1);
- add_disk(disk);
- }
-
- init_battery_timer();
- printk(KERN_INFO "MM: desc_per_page = %ld\n", DESC_PER_PAGE);
-/* printk("mm_init: Done. 10-19-01 9:00\n"); */
- return 0;
-
-out:
- pci_unregister_driver(&mm_pci_driver);
- unregister_blkdev(major_nr, DRIVER_NAME);
- while (i--)
- put_disk(mm_gendisk[i]);
- return -ENOMEM;
-}
-
-static void __exit mm_cleanup(void)
-{
- int i;
-
- del_battery_timer();
-
- for (i = 0; i < num_cards ; i++) {
- del_gendisk(mm_gendisk[i]);
- put_disk(mm_gendisk[i]);
- }
-
- pci_unregister_driver(&mm_pci_driver);
-
- unregister_blkdev(major_nr, DRIVER_NAME);
-}
-
-module_init(mm_init);
-module_exit(mm_cleanup);
-
-MODULE_AUTHOR(DRIVER_AUTHOR);
-MODULE_DESCRIPTION(DRIVER_DESC);
-MODULE_LICENSE("GPL");
diff --git a/drivers/block/umem.h b/drivers/block/umem.h
deleted file mode 100644
index 58384978ff05..000000000000
--- a/drivers/block/umem.h
+++ /dev/null
@@ -1,132 +0,0 @@
-/* SPDX-License-Identifier: GPL-2.0-only */
-
-/*
- * This file contains defines for the
- * Micro Memory MM5415
- * family PCI Memory Module with Battery Backup.
- *
- * Copyright Micro Memory INC 2001. All rights reserved.
- */
-
-#ifndef _DRIVERS_BLOCK_MM_H
-#define _DRIVERS_BLOCK_MM_H
-
-
-#define IRQ_TIMEOUT (1 * HZ)
-
-/* CSR register definition */
-#define MEMCTRLSTATUS_MAGIC 0x00
-#define MM_MAGIC_VALUE (unsigned char)0x59
-
-#define MEMCTRLSTATUS_BATTERY 0x04
-#define BATTERY_1_DISABLED 0x01
-#define BATTERY_1_FAILURE 0x02
-#define BATTERY_2_DISABLED 0x04
-#define BATTERY_2_FAILURE 0x08
-
-#define MEMCTRLSTATUS_MEMORY 0x07
-#define MEM_128_MB 0xfe
-#define MEM_256_MB 0xfc
-#define MEM_512_MB 0xf8
-#define MEM_1_GB 0xf0
-#define MEM_2_GB 0xe0
-
-#define MEMCTRLCMD_LEDCTRL 0x08
-#define LED_REMOVE 2
-#define LED_FAULT 4
-#define LED_POWER 6
-#define LED_FLIP 255
-#define LED_OFF 0x00
-#define LED_ON 0x01
-#define LED_FLASH_3_5 0x02
-#define LED_FLASH_7_0 0x03
-#define LED_POWER_ON 0x00
-#define LED_POWER_OFF 0x01
-#define USER_BIT1 0x01
-#define USER_BIT2 0x02
-
-#define MEMORY_INITIALIZED USER_BIT1
-
-#define MEMCTRLCMD_ERRCTRL 0x0C
-#define EDC_NONE_DEFAULT 0x00
-#define EDC_NONE 0x01
-#define EDC_STORE_READ 0x02
-#define EDC_STORE_CORRECT 0x03
-
-#define MEMCTRLCMD_ERRCNT 0x0D
-#define MEMCTRLCMD_ERRSTATUS 0x0E
-
-#define ERROR_DATA_LOG 0x20
-#define ERROR_ADDR_LOG 0x28
-#define ERROR_COUNT 0x3D
-#define ERROR_SYNDROME 0x3E
-#define ERROR_CHECK 0x3F
-
-#define DMA_PCI_ADDR 0x40
-#define DMA_LOCAL_ADDR 0x48
-#define DMA_TRANSFER_SIZE 0x50
-#define DMA_DESCRIPTOR_ADDR 0x58
-#define DMA_SEMAPHORE_ADDR 0x60
-#define DMA_STATUS_CTRL 0x68
-#define DMASCR_GO 0x00001
-#define DMASCR_TRANSFER_READ 0x00002
-#define DMASCR_CHAIN_EN 0x00004
-#define DMASCR_SEM_EN 0x00010
-#define DMASCR_DMA_COMP_EN 0x00020
-#define DMASCR_CHAIN_COMP_EN 0x00040
-#define DMASCR_ERR_INT_EN 0x00080
-#define DMASCR_PARITY_INT_EN 0x00100
-#define DMASCR_ANY_ERR 0x00800
-#define DMASCR_MBE_ERR 0x01000
-#define DMASCR_PARITY_ERR_REP 0x02000
-#define DMASCR_PARITY_ERR_DET 0x04000
-#define DMASCR_SYSTEM_ERR_SIG 0x08000
-#define DMASCR_TARGET_ABT 0x10000
-#define DMASCR_MASTER_ABT 0x20000
-#define DMASCR_DMA_COMPLETE 0x40000
-#define DMASCR_CHAIN_COMPLETE 0x80000
-
-/*
-3.SOME PCs HAVE HOST BRIDGES WHICH APPARENTLY DO NOT CORRECTLY HANDLE
-READ-LINE (0xE) OR READ-MULTIPLE (0xC) PCI COMMAND CODES DURING DMA
-TRANSFERS. IN OTHER SYSTEMS THESE COMMAND CODES WILL CAUSE THE HOST BRIDGE
-TO ALLOW LONGER BURSTS DURING DMA READ OPERATIONS. THE UPPER FOUR BITS
-(31..28) OF THE DMA CSR HAVE BEEN MADE PROGRAMMABLE, SO THAT EITHER A 0x6,
-AN 0xE OR A 0xC CAN BE WRITTEN TO THEM TO SET THE COMMAND CODE USED DURING
-DMA READ OPERATIONS.
-*/
-#define DMASCR_READ 0x60000000
-#define DMASCR_READLINE 0xE0000000
-#define DMASCR_READMULTI 0xC0000000
-
-
-#define DMASCR_ERROR_MASK (DMASCR_MASTER_ABT | DMASCR_TARGET_ABT | DMASCR_SYSTEM_ERR_SIG | DMASCR_PARITY_ERR_DET | DMASCR_MBE_ERR | DMASCR_ANY_ERR)
-#define DMASCR_HARD_ERROR (DMASCR_MASTER_ABT | DMASCR_TARGET_ABT | DMASCR_SYSTEM_ERR_SIG | DMASCR_PARITY_ERR_DET | DMASCR_MBE_ERR)
-
-#define WINDOWMAP_WINNUM 0x7B
-
-#define DMA_READ_FROM_HOST 0
-#define DMA_WRITE_TO_HOST 1
-
-struct mm_dma_desc {
- __le64 pci_addr;
- __le64 local_addr;
- __le32 transfer_size;
- u32 zero1;
- __le64 next_desc_addr;
- __le64 sem_addr;
- __le32 control_bits;
- u32 zero2;
-
- dma_addr_t data_dma_handle;
-
- /* Copy of the bits */
- __le64 sem_control_bits;
-} __attribute__((aligned(8)));
-
-/* bits for card->flags */
-#define UM_FLAG_DMA_IN_REGS 1
-#define UM_FLAG_NO_BYTE_STATUS 2
-#define UM_FLAG_NO_BATTREG 4
-#define UM_FLAG_NO_BATT 8
-#endif
diff --git a/drivers/block/xen-blkback/blkback.c b/drivers/block/xen-blkback/blkback.c
index 1cdf09ff67b6..14e452896d04 100644
--- a/drivers/block/xen-blkback/blkback.c
+++ b/drivers/block/xen-blkback/blkback.c
@@ -891,7 +891,7 @@ next:
out:
for (i = last_map; i < num; i++) {
/* Don't zap current batch's valid persistent grants. */
- if(i >= last_map + segs_to_map)
+ if(i >= map_until)
pages[i]->persistent_gnt = NULL;
pages[i]->handle = BLKBACK_INVALID_HANDLE;
}
diff --git a/drivers/block/xen-blkback/common.h b/drivers/block/xen-blkback/common.h
index b0c71d3a81a0..bda5c815e441 100644
--- a/drivers/block/xen-blkback/common.h
+++ b/drivers/block/xen-blkback/common.h
@@ -313,6 +313,7 @@ struct xen_blkif {
struct work_struct free_work;
unsigned int nr_ring_pages;
+ bool multi_ref;
/* All rings for this device. */
struct xen_blkif_ring *rings;
unsigned int nr_rings;
diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
index c2aaf690352c..125b22205d38 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -998,14 +998,17 @@ static int read_per_ring_refs(struct xen_blkif_ring *ring, const char *dir)
for (i = 0; i < nr_grefs; i++) {
char ring_ref_name[RINGREF_NAME_LEN];
- snprintf(ring_ref_name, RINGREF_NAME_LEN, "ring-ref%u", i);
+ if (blkif->multi_ref)
+ snprintf(ring_ref_name, RINGREF_NAME_LEN, "ring-ref%u", i);
+ else {
+ WARN_ON(i != 0);
+ snprintf(ring_ref_name, RINGREF_NAME_LEN, "ring-ref");
+ }
+
err = xenbus_scanf(XBT_NIL, dir, ring_ref_name,
"%u", &ring_ref[i]);
if (err != 1) {
- if (nr_grefs == 1)
- break;
-
err = -EINVAL;
xenbus_dev_fatal(dev, err, "reading %s/%s",
dir, ring_ref_name);
@@ -1013,18 +1016,6 @@ static int read_per_ring_refs(struct xen_blkif_ring *ring, const char *dir)
}
}
- if (err != 1) {
- WARN_ON(nr_grefs != 1);
-
- err = xenbus_scanf(XBT_NIL, dir, "ring-ref", "%u",
- &ring_ref[0]);
- if (err != 1) {
- err = -EINVAL;
- xenbus_dev_fatal(dev, err, "reading %s/ring-ref", dir);
- return err;
- }
- }
-
err = -ENOMEM;
for (i = 0; i < nr_grefs * XEN_BLKIF_REQS_PER_PAGE; i++) {
req = kzalloc(sizeof(*req), GFP_KERNEL);
@@ -1129,10 +1120,15 @@ static int connect_ring(struct backend_info *be)
blkif->nr_rings, blkif->blk_protocol, protocol,
blkif->vbd.feature_gnt_persistent ? "persistent grants" : "");
- ring_page_order = xenbus_read_unsigned(dev->otherend,
- "ring-page-order", 0);
-
- if (ring_page_order > xen_blkif_max_ring_order) {
+ err = xenbus_scanf(XBT_NIL, dev->otherend, "ring-page-order", "%u",
+ &ring_page_order);
+ if (err != 1) {
+ blkif->nr_ring_pages = 1;
+ blkif->multi_ref = false;
+ } else if (ring_page_order <= xen_blkif_max_ring_order) {
+ blkif->nr_ring_pages = 1 << ring_page_order;
+ blkif->multi_ref = true;
+ } else {
err = -EINVAL;
xenbus_dev_fatal(dev, err,
"requested ring page order %d exceed max:%d",
@@ -1141,8 +1137,6 @@ static int connect_ring(struct backend_info *be)
return err;
}
- blkif->nr_ring_pages = 1 << ring_page_order;
-
if (blkif->nr_rings == 1)
return read_per_ring_refs(&blkif->rings[0], dev->otherend);
else {
diff --git a/drivers/block/xen-blkfront.c b/drivers/block/xen-blkfront.c
index e1c6798889f4..10df39a8b18d 100644
--- a/drivers/block/xen-blkfront.c
+++ b/drivers/block/xen-blkfront.c
@@ -1949,7 +1949,7 @@ module_param(feature_persistent, bool, 0644);
MODULE_PARM_DESC(feature_persistent,
"Enables the persistent grants feature");
-/**
+/*
* Entry point to this code when a new device is created. Allocate the basic
* structures and the ring buffer for communication with the backend, and
* inform the backend of the appropriate details for those. Switch to
@@ -2075,7 +2075,7 @@ static int blkif_recover(struct blkfront_info *info)
return 0;
}
-/**
+/*
* We are reconnecting to the backend, due to a suspend/resume, or a backend
* driver restart. We tear down our blkif structure and recreate it, but
* leave the device-layer structures intact so that this is transparent to the
@@ -2397,7 +2397,7 @@ static void blkfront_connect(struct blkfront_info *info)
}
/*
- * physcial-sector-size is a newer field, so old backends may not
+ * physical-sector-size is a newer field, so old backends may not
* provide this. Assume physical sector size to be the same as
* sector_size in that case.
*/
@@ -2440,7 +2440,7 @@ fail:
return;
}
-/**
+/*
* Callback received when the backend's state changes.
*/
static void blkback_changed(struct xenbus_device *dev,
diff --git a/drivers/block/xsysace.c b/drivers/block/xsysace.c
deleted file mode 100644
index eb8ef65778c3..000000000000
--- a/drivers/block/xsysace.c
+++ /dev/null
@@ -1,1273 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-only
-/*
- * Xilinx SystemACE device driver
- *
- * Copyright 2007 Secret Lab Technologies Ltd.
- */
-
-/*
- * The SystemACE chip is designed to configure FPGAs by loading an FPGA
- * bitstream from a file on a CF card and squirting it into FPGAs connected
- * to the SystemACE JTAG chain. It also has the advantage of providing an
- * MPU interface which can be used to control the FPGA configuration process
- * and to use the attached CF card for general purpose storage.
- *
- * This driver is a block device driver for the SystemACE.
- *
- * Initialization:
- * The driver registers itself as a platform_device driver at module
- * load time. The platform bus will take care of calling the
- * ace_probe() method for all SystemACE instances in the system. Any
- * number of SystemACE instances are supported. ace_probe() calls
- * ace_setup() which initialized all data structures, reads the CF
- * id structure and registers the device.
- *
- * Processing:
- * Just about all of the heavy lifting in this driver is performed by
- * a Finite State Machine (FSM). The driver needs to wait on a number
- * of events; some raised by interrupts, some which need to be polled
- * for. Describing all of the behaviour in a FSM seems to be the
- * easiest way to keep the complexity low and make it easy to
- * understand what the driver is doing. If the block ops or the
- * request function need to interact with the hardware, then they
- * simply need to flag the request and kick of FSM processing.
- *
- * The FSM itself is atomic-safe code which can be run from any
- * context. The general process flow is:
- * 1. obtain the ace->lock spinlock.
- * 2. loop on ace_fsm_dostate() until the ace->fsm_continue flag is
- * cleared.
- * 3. release the lock.
- *
- * Individual states do not sleep in any way. If a condition needs to
- * be waited for then the state much clear the fsm_continue flag and
- * either schedule the FSM to be run again at a later time, or expect
- * an interrupt to call the FSM when the desired condition is met.
- *
- * In normal operation, the FSM is processed at interrupt context
- * either when the driver's tasklet is scheduled, or when an irq is
- * raised by the hardware. The tasklet can be scheduled at any time.
- * The request method in particular schedules the tasklet when a new
- * request has been indicated by the block layer. Once started, the
- * FSM proceeds as far as it can processing the request until it
- * needs on a hardware event. At this point, it must yield execution.
- *
- * A state has two options when yielding execution:
- * 1. ace_fsm_yield()
- * - Call if need to poll for event.
- * - clears the fsm_continue flag to exit the processing loop
- * - reschedules the tasklet to run again as soon as possible
- * 2. ace_fsm_yieldirq()
- * - Call if an irq is expected from the HW
- * - clears the fsm_continue flag to exit the processing loop
- * - does not reschedule the tasklet so the FSM will not be processed
- * again until an irq is received.
- * After calling a yield function, the state must return control back
- * to the FSM main loop.
- *
- * Additionally, the driver maintains a kernel timer which can process
- * the FSM. If the FSM gets stalled, typically due to a missed
- * interrupt, then the kernel timer will expire and the driver can
- * continue where it left off.
- *
- * To Do:
- * - Add FPGA configuration control interface.
- * - Request major number from lanana
- */
-
-#undef DEBUG
-
-#include <linux/module.h>
-#include <linux/ctype.h>
-#include <linux/init.h>
-#include <linux/interrupt.h>
-#include <linux/errno.h>
-#include <linux/kernel.h>
-#include <linux/delay.h>
-#include <linux/slab.h>
-#include <linux/blk-mq.h>
-#include <linux/mutex.h>
-#include <linux/ata.h>
-#include <linux/hdreg.h>
-#include <linux/platform_device.h>
-#if defined(CONFIG_OF)
-#include <linux/of_address.h>
-#include <linux/of_device.h>
-#include <linux/of_platform.h>
-#endif
-
-MODULE_AUTHOR("Grant Likely <grant.likely@secretlab.ca>");
-MODULE_DESCRIPTION("Xilinx SystemACE device driver");
-MODULE_LICENSE("GPL");
-
-/* SystemACE register definitions */
-#define ACE_BUSMODE (0x00)
-
-#define ACE_STATUS (0x04)
-#define ACE_STATUS_CFGLOCK (0x00000001)
-#define ACE_STATUS_MPULOCK (0x00000002)
-#define ACE_STATUS_CFGERROR (0x00000004) /* config controller error */
-#define ACE_STATUS_CFCERROR (0x00000008) /* CF controller error */
-#define ACE_STATUS_CFDETECT (0x00000010)
-#define ACE_STATUS_DATABUFRDY (0x00000020)
-#define ACE_STATUS_DATABUFMODE (0x00000040)
-#define ACE_STATUS_CFGDONE (0x00000080)
-#define ACE_STATUS_RDYFORCFCMD (0x00000100)
-#define ACE_STATUS_CFGMODEPIN (0x00000200)
-#define ACE_STATUS_CFGADDR_MASK (0x0000e000)
-#define ACE_STATUS_CFBSY (0x00020000)
-#define ACE_STATUS_CFRDY (0x00040000)
-#define ACE_STATUS_CFDWF (0x00080000)
-#define ACE_STATUS_CFDSC (0x00100000)
-#define ACE_STATUS_CFDRQ (0x00200000)
-#define ACE_STATUS_CFCORR (0x00400000)
-#define ACE_STATUS_CFERR (0x00800000)
-
-#define ACE_ERROR (0x08)
-#define ACE_CFGLBA (0x0c)
-#define ACE_MPULBA (0x10)
-
-#define ACE_SECCNTCMD (0x14)
-#define ACE_SECCNTCMD_RESET (0x0100)
-#define ACE_SECCNTCMD_IDENTIFY (0x0200)
-#define ACE_SECCNTCMD_READ_DATA (0x0300)
-#define ACE_SECCNTCMD_WRITE_DATA (0x0400)
-#define ACE_SECCNTCMD_ABORT (0x0600)
-
-#define ACE_VERSION (0x16)
-#define ACE_VERSION_REVISION_MASK (0x00FF)
-#define ACE_VERSION_MINOR_MASK (0x0F00)
-#define ACE_VERSION_MAJOR_MASK (0xF000)
-
-#define ACE_CTRL (0x18)
-#define ACE_CTRL_FORCELOCKREQ (0x0001)
-#define ACE_CTRL_LOCKREQ (0x0002)
-#define ACE_CTRL_FORCECFGADDR (0x0004)
-#define ACE_CTRL_FORCECFGMODE (0x0008)
-#define ACE_CTRL_CFGMODE (0x0010)
-#define ACE_CTRL_CFGSTART (0x0020)
-#define ACE_CTRL_CFGSEL (0x0040)
-#define ACE_CTRL_CFGRESET (0x0080)
-#define ACE_CTRL_DATABUFRDYIRQ (0x0100)
-#define ACE_CTRL_ERRORIRQ (0x0200)
-#define ACE_CTRL_CFGDONEIRQ (0x0400)
-#define ACE_CTRL_RESETIRQ (0x0800)
-#define ACE_CTRL_CFGPROG (0x1000)
-#define ACE_CTRL_CFGADDR_MASK (0xe000)
-
-#define ACE_FATSTAT (0x1c)
-
-#define ACE_NUM_MINORS 16
-#define ACE_SECTOR_SIZE (512)
-#define ACE_FIFO_SIZE (32)
-#define ACE_BUF_PER_SECTOR (ACE_SECTOR_SIZE / ACE_FIFO_SIZE)
-
-#define ACE_BUS_WIDTH_8 0
-#define ACE_BUS_WIDTH_16 1
-
-struct ace_reg_ops;
-
-struct ace_device {
- /* driver state data */
- int id;
- int media_change;
- int users;
- struct list_head list;
-
- /* finite state machine data */
- struct tasklet_struct fsm_tasklet;
- uint fsm_task; /* Current activity (ACE_TASK_*) */
- uint fsm_state; /* Current state (ACE_FSM_STATE_*) */
- uint fsm_continue_flag; /* cleared to exit FSM mainloop */
- uint fsm_iter_num;
- struct timer_list stall_timer;
-
- /* Transfer state/result, use for both id and block request */
- struct request *req; /* request being processed */
- void *data_ptr; /* pointer to I/O buffer */
- int data_count; /* number of buffers remaining */
- int data_result; /* Result of transfer; 0 := success */
-
- int id_req_count; /* count of id requests */
- int id_result;
- struct completion id_completion; /* used when id req finishes */
- int in_irq;
-
- /* Details of hardware device */
- resource_size_t physaddr;
- void __iomem *baseaddr;
- int irq;
- int bus_width; /* 0 := 8 bit; 1 := 16 bit */
- struct ace_reg_ops *reg_ops;
- int lock_count;
-
- /* Block device data structures */
- spinlock_t lock;
- struct device *dev;
- struct request_queue *queue;
- struct gendisk *gd;
- struct blk_mq_tag_set tag_set;
- struct list_head rq_list;
-
- /* Inserted CF card parameters */
- u16 cf_id[ATA_ID_WORDS];
-};
-
-static DEFINE_MUTEX(xsysace_mutex);
-static int ace_major;
-
-/* ---------------------------------------------------------------------
- * Low level register access
- */
-
-struct ace_reg_ops {
- u16(*in) (struct ace_device * ace, int reg);
- void (*out) (struct ace_device * ace, int reg, u16 val);
- void (*datain) (struct ace_device * ace);
- void (*dataout) (struct ace_device * ace);
-};
-
-/* 8 Bit bus width */
-static u16 ace_in_8(struct ace_device *ace, int reg)
-{
- void __iomem *r = ace->baseaddr + reg;
- return in_8(r) | (in_8(r + 1) << 8);
-}
-
-static void ace_out_8(struct ace_device *ace, int reg, u16 val)
-{
- void __iomem *r = ace->baseaddr + reg;
- out_8(r, val);
- out_8(r + 1, val >> 8);
-}
-
-static void ace_datain_8(struct ace_device *ace)
-{
- void __iomem *r = ace->baseaddr + 0x40;
- u8 *dst = ace->data_ptr;
- int i = ACE_FIFO_SIZE;
- while (i--)
- *dst++ = in_8(r++);
- ace->data_ptr = dst;
-}
-
-static void ace_dataout_8(struct ace_device *ace)
-{
- void __iomem *r = ace->baseaddr + 0x40;
- u8 *src = ace->data_ptr;
- int i = ACE_FIFO_SIZE;
- while (i--)
- out_8(r++, *src++);
- ace->data_ptr = src;
-}
-
-static struct ace_reg_ops ace_reg_8_ops = {
- .in = ace_in_8,
- .out = ace_out_8,
- .datain = ace_datain_8,
- .dataout = ace_dataout_8,
-};
-
-/* 16 bit big endian bus attachment */
-static u16 ace_in_be16(struct ace_device *ace, int reg)
-{
- return in_be16(ace->baseaddr + reg);
-}
-
-static void ace_out_be16(struct ace_device *ace, int reg, u16 val)
-{
- out_be16(ace->baseaddr + reg, val);
-}
-
-static void ace_datain_be16(struct ace_device *ace)
-{
- int i = ACE_FIFO_SIZE / 2;
- u16 *dst = ace->data_ptr;
- while (i--)
- *dst++ = in_le16(ace->baseaddr + 0x40);
- ace->data_ptr = dst;
-}
-
-static void ace_dataout_be16(struct ace_device *ace)
-{
- int i = ACE_FIFO_SIZE / 2;
- u16 *src = ace->data_ptr;
- while (i--)
- out_le16(ace->baseaddr + 0x40, *src++);
- ace->data_ptr = src;
-}
-
-/* 16 bit little endian bus attachment */
-static u16 ace_in_le16(struct ace_device *ace, int reg)
-{
- return in_le16(ace->baseaddr + reg);
-}
-
-static void ace_out_le16(struct ace_device *ace, int reg, u16 val)
-{
- out_le16(ace->baseaddr + reg, val);
-}
-
-static void ace_datain_le16(struct ace_device *ace)
-{
- int i = ACE_FIFO_SIZE / 2;
- u16 *dst = ace->data_ptr;
- while (i--)
- *dst++ = in_be16(ace->baseaddr + 0x40);
- ace->data_ptr = dst;
-}
-
-static void ace_dataout_le16(struct ace_device *ace)
-{
- int i = ACE_FIFO_SIZE / 2;
- u16 *src = ace->data_ptr;
- while (i--)
- out_be16(ace->baseaddr + 0x40, *src++);
- ace->data_ptr = src;
-}
-
-static struct ace_reg_ops ace_reg_be16_ops = {
- .in = ace_in_be16,
- .out = ace_out_be16,
- .datain = ace_datain_be16,
- .dataout = ace_dataout_be16,
-};
-
-static struct ace_reg_ops ace_reg_le16_ops = {
- .in = ace_in_le16,
- .out = ace_out_le16,
- .datain = ace_datain_le16,
- .dataout = ace_dataout_le16,
-};
-
-static inline u16 ace_in(struct ace_device *ace, int reg)
-{
- return ace->reg_ops->in(ace, reg);
-}
-
-static inline u32 ace_in32(struct ace_device *ace, int reg)
-{
- return ace_in(ace, reg) | (ace_in(ace, reg + 2) << 16);
-}
-
-static inline void ace_out(struct ace_device *ace, int reg, u16 val)
-{
- ace->reg_ops->out(ace, reg, val);
-}
-
-static inline void ace_out32(struct ace_device *ace, int reg, u32 val)
-{
- ace_out(ace, reg, val);
- ace_out(ace, reg + 2, val >> 16);
-}
-
-/* ---------------------------------------------------------------------
- * Debug support functions
- */
-
-#if defined(DEBUG)
-static void ace_dump_mem(void *base, int len)
-{
- const char *ptr = base;
- int i, j;
-
- for (i = 0; i < len; i += 16) {
- printk(KERN_INFO "%.8x:", i);
- for (j = 0; j < 16; j++) {
- if (!(j % 4))
- printk(" ");
- printk("%.2x", ptr[i + j]);
- }
- printk(" ");
- for (j = 0; j < 16; j++)
- printk("%c", isprint(ptr[i + j]) ? ptr[i + j] : '.');
- printk("\n");
- }
-}
-#else
-static inline void ace_dump_mem(void *base, int len)
-{
-}
-#endif
-
-static void ace_dump_regs(struct ace_device *ace)
-{
- dev_info(ace->dev,
- " ctrl: %.8x seccnt/cmd: %.4x ver:%.4x\n"
- " status:%.8x mpu_lba:%.8x busmode:%4x\n"
- " error: %.8x cfg_lba:%.8x fatstat:%.4x\n",
- ace_in32(ace, ACE_CTRL),
- ace_in(ace, ACE_SECCNTCMD),
- ace_in(ace, ACE_VERSION),
- ace_in32(ace, ACE_STATUS),
- ace_in32(ace, ACE_MPULBA),
- ace_in(ace, ACE_BUSMODE),
- ace_in32(ace, ACE_ERROR),
- ace_in32(ace, ACE_CFGLBA), ace_in(ace, ACE_FATSTAT));
-}
-
-static void ace_fix_driveid(u16 *id)
-{
-#if defined(__BIG_ENDIAN)
- int i;
-
- /* All half words have wrong byte order; swap the bytes */
- for (i = 0; i < ATA_ID_WORDS; i++, id++)
- *id = le16_to_cpu(*id);
-#endif
-}
-
-/* ---------------------------------------------------------------------
- * Finite State Machine (FSM) implementation
- */
-
-/* FSM tasks; used to direct state transitions */
-#define ACE_TASK_IDLE 0
-#define ACE_TASK_IDENTIFY 1
-#define ACE_TASK_READ 2
-#define ACE_TASK_WRITE 3
-#define ACE_FSM_NUM_TASKS 4
-
-/* FSM state definitions */
-#define ACE_FSM_STATE_IDLE 0
-#define ACE_FSM_STATE_REQ_LOCK 1
-#define ACE_FSM_STATE_WAIT_LOCK 2
-#define ACE_FSM_STATE_WAIT_CFREADY 3
-#define ACE_FSM_STATE_IDENTIFY_PREPARE 4
-#define ACE_FSM_STATE_IDENTIFY_TRANSFER 5
-#define ACE_FSM_STATE_IDENTIFY_COMPLETE 6
-#define ACE_FSM_STATE_REQ_PREPARE 7
-#define ACE_FSM_STATE_REQ_TRANSFER 8
-#define ACE_FSM_STATE_REQ_COMPLETE 9
-#define ACE_FSM_STATE_ERROR 10
-#define ACE_FSM_NUM_STATES 11
-
-/* Set flag to exit FSM loop and reschedule tasklet */
-static inline void ace_fsm_yieldpoll(struct ace_device *ace)
-{
- tasklet_schedule(&ace->fsm_tasklet);
- ace->fsm_continue_flag = 0;
-}
-
-static inline void ace_fsm_yield(struct ace_device *ace)
-{
- dev_dbg(ace->dev, "%s()\n", __func__);
- ace_fsm_yieldpoll(ace);
-}
-
-/* Set flag to exit FSM loop and wait for IRQ to reschedule tasklet */
-static inline void ace_fsm_yieldirq(struct ace_device *ace)
-{
- dev_dbg(ace->dev, "ace_fsm_yieldirq()\n");
-
- if (ace->irq > 0)
- ace->fsm_continue_flag = 0;
- else
- ace_fsm_yieldpoll(ace);
-}
-
-static bool ace_has_next_request(struct request_queue *q)
-{
- struct ace_device *ace = q->queuedata;
-
- return !list_empty(&ace->rq_list);
-}
-
-/* Get the next read/write request; ending requests that we don't handle */
-static struct request *ace_get_next_request(struct request_queue *q)
-{
- struct ace_device *ace = q->queuedata;
- struct request *rq;
-
- rq = list_first_entry_or_null(&ace->rq_list, struct request, queuelist);
- if (rq) {
- list_del_init(&rq->queuelist);
- blk_mq_start_request(rq);
- }
-
- return NULL;
-}
-
-static void ace_fsm_dostate(struct ace_device *ace)
-{
- struct request *req;
- u32 status;
- u16 val;
- int count;
-
-#if defined(DEBUG)
- dev_dbg(ace->dev, "fsm_state=%i, id_req_count=%i\n",
- ace->fsm_state, ace->id_req_count);
-#endif
-
- /* Verify that there is actually a CF in the slot. If not, then
- * bail out back to the idle state and wake up all the waiters */
- status = ace_in32(ace, ACE_STATUS);
- if ((status & ACE_STATUS_CFDETECT) == 0) {
- ace->fsm_state = ACE_FSM_STATE_IDLE;
- ace->media_change = 1;
- set_capacity(ace->gd, 0);
- dev_info(ace->dev, "No CF in slot\n");
-
- /* Drop all in-flight and pending requests */
- if (ace->req) {
- blk_mq_end_request(ace->req, BLK_STS_IOERR);
- ace->req = NULL;
- }
- while ((req = ace_get_next_request(ace->queue)) != NULL)
- blk_mq_end_request(req, BLK_STS_IOERR);
-
- /* Drop back to IDLE state and notify waiters */
- ace->fsm_state = ACE_FSM_STATE_IDLE;
- ace->id_result = -EIO;
- while (ace->id_req_count) {
- complete(&ace->id_completion);
- ace->id_req_count--;
- }
- }
-
- switch (ace->fsm_state) {
- case ACE_FSM_STATE_IDLE:
- /* See if there is anything to do */
- if (ace->id_req_count || ace_has_next_request(ace->queue)) {
- ace->fsm_iter_num++;
- ace->fsm_state = ACE_FSM_STATE_REQ_LOCK;
- mod_timer(&ace->stall_timer, jiffies + HZ);
- if (!timer_pending(&ace->stall_timer))
- add_timer(&ace->stall_timer);
- break;
- }
- del_timer(&ace->stall_timer);
- ace->fsm_continue_flag = 0;
- break;
-
- case ACE_FSM_STATE_REQ_LOCK:
- if (ace_in(ace, ACE_STATUS) & ACE_STATUS_MPULOCK) {
- /* Already have the lock, jump to next state */
- ace->fsm_state = ACE_FSM_STATE_WAIT_CFREADY;
- break;
- }
-
- /* Request the lock */
- val = ace_in(ace, ACE_CTRL);
- ace_out(ace, ACE_CTRL, val | ACE_CTRL_LOCKREQ);
- ace->fsm_state = ACE_FSM_STATE_WAIT_LOCK;
- break;
-
- case ACE_FSM_STATE_WAIT_LOCK:
- if (ace_in(ace, ACE_STATUS) & ACE_STATUS_MPULOCK) {
- /* got the lock; move to next state */
- ace->fsm_state = ACE_FSM_STATE_WAIT_CFREADY;
- break;
- }
-
- /* wait a bit for the lock */
- ace_fsm_yield(ace);
- break;
-
- case ACE_FSM_STATE_WAIT_CFREADY:
- status = ace_in32(ace, ACE_STATUS);
- if (!(status & ACE_STATUS_RDYFORCFCMD) ||
- (status & ACE_STATUS_CFBSY)) {
- /* CF card isn't ready; it needs to be polled */
- ace_fsm_yield(ace);
- break;
- }
-
- /* Device is ready for command; determine what to do next */
- if (ace->id_req_count)
- ace->fsm_state = ACE_FSM_STATE_IDENTIFY_PREPARE;
- else
- ace->fsm_state = ACE_FSM_STATE_REQ_PREPARE;
- break;
-
- case ACE_FSM_STATE_IDENTIFY_PREPARE:
- /* Send identify command */
- ace->fsm_task = ACE_TASK_IDENTIFY;
- ace->data_ptr = ace->cf_id;
- ace->data_count = ACE_BUF_PER_SECTOR;
- ace_out(ace, ACE_SECCNTCMD, ACE_SECCNTCMD_IDENTIFY);
-
- /* As per datasheet, put config controller in reset */
- val = ace_in(ace, ACE_CTRL);
- ace_out(ace, ACE_CTRL, val | ACE_CTRL_CFGRESET);
-
- /* irq handler takes over from this point; wait for the
- * transfer to complete */
- ace->fsm_state = ACE_FSM_STATE_IDENTIFY_TRANSFER;
- ace_fsm_yieldirq(ace);
- break;
-
- case ACE_FSM_STATE_IDENTIFY_TRANSFER:
- /* Check that the sysace is ready to receive data */
- status = ace_in32(ace, ACE_STATUS);
- if (status & ACE_STATUS_CFBSY) {
- dev_dbg(ace->dev, "CFBSY set; t=%i iter=%i dc=%i\n",
- ace->fsm_task, ace->fsm_iter_num,
- ace->data_count);
- ace_fsm_yield(ace);
- break;
- }
- if (!(status & ACE_STATUS_DATABUFRDY)) {
- ace_fsm_yield(ace);
- break;
- }
-
- /* Transfer the next buffer */
- ace->reg_ops->datain(ace);
- ace->data_count--;
-
- /* If there are still buffers to be transfers; jump out here */
- if (ace->data_count != 0) {
- ace_fsm_yieldirq(ace);
- break;
- }
-
- /* transfer finished; kick state machine */
- dev_dbg(ace->dev, "identify finished\n");
- ace->fsm_state = ACE_FSM_STATE_IDENTIFY_COMPLETE;
- break;
-
- case ACE_FSM_STATE_IDENTIFY_COMPLETE:
- ace_fix_driveid(ace->cf_id);
- ace_dump_mem(ace->cf_id, 512); /* Debug: Dump out disk ID */
-
- if (ace->data_result) {
- /* Error occurred, disable the disk */
- ace->media_change = 1;
- set_capacity(ace->gd, 0);
- dev_err(ace->dev, "error fetching CF id (%i)\n",
- ace->data_result);
- } else {
- ace->media_change = 0;
-
- /* Record disk parameters */
- set_capacity(ace->gd,
- ata_id_u32(ace->cf_id, ATA_ID_LBA_CAPACITY));
- dev_info(ace->dev, "capacity: %i sectors\n",
- ata_id_u32(ace->cf_id, ATA_ID_LBA_CAPACITY));
- }
-
- /* We're done, drop to IDLE state and notify waiters */
- ace->fsm_state = ACE_FSM_STATE_IDLE;
- ace->id_result = ace->data_result;
- while (ace->id_req_count) {
- complete(&ace->id_completion);
- ace->id_req_count--;
- }
- break;
-
- case ACE_FSM_STATE_REQ_PREPARE:
- req = ace_get_next_request(ace->queue);
- if (!req) {
- ace->fsm_state = ACE_FSM_STATE_IDLE;
- break;
- }
-
- /* Okay, it's a data request, set it up for transfer */
- dev_dbg(ace->dev,
- "request: sec=%llx hcnt=%x, ccnt=%x, dir=%i\n",
- (unsigned long long)blk_rq_pos(req),
- blk_rq_sectors(req), blk_rq_cur_sectors(req),
- rq_data_dir(req));
-
- ace->req = req;
- ace->data_ptr = bio_data(req->bio);
- ace->data_count = blk_rq_cur_sectors(req) * ACE_BUF_PER_SECTOR;
- ace_out32(ace, ACE_MPULBA, blk_rq_pos(req) & 0x0FFFFFFF);
-
- count = blk_rq_sectors(req);
- if (rq_data_dir(req)) {
- /* Kick off write request */
- dev_dbg(ace->dev, "write data\n");
- ace->fsm_task = ACE_TASK_WRITE;
- ace_out(ace, ACE_SECCNTCMD,
- count | ACE_SECCNTCMD_WRITE_DATA);
- } else {
- /* Kick off read request */
- dev_dbg(ace->dev, "read data\n");
- ace->fsm_task = ACE_TASK_READ;
- ace_out(ace, ACE_SECCNTCMD,
- count | ACE_SECCNTCMD_READ_DATA);
- }
-
- /* As per datasheet, put config controller in reset */
- val = ace_in(ace, ACE_CTRL);
- ace_out(ace, ACE_CTRL, val | ACE_CTRL_CFGRESET);
-
- /* Move to the transfer state. The systemace will raise
- * an interrupt once there is something to do
- */
- ace->fsm_state = ACE_FSM_STATE_REQ_TRANSFER;
- if (ace->fsm_task == ACE_TASK_READ)
- ace_fsm_yieldirq(ace); /* wait for data ready */
- break;
-
- case ACE_FSM_STATE_REQ_TRANSFER:
- /* Check that the sysace is ready to receive data */
- status = ace_in32(ace, ACE_STATUS);
- if (status & ACE_STATUS_CFBSY) {
- dev_dbg(ace->dev,
- "CFBSY set; t=%i iter=%i c=%i dc=%i irq=%i\n",
- ace->fsm_task, ace->fsm_iter_num,
- blk_rq_cur_sectors(ace->req) * 16,
- ace->data_count, ace->in_irq);
- ace_fsm_yield(ace); /* need to poll CFBSY bit */
- break;
- }
- if (!(status & ACE_STATUS_DATABUFRDY)) {
- dev_dbg(ace->dev,
- "DATABUF not set; t=%i iter=%i c=%i dc=%i irq=%i\n",
- ace->fsm_task, ace->fsm_iter_num,
- blk_rq_cur_sectors(ace->req) * 16,
- ace->data_count, ace->in_irq);
- ace_fsm_yieldirq(ace);
- break;
- }
-
- /* Transfer the next buffer */
- if (ace->fsm_task == ACE_TASK_WRITE)
- ace->reg_ops->dataout(ace);
- else
- ace->reg_ops->datain(ace);
- ace->data_count--;
-
- /* If there are still buffers to be transfers; jump out here */
- if (ace->data_count != 0) {
- ace_fsm_yieldirq(ace);
- break;
- }
-
- /* bio finished; is there another one? */
- if (blk_update_request(ace->req, BLK_STS_OK,
- blk_rq_cur_bytes(ace->req))) {
- /* dev_dbg(ace->dev, "next block; h=%u c=%u\n",
- * blk_rq_sectors(ace->req),
- * blk_rq_cur_sectors(ace->req));
- */
- ace->data_ptr = bio_data(ace->req->bio);
- ace->data_count = blk_rq_cur_sectors(ace->req) * 16;
- ace_fsm_yieldirq(ace);
- break;
- }
-
- ace->fsm_state = ACE_FSM_STATE_REQ_COMPLETE;
- break;
-
- case ACE_FSM_STATE_REQ_COMPLETE:
- ace->req = NULL;
-
- /* Finished request; go to idle state */
- ace->fsm_state = ACE_FSM_STATE_IDLE;
- break;
-
- default:
- ace->fsm_state = ACE_FSM_STATE_IDLE;
- break;
- }
-}
-
-static void ace_fsm_tasklet(unsigned long data)
-{
- struct ace_device *ace = (void *)data;
- unsigned long flags;
-
- spin_lock_irqsave(&ace->lock, flags);
-
- /* Loop over state machine until told to stop */
- ace->fsm_continue_flag = 1;
- while (ace->fsm_continue_flag)
- ace_fsm_dostate(ace);
-
- spin_unlock_irqrestore(&ace->lock, flags);
-}
-
-static void ace_stall_timer(struct timer_list *t)
-{
- struct ace_device *ace = from_timer(ace, t, stall_timer);
- unsigned long flags;
-
- dev_warn(ace->dev,
- "kicking stalled fsm; state=%i task=%i iter=%i dc=%i\n",
- ace->fsm_state, ace->fsm_task, ace->fsm_iter_num,
- ace->data_count);
- spin_lock_irqsave(&ace->lock, flags);
-
- /* Rearm the stall timer *before* entering FSM (which may then
- * delete the timer) */
- mod_timer(&ace->stall_timer, jiffies + HZ);
-
- /* Loop over state machine until told to stop */
- ace->fsm_continue_flag = 1;
- while (ace->fsm_continue_flag)
- ace_fsm_dostate(ace);
-
- spin_unlock_irqrestore(&ace->lock, flags);
-}
-
-/* ---------------------------------------------------------------------
- * Interrupt handling routines
- */
-static int ace_interrupt_checkstate(struct ace_device *ace)
-{
- u32 sreg = ace_in32(ace, ACE_STATUS);
- u16 creg = ace_in(ace, ACE_CTRL);
-
- /* Check for error occurrence */
- if ((sreg & (ACE_STATUS_CFGERROR | ACE_STATUS_CFCERROR)) &&
- (creg & ACE_CTRL_ERRORIRQ)) {
- dev_err(ace->dev, "transfer failure\n");
- ace_dump_regs(ace);
- return -EIO;
- }
-
- return 0;
-}
-
-static irqreturn_t ace_interrupt(int irq, void *dev_id)
-{
- u16 creg;
- struct ace_device *ace = dev_id;
-
- /* be safe and get the lock */
- spin_lock(&ace->lock);
- ace->in_irq = 1;
-
- /* clear the interrupt */
- creg = ace_in(ace, ACE_CTRL);
- ace_out(ace, ACE_CTRL, creg | ACE_CTRL_RESETIRQ);
- ace_out(ace, ACE_CTRL, creg);
-
- /* check for IO failures */
- if (ace_interrupt_checkstate(ace))
- ace->data_result = -EIO;
-
- if (ace->fsm_task == 0) {
- dev_err(ace->dev,
- "spurious irq; stat=%.8x ctrl=%.8x cmd=%.4x\n",
- ace_in32(ace, ACE_STATUS), ace_in32(ace, ACE_CTRL),
- ace_in(ace, ACE_SECCNTCMD));
- dev_err(ace->dev, "fsm_task=%i fsm_state=%i data_count=%i\n",
- ace->fsm_task, ace->fsm_state, ace->data_count);
- }
-
- /* Loop over state machine until told to stop */
- ace->fsm_continue_flag = 1;
- while (ace->fsm_continue_flag)
- ace_fsm_dostate(ace);
-
- /* done with interrupt; drop the lock */
- ace->in_irq = 0;
- spin_unlock(&ace->lock);
-
- return IRQ_HANDLED;
-}
-
-/* ---------------------------------------------------------------------
- * Block ops
- */
-static blk_status_t ace_queue_rq(struct blk_mq_hw_ctx *hctx,
- const struct blk_mq_queue_data *bd)
-{
- struct ace_device *ace = hctx->queue->queuedata;
- struct request *req = bd->rq;
-
- if (blk_rq_is_passthrough(req)) {
- blk_mq_start_request(req);
- return BLK_STS_IOERR;
- }
-
- spin_lock_irq(&ace->lock);
- list_add_tail(&req->queuelist, &ace->rq_list);
- spin_unlock_irq(&ace->lock);
-
- tasklet_schedule(&ace->fsm_tasklet);
- return BLK_STS_OK;
-}
-
-static unsigned int ace_check_events(struct gendisk *gd, unsigned int clearing)
-{
- struct ace_device *ace = gd->private_data;
- dev_dbg(ace->dev, "ace_check_events(): %i\n", ace->media_change);
-
- return ace->media_change ? DISK_EVENT_MEDIA_CHANGE : 0;
-}
-
-static void ace_media_changed(struct ace_device *ace)
-{
- unsigned long flags;
-
- dev_dbg(ace->dev, "requesting cf id and scheduling tasklet\n");
-
- spin_lock_irqsave(&ace->lock, flags);
- ace->id_req_count++;
- spin_unlock_irqrestore(&ace->lock, flags);
-
- tasklet_schedule(&ace->fsm_tasklet);
- wait_for_completion(&ace->id_completion);
-
- dev_dbg(ace->dev, "revalidate complete\n");
-}
-
-static int ace_open(struct block_device *bdev, fmode_t mode)
-{
- struct ace_device *ace = bdev->bd_disk->private_data;
- unsigned long flags;
-
- dev_dbg(ace->dev, "ace_open() users=%i\n", ace->users + 1);
-
- mutex_lock(&xsysace_mutex);
- spin_lock_irqsave(&ace->lock, flags);
- ace->users++;
- spin_unlock_irqrestore(&ace->lock, flags);
-
- if (bdev_check_media_change(bdev) && ace->media_change)
- ace_media_changed(ace);
- mutex_unlock(&xsysace_mutex);
-
- return 0;
-}
-
-static void ace_release(struct gendisk *disk, fmode_t mode)
-{
- struct ace_device *ace = disk->private_data;
- unsigned long flags;
- u16 val;
-
- dev_dbg(ace->dev, "ace_release() users=%i\n", ace->users - 1);
-
- mutex_lock(&xsysace_mutex);
- spin_lock_irqsave(&ace->lock, flags);
- ace->users--;
- if (ace->users == 0) {
- val = ace_in(ace, ACE_CTRL);
- ace_out(ace, ACE_CTRL, val & ~ACE_CTRL_LOCKREQ);
- }
- spin_unlock_irqrestore(&ace->lock, flags);
- mutex_unlock(&xsysace_mutex);
-}
-
-static int ace_getgeo(struct block_device *bdev, struct hd_geometry *geo)
-{
- struct ace_device *ace = bdev->bd_disk->private_data;
- u16 *cf_id = ace->cf_id;
-
- dev_dbg(ace->dev, "ace_getgeo()\n");
-
- geo->heads = cf_id[ATA_ID_HEADS];
- geo->sectors = cf_id[ATA_ID_SECTORS];
- geo->cylinders = cf_id[ATA_ID_CYLS];
-
- return 0;
-}
-
-static const struct block_device_operations ace_fops = {
- .owner = THIS_MODULE,
- .open = ace_open,
- .release = ace_release,
- .check_events = ace_check_events,
- .getgeo = ace_getgeo,
-};
-
-static const struct blk_mq_ops ace_mq_ops = {
- .queue_rq = ace_queue_rq,
-};
-
-/* --------------------------------------------------------------------
- * SystemACE device setup/teardown code
- */
-static int ace_setup(struct ace_device *ace)
-{
- u16 version;
- u16 val;
- int rc;
-
- dev_dbg(ace->dev, "ace_setup(ace=0x%p)\n", ace);
- dev_dbg(ace->dev, "physaddr=0x%llx irq=%i\n",
- (unsigned long long)ace->physaddr, ace->irq);
-
- spin_lock_init(&ace->lock);
- init_completion(&ace->id_completion);
- INIT_LIST_HEAD(&ace->rq_list);
-
- /*
- * Map the device
- */
- ace->baseaddr = ioremap(ace->physaddr, 0x80);
- if (!ace->baseaddr)
- goto err_ioremap;
-
- /*
- * Initialize the state machine tasklet and stall timer
- */
- tasklet_init(&ace->fsm_tasklet, ace_fsm_tasklet, (unsigned long)ace);
- timer_setup(&ace->stall_timer, ace_stall_timer, 0);
-
- /*
- * Initialize the request queue
- */
- ace->queue = blk_mq_init_sq_queue(&ace->tag_set, &ace_mq_ops, 2,
- BLK_MQ_F_SHOULD_MERGE);
- if (IS_ERR(ace->queue)) {
- rc = PTR_ERR(ace->queue);
- ace->queue = NULL;
- goto err_blk_initq;
- }
- ace->queue->queuedata = ace;
-
- blk_queue_logical_block_size(ace->queue, 512);
- blk_queue_bounce_limit(ace->queue, BLK_BOUNCE_HIGH);
-
- /*
- * Allocate and initialize GD structure
- */
- ace->gd = alloc_disk(ACE_NUM_MINORS);
- if (!ace->gd)
- goto err_alloc_disk;
-
- ace->gd->major = ace_major;
- ace->gd->first_minor = ace->id * ACE_NUM_MINORS;
- ace->gd->fops = &ace_fops;
- ace->gd->events = DISK_EVENT_MEDIA_CHANGE;
- ace->gd->queue = ace->queue;
- ace->gd->private_data = ace;
- snprintf(ace->gd->disk_name, 32, "xs%c", ace->id + 'a');
-
- /* set bus width */
- if (ace->bus_width == ACE_BUS_WIDTH_16) {
- /* 0x0101 should work regardless of endianess */
- ace_out_le16(ace, ACE_BUSMODE, 0x0101);
-
- /* read it back to determine endianess */
- if (ace_in_le16(ace, ACE_BUSMODE) == 0x0001)
- ace->reg_ops = &ace_reg_le16_ops;
- else
- ace->reg_ops = &ace_reg_be16_ops;
- } else {
- ace_out_8(ace, ACE_BUSMODE, 0x00);
- ace->reg_ops = &ace_reg_8_ops;
- }
-
- /* Make sure version register is sane */
- version = ace_in(ace, ACE_VERSION);
- if ((version == 0) || (version == 0xFFFF))
- goto err_read;
-
- /* Put sysace in a sane state by clearing most control reg bits */
- ace_out(ace, ACE_CTRL, ACE_CTRL_FORCECFGMODE |
- ACE_CTRL_DATABUFRDYIRQ | ACE_CTRL_ERRORIRQ);
-
- /* Now we can hook up the irq handler */
- if (ace->irq > 0) {
- rc = request_irq(ace->irq, ace_interrupt, 0, "systemace", ace);
- if (rc) {
- /* Failure - fall back to polled mode */
- dev_err(ace->dev, "request_irq failed\n");
- ace->irq = rc;
- }
- }
-
- /* Enable interrupts */
- val = ace_in(ace, ACE_CTRL);
- val |= ACE_CTRL_DATABUFRDYIRQ | ACE_CTRL_ERRORIRQ;
- ace_out(ace, ACE_CTRL, val);
-
- /* Print the identification */
- dev_info(ace->dev, "Xilinx SystemACE revision %i.%i.%i\n",
- (version >> 12) & 0xf, (version >> 8) & 0x0f, version & 0xff);
- dev_dbg(ace->dev, "physaddr 0x%llx, mapped to 0x%p, irq=%i\n",
- (unsigned long long) ace->physaddr, ace->baseaddr, ace->irq);
-
- ace->media_change = 1;
- ace_media_changed(ace);
-
- /* Make the sysace device 'live' */
- add_disk(ace->gd);
-
- return 0;
-
-err_read:
- /* prevent double queue cleanup */
- ace->gd->queue = NULL;
- put_disk(ace->gd);
-err_alloc_disk:
- blk_cleanup_queue(ace->queue);
- blk_mq_free_tag_set(&ace->tag_set);
-err_blk_initq:
- iounmap(ace->baseaddr);
-err_ioremap:
- dev_info(ace->dev, "xsysace: error initializing device at 0x%llx\n",
- (unsigned long long) ace->physaddr);
- return -ENOMEM;
-}
-
-static void ace_teardown(struct ace_device *ace)
-{
- if (ace->gd) {
- del_gendisk(ace->gd);
- put_disk(ace->gd);
- }
-
- if (ace->queue) {
- blk_cleanup_queue(ace->queue);
- blk_mq_free_tag_set(&ace->tag_set);
- }
-
- tasklet_kill(&ace->fsm_tasklet);
-
- if (ace->irq > 0)
- free_irq(ace->irq, ace);
-
- iounmap(ace->baseaddr);
-}
-
-static int ace_alloc(struct device *dev, int id, resource_size_t physaddr,
- int irq, int bus_width)
-{
- struct ace_device *ace;
- int rc;
- dev_dbg(dev, "ace_alloc(%p)\n", dev);
-
- /* Allocate and initialize the ace device structure */
- ace = kzalloc(sizeof(struct ace_device), GFP_KERNEL);
- if (!ace) {
- rc = -ENOMEM;
- goto err_alloc;
- }
-
- ace->dev = dev;
- ace->id = id;
- ace->physaddr = physaddr;
- ace->irq = irq;
- ace->bus_width = bus_width;
-
- /* Call the setup code */
- rc = ace_setup(ace);
- if (rc)
- goto err_setup;
-
- dev_set_drvdata(dev, ace);
- return 0;
-
-err_setup:
- dev_set_drvdata(dev, NULL);
- kfree(ace);
-err_alloc:
- dev_err(dev, "could not initialize device, err=%i\n", rc);
- return rc;
-}
-
-static void ace_free(struct device *dev)
-{
- struct ace_device *ace = dev_get_drvdata(dev);
- dev_dbg(dev, "ace_free(%p)\n", dev);
-
- if (ace) {
- ace_teardown(ace);
- dev_set_drvdata(dev, NULL);
- kfree(ace);
- }
-}
-
-/* ---------------------------------------------------------------------
- * Platform Bus Support
- */
-
-static int ace_probe(struct platform_device *dev)
-{
- int bus_width = ACE_BUS_WIDTH_16; /* FIXME: should not be hard coded */
- resource_size_t physaddr;
- struct resource *res;
- u32 id = dev->id;
- int irq;
- int i;
-
- dev_dbg(&dev->dev, "ace_probe(%p)\n", dev);
-
- /* device id and bus width */
- if (of_property_read_u32(dev->dev.of_node, "port-number", &id))
- id = 0;
- if (of_find_property(dev->dev.of_node, "8-bit", NULL))
- bus_width = ACE_BUS_WIDTH_8;
-
- res = platform_get_resource(dev, IORESOURCE_MEM, 0);
- if (!res)
- return -EINVAL;
-
- physaddr = res->start;
- if (!physaddr)
- return -ENODEV;
-
- irq = platform_get_irq_optional(dev, 0);
-
- /* Call the bus-independent setup code */
- return ace_alloc(&dev->dev, id, physaddr, irq, bus_width);
-}
-
-/*
- * Platform bus remove() method
- */
-static int ace_remove(struct platform_device *dev)
-{
- ace_free(&dev->dev);
- return 0;
-}
-
-#if defined(CONFIG_OF)
-/* Match table for of_platform binding */
-static const struct of_device_id ace_of_match[] = {
- { .compatible = "xlnx,opb-sysace-1.00.b", },
- { .compatible = "xlnx,opb-sysace-1.00.c", },
- { .compatible = "xlnx,xps-sysace-1.00.a", },
- { .compatible = "xlnx,sysace", },
- {},
-};
-MODULE_DEVICE_TABLE(of, ace_of_match);
-#else /* CONFIG_OF */
-#define ace_of_match NULL
-#endif /* CONFIG_OF */
-
-static struct platform_driver ace_platform_driver = {
- .probe = ace_probe,
- .remove = ace_remove,
- .driver = {
- .name = "xsysace",
- .of_match_table = ace_of_match,
- },
-};
-
-/* ---------------------------------------------------------------------
- * Module init/exit routines
- */
-static int __init ace_init(void)
-{
- int rc;
-
- ace_major = register_blkdev(ace_major, "xsysace");
- if (ace_major <= 0) {
- rc = -ENOMEM;
- goto err_blk;
- }
-
- rc = platform_driver_register(&ace_platform_driver);
- if (rc)
- goto err_plat;
-
- pr_info("Xilinx SystemACE device driver, major=%i\n", ace_major);
- return 0;
-
-err_plat:
- unregister_blkdev(ace_major, "xsysace");
-err_blk:
- printk(KERN_ERR "xsysace: registration failed; err=%i\n", rc);
- return rc;
-}
-module_init(ace_init);
-
-static void __exit ace_exit(void)
-{
- pr_debug("Unregistering Xilinx SystemACE driver\n");
- platform_driver_unregister(&ace_platform_driver);
- unregister_blkdev(ace_major, "xsysace");
-}
-module_exit(ace_exit);
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index a711a2e2a794..cf8deecc39ef 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -627,7 +627,7 @@ static ssize_t writeback_store(struct device *dev,
struct bio_vec bio_vec;
struct page *page;
ssize_t ret = len;
- int mode;
+ int mode, err;
unsigned long blk_idx = 0;
if (sysfs_streq(buf, "idle"))
@@ -638,8 +638,8 @@ static ssize_t writeback_store(struct device *dev,
if (strncmp(buf, PAGE_WB_SIG, sizeof(PAGE_WB_SIG) - 1))
return -EINVAL;
- ret = kstrtol(buf + sizeof(PAGE_WB_SIG) - 1, 10, &index);
- if (ret || index >= nr_pages)
+ if (kstrtol(buf + sizeof(PAGE_WB_SIG) - 1, 10, &index) ||
+ index >= nr_pages)
return -EINVAL;
nr_pages = 1;
@@ -663,7 +663,7 @@ static ssize_t writeback_store(struct device *dev,
goto release_init_lock;
}
- while (nr_pages--) {
+ for (; nr_pages != 0; index++, nr_pages--) {
struct bio_vec bvec;
bvec.bv_page = page;
@@ -728,12 +728,17 @@ static ssize_t writeback_store(struct device *dev,
* XXX: A single page IO would be inefficient for write
* but it would be not bad as starter.
*/
- ret = submit_bio_wait(&bio);
- if (ret) {
+ err = submit_bio_wait(&bio);
+ if (err) {
zram_slot_lock(zram, index);
zram_clear_flag(zram, index, ZRAM_UNDER_WB);
zram_clear_flag(zram, index, ZRAM_IDLE);
zram_slot_unlock(zram, index);
+ /*
+ * Return last IO error unless every IO were
+ * not suceeded.
+ */
+ ret = err;
continue;
}
diff --git a/drivers/bluetooth/Kconfig b/drivers/bluetooth/Kconfig
index 4e73a531b377..851842372c9b 100644
--- a/drivers/bluetooth/Kconfig
+++ b/drivers/bluetooth/Kconfig
@@ -425,4 +425,14 @@ config BT_HCIRSI
Say Y here to compile support for HCI over Redpine into the
kernel or say M to compile as a module.
+config BT_VIRTIO
+ tristate "Virtio Bluetooth driver"
+ depends on VIRTIO
+ help
+ Virtio Bluetooth support driver.
+ This driver supports Virtio Bluetooth devices.
+
+ Say Y here to compile support for HCI over Virtio into the
+ kernel or say M to compile as a module.
+
endmenu
diff --git a/drivers/bluetooth/Makefile b/drivers/bluetooth/Makefile
index 1a58a3ae142c..16286ea2655d 100644
--- a/drivers/bluetooth/Makefile
+++ b/drivers/bluetooth/Makefile
@@ -26,6 +26,8 @@ obj-$(CONFIG_BT_BCM) += btbcm.o
obj-$(CONFIG_BT_RTL) += btrtl.o
obj-$(CONFIG_BT_QCA) += btqca.o
+obj-$(CONFIG_BT_VIRTIO) += virtio_bt.o
+
obj-$(CONFIG_BT_HCIUART_NOKIA) += hci_nokia.o
obj-$(CONFIG_BT_HCIRSI) += btrsi.o
diff --git a/drivers/bluetooth/btintel.c b/drivers/bluetooth/btintel.c
index 88ce5f0ffc4b..e44b6993cf91 100644
--- a/drivers/bluetooth/btintel.c
+++ b/drivers/bluetooth/btintel.c
@@ -24,6 +24,14 @@
#define ECDSA_OFFSET 644
#define ECDSA_HEADER_LEN 320
+#define CMD_WRITE_BOOT_PARAMS 0xfc0e
+struct cmd_write_boot_params {
+ u32 boot_addr;
+ u8 fw_build_num;
+ u8 fw_build_ww;
+ u8 fw_build_yy;
+} __packed;
+
int btintel_check_bdaddr(struct hci_dev *hdev)
{
struct hci_rp_read_bd_addr *bda;
@@ -208,10 +216,39 @@ void btintel_hw_error(struct hci_dev *hdev, u8 code)
}
EXPORT_SYMBOL_GPL(btintel_hw_error);
-void btintel_version_info(struct hci_dev *hdev, struct intel_version *ver)
+int btintel_version_info(struct hci_dev *hdev, struct intel_version *ver)
{
const char *variant;
+ /* The hardware platform number has a fixed value of 0x37 and
+ * for now only accept this single value.
+ */
+ if (ver->hw_platform != 0x37) {
+ bt_dev_err(hdev, "Unsupported Intel hardware platform (%u)",
+ ver->hw_platform);
+ return -EINVAL;
+ }
+
+ /* Check for supported iBT hardware variants of this firmware
+ * loading method.
+ *
+ * This check has been put in place to ensure correct forward
+ * compatibility options when newer hardware variants come along.
+ */
+ switch (ver->hw_variant) {
+ case 0x0b: /* SfP */
+ case 0x0c: /* WsP */
+ case 0x11: /* JfP */
+ case 0x12: /* ThP */
+ case 0x13: /* HrP */
+ case 0x14: /* CcP */
+ break;
+ default:
+ bt_dev_err(hdev, "Unsupported Intel hardware variant (%u)",
+ ver->hw_variant);
+ return -EINVAL;
+ }
+
switch (ver->fw_variant) {
case 0x06:
variant = "Bootloader";
@@ -220,13 +257,16 @@ void btintel_version_info(struct hci_dev *hdev, struct intel_version *ver)
variant = "Firmware";
break;
default:
- return;
+ bt_dev_err(hdev, "Unsupported firmware variant(%02x)", ver->fw_variant);
+ return -EINVAL;
}
bt_dev_info(hdev, "%s revision %u.%u build %u week %u %u",
variant, ver->fw_revision >> 4, ver->fw_revision & 0x0f,
ver->fw_build_num, ver->fw_build_ww,
2000 + ver->fw_build_yy);
+
+ return 0;
}
EXPORT_SYMBOL_GPL(btintel_version_info);
@@ -364,13 +404,56 @@ int btintel_read_version(struct hci_dev *hdev, struct intel_version *ver)
}
EXPORT_SYMBOL_GPL(btintel_read_version);
-void btintel_version_info_tlv(struct hci_dev *hdev, struct intel_version_tlv *version)
+int btintel_version_info_tlv(struct hci_dev *hdev, struct intel_version_tlv *version)
{
const char *variant;
+ /* The hardware platform number has a fixed value of 0x37 and
+ * for now only accept this single value.
+ */
+ if (INTEL_HW_PLATFORM(version->cnvi_bt) != 0x37) {
+ bt_dev_err(hdev, "Unsupported Intel hardware platform (0x%2x)",
+ INTEL_HW_PLATFORM(version->cnvi_bt));
+ return -EINVAL;
+ }
+
+ /* Check for supported iBT hardware variants of this firmware
+ * loading method.
+ *
+ * This check has been put in place to ensure correct forward
+ * compatibility options when newer hardware variants come along.
+ */
+ switch (INTEL_HW_VARIANT(version->cnvi_bt)) {
+ case 0x17: /* TyP */
+ case 0x18: /* Slr */
+ case 0x19: /* Slr-F */
+ break;
+ default:
+ bt_dev_err(hdev, "Unsupported Intel hardware variant (0x%x)",
+ INTEL_HW_VARIANT(version->cnvi_bt));
+ return -EINVAL;
+ }
+
switch (version->img_type) {
case 0x01:
variant = "Bootloader";
+ /* It is required that every single firmware fragment is acknowledged
+ * with a command complete event. If the boot parameters indicate
+ * that this bootloader does not send them, then abort the setup.
+ */
+ if (version->limited_cce != 0x00) {
+ bt_dev_err(hdev, "Unsupported Intel firmware loading method (0x%x)",
+ version->limited_cce);
+ return -EINVAL;
+ }
+
+ /* Secure boot engine type should be either 1 (ECDSA) or 0 (RSA) */
+ if (version->sbe_type > 0x01) {
+ bt_dev_err(hdev, "Unsupported Intel secure boot engine type (0x%x)",
+ version->sbe_type);
+ return -EINVAL;
+ }
+
bt_dev_info(hdev, "Device revision is %u", version->dev_rev_id);
bt_dev_info(hdev, "Secure boot is %s",
version->secure_boot ? "enabled" : "disabled");
@@ -389,15 +472,14 @@ void btintel_version_info_tlv(struct hci_dev *hdev, struct intel_version_tlv *ve
break;
default:
bt_dev_err(hdev, "Unsupported image type(%02x)", version->img_type);
- goto done;
+ return -EINVAL;
}
bt_dev_info(hdev, "%s timestamp %u.%u buildtype %u build %u", variant,
2000 + (version->timestamp >> 8), version->timestamp & 0xff,
version->build_type, version->build_num);
-done:
- return;
+ return 0;
}
EXPORT_SYMBOL_GPL(btintel_version_info_tlv);
@@ -455,12 +537,23 @@ int btintel_read_version_tlv(struct hci_dev *hdev, struct intel_version_tlv *ver
version->img_type = tlv->val[0];
break;
case INTEL_TLV_TIME_STAMP:
+ /* If image type is Operational firmware (0x03), then
+ * running FW Calendar Week and Year information can
+ * be extracted from Timestamp information
+ */
+ version->min_fw_build_cw = tlv->val[0];
+ version->min_fw_build_yy = tlv->val[1];
version->timestamp = get_unaligned_le16(tlv->val);
break;
case INTEL_TLV_BUILD_TYPE:
version->build_type = tlv->val[0];
break;
case INTEL_TLV_BUILD_NUM:
+ /* If image type is Operational firmware (0x03), then
+ * running FW build number can be extracted from the
+ * Build information
+ */
+ version->min_fw_build_nn = tlv->val[0];
version->build_num = get_unaligned_le32(tlv->val);
break;
case INTEL_TLV_SECURE_BOOT:
@@ -841,7 +934,7 @@ static int btintel_sfi_ecdsa_header_secure_send(struct hci_dev *hdev,
static int btintel_download_firmware_payload(struct hci_dev *hdev,
const struct firmware *fw,
- u32 *boot_param, size_t offset)
+ size_t offset)
{
int err;
const u8 *fw_ptr;
@@ -854,20 +947,6 @@ static int btintel_download_firmware_payload(struct hci_dev *hdev,
while (fw_ptr - fw->data < fw->size) {
struct hci_command_hdr *cmd = (void *)(fw_ptr + frag_len);
- /* Each SKU has a different reset parameter to use in the
- * HCI_Intel_Reset command and it is embedded in the firmware
- * data. So, instead of using static value per SKU, check
- * the firmware data and save it for later use.
- */
- if (le16_to_cpu(cmd->opcode) == 0xfc0e) {
- /* The boot parameter is the first 32-bit value
- * and rest of 3 octets are reserved.
- */
- *boot_param = get_unaligned_le32(fw_ptr + sizeof(*cmd));
-
- bt_dev_dbg(hdev, "boot_param=0x%x", *boot_param);
- }
-
frag_len += sizeof(*cmd) + cmd->plen;
/* The parameter length of the secure send command requires
@@ -896,28 +975,131 @@ done:
return err;
}
+static bool btintel_firmware_version(struct hci_dev *hdev,
+ u8 num, u8 ww, u8 yy,
+ const struct firmware *fw,
+ u32 *boot_addr)
+{
+ const u8 *fw_ptr;
+
+ fw_ptr = fw->data;
+
+ while (fw_ptr - fw->data < fw->size) {
+ struct hci_command_hdr *cmd = (void *)(fw_ptr);
+
+ /* Each SKU has a different reset parameter to use in the
+ * HCI_Intel_Reset command and it is embedded in the firmware
+ * data. So, instead of using static value per SKU, check
+ * the firmware data and save it for later use.
+ */
+ if (le16_to_cpu(cmd->opcode) == CMD_WRITE_BOOT_PARAMS) {
+ struct cmd_write_boot_params *params;
+
+ params = (void *)(fw_ptr + sizeof(*cmd));
+
+ bt_dev_info(hdev, "Boot Address: 0x%x",
+ le32_to_cpu(params->boot_addr));
+
+ bt_dev_info(hdev, "Firmware Version: %u-%u.%u",
+ params->fw_build_num, params->fw_build_ww,
+ params->fw_build_yy);
+
+ return (num == params->fw_build_num &&
+ ww == params->fw_build_ww &&
+ yy == params->fw_build_yy);
+ }
+
+ fw_ptr += sizeof(*cmd) + cmd->plen;
+ }
+
+ return false;
+}
+
int btintel_download_firmware(struct hci_dev *hdev,
+ struct intel_version *ver,
const struct firmware *fw,
u32 *boot_param)
{
int err;
+ /* SfP and WsP don't seem to update the firmware version on file
+ * so version checking is currently not possible.
+ */
+ switch (ver->hw_variant) {
+ case 0x0b: /* SfP */
+ case 0x0c: /* WsP */
+ /* Skip version checking */
+ break;
+ default:
+ /* Skip reading firmware file version in bootloader mode */
+ if (ver->fw_variant == 0x06)
+ break;
+
+ /* Skip download if firmware has the same version */
+ if (btintel_firmware_version(hdev, ver->fw_build_num,
+ ver->fw_build_ww, ver->fw_build_yy,
+ fw, boot_param)) {
+ bt_dev_info(hdev, "Firmware already loaded");
+ /* Return -EALREADY to indicate that the firmware has
+ * already been loaded.
+ */
+ return -EALREADY;
+ }
+ }
+
+ /* The firmware variant determines if the device is in bootloader
+ * mode or is running operational firmware. The value 0x06 identifies
+ * the bootloader and the value 0x23 identifies the operational
+ * firmware.
+ *
+ * If the firmware version has changed that means it needs to be reset
+ * to bootloader when operational so the new firmware can be loaded.
+ */
+ if (ver->fw_variant == 0x23)
+ return -EINVAL;
+
err = btintel_sfi_rsa_header_secure_send(hdev, fw);
if (err)
return err;
- return btintel_download_firmware_payload(hdev, fw, boot_param,
- RSA_HEADER_LEN);
+ return btintel_download_firmware_payload(hdev, fw, RSA_HEADER_LEN);
}
EXPORT_SYMBOL_GPL(btintel_download_firmware);
int btintel_download_firmware_newgen(struct hci_dev *hdev,
+ struct intel_version_tlv *ver,
const struct firmware *fw, u32 *boot_param,
u8 hw_variant, u8 sbe_type)
{
int err;
u32 css_header_ver;
+ /* Skip reading firmware file version in bootloader mode */
+ if (ver->img_type != 0x01) {
+ /* Skip download if firmware has the same version */
+ if (btintel_firmware_version(hdev, ver->min_fw_build_nn,
+ ver->min_fw_build_cw,
+ ver->min_fw_build_yy,
+ fw, boot_param)) {
+ bt_dev_info(hdev, "Firmware already loaded");
+ /* Return -EALREADY to indicate that firmware has
+ * already been loaded.
+ */
+ return -EALREADY;
+ }
+ }
+
+ /* The firmware variant determines if the device is in bootloader
+ * mode or is running operational firmware. The value 0x01 identifies
+ * the bootloader and the value 0x03 identifies the operational
+ * firmware.
+ *
+ * If the firmware version has changed that means it needs to be reset
+ * to bootloader when operational so the new firmware can be loaded.
+ */
+ if (ver->img_type == 0x03)
+ return -EINVAL;
+
/* iBT hardware variants 0x0b, 0x0c, 0x11, 0x12, 0x13, 0x14 support
* only RSA secure boot engine. Hence, the corresponding sfi file will
* have RSA header of 644 bytes followed by Command Buffer.
@@ -947,7 +1129,7 @@ int btintel_download_firmware_newgen(struct hci_dev *hdev,
if (err)
return err;
- err = btintel_download_firmware_payload(hdev, fw, boot_param, RSA_HEADER_LEN);
+ err = btintel_download_firmware_payload(hdev, fw, RSA_HEADER_LEN);
if (err)
return err;
} else if (hw_variant >= 0x17) {
@@ -968,7 +1150,6 @@ int btintel_download_firmware_newgen(struct hci_dev *hdev,
return err;
err = btintel_download_firmware_payload(hdev, fw,
- boot_param,
RSA_HEADER_LEN + ECDSA_HEADER_LEN);
if (err)
return err;
@@ -978,7 +1159,6 @@ int btintel_download_firmware_newgen(struct hci_dev *hdev,
return err;
err = btintel_download_firmware_payload(hdev, fw,
- boot_param,
RSA_HEADER_LEN + ECDSA_HEADER_LEN);
if (err)
return err;
diff --git a/drivers/bluetooth/btintel.h b/drivers/bluetooth/btintel.h
index 6511b091caf5..d184064a5e7c 100644
--- a/drivers/bluetooth/btintel.h
+++ b/drivers/bluetooth/btintel.h
@@ -148,8 +148,8 @@ int btintel_set_diag(struct hci_dev *hdev, bool enable);
int btintel_set_diag_mfg(struct hci_dev *hdev, bool enable);
void btintel_hw_error(struct hci_dev *hdev, u8 code);
-void btintel_version_info(struct hci_dev *hdev, struct intel_version *ver);
-void btintel_version_info_tlv(struct hci_dev *hdev, struct intel_version_tlv *version);
+int btintel_version_info(struct hci_dev *hdev, struct intel_version *ver);
+int btintel_version_info_tlv(struct hci_dev *hdev, struct intel_version_tlv *version);
int btintel_secure_send(struct hci_dev *hdev, u8 fragment_type, u32 plen,
const void *param);
int btintel_load_ddc_config(struct hci_dev *hdev, const char *ddc_name);
@@ -163,9 +163,10 @@ struct regmap *btintel_regmap_init(struct hci_dev *hdev, u16 opcode_read,
int btintel_send_intel_reset(struct hci_dev *hdev, u32 boot_param);
int btintel_read_boot_params(struct hci_dev *hdev,
struct intel_boot_params *params);
-int btintel_download_firmware(struct hci_dev *dev, const struct firmware *fw,
- u32 *boot_param);
+int btintel_download_firmware(struct hci_dev *dev, struct intel_version *ver,
+ const struct firmware *fw, u32 *boot_param);
int btintel_download_firmware_newgen(struct hci_dev *hdev,
+ struct intel_version_tlv *ver,
const struct firmware *fw,
u32 *boot_param, u8 hw_variant,
u8 sbe_type);
@@ -210,14 +211,16 @@ static inline void btintel_hw_error(struct hci_dev *hdev, u8 code)
{
}
-static inline void btintel_version_info(struct hci_dev *hdev,
- struct intel_version *ver)
+static inline int btintel_version_info(struct hci_dev *hdev,
+ struct intel_version *ver)
{
+ return -EOPNOTSUPP;
}
-static inline void btintel_version_info_tlv(struct hci_dev *hdev,
- struct intel_version_tlv *version)
+static inline int btintel_version_info_tlv(struct hci_dev *hdev,
+ struct intel_version_tlv *version)
{
+ return -EOPNOTSUPP;
}
static inline int btintel_secure_send(struct hci_dev *hdev, u8 fragment_type,
diff --git a/drivers/bluetooth/btrsi.c b/drivers/bluetooth/btrsi.c
index 3951f7b23840..bea1595f6432 100644
--- a/drivers/bluetooth/btrsi.c
+++ b/drivers/bluetooth/btrsi.c
@@ -194,5 +194,4 @@ module_init(rsi_91x_bt_module_init);
module_exit(rsi_91x_bt_module_exit);
MODULE_AUTHOR("Redpine Signals Inc");
MODULE_DESCRIPTION("RSI BT driver");
-MODULE_SUPPORTED_DEVICE("RSI-BT");
MODULE_LICENSE("Dual BSD/GPL");
diff --git a/drivers/bluetooth/btusb.c b/drivers/bluetooth/btusb.c
index 52683fd22e05..5d603ef39bad 100644
--- a/drivers/bluetooth/btusb.c
+++ b/drivers/bluetooth/btusb.c
@@ -399,7 +399,9 @@ static const struct usb_device_id blacklist_table[] = {
/* MediaTek Bluetooth devices */
{ USB_VENDOR_AND_INTERFACE_INFO(0x0e8d, 0xe0, 0x01, 0x01),
- .driver_info = BTUSB_MEDIATEK },
+ .driver_info = BTUSB_MEDIATEK |
+ BTUSB_WIDEBAND_SPEECH |
+ BTUSB_VALID_LE_STATES },
/* Additional MediaTek MT7615E Bluetooth devices */
{ USB_DEVICE(0x13d3, 0x3560), .driver_info = BTUSB_MEDIATEK},
@@ -455,6 +457,8 @@ static const struct usb_device_id blacklist_table[] = {
BTUSB_WIDEBAND_SPEECH },
{ USB_DEVICE(0x0bda, 0xc123), .driver_info = BTUSB_REALTEK |
BTUSB_WIDEBAND_SPEECH },
+ { USB_DEVICE(0x0cb5, 0xc547), .driver_info = BTUSB_REALTEK |
+ BTUSB_WIDEBAND_SPEECH },
/* Silicon Wave based devices */
{ USB_DEVICE(0x0c10, 0x0000), .driver_info = BTUSB_SWAVE },
@@ -2400,7 +2404,7 @@ static int btusb_send_frame_intel(struct hci_dev *hdev, struct sk_buff *skb)
return -EILSEQ;
}
-static bool btusb_setup_intel_new_get_fw_name(struct intel_version *ver,
+static int btusb_setup_intel_new_get_fw_name(struct intel_version *ver,
struct intel_boot_params *params,
char *fw_name, size_t len,
const char *suffix)
@@ -2424,9 +2428,10 @@ static bool btusb_setup_intel_new_get_fw_name(struct intel_version *ver,
suffix);
break;
default:
- return false;
+ return -EINVAL;
}
- return true;
+
+ return 0;
}
static void btusb_setup_intel_newgen_get_fw_name(const struct intel_version_tlv *ver_tlv,
@@ -2444,6 +2449,44 @@ static void btusb_setup_intel_newgen_get_fw_name(const struct intel_version_tlv
suffix);
}
+static int btusb_download_wait(struct hci_dev *hdev, ktime_t calltime, int msec)
+{
+ struct btusb_data *data = hci_get_drvdata(hdev);
+ ktime_t delta, rettime;
+ unsigned long long duration;
+ int err;
+
+ set_bit(BTUSB_FIRMWARE_LOADED, &data->flags);
+
+ bt_dev_info(hdev, "Waiting for firmware download to complete");
+
+ err = wait_on_bit_timeout(&data->flags, BTUSB_DOWNLOADING,
+ TASK_INTERRUPTIBLE,
+ msecs_to_jiffies(msec));
+ if (err == -EINTR) {
+ bt_dev_err(hdev, "Firmware loading interrupted");
+ return err;
+ }
+
+ if (err) {
+ bt_dev_err(hdev, "Firmware loading timeout");
+ return -ETIMEDOUT;
+ }
+
+ if (test_bit(BTUSB_FIRMWARE_FAILED, &data->flags)) {
+ bt_dev_err(hdev, "Firmware loading failed");
+ return -ENOEXEC;
+ }
+
+ rettime = ktime_get();
+ delta = ktime_sub(rettime, calltime);
+ duration = (unsigned long long)ktime_to_ns(delta) >> 10;
+
+ bt_dev_info(hdev, "Firmware loaded in %llu usecs", duration);
+
+ return 0;
+}
+
static int btusb_intel_download_firmware_newgen(struct hci_dev *hdev,
struct intel_version_tlv *ver,
u32 *boot_param)
@@ -2452,19 +2495,11 @@ static int btusb_intel_download_firmware_newgen(struct hci_dev *hdev,
char fwname[64];
int err;
struct btusb_data *data = hci_get_drvdata(hdev);
+ ktime_t calltime;
if (!ver || !boot_param)
return -EINVAL;
- /* The hardware platform number has a fixed value of 0x37 and
- * for now only accept this single value.
- */
- if (INTEL_HW_PLATFORM(ver->cnvi_bt) != 0x37) {
- bt_dev_err(hdev, "Unsupported Intel hardware platform (0x%2x)",
- INTEL_HW_PLATFORM(ver->cnvi_bt));
- return -EINVAL;
- }
-
/* The firmware variant determines if the device is in bootloader
* mode or is running operational firmware. The value 0x03 identifies
* the bootloader and the value 0x23 identifies the operational
@@ -2481,50 +2516,6 @@ static int btusb_intel_download_firmware_newgen(struct hci_dev *hdev,
if (ver->img_type == 0x03) {
clear_bit(BTUSB_BOOTLOADER, &data->flags);
btintel_check_bdaddr(hdev);
- return 0;
- }
-
- /* Check for supported iBT hardware variants of this firmware
- * loading method.
- *
- * This check has been put in place to ensure correct forward
- * compatibility options when newer hardware variants come along.
- */
- switch (INTEL_HW_VARIANT(ver->cnvi_bt)) {
- case 0x17: /* TyP */
- case 0x18: /* Slr */
- case 0x19: /* Slr-F */
- break;
- default:
- bt_dev_err(hdev, "Unsupported Intel hardware variant (0x%x)",
- INTEL_HW_VARIANT(ver->cnvi_bt));
- return -EINVAL;
- }
-
- /* If the device is not in bootloader mode, then the only possible
- * choice is to return an error and abort the device initialization.
- */
- if (ver->img_type != 0x01) {
- bt_dev_err(hdev, "Unsupported Intel firmware variant (0x%x)",
- ver->img_type);
- return -ENODEV;
- }
-
- /* It is required that every single firmware fragment is acknowledged
- * with a command complete event. If the boot parameters indicate
- * that this bootloader does not send them, then abort the setup.
- */
- if (ver->limited_cce != 0x00) {
- bt_dev_err(hdev, "Unsupported Intel firmware loading method (0x%x)",
- ver->limited_cce);
- return -EINVAL;
- }
-
- /* Secure boot engine type should be either 1 (ECDSA) or 0 (RSA) */
- if (ver->sbe_type > 0x01) {
- bt_dev_err(hdev, "Unsupported Intel secure boot engine type (0x%x)",
- ver->sbe_type);
- return -EINVAL;
}
/* If the OTP has no valid Bluetooth device address, then there will
@@ -2538,7 +2529,8 @@ static int btusb_intel_download_firmware_newgen(struct hci_dev *hdev,
btusb_setup_intel_newgen_get_fw_name(ver, fwname, sizeof(fwname), "sfi");
err = request_firmware(&fw, fwname, &hdev->dev);
if (err < 0) {
- bt_dev_err(hdev, "Failed to load Intel firmware file (%d)", err);
+ bt_dev_err(hdev, "Failed to load Intel firmware file %s (%d)",
+ fwname, err);
return err;
}
@@ -2551,22 +2543,28 @@ static int btusb_intel_download_firmware_newgen(struct hci_dev *hdev,
goto done;
}
+ calltime = ktime_get();
+
set_bit(BTUSB_DOWNLOADING, &data->flags);
/* Start firmware downloading and get boot parameter */
- err = btintel_download_firmware_newgen(hdev, fw, boot_param,
+ err = btintel_download_firmware_newgen(hdev, ver, fw, boot_param,
INTEL_HW_VARIANT(ver->cnvi_bt),
ver->sbe_type);
if (err < 0) {
+ if (err == -EALREADY) {
+ /* Firmware has already been loaded */
+ set_bit(BTUSB_FIRMWARE_LOADED, &data->flags);
+ err = 0;
+ goto done;
+ }
+
/* When FW download fails, send Intel Reset to retry
* FW download.
*/
btintel_reset_to_bootloader(hdev);
goto done;
}
- set_bit(BTUSB_FIRMWARE_LOADED, &data->flags);
-
- bt_dev_info(hdev, "Waiting for firmware download to complete");
/* Before switching the device into operational mode and with that
* booting the loaded firmware, wait for the bootloader notification
@@ -2579,26 +2577,9 @@ static int btusb_intel_download_firmware_newgen(struct hci_dev *hdev,
* and thus just timeout if that happens and fail the setup
* of this device.
*/
- err = wait_on_bit_timeout(&data->flags, BTUSB_DOWNLOADING,
- TASK_INTERRUPTIBLE,
- msecs_to_jiffies(5000));
- if (err == -EINTR) {
- bt_dev_err(hdev, "Firmware loading interrupted");
- goto done;
- }
-
- if (err) {
- bt_dev_err(hdev, "Firmware loading timeout");
- err = -ETIMEDOUT;
+ err = btusb_download_wait(hdev, calltime, 5000);
+ if (err == -ETIMEDOUT)
btintel_reset_to_bootloader(hdev);
- goto done;
- }
-
- if (test_bit(BTUSB_FIRMWARE_FAILED, &data->flags)) {
- bt_dev_err(hdev, "Firmware loading failed");
- err = -ENOEXEC;
- goto done;
- }
done:
release_firmware(fw);
@@ -2614,41 +2595,11 @@ static int btusb_intel_download_firmware(struct hci_dev *hdev,
char fwname[64];
int err;
struct btusb_data *data = hci_get_drvdata(hdev);
+ ktime_t calltime;
if (!ver || !params)
return -EINVAL;
- /* The hardware platform number has a fixed value of 0x37 and
- * for now only accept this single value.
- */
- if (ver->hw_platform != 0x37) {
- bt_dev_err(hdev, "Unsupported Intel hardware platform (%u)",
- ver->hw_platform);
- return -EINVAL;
- }
-
- /* Check for supported iBT hardware variants of this firmware
- * loading method.
- *
- * This check has been put in place to ensure correct forward
- * compatibility options when newer hardware variants come along.
- */
- switch (ver->hw_variant) {
- case 0x0b: /* SfP */
- case 0x0c: /* WsP */
- case 0x11: /* JfP */
- case 0x12: /* ThP */
- case 0x13: /* HrP */
- case 0x14: /* CcP */
- break;
- default:
- bt_dev_err(hdev, "Unsupported Intel hardware variant (%u)",
- ver->hw_variant);
- return -EINVAL;
- }
-
- btintel_version_info(hdev, ver);
-
/* The firmware variant determines if the device is in bootloader
* mode or is running operational firmware. The value 0x06 identifies
* the bootloader and the value 0x23 identifies the operational
@@ -2665,16 +2616,18 @@ static int btusb_intel_download_firmware(struct hci_dev *hdev,
if (ver->fw_variant == 0x23) {
clear_bit(BTUSB_BOOTLOADER, &data->flags);
btintel_check_bdaddr(hdev);
- return 0;
- }
- /* If the device is not in bootloader mode, then the only possible
- * choice is to return an error and abort the device initialization.
- */
- if (ver->fw_variant != 0x06) {
- bt_dev_err(hdev, "Unsupported Intel firmware variant (%u)",
- ver->fw_variant);
- return -ENODEV;
+ /* SfP and WsP don't seem to update the firmware version on file
+ * so version checking is currently possible.
+ */
+ switch (ver->hw_variant) {
+ case 0x0b: /* SfP */
+ case 0x0c: /* WsP */
+ return 0;
+ }
+
+ /* Proceed to download to check if the version matches */
+ goto download;
}
/* Read the secure boot parameters to identify the operating
@@ -2702,6 +2655,7 @@ static int btusb_intel_download_firmware(struct hci_dev *hdev,
set_bit(HCI_QUIRK_INVALID_BDADDR, &hdev->quirks);
}
+download:
/* With this Intel bootloader only the hardware variant and device
* revision information are used to select the right firmware for SfP
* and WsP.
@@ -2725,14 +2679,15 @@ static int btusb_intel_download_firmware(struct hci_dev *hdev,
*/
err = btusb_setup_intel_new_get_fw_name(ver, params, fwname,
sizeof(fwname), "sfi");
- if (!err) {
+ if (err < 0) {
bt_dev_err(hdev, "Unsupported Intel firmware naming");
return -EINVAL;
}
err = request_firmware(&fw, fwname, &hdev->dev);
if (err < 0) {
- bt_dev_err(hdev, "Failed to load Intel firmware file (%d)", err);
+ bt_dev_err(hdev, "Failed to load Intel firmware file %s (%d)",
+ fwname, err);
return err;
}
@@ -2745,20 +2700,26 @@ static int btusb_intel_download_firmware(struct hci_dev *hdev,
goto done;
}
+ calltime = ktime_get();
+
set_bit(BTUSB_DOWNLOADING, &data->flags);
/* Start firmware downloading and get boot parameter */
- err = btintel_download_firmware(hdev, fw, boot_param);
+ err = btintel_download_firmware(hdev, ver, fw, boot_param);
if (err < 0) {
+ if (err == -EALREADY) {
+ /* Firmware has already been loaded */
+ set_bit(BTUSB_FIRMWARE_LOADED, &data->flags);
+ err = 0;
+ goto done;
+ }
+
/* When FW download fails, send Intel Reset to retry
* FW download.
*/
btintel_reset_to_bootloader(hdev);
goto done;
}
- set_bit(BTUSB_FIRMWARE_LOADED, &data->flags);
-
- bt_dev_info(hdev, "Waiting for firmware download to complete");
/* Before switching the device into operational mode and with that
* booting the loaded firmware, wait for the bootloader notification
@@ -2771,29 +2732,74 @@ static int btusb_intel_download_firmware(struct hci_dev *hdev,
* and thus just timeout if that happens and fail the setup
* of this device.
*/
- err = wait_on_bit_timeout(&data->flags, BTUSB_DOWNLOADING,
+ err = btusb_download_wait(hdev, calltime, 5000);
+ if (err == -ETIMEDOUT)
+ btintel_reset_to_bootloader(hdev);
+
+done:
+ release_firmware(fw);
+ return err;
+}
+
+static int btusb_boot_wait(struct hci_dev *hdev, ktime_t calltime, int msec)
+{
+ struct btusb_data *data = hci_get_drvdata(hdev);
+ ktime_t delta, rettime;
+ unsigned long long duration;
+ int err;
+
+ bt_dev_info(hdev, "Waiting for device to boot");
+
+ err = wait_on_bit_timeout(&data->flags, BTUSB_BOOTING,
TASK_INTERRUPTIBLE,
- msecs_to_jiffies(5000));
+ msecs_to_jiffies(msec));
if (err == -EINTR) {
- bt_dev_err(hdev, "Firmware loading interrupted");
- goto done;
+ bt_dev_err(hdev, "Device boot interrupted");
+ return -EINTR;
}
if (err) {
- bt_dev_err(hdev, "Firmware loading timeout");
- err = -ETIMEDOUT;
- btintel_reset_to_bootloader(hdev);
- goto done;
+ bt_dev_err(hdev, "Device boot timeout");
+ return -ETIMEDOUT;
}
- if (test_bit(BTUSB_FIRMWARE_FAILED, &data->flags)) {
- bt_dev_err(hdev, "Firmware loading failed");
- err = -ENOEXEC;
- goto done;
+ rettime = ktime_get();
+ delta = ktime_sub(rettime, calltime);
+ duration = (unsigned long long) ktime_to_ns(delta) >> 10;
+
+ bt_dev_info(hdev, "Device booted in %llu usecs", duration);
+
+ return 0;
+}
+
+static int btusb_intel_boot(struct hci_dev *hdev, u32 boot_addr)
+{
+ struct btusb_data *data = hci_get_drvdata(hdev);
+ ktime_t calltime;
+ int err;
+
+ calltime = ktime_get();
+
+ set_bit(BTUSB_BOOTING, &data->flags);
+
+ err = btintel_send_intel_reset(hdev, boot_addr);
+ if (err) {
+ bt_dev_err(hdev, "Intel Soft Reset failed (%d)", err);
+ btintel_reset_to_bootloader(hdev);
+ return err;
}
-done:
- release_firmware(fw);
+ /* The bootloader will not indicate when the device is ready. This
+ * is done by the operational firmware sending bootup notification.
+ *
+ * Booting into operational firmware should not take longer than
+ * 1 second. However if that happens, then just fail the setup
+ * since something went wrong.
+ */
+ err = btusb_boot_wait(hdev, calltime, 1000);
+ if (err == -ETIMEDOUT)
+ btintel_reset_to_bootloader(hdev);
+
return err;
}
@@ -2804,8 +2810,6 @@ static int btusb_setup_intel_new(struct hci_dev *hdev)
struct intel_boot_params params;
u32 boot_param;
char ddcname[64];
- ktime_t calltime, delta, rettime;
- unsigned long long duration;
int err;
struct intel_debug_features features;
@@ -2817,8 +2821,6 @@ static int btusb_setup_intel_new(struct hci_dev *hdev)
*/
boot_param = 0x00000000;
- calltime = ktime_get();
-
/* Read the Intel version information to determine if the device
* is in bootloader mode or if it already has operational firmware
* loaded.
@@ -2830,6 +2832,10 @@ static int btusb_setup_intel_new(struct hci_dev *hdev)
return err;
}
+ err = btintel_version_info(hdev, &ver);
+ if (err)
+ return err;
+
err = btusb_intel_download_firmware(hdev, &ver, &params, &boot_param);
if (err)
return err;
@@ -2838,59 +2844,16 @@ static int btusb_setup_intel_new(struct hci_dev *hdev)
if (ver.fw_variant == 0x23)
goto finish;
- rettime = ktime_get();
- delta = ktime_sub(rettime, calltime);
- duration = (unsigned long long) ktime_to_ns(delta) >> 10;
-
- bt_dev_info(hdev, "Firmware loaded in %llu usecs", duration);
-
- calltime = ktime_get();
-
- set_bit(BTUSB_BOOTING, &data->flags);
-
- err = btintel_send_intel_reset(hdev, boot_param);
- if (err) {
- bt_dev_err(hdev, "Intel Soft Reset failed (%d)", err);
- btintel_reset_to_bootloader(hdev);
+ err = btusb_intel_boot(hdev, boot_param);
+ if (err)
return err;
- }
-
- /* The bootloader will not indicate when the device is ready. This
- * is done by the operational firmware sending bootup notification.
- *
- * Booting into operational firmware should not take longer than
- * 1 second. However if that happens, then just fail the setup
- * since something went wrong.
- */
- bt_dev_info(hdev, "Waiting for device to boot");
-
- err = wait_on_bit_timeout(&data->flags, BTUSB_BOOTING,
- TASK_INTERRUPTIBLE,
- msecs_to_jiffies(1000));
-
- if (err == -EINTR) {
- bt_dev_err(hdev, "Device boot interrupted");
- return -EINTR;
- }
-
- if (err) {
- bt_dev_err(hdev, "Device boot timeout");
- btintel_reset_to_bootloader(hdev);
- return -ETIMEDOUT;
- }
-
- rettime = ktime_get();
- delta = ktime_sub(rettime, calltime);
- duration = (unsigned long long) ktime_to_ns(delta) >> 10;
-
- bt_dev_info(hdev, "Device booted in %llu usecs", duration);
clear_bit(BTUSB_BOOTLOADER, &data->flags);
err = btusb_setup_intel_new_get_fw_name(&ver, &params, ddcname,
sizeof(ddcname), "ddc");
- if (!err) {
+ if (err < 0) {
bt_dev_err(hdev, "Unsupported Intel firmware naming");
} else {
/* Once the device is running in operational mode, it needs to
@@ -2947,8 +2910,6 @@ static int btusb_setup_intel_newgen(struct hci_dev *hdev)
struct btusb_data *data = hci_get_drvdata(hdev);
u32 boot_param;
char ddcname[64];
- ktime_t calltime, delta, rettime;
- unsigned long long duration;
int err;
struct intel_debug_features features;
struct intel_version_tlv version;
@@ -2961,8 +2922,6 @@ static int btusb_setup_intel_newgen(struct hci_dev *hdev)
*/
boot_param = 0x00000000;
- calltime = ktime_get();
-
/* Read the Intel version information to determine if the device
* is in bootloader mode or if it already has operational firmware
* loaded.
@@ -2974,7 +2933,9 @@ static int btusb_setup_intel_newgen(struct hci_dev *hdev)
return err;
}
- btintel_version_info_tlv(hdev, &version);
+ err = btintel_version_info_tlv(hdev, &version);
+ if (err)
+ return err;
err = btusb_intel_download_firmware_newgen(hdev, &version, &boot_param);
if (err)
@@ -2984,52 +2945,9 @@ static int btusb_setup_intel_newgen(struct hci_dev *hdev)
if (version.img_type == 0x03)
goto finish;
- rettime = ktime_get();
- delta = ktime_sub(rettime, calltime);
- duration = (unsigned long long)ktime_to_ns(delta) >> 10;
-
- bt_dev_info(hdev, "Firmware loaded in %llu usecs", duration);
-
- calltime = ktime_get();
-
- set_bit(BTUSB_BOOTING, &data->flags);
-
- err = btintel_send_intel_reset(hdev, boot_param);
- if (err) {
- bt_dev_err(hdev, "Intel Soft Reset failed (%d)", err);
- btintel_reset_to_bootloader(hdev);
+ err = btusb_intel_boot(hdev, boot_param);
+ if (err)
return err;
- }
-
- /* The bootloader will not indicate when the device is ready. This
- * is done by the operational firmware sending bootup notification.
- *
- * Booting into operational firmware should not take longer than
- * 1 second. However if that happens, then just fail the setup
- * since something went wrong.
- */
- bt_dev_info(hdev, "Waiting for device to boot");
-
- err = wait_on_bit_timeout(&data->flags, BTUSB_BOOTING,
- TASK_INTERRUPTIBLE,
- msecs_to_jiffies(1000));
-
- if (err == -EINTR) {
- bt_dev_err(hdev, "Device boot interrupted");
- return -EINTR;
- }
-
- if (err) {
- bt_dev_err(hdev, "Device boot timeout");
- btintel_reset_to_bootloader(hdev);
- return -ETIMEDOUT;
- }
-
- rettime = ktime_get();
- delta = ktime_sub(rettime, calltime);
- duration = (unsigned long long)ktime_to_ns(delta) >> 10;
-
- bt_dev_info(hdev, "Device booted in %llu usecs", duration);
clear_bit(BTUSB_BOOTLOADER, &data->flags);
@@ -3495,7 +3413,7 @@ static int btusb_mtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwnam
fw_ptr = fw->data;
fw_bin_ptr = fw_ptr;
globaldesc = (struct btmtk_global_desc *)(fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE);
- section_num = globaldesc->section_num;
+ section_num = le32_to_cpu(globaldesc->section_num);
for (i = 0; i < section_num; i++) {
first_block = 1;
@@ -3503,8 +3421,8 @@ static int btusb_mtk_setup_firmware_79xx(struct hci_dev *hdev, const char *fwnam
sectionmap = (struct btmtk_section_map *)(fw_ptr + MTK_FW_ROM_PATCH_HEADER_SIZE +
MTK_FW_ROM_PATCH_GD_SIZE + MTK_FW_ROM_PATCH_SEC_MAP_SIZE * i);
- section_offset = sectionmap->secoffset;
- dl_size = sectionmap->bin_info_spec.dlsize;
+ section_offset = le32_to_cpu(sectionmap->secoffset);
+ dl_size = le32_to_cpu(sectionmap->bin_info_spec.dlsize);
if (dl_size > 0) {
retry = 20;
@@ -3740,7 +3658,7 @@ static int btusb_mtk_setup(struct hci_dev *hdev)
int err, status;
u32 dev_id;
char fw_bin_name[64];
- u32 fw_version;
+ u32 fw_version = 0;
u8 param;
calltime = ktime_get();
@@ -4849,8 +4767,8 @@ static int btusb_probe(struct usb_interface *intf,
data->diag = NULL;
}
- if (!enable_autosuspend)
- usb_disable_autosuspend(data->udev);
+ if (enable_autosuspend)
+ usb_enable_autosuspend(data->udev);
err = hci_register_dev(hdev);
if (err < 0)
@@ -4910,9 +4828,6 @@ static void btusb_disconnect(struct usb_interface *intf)
gpiod_put(data->reset_gpio);
hci_free_dev(hdev);
-
- if (!enable_autosuspend)
- usb_enable_autosuspend(data->udev);
}
#ifdef CONFIG_PM
diff --git a/drivers/bluetooth/hci_bcm.c b/drivers/bluetooth/hci_bcm.c
index 3764ceb6fa0d..3cd57fc56ade 100644
--- a/drivers/bluetooth/hci_bcm.c
+++ b/drivers/bluetooth/hci_bcm.c
@@ -68,6 +68,8 @@ struct bcm_device_data {
* deassert = Bluetooth device may sleep when sleep criteria are met
* @shutdown: BT_REG_ON pin,
* power up or power down Bluetooth device internal regulators
+ * @reset: BT_RST_N pin,
+ * active low resets the Bluetooth logic core
* @set_device_wakeup: callback to toggle BT_WAKE pin
* either by accessing @device_wakeup or by calling @btlp
* @set_shutdown: callback to toggle BT_REG_ON pin
@@ -101,6 +103,7 @@ struct bcm_device {
const char *name;
struct gpio_desc *device_wakeup;
struct gpio_desc *shutdown;
+ struct gpio_desc *reset;
int (*set_device_wakeup)(struct bcm_device *, bool);
int (*set_shutdown)(struct bcm_device *, bool);
#ifdef CONFIG_ACPI
@@ -985,6 +988,15 @@ static int bcm_gpio_set_device_wakeup(struct bcm_device *dev, bool awake)
static int bcm_gpio_set_shutdown(struct bcm_device *dev, bool powered)
{
gpiod_set_value_cansleep(dev->shutdown, powered);
+ if (dev->reset)
+ /*
+ * The reset line is asserted on powerdown and deasserted
+ * on poweron so the inverse of powered is used. Notice
+ * that the GPIO line BT_RST_N needs to be specified as
+ * active low in the device tree or similar system
+ * description.
+ */
+ gpiod_set_value_cansleep(dev->reset, !powered);
return 0;
}
@@ -1050,6 +1062,11 @@ static int bcm_get_resources(struct bcm_device *dev)
if (IS_ERR(dev->shutdown))
return PTR_ERR(dev->shutdown);
+ dev->reset = devm_gpiod_get_optional(dev->dev, "reset",
+ GPIOD_OUT_LOW);
+ if (IS_ERR(dev->reset))
+ return PTR_ERR(dev->reset);
+
dev->set_device_wakeup = bcm_gpio_set_device_wakeup;
dev->set_shutdown = bcm_gpio_set_shutdown;
@@ -1482,6 +1499,8 @@ static struct bcm_device_data bcm43438_device_data = {
static const struct of_device_id bcm_bluetooth_of_match[] = {
{ .compatible = "brcm,bcm20702a1" },
{ .compatible = "brcm,bcm4329-bt" },
+ { .compatible = "brcm,bcm4330-bt" },
+ { .compatible = "brcm,bcm4334-bt" },
{ .compatible = "brcm,bcm4345c5" },
{ .compatible = "brcm,bcm4330-bt" },
{ .compatible = "brcm,bcm43438-bt", .data = &bcm43438_device_data },
diff --git a/drivers/bluetooth/hci_intel.c b/drivers/bluetooth/hci_intel.c
index b20a40fab83e..7249b91d9b91 100644
--- a/drivers/bluetooth/hci_intel.c
+++ b/drivers/bluetooth/hci_intel.c
@@ -735,7 +735,7 @@ static int intel_setup(struct hci_uart *hu)
set_bit(STATE_DOWNLOADING, &intel->flags);
/* Start firmware downloading and get boot parameter */
- err = btintel_download_firmware(hdev, fw, &boot_param);
+ err = btintel_download_firmware(hdev, &ver, fw, &boot_param);
if (err < 0)
goto done;
@@ -784,7 +784,10 @@ static int intel_setup(struct hci_uart *hu)
done:
release_firmware(fw);
- if (err < 0)
+ /* Check if there was an error and if is not -EALREADY which means the
+ * firmware has already been loaded.
+ */
+ if (err < 0 && err != -EALREADY)
return err;
/* We need to restore the default speed before Intel reset */
diff --git a/drivers/bluetooth/hci_ldisc.c b/drivers/bluetooth/hci_ldisc.c
index 637c5b8c2aa1..71a4ca505e09 100644
--- a/drivers/bluetooth/hci_ldisc.c
+++ b/drivers/bluetooth/hci_ldisc.c
@@ -821,7 +821,6 @@ static __poll_t hci_uart_tty_poll(struct tty_struct *tty,
static struct tty_ldisc_ops hci_uart_ldisc = {
.owner = THIS_MODULE,
- .magic = TTY_LDISC_MAGIC,
.name = "n_hci",
.open = hci_uart_tty_open,
.close = hci_uart_tty_close,
diff --git a/drivers/bluetooth/hci_qca.c b/drivers/bluetooth/hci_qca.c
index de36af63e182..0a0056912d51 100644
--- a/drivers/bluetooth/hci_qca.c
+++ b/drivers/bluetooth/hci_qca.c
@@ -1066,7 +1066,7 @@ static void qca_controller_memdump(struct work_struct *work)
* packets in the buffer.
*/
/* For QCA6390, controller does not lost packets but
- * sequence number field of packat sometimes has error
+ * sequence number field of packet sometimes has error
* bits, so skip this checking for missing packet.
*/
while ((seq_no > qca_memdump->current_seq_no + 1) &&
@@ -1571,6 +1571,20 @@ static void qca_cmd_timeout(struct hci_dev *hdev)
mutex_unlock(&qca->hci_memdump_lock);
}
+static bool qca_prevent_wake(struct hci_dev *hdev)
+{
+ struct hci_uart *hu = hci_get_drvdata(hdev);
+ bool wakeup;
+
+ /* UART driver handles the interrupt from BT SoC.So we need to use
+ * device handle of UART driver to get the status of device may wakeup.
+ */
+ wakeup = device_may_wakeup(hu->serdev->ctrl->dev.parent);
+ bt_dev_dbg(hu->hdev, "wakeup status : %d", wakeup);
+
+ return !wakeup;
+}
+
static int qca_wcn3990_init(struct hci_uart *hu)
{
struct qca_serdev *qcadev;
@@ -1721,6 +1735,7 @@ retry:
qca_debugfs_init(hdev);
hu->hdev->hw_error = qca_hw_error;
hu->hdev->cmd_timeout = qca_cmd_timeout;
+ hu->hdev->prevent_wake = qca_prevent_wake;
} else if (ret == -ENOENT) {
/* No patch/nvm-config found, run with original fw/config */
set_bit(QCA_ROM_FW, &qca->flags);
diff --git a/drivers/bluetooth/virtio_bt.c b/drivers/bluetooth/virtio_bt.c
new file mode 100644
index 000000000000..c804db7e90f8
--- /dev/null
+++ b/drivers/bluetooth/virtio_bt.c
@@ -0,0 +1,401 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <linux/module.h>
+#include <linux/virtio.h>
+#include <linux/virtio_config.h>
+#include <linux/skbuff.h>
+
+#include <uapi/linux/virtio_ids.h>
+#include <uapi/linux/virtio_bt.h>
+
+#include <net/bluetooth/bluetooth.h>
+#include <net/bluetooth/hci_core.h>
+
+#define VERSION "0.1"
+
+enum {
+ VIRTBT_VQ_TX,
+ VIRTBT_VQ_RX,
+ VIRTBT_NUM_VQS,
+};
+
+struct virtio_bluetooth {
+ struct virtio_device *vdev;
+ struct virtqueue *vqs[VIRTBT_NUM_VQS];
+ struct work_struct rx;
+ struct hci_dev *hdev;
+};
+
+static int virtbt_add_inbuf(struct virtio_bluetooth *vbt)
+{
+ struct virtqueue *vq = vbt->vqs[VIRTBT_VQ_RX];
+ struct scatterlist sg[1];
+ struct sk_buff *skb;
+ int err;
+
+ skb = alloc_skb(1000, GFP_KERNEL);
+ sg_init_one(sg, skb->data, 1000);
+
+ err = virtqueue_add_inbuf(vq, sg, 1, skb, GFP_KERNEL);
+ if (err < 0) {
+ kfree_skb(skb);
+ return err;
+ }
+
+ return 0;
+}
+
+static int virtbt_open(struct hci_dev *hdev)
+{
+ struct virtio_bluetooth *vbt = hci_get_drvdata(hdev);
+
+ if (virtbt_add_inbuf(vbt) < 0)
+ return -EIO;
+
+ virtqueue_kick(vbt->vqs[VIRTBT_VQ_RX]);
+ return 0;
+}
+
+static int virtbt_close(struct hci_dev *hdev)
+{
+ struct virtio_bluetooth *vbt = hci_get_drvdata(hdev);
+ int i;
+
+ cancel_work_sync(&vbt->rx);
+
+ for (i = 0; i < ARRAY_SIZE(vbt->vqs); i++) {
+ struct virtqueue *vq = vbt->vqs[i];
+ struct sk_buff *skb;
+
+ while ((skb = virtqueue_detach_unused_buf(vq)))
+ kfree_skb(skb);
+ }
+
+ return 0;
+}
+
+static int virtbt_flush(struct hci_dev *hdev)
+{
+ return 0;
+}
+
+static int virtbt_send_frame(struct hci_dev *hdev, struct sk_buff *skb)
+{
+ struct virtio_bluetooth *vbt = hci_get_drvdata(hdev);
+ struct scatterlist sg[1];
+ int err;
+
+ memcpy(skb_push(skb, 1), &hci_skb_pkt_type(skb), 1);
+
+ sg_init_one(sg, skb->data, skb->len);
+ err = virtqueue_add_outbuf(vbt->vqs[VIRTBT_VQ_TX], sg, 1, skb,
+ GFP_KERNEL);
+ if (err) {
+ kfree_skb(skb);
+ return err;
+ }
+
+ virtqueue_kick(vbt->vqs[VIRTBT_VQ_TX]);
+ return 0;
+}
+
+static int virtbt_setup_zephyr(struct hci_dev *hdev)
+{
+ struct sk_buff *skb;
+
+ /* Read Build Information */
+ skb = __hci_cmd_sync(hdev, 0xfc08, 0, NULL, HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ bt_dev_info(hdev, "%s", (char *)(skb->data + 1));
+
+ hci_set_fw_info(hdev, "%s", skb->data + 1);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static int virtbt_set_bdaddr_zephyr(struct hci_dev *hdev,
+ const bdaddr_t *bdaddr)
+{
+ struct sk_buff *skb;
+
+ /* Write BD_ADDR */
+ skb = __hci_cmd_sync(hdev, 0xfc06, 6, bdaddr, HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static int virtbt_setup_intel(struct hci_dev *hdev)
+{
+ struct sk_buff *skb;
+
+ /* Intel Read Version */
+ skb = __hci_cmd_sync(hdev, 0xfc05, 0, NULL, HCI_CMD_TIMEOUT);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static int virtbt_set_bdaddr_intel(struct hci_dev *hdev, const bdaddr_t *bdaddr)
+{
+ struct sk_buff *skb;
+
+ /* Intel Write BD Address */
+ skb = __hci_cmd_sync(hdev, 0xfc31, 6, bdaddr, HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static int virtbt_setup_realtek(struct hci_dev *hdev)
+{
+ struct sk_buff *skb;
+
+ /* Read ROM Version */
+ skb = __hci_cmd_sync(hdev, 0xfc6d, 0, NULL, HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ bt_dev_info(hdev, "ROM version %u", *((__u8 *) (skb->data + 1)));
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static int virtbt_shutdown_generic(struct hci_dev *hdev)
+{
+ struct sk_buff *skb;
+
+ /* Reset */
+ skb = __hci_cmd_sync(hdev, HCI_OP_RESET, 0, NULL, HCI_INIT_TIMEOUT);
+ if (IS_ERR(skb))
+ return PTR_ERR(skb);
+
+ kfree_skb(skb);
+ return 0;
+}
+
+static void virtbt_rx_handle(struct virtio_bluetooth *vbt, struct sk_buff *skb)
+{
+ __u8 pkt_type;
+
+ pkt_type = *((__u8 *) skb->data);
+ skb_pull(skb, 1);
+
+ switch (pkt_type) {
+ case HCI_EVENT_PKT:
+ case HCI_ACLDATA_PKT:
+ case HCI_SCODATA_PKT:
+ case HCI_ISODATA_PKT:
+ hci_skb_pkt_type(skb) = pkt_type;
+ hci_recv_frame(vbt->hdev, skb);
+ break;
+ }
+}
+
+static void virtbt_rx_work(struct work_struct *work)
+{
+ struct virtio_bluetooth *vbt = container_of(work,
+ struct virtio_bluetooth, rx);
+ struct sk_buff *skb;
+ unsigned int len;
+
+ skb = virtqueue_get_buf(vbt->vqs[VIRTBT_VQ_RX], &len);
+ if (!skb)
+ return;
+
+ skb->len = len;
+ virtbt_rx_handle(vbt, skb);
+
+ if (virtbt_add_inbuf(vbt) < 0)
+ return;
+
+ virtqueue_kick(vbt->vqs[VIRTBT_VQ_RX]);
+}
+
+static void virtbt_tx_done(struct virtqueue *vq)
+{
+ struct sk_buff *skb;
+ unsigned int len;
+
+ while ((skb = virtqueue_get_buf(vq, &len)))
+ kfree_skb(skb);
+}
+
+static void virtbt_rx_done(struct virtqueue *vq)
+{
+ struct virtio_bluetooth *vbt = vq->vdev->priv;
+
+ schedule_work(&vbt->rx);
+}
+
+static int virtbt_probe(struct virtio_device *vdev)
+{
+ vq_callback_t *callbacks[VIRTBT_NUM_VQS] = {
+ [VIRTBT_VQ_TX] = virtbt_tx_done,
+ [VIRTBT_VQ_RX] = virtbt_rx_done,
+ };
+ const char *names[VIRTBT_NUM_VQS] = {
+ [VIRTBT_VQ_TX] = "tx",
+ [VIRTBT_VQ_RX] = "rx",
+ };
+ struct virtio_bluetooth *vbt;
+ struct hci_dev *hdev;
+ int err;
+ __u8 type;
+
+ if (!virtio_has_feature(vdev, VIRTIO_F_VERSION_1))
+ return -ENODEV;
+
+ type = virtio_cread8(vdev, offsetof(struct virtio_bt_config, type));
+
+ switch (type) {
+ case VIRTIO_BT_CONFIG_TYPE_PRIMARY:
+ case VIRTIO_BT_CONFIG_TYPE_AMP:
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ vbt = kzalloc(sizeof(*vbt), GFP_KERNEL);
+ if (!vbt)
+ return -ENOMEM;
+
+ vdev->priv = vbt;
+ vbt->vdev = vdev;
+
+ INIT_WORK(&vbt->rx, virtbt_rx_work);
+
+ err = virtio_find_vqs(vdev, VIRTBT_NUM_VQS, vbt->vqs, callbacks,
+ names, NULL);
+ if (err)
+ return err;
+
+ hdev = hci_alloc_dev();
+ if (!hdev) {
+ err = -ENOMEM;
+ goto failed;
+ }
+
+ vbt->hdev = hdev;
+
+ hdev->bus = HCI_VIRTIO;
+ hdev->dev_type = type;
+ hci_set_drvdata(hdev, vbt);
+
+ hdev->open = virtbt_open;
+ hdev->close = virtbt_close;
+ hdev->flush = virtbt_flush;
+ hdev->send = virtbt_send_frame;
+
+ if (virtio_has_feature(vdev, VIRTIO_BT_F_VND_HCI)) {
+ __u16 vendor;
+
+ virtio_cread(vdev, struct virtio_bt_config, vendor, &vendor);
+
+ switch (vendor) {
+ case VIRTIO_BT_CONFIG_VENDOR_ZEPHYR:
+ hdev->manufacturer = 1521;
+ hdev->setup = virtbt_setup_zephyr;
+ hdev->shutdown = virtbt_shutdown_generic;
+ hdev->set_bdaddr = virtbt_set_bdaddr_zephyr;
+ break;
+
+ case VIRTIO_BT_CONFIG_VENDOR_INTEL:
+ hdev->manufacturer = 2;
+ hdev->setup = virtbt_setup_intel;
+ hdev->shutdown = virtbt_shutdown_generic;
+ hdev->set_bdaddr = virtbt_set_bdaddr_intel;
+ set_bit(HCI_QUIRK_STRICT_DUPLICATE_FILTER, &hdev->quirks);
+ set_bit(HCI_QUIRK_SIMULTANEOUS_DISCOVERY, &hdev->quirks);
+ set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks);
+ break;
+
+ case VIRTIO_BT_CONFIG_VENDOR_REALTEK:
+ hdev->manufacturer = 93;
+ hdev->setup = virtbt_setup_realtek;
+ hdev->shutdown = virtbt_shutdown_generic;
+ set_bit(HCI_QUIRK_SIMULTANEOUS_DISCOVERY, &hdev->quirks);
+ set_bit(HCI_QUIRK_WIDEBAND_SPEECH_SUPPORTED, &hdev->quirks);
+ break;
+ }
+ }
+
+ if (virtio_has_feature(vdev, VIRTIO_BT_F_MSFT_EXT)) {
+ __u16 msft_opcode;
+
+ virtio_cread(vdev, struct virtio_bt_config,
+ msft_opcode, &msft_opcode);
+
+ hci_set_msft_opcode(hdev, msft_opcode);
+ }
+
+ if (virtio_has_feature(vdev, VIRTIO_BT_F_AOSP_EXT))
+ hci_set_aosp_capable(hdev);
+
+ if (hci_register_dev(hdev) < 0) {
+ hci_free_dev(hdev);
+ err = -EBUSY;
+ goto failed;
+ }
+
+ return 0;
+
+failed:
+ vdev->config->del_vqs(vdev);
+ return err;
+}
+
+static void virtbt_remove(struct virtio_device *vdev)
+{
+ struct virtio_bluetooth *vbt = vdev->priv;
+ struct hci_dev *hdev = vbt->hdev;
+
+ hci_unregister_dev(hdev);
+ vdev->config->reset(vdev);
+
+ hci_free_dev(hdev);
+ vbt->hdev = NULL;
+
+ vdev->config->del_vqs(vdev);
+ kfree(vbt);
+}
+
+static struct virtio_device_id virtbt_table[] = {
+ { VIRTIO_ID_BT, VIRTIO_DEV_ANY_ID },
+ { 0 },
+};
+
+MODULE_DEVICE_TABLE(virtio, virtbt_table);
+
+static const unsigned int virtbt_features[] = {
+ VIRTIO_BT_F_VND_HCI,
+ VIRTIO_BT_F_MSFT_EXT,
+ VIRTIO_BT_F_AOSP_EXT,
+};
+
+static struct virtio_driver virtbt_driver = {
+ .driver.name = KBUILD_MODNAME,
+ .driver.owner = THIS_MODULE,
+ .feature_table = virtbt_features,
+ .feature_table_size = ARRAY_SIZE(virtbt_features),
+ .id_table = virtbt_table,
+ .probe = virtbt_probe,
+ .remove = virtbt_remove,
+};
+
+module_virtio_driver(virtbt_driver);
+
+MODULE_AUTHOR("Marcel Holtmann <marcel@holtmann.org>");
+MODULE_DESCRIPTION("Generic Bluetooth VIRTIO driver ver " VERSION);
+MODULE_VERSION(VERSION);
+MODULE_LICENSE("GPL");
diff --git a/drivers/bus/mhi/core/boot.c b/drivers/bus/mhi/core/boot.c
index c2546bf229fb..8100cf51cd09 100644
--- a/drivers/bus/mhi/core/boot.c
+++ b/drivers/bus/mhi/core/boot.c
@@ -389,7 +389,6 @@ static void mhi_firmware_copy(struct mhi_controller *mhi_cntrl,
void mhi_fw_load_handler(struct mhi_controller *mhi_cntrl)
{
const struct firmware *firmware = NULL;
- struct image_info *image_info;
struct device *dev = &mhi_cntrl->mhi_dev->dev;
const char *fw_name;
void *buf;
@@ -417,9 +416,9 @@ void mhi_fw_load_handler(struct mhi_controller *mhi_cntrl)
}
}
- /* If device is in pass through, do reset to ready state transition */
- if (mhi_cntrl->ee == MHI_EE_PTHRU)
- goto fw_load_ee_pthru;
+ /* wait for ready on pass through or any other execution environment */
+ if (mhi_cntrl->ee != MHI_EE_EDL && mhi_cntrl->ee != MHI_EE_PBL)
+ goto fw_load_ready_state;
fw_name = (mhi_cntrl->ee == MHI_EE_EDL) ?
mhi_cntrl->edl_image : mhi_cntrl->fw_image;
@@ -461,9 +460,10 @@ void mhi_fw_load_handler(struct mhi_controller *mhi_cntrl)
goto error_fw_load;
}
- if (mhi_cntrl->ee == MHI_EE_EDL) {
+ /* Wait for ready since EDL image was loaded */
+ if (fw_name == mhi_cntrl->edl_image) {
release_firmware(firmware);
- return;
+ goto fw_load_ready_state;
}
write_lock_irq(&mhi_cntrl->pm_lock);
@@ -488,47 +488,45 @@ void mhi_fw_load_handler(struct mhi_controller *mhi_cntrl)
release_firmware(firmware);
-fw_load_ee_pthru:
+fw_load_ready_state:
/* Transitioning into MHI RESET->READY state */
ret = mhi_ready_state_transition(mhi_cntrl);
-
- if (!mhi_cntrl->fbc_download)
- return;
-
if (ret) {
dev_err(dev, "MHI did not enter READY state\n");
goto error_ready_state;
}
- /* Wait for the SBL event */
- ret = wait_event_timeout(mhi_cntrl->state_event,
- mhi_cntrl->ee == MHI_EE_SBL ||
- MHI_PM_IN_ERROR_STATE(mhi_cntrl->pm_state),
- msecs_to_jiffies(mhi_cntrl->timeout_ms));
+ dev_info(dev, "Wait for device to enter SBL or Mission mode\n");
+ return;
- if (!ret || MHI_PM_IN_ERROR_STATE(mhi_cntrl->pm_state)) {
- dev_err(dev, "MHI did not enter SBL\n");
- goto error_ready_state;
+error_ready_state:
+ if (mhi_cntrl->fbc_download) {
+ mhi_free_bhie_table(mhi_cntrl, mhi_cntrl->fbc_image);
+ mhi_cntrl->fbc_image = NULL;
}
- /* Start full firmware image download */
- image_info = mhi_cntrl->fbc_image;
+error_fw_load:
+ mhi_cntrl->pm_state = MHI_PM_FW_DL_ERR;
+ wake_up_all(&mhi_cntrl->state_event);
+}
+
+int mhi_download_amss_image(struct mhi_controller *mhi_cntrl)
+{
+ struct image_info *image_info = mhi_cntrl->fbc_image;
+ struct device *dev = &mhi_cntrl->mhi_dev->dev;
+ int ret;
+
+ if (!image_info)
+ return -EIO;
+
ret = mhi_fw_load_bhie(mhi_cntrl,
/* Vector table is the last entry */
&image_info->mhi_buf[image_info->entries - 1]);
if (ret) {
- dev_err(dev, "MHI did not load image over BHIe, ret: %d\n",
- ret);
- goto error_fw_load;
+ dev_err(dev, "MHI did not load AMSS, ret:%d\n", ret);
+ mhi_cntrl->pm_state = MHI_PM_FW_DL_ERR;
+ wake_up_all(&mhi_cntrl->state_event);
}
- return;
-
-error_ready_state:
- mhi_free_bhie_table(mhi_cntrl, mhi_cntrl->fbc_image);
- mhi_cntrl->fbc_image = NULL;
-
-error_fw_load:
- mhi_cntrl->pm_state = MHI_PM_FW_DL_ERR;
- wake_up_all(&mhi_cntrl->state_event);
+ return ret;
}
diff --git a/drivers/bus/mhi/core/debugfs.c b/drivers/bus/mhi/core/debugfs.c
index 7d43138ce66d..858d7516410b 100644
--- a/drivers/bus/mhi/core/debugfs.c
+++ b/drivers/bus/mhi/core/debugfs.c
@@ -377,7 +377,7 @@ static struct dentry *mhi_debugfs_root;
void mhi_create_debugfs(struct mhi_controller *mhi_cntrl)
{
mhi_cntrl->debugfs_dentry =
- debugfs_create_dir(dev_name(mhi_cntrl->cntrl_dev),
+ debugfs_create_dir(dev_name(&mhi_cntrl->mhi_dev->dev),
mhi_debugfs_root);
debugfs_create_file("states", 0444, mhi_cntrl->debugfs_dentry,
diff --git a/drivers/bus/mhi/core/init.c b/drivers/bus/mhi/core/init.c
index be4eebb0971b..c81b377fca8f 100644
--- a/drivers/bus/mhi/core/init.c
+++ b/drivers/bus/mhi/core/init.c
@@ -22,13 +22,14 @@
static DEFINE_IDA(mhi_controller_ida);
const char * const mhi_ee_str[MHI_EE_MAX] = {
- [MHI_EE_PBL] = "PBL",
- [MHI_EE_SBL] = "SBL",
- [MHI_EE_AMSS] = "AMSS",
- [MHI_EE_RDDM] = "RDDM",
- [MHI_EE_WFW] = "WFW",
- [MHI_EE_PTHRU] = "PASS THRU",
- [MHI_EE_EDL] = "EDL",
+ [MHI_EE_PBL] = "PRIMARY BOOTLOADER",
+ [MHI_EE_SBL] = "SECONDARY BOOTLOADER",
+ [MHI_EE_AMSS] = "MISSION MODE",
+ [MHI_EE_RDDM] = "RAMDUMP DOWNLOAD MODE",
+ [MHI_EE_WFW] = "WLAN FIRMWARE",
+ [MHI_EE_PTHRU] = "PASS THROUGH",
+ [MHI_EE_EDL] = "EMERGENCY DOWNLOAD",
+ [MHI_EE_FP] = "FLASH PROGRAMMER",
[MHI_EE_DISABLE_TRANSITION] = "DISABLE",
[MHI_EE_NOT_SUPPORTED] = "NOT SUPPORTED",
};
@@ -37,8 +38,9 @@ const char * const dev_state_tran_str[DEV_ST_TRANSITION_MAX] = {
[DEV_ST_TRANSITION_PBL] = "PBL",
[DEV_ST_TRANSITION_READY] = "READY",
[DEV_ST_TRANSITION_SBL] = "SBL",
- [DEV_ST_TRANSITION_MISSION_MODE] = "MISSION_MODE",
- [DEV_ST_TRANSITION_SYS_ERR] = "SYS_ERR",
+ [DEV_ST_TRANSITION_MISSION_MODE] = "MISSION MODE",
+ [DEV_ST_TRANSITION_FP] = "FLASH PROGRAMMER",
+ [DEV_ST_TRANSITION_SYS_ERR] = "SYS ERROR",
[DEV_ST_TRANSITION_DISABLE] = "DISABLE",
};
@@ -49,24 +51,30 @@ const char * const mhi_state_str[MHI_STATE_MAX] = {
[MHI_STATE_M1] = "M1",
[MHI_STATE_M2] = "M2",
[MHI_STATE_M3] = "M3",
- [MHI_STATE_M3_FAST] = "M3_FAST",
+ [MHI_STATE_M3_FAST] = "M3 FAST",
[MHI_STATE_BHI] = "BHI",
- [MHI_STATE_SYS_ERR] = "SYS_ERR",
+ [MHI_STATE_SYS_ERR] = "SYS ERROR",
+};
+
+const char * const mhi_ch_state_type_str[MHI_CH_STATE_TYPE_MAX] = {
+ [MHI_CH_STATE_TYPE_RESET] = "RESET",
+ [MHI_CH_STATE_TYPE_STOP] = "STOP",
+ [MHI_CH_STATE_TYPE_START] = "START",
};
static const char * const mhi_pm_state_str[] = {
[MHI_PM_STATE_DISABLE] = "DISABLE",
- [MHI_PM_STATE_POR] = "POR",
+ [MHI_PM_STATE_POR] = "POWER ON RESET",
[MHI_PM_STATE_M0] = "M0",
[MHI_PM_STATE_M2] = "M2",
[MHI_PM_STATE_M3_ENTER] = "M?->M3",
[MHI_PM_STATE_M3] = "M3",
[MHI_PM_STATE_M3_EXIT] = "M3->M0",
- [MHI_PM_STATE_FW_DL_ERR] = "FW DL Error",
- [MHI_PM_STATE_SYS_ERR_DETECT] = "SYS_ERR Detect",
- [MHI_PM_STATE_SYS_ERR_PROCESS] = "SYS_ERR Process",
+ [MHI_PM_STATE_FW_DL_ERR] = "Firmware Download Error",
+ [MHI_PM_STATE_SYS_ERR_DETECT] = "SYS ERROR Detect",
+ [MHI_PM_STATE_SYS_ERR_PROCESS] = "SYS ERROR Process",
[MHI_PM_STATE_SHUTDOWN_PROCESS] = "SHUTDOWN Process",
- [MHI_PM_STATE_LD_ERR_FATAL_DETECT] = "LD or Error Fatal Detect",
+ [MHI_PM_STATE_LD_ERR_FATAL_DETECT] = "Linkdown or Error Fatal Detect",
};
const char *to_mhi_pm_state_str(enum mhi_pm_state state)
@@ -508,8 +516,6 @@ int mhi_init_mmio(struct mhi_controller *mhi_cntrl)
/* Setup wake db */
mhi_cntrl->wake_db = base + val + (8 * MHI_DEV_WAKE_DB);
- mhi_write_reg(mhi_cntrl, mhi_cntrl->wake_db, 4, 0);
- mhi_write_reg(mhi_cntrl, mhi_cntrl->wake_db, 0, 0);
mhi_cntrl->wake_set = false;
/* Setup channel db address for each channel in tre_ring */
@@ -552,6 +558,7 @@ void mhi_deinit_chan_ctxt(struct mhi_controller *mhi_cntrl,
struct mhi_ring *buf_ring;
struct mhi_ring *tre_ring;
struct mhi_chan_ctxt *chan_ctxt;
+ u32 tmp;
buf_ring = &mhi_chan->buf_ring;
tre_ring = &mhi_chan->tre_ring;
@@ -565,7 +572,19 @@ void mhi_deinit_chan_ctxt(struct mhi_controller *mhi_cntrl,
vfree(buf_ring->base);
buf_ring->base = tre_ring->base = NULL;
+ tre_ring->ctxt_wp = NULL;
chan_ctxt->rbase = 0;
+ chan_ctxt->rlen = 0;
+ chan_ctxt->rp = 0;
+ chan_ctxt->wp = 0;
+
+ tmp = chan_ctxt->chcfg;
+ tmp &= ~CHAN_CTX_CHSTATE_MASK;
+ tmp |= (MHI_CH_STATE_DISABLED << CHAN_CTX_CHSTATE_SHIFT);
+ chan_ctxt->chcfg = tmp;
+
+ /* Update to all cores */
+ smp_wmb();
}
int mhi_init_chan_ctxt(struct mhi_controller *mhi_cntrl,
@@ -863,12 +882,10 @@ int mhi_register_controller(struct mhi_controller *mhi_cntrl,
u32 soc_info;
int ret, i;
- if (!mhi_cntrl)
- return -EINVAL;
-
- if (!mhi_cntrl->runtime_get || !mhi_cntrl->runtime_put ||
+ if (!mhi_cntrl || !mhi_cntrl->cntrl_dev || !mhi_cntrl->regs ||
+ !mhi_cntrl->runtime_get || !mhi_cntrl->runtime_put ||
!mhi_cntrl->status_cb || !mhi_cntrl->read_reg ||
- !mhi_cntrl->write_reg || !mhi_cntrl->nr_irqs)
+ !mhi_cntrl->write_reg || !mhi_cntrl->nr_irqs || !mhi_cntrl->irq)
return -EINVAL;
ret = parse_config(mhi_cntrl, config);
@@ -890,8 +907,7 @@ int mhi_register_controller(struct mhi_controller *mhi_cntrl,
INIT_WORK(&mhi_cntrl->st_worker, mhi_pm_st_worker);
init_waitqueue_head(&mhi_cntrl->state_event);
- mhi_cntrl->hiprio_wq = alloc_ordered_workqueue
- ("mhi_hiprio_wq", WQ_MEM_RECLAIM | WQ_HIGHPRI);
+ mhi_cntrl->hiprio_wq = alloc_ordered_workqueue("mhi_hiprio_wq", WQ_HIGHPRI);
if (!mhi_cntrl->hiprio_wq) {
dev_err(mhi_cntrl->cntrl_dev, "Failed to allocate workqueue\n");
ret = -ENOMEM;
@@ -1083,8 +1099,6 @@ int mhi_prepare_for_power_up(struct mhi_controller *mhi_cntrl)
mhi_rddm_prepare(mhi_cntrl, mhi_cntrl->rddm_image);
}
- mhi_cntrl->pre_init = true;
-
mutex_unlock(&mhi_cntrl->pm_mutex);
return 0;
@@ -1115,7 +1129,6 @@ void mhi_unprepare_after_power_down(struct mhi_controller *mhi_cntrl)
}
mhi_deinit_dev_ctxt(mhi_cntrl);
- mhi_cntrl->pre_init = false;
}
EXPORT_SYMBOL_GPL(mhi_unprepare_after_power_down);
@@ -1296,7 +1309,8 @@ static int mhi_driver_remove(struct device *dev)
mutex_lock(&mhi_chan->mutex);
- if (ch_state[dir] == MHI_CH_STATE_ENABLED &&
+ if ((ch_state[dir] == MHI_CH_STATE_ENABLED ||
+ ch_state[dir] == MHI_CH_STATE_STOP) &&
!mhi_chan->offload_ch)
mhi_deinit_chan_ctxt(mhi_cntrl, mhi_chan);
diff --git a/drivers/bus/mhi/core/internal.h b/drivers/bus/mhi/core/internal.h
index 6f80ec30c0cd..5b9ea66b92dc 100644
--- a/drivers/bus/mhi/core/internal.h
+++ b/drivers/bus/mhi/core/internal.h
@@ -369,6 +369,18 @@ enum mhi_ch_state {
MHI_CH_STATE_ERROR = 0x5,
};
+enum mhi_ch_state_type {
+ MHI_CH_STATE_TYPE_RESET,
+ MHI_CH_STATE_TYPE_STOP,
+ MHI_CH_STATE_TYPE_START,
+ MHI_CH_STATE_TYPE_MAX,
+};
+
+extern const char * const mhi_ch_state_type_str[MHI_CH_STATE_TYPE_MAX];
+#define TO_CH_STATE_TYPE_STR(state) (((state) >= MHI_CH_STATE_TYPE_MAX) ? \
+ "INVALID_STATE" : \
+ mhi_ch_state_type_str[(state)])
+
#define MHI_INVALID_BRSTMODE(mode) (mode != MHI_DB_BRST_DISABLE && \
mode != MHI_DB_BRST_ENABLE)
@@ -379,13 +391,15 @@ extern const char * const mhi_ee_str[MHI_EE_MAX];
#define MHI_IN_PBL(ee) (ee == MHI_EE_PBL || ee == MHI_EE_PTHRU || \
ee == MHI_EE_EDL)
-#define MHI_IN_MISSION_MODE(ee) (ee == MHI_EE_AMSS || ee == MHI_EE_WFW)
+#define MHI_IN_MISSION_MODE(ee) (ee == MHI_EE_AMSS || ee == MHI_EE_WFW || \
+ ee == MHI_EE_FP)
enum dev_st_transition {
DEV_ST_TRANSITION_PBL,
DEV_ST_TRANSITION_READY,
DEV_ST_TRANSITION_SBL,
DEV_ST_TRANSITION_MISSION_MODE,
+ DEV_ST_TRANSITION_FP,
DEV_ST_TRANSITION_SYS_ERR,
DEV_ST_TRANSITION_DISABLE,
DEV_ST_TRANSITION_MAX,
@@ -619,6 +633,7 @@ int mhi_pm_m3_transition(struct mhi_controller *mhi_cntrl);
int __mhi_device_get_sync(struct mhi_controller *mhi_cntrl);
int mhi_send_cmd(struct mhi_controller *mhi_cntrl, struct mhi_chan *mhi_chan,
enum mhi_cmd_type cmd);
+int mhi_download_amss_image(struct mhi_controller *mhi_cntrl);
static inline bool mhi_is_active(struct mhi_controller *mhi_cntrl)
{
return (mhi_cntrl->dev_state >= MHI_STATE_M0 &&
@@ -643,6 +658,9 @@ int __must_check mhi_read_reg(struct mhi_controller *mhi_cntrl,
int __must_check mhi_read_reg_field(struct mhi_controller *mhi_cntrl,
void __iomem *base, u32 offset, u32 mask,
u32 shift, u32 *out);
+int __must_check mhi_poll_reg_field(struct mhi_controller *mhi_cntrl,
+ void __iomem *base, u32 offset, u32 mask,
+ u32 shift, u32 val, u32 delayus);
void mhi_write_reg(struct mhi_controller *mhi_cntrl, void __iomem *base,
u32 offset, u32 val);
void mhi_write_reg_field(struct mhi_controller *mhi_cntrl, void __iomem *base,
diff --git a/drivers/bus/mhi/core/main.c b/drivers/bus/mhi/core/main.c
index 4e0131b94056..22acde118bc3 100644
--- a/drivers/bus/mhi/core/main.c
+++ b/drivers/bus/mhi/core/main.c
@@ -4,6 +4,7 @@
*
*/
+#include <linux/delay.h>
#include <linux/device.h>
#include <linux/dma-direction.h>
#include <linux/dma-mapping.h>
@@ -37,6 +38,28 @@ int __must_check mhi_read_reg_field(struct mhi_controller *mhi_cntrl,
return 0;
}
+int __must_check mhi_poll_reg_field(struct mhi_controller *mhi_cntrl,
+ void __iomem *base, u32 offset,
+ u32 mask, u32 shift, u32 val, u32 delayus)
+{
+ int ret;
+ u32 out, retry = (mhi_cntrl->timeout_ms * 1000) / delayus;
+
+ while (retry--) {
+ ret = mhi_read_reg_field(mhi_cntrl, base, offset, mask, shift,
+ &out);
+ if (ret)
+ return ret;
+
+ if (out == val)
+ return 0;
+
+ fsleep(delayus);
+ }
+
+ return -ETIMEDOUT;
+}
+
void mhi_write_reg(struct mhi_controller *mhi_cntrl, void __iomem *base,
u32 offset, u32 val)
{
@@ -242,10 +265,17 @@ static void mhi_del_ring_element(struct mhi_controller *mhi_cntrl,
smp_wmb();
}
+static bool is_valid_ring_ptr(struct mhi_ring *ring, dma_addr_t addr)
+{
+ return addr >= ring->iommu_base && addr < ring->iommu_base + ring->len;
+}
+
int mhi_destroy_device(struct device *dev, void *data)
{
+ struct mhi_chan *ul_chan, *dl_chan;
struct mhi_device *mhi_dev;
struct mhi_controller *mhi_cntrl;
+ enum mhi_ee_type ee = MHI_EE_MAX;
if (dev->bus != &mhi_bus_type)
return 0;
@@ -257,6 +287,17 @@ int mhi_destroy_device(struct device *dev, void *data)
if (mhi_dev->dev_type == MHI_DEVICE_CONTROLLER)
return 0;
+ ul_chan = mhi_dev->ul_chan;
+ dl_chan = mhi_dev->dl_chan;
+
+ /*
+ * If execution environment is specified, remove only those devices that
+ * started in them based on ee_mask for the channels as we move on to a
+ * different execution environment
+ */
+ if (data)
+ ee = *(enum mhi_ee_type *)data;
+
/*
* For the suspend and resume case, this function will get called
* without mhi_unregister_controller(). Hence, we need to drop the
@@ -264,11 +305,19 @@ int mhi_destroy_device(struct device *dev, void *data)
* be sure that there will be no instances of mhi_dev left after
* this.
*/
- if (mhi_dev->ul_chan)
- put_device(&mhi_dev->ul_chan->mhi_dev->dev);
+ if (ul_chan) {
+ if (ee != MHI_EE_MAX && !(ul_chan->ee_mask & BIT(ee)))
+ return 0;
- if (mhi_dev->dl_chan)
- put_device(&mhi_dev->dl_chan->mhi_dev->dev);
+ put_device(&ul_chan->mhi_dev->dev);
+ }
+
+ if (dl_chan) {
+ if (ee != MHI_EE_MAX && !(dl_chan->ee_mask & BIT(ee)))
+ return 0;
+
+ put_device(&dl_chan->mhi_dev->dev);
+ }
dev_dbg(&mhi_cntrl->mhi_dev->dev, "destroy device for chan:%s\n",
mhi_dev->name);
@@ -383,7 +432,16 @@ irqreturn_t mhi_irq_handler(int irq_number, void *dev)
struct mhi_event_ctxt *er_ctxt =
&mhi_cntrl->mhi_ctxt->er_ctxt[mhi_event->er_index];
struct mhi_ring *ev_ring = &mhi_event->ring;
- void *dev_rp = mhi_to_virtual(ev_ring, er_ctxt->rp);
+ dma_addr_t ptr = er_ctxt->rp;
+ void *dev_rp;
+
+ if (!is_valid_ring_ptr(ev_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event ring rp points outside of the event ring\n");
+ return IRQ_HANDLED;
+ }
+
+ dev_rp = mhi_to_virtual(ev_ring, ptr);
/* Only proceed if event ring has pending events */
if (ev_ring->rp == dev_rp)
@@ -407,9 +465,9 @@ irqreturn_t mhi_intvec_threaded_handler(int irq_number, void *priv)
{
struct mhi_controller *mhi_cntrl = priv;
struct device *dev = &mhi_cntrl->mhi_dev->dev;
- enum mhi_state state = MHI_STATE_MAX;
+ enum mhi_state state;
enum mhi_pm_state pm_state = 0;
- enum mhi_ee_type ee = 0;
+ enum mhi_ee_type ee;
write_lock_irq(&mhi_cntrl->pm_lock);
if (!MHI_REG_ACCESS_VALID(mhi_cntrl->pm_state)) {
@@ -418,11 +476,11 @@ irqreturn_t mhi_intvec_threaded_handler(int irq_number, void *priv)
}
state = mhi_get_mhi_state(mhi_cntrl);
- ee = mhi_cntrl->ee;
- mhi_cntrl->ee = mhi_get_exec_env(mhi_cntrl);
- dev_dbg(dev, "local ee:%s device ee:%s dev_state:%s\n",
- TO_MHI_EXEC_STR(mhi_cntrl->ee), TO_MHI_EXEC_STR(ee),
- TO_MHI_STATE_STR(state));
+ ee = mhi_get_exec_env(mhi_cntrl);
+ dev_dbg(dev, "local ee: %s state: %s device ee: %s state: %s\n",
+ TO_MHI_EXEC_STR(mhi_cntrl->ee),
+ TO_MHI_STATE_STR(mhi_cntrl->dev_state),
+ TO_MHI_EXEC_STR(ee), TO_MHI_STATE_STR(state));
if (state == MHI_STATE_SYS_ERR) {
dev_dbg(dev, "System error detected\n");
@@ -431,27 +489,30 @@ irqreturn_t mhi_intvec_threaded_handler(int irq_number, void *priv)
}
write_unlock_irq(&mhi_cntrl->pm_lock);
- /* If device supports RDDM don't bother processing SYS error */
- if (mhi_cntrl->rddm_image) {
- /* host may be performing a device power down already */
- if (!mhi_is_active(mhi_cntrl))
- goto exit_intvec;
+ if (pm_state != MHI_PM_SYS_ERR_DETECT || ee == mhi_cntrl->ee)
+ goto exit_intvec;
- if (mhi_cntrl->ee == MHI_EE_RDDM && mhi_cntrl->ee != ee) {
+ switch (ee) {
+ case MHI_EE_RDDM:
+ /* proceed if power down is not already in progress */
+ if (mhi_cntrl->rddm_image && mhi_is_active(mhi_cntrl)) {
mhi_cntrl->status_cb(mhi_cntrl, MHI_CB_EE_RDDM);
+ mhi_cntrl->ee = ee;
wake_up_all(&mhi_cntrl->state_event);
}
- goto exit_intvec;
- }
-
- if (pm_state == MHI_PM_SYS_ERR_DETECT) {
+ break;
+ case MHI_EE_PBL:
+ case MHI_EE_EDL:
+ case MHI_EE_PTHRU:
+ mhi_cntrl->status_cb(mhi_cntrl, MHI_CB_FATAL_ERROR);
+ mhi_cntrl->ee = ee;
wake_up_all(&mhi_cntrl->state_event);
-
- /* For fatal errors, we let controller decide next step */
- if (MHI_IN_PBL(ee))
- mhi_cntrl->status_cb(mhi_cntrl, MHI_CB_FATAL_ERROR);
- else
- mhi_pm_sys_err_handler(mhi_cntrl);
+ mhi_pm_sys_err_handler(mhi_cntrl);
+ break;
+ default:
+ wake_up_all(&mhi_cntrl->state_event);
+ mhi_pm_sys_err_handler(mhi_cntrl);
+ break;
}
exit_intvec:
@@ -536,6 +597,11 @@ static int parse_xfer_event(struct mhi_controller *mhi_cntrl,
struct mhi_buf_info *buf_info;
u16 xfer_len;
+ if (!is_valid_ring_ptr(tre_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event element points outside of the tre ring\n");
+ break;
+ }
/* Get the TRB this event points to */
ev_tre = mhi_to_virtual(tre_ring, ptr);
@@ -570,8 +636,11 @@ static int parse_xfer_event(struct mhi_controller *mhi_cntrl,
/* notify client */
mhi_chan->xfer_cb(mhi_chan->mhi_dev, &result);
- if (mhi_chan->dir == DMA_TO_DEVICE)
+ if (mhi_chan->dir == DMA_TO_DEVICE) {
atomic_dec(&mhi_cntrl->pending_pkts);
+ /* Release the reference got from mhi_queue() */
+ mhi_cntrl->runtime_put(mhi_cntrl);
+ }
/*
* Recycle the buffer if buffer is pre-allocated,
@@ -595,15 +664,15 @@ static int parse_xfer_event(struct mhi_controller *mhi_cntrl,
case MHI_EV_CC_OOB:
case MHI_EV_CC_DB_MODE:
{
- unsigned long flags;
+ unsigned long pm_lock_flags;
mhi_chan->db_cfg.db_mode = 1;
- read_lock_irqsave(&mhi_cntrl->pm_lock, flags);
+ read_lock_irqsave(&mhi_cntrl->pm_lock, pm_lock_flags);
if (tre_ring->wp != tre_ring->rp &&
MHI_DB_ACCESS_VALID(mhi_cntrl)) {
mhi_ring_chan_db(mhi_cntrl, mhi_chan);
}
- read_unlock_irqrestore(&mhi_cntrl->pm_lock, flags);
+ read_unlock_irqrestore(&mhi_cntrl->pm_lock, pm_lock_flags);
break;
}
case MHI_EV_CC_BAD_TRE:
@@ -695,6 +764,12 @@ static void mhi_process_cmd_completion(struct mhi_controller *mhi_cntrl,
struct mhi_chan *mhi_chan;
u32 chan;
+ if (!is_valid_ring_ptr(mhi_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event element points outside of the cmd ring\n");
+ return;
+ }
+
cmd_pkt = mhi_to_virtual(mhi_ring, ptr);
chan = MHI_TRE_GET_CMD_CHID(cmd_pkt);
@@ -719,6 +794,7 @@ int mhi_process_ctrl_ev_ring(struct mhi_controller *mhi_cntrl,
struct device *dev = &mhi_cntrl->mhi_dev->dev;
u32 chan;
int count = 0;
+ dma_addr_t ptr = er_ctxt->rp;
/*
* This is a quick check to avoid unnecessary event processing
@@ -728,7 +804,13 @@ int mhi_process_ctrl_ev_ring(struct mhi_controller *mhi_cntrl,
if (unlikely(MHI_EVENT_ACCESS_INVALID(mhi_cntrl->pm_state)))
return -EIO;
- dev_rp = mhi_to_virtual(ev_ring, er_ctxt->rp);
+ if (!is_valid_ring_ptr(ev_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event ring rp points outside of the event ring\n");
+ return -EIO;
+ }
+
+ dev_rp = mhi_to_virtual(ev_ring, ptr);
local_rp = ev_ring->rp;
while (dev_rp != local_rp) {
@@ -771,14 +853,14 @@ int mhi_process_ctrl_ev_ring(struct mhi_controller *mhi_cntrl,
break;
case MHI_STATE_SYS_ERR:
{
- enum mhi_pm_state new_state;
+ enum mhi_pm_state pm_state;
dev_dbg(dev, "System error detected\n");
write_lock_irq(&mhi_cntrl->pm_lock);
- new_state = mhi_tryset_pm_state(mhi_cntrl,
+ pm_state = mhi_tryset_pm_state(mhi_cntrl,
MHI_PM_SYS_ERR_DETECT);
write_unlock_irq(&mhi_cntrl->pm_lock);
- if (new_state == MHI_PM_SYS_ERR_DETECT)
+ if (pm_state == MHI_PM_SYS_ERR_DETECT)
mhi_pm_sys_err_handler(mhi_cntrl);
break;
}
@@ -807,6 +889,9 @@ int mhi_process_ctrl_ev_ring(struct mhi_controller *mhi_cntrl,
case MHI_EE_AMSS:
st = DEV_ST_TRANSITION_MISSION_MODE;
break;
+ case MHI_EE_FP:
+ st = DEV_ST_TRANSITION_FP;
+ break;
case MHI_EE_RDDM:
mhi_cntrl->status_cb(mhi_cntrl, MHI_CB_EE_RDDM);
write_lock_irq(&mhi_cntrl->pm_lock);
@@ -834,6 +919,8 @@ int mhi_process_ctrl_ev_ring(struct mhi_controller *mhi_cntrl,
*/
if (chan < mhi_cntrl->max_chan) {
mhi_chan = &mhi_cntrl->mhi_chan[chan];
+ if (!mhi_chan->configured)
+ break;
parse_xfer_event(mhi_cntrl, local_rp, mhi_chan);
event_quota--;
}
@@ -845,7 +932,15 @@ int mhi_process_ctrl_ev_ring(struct mhi_controller *mhi_cntrl,
mhi_recycle_ev_ring_element(mhi_cntrl, ev_ring);
local_rp = ev_ring->rp;
- dev_rp = mhi_to_virtual(ev_ring, er_ctxt->rp);
+
+ ptr = er_ctxt->rp;
+ if (!is_valid_ring_ptr(ev_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event ring rp points outside of the event ring\n");
+ return -EIO;
+ }
+
+ dev_rp = mhi_to_virtual(ev_ring, ptr);
count++;
}
@@ -868,11 +963,18 @@ int mhi_process_data_event_ring(struct mhi_controller *mhi_cntrl,
int count = 0;
u32 chan;
struct mhi_chan *mhi_chan;
+ dma_addr_t ptr = er_ctxt->rp;
if (unlikely(MHI_EVENT_ACCESS_INVALID(mhi_cntrl->pm_state)))
return -EIO;
- dev_rp = mhi_to_virtual(ev_ring, er_ctxt->rp);
+ if (!is_valid_ring_ptr(ev_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event ring rp points outside of the event ring\n");
+ return -EIO;
+ }
+
+ dev_rp = mhi_to_virtual(ev_ring, ptr);
local_rp = ev_ring->rp;
while (dev_rp != local_rp && event_quota > 0) {
@@ -886,7 +988,8 @@ int mhi_process_data_event_ring(struct mhi_controller *mhi_cntrl,
* Only process the event ring elements whose channel
* ID is within the maximum supported range.
*/
- if (chan < mhi_cntrl->max_chan) {
+ if (chan < mhi_cntrl->max_chan &&
+ mhi_cntrl->mhi_chan[chan].configured) {
mhi_chan = &mhi_cntrl->mhi_chan[chan];
if (likely(type == MHI_PKT_TYPE_TX_EVENT)) {
@@ -900,7 +1003,15 @@ int mhi_process_data_event_ring(struct mhi_controller *mhi_cntrl,
mhi_recycle_ev_ring_element(mhi_cntrl, ev_ring);
local_rp = ev_ring->rp;
- dev_rp = mhi_to_virtual(ev_ring, er_ctxt->rp);
+
+ ptr = er_ctxt->rp;
+ if (!is_valid_ring_ptr(ev_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event ring rp points outside of the event ring\n");
+ return -EIO;
+ }
+
+ dev_rp = mhi_to_virtual(ev_ring, ptr);
count++;
}
read_lock_bh(&mhi_cntrl->pm_lock);
@@ -996,7 +1107,7 @@ static int mhi_queue(struct mhi_device *mhi_dev, struct mhi_buf_info *buf_info,
ret = mhi_is_ring_full(mhi_cntrl, tre_ring);
if (unlikely(ret)) {
- ret = -ENOMEM;
+ ret = -EAGAIN;
goto exit_unlock;
}
@@ -1004,9 +1115,11 @@ static int mhi_queue(struct mhi_device *mhi_dev, struct mhi_buf_info *buf_info,
if (unlikely(ret))
goto exit_unlock;
- /* trigger M3 exit if necessary */
- if (MHI_PM_IN_SUSPEND_STATE(mhi_cntrl->pm_state))
- mhi_trigger_resume(mhi_cntrl);
+ /* Packet is queued, take a usage ref to exit M3 if necessary
+ * for host->device buffer, balanced put is done on buffer completion
+ * for device->host buffer, balanced put is after ringing the DB
+ */
+ mhi_cntrl->runtime_get(mhi_cntrl);
/* Assert dev_wake (to exit/prevent M1/M2)*/
mhi_cntrl->wake_toggle(mhi_cntrl);
@@ -1014,12 +1127,11 @@ static int mhi_queue(struct mhi_device *mhi_dev, struct mhi_buf_info *buf_info,
if (mhi_chan->dir == DMA_TO_DEVICE)
atomic_inc(&mhi_cntrl->pending_pkts);
- if (unlikely(!MHI_DB_ACCESS_VALID(mhi_cntrl))) {
- ret = -EIO;
- goto exit_unlock;
- }
+ if (likely(MHI_DB_ACCESS_VALID(mhi_cntrl)))
+ mhi_ring_chan_db(mhi_cntrl, mhi_chan);
- mhi_ring_chan_db(mhi_cntrl, mhi_chan);
+ if (dir == DMA_FROM_DEVICE)
+ mhi_cntrl->runtime_put(mhi_cntrl);
exit_unlock:
read_unlock_irqrestore(&mhi_cntrl->pm_lock, flags);
@@ -1162,6 +1274,11 @@ int mhi_send_cmd(struct mhi_controller *mhi_cntrl,
cmd_tre->dword[0] = MHI_TRE_CMD_RESET_DWORD0;
cmd_tre->dword[1] = MHI_TRE_CMD_RESET_DWORD1(chan);
break;
+ case MHI_CMD_STOP_CHAN:
+ cmd_tre->ptr = MHI_TRE_CMD_STOP_PTR;
+ cmd_tre->dword[0] = MHI_TRE_CMD_STOP_DWORD0;
+ cmd_tre->dword[1] = MHI_TRE_CMD_STOP_DWORD1(chan);
+ break;
case MHI_CMD_START_CHAN:
cmd_tre->ptr = MHI_TRE_CMD_START_PTR;
cmd_tre->dword[0] = MHI_TRE_CMD_START_DWORD0;
@@ -1183,56 +1300,125 @@ int mhi_send_cmd(struct mhi_controller *mhi_cntrl,
return 0;
}
-static void __mhi_unprepare_channel(struct mhi_controller *mhi_cntrl,
- struct mhi_chan *mhi_chan)
+static int mhi_update_channel_state(struct mhi_controller *mhi_cntrl,
+ struct mhi_chan *mhi_chan,
+ enum mhi_ch_state_type to_state)
{
+ struct device *dev = &mhi_chan->mhi_dev->dev;
+ enum mhi_cmd_type cmd = MHI_CMD_NOP;
int ret;
- struct device *dev = &mhi_cntrl->mhi_dev->dev;
-
- dev_dbg(dev, "Entered: unprepare channel:%d\n", mhi_chan->chan);
- /* no more processing events for this channel */
- mutex_lock(&mhi_chan->mutex);
- write_lock_irq(&mhi_chan->lock);
- if (mhi_chan->ch_state != MHI_CH_STATE_ENABLED &&
- mhi_chan->ch_state != MHI_CH_STATE_SUSPENDED) {
+ dev_dbg(dev, "%d: Updating channel state to: %s\n", mhi_chan->chan,
+ TO_CH_STATE_TYPE_STR(to_state));
+
+ switch (to_state) {
+ case MHI_CH_STATE_TYPE_RESET:
+ write_lock_irq(&mhi_chan->lock);
+ if (mhi_chan->ch_state != MHI_CH_STATE_STOP &&
+ mhi_chan->ch_state != MHI_CH_STATE_ENABLED &&
+ mhi_chan->ch_state != MHI_CH_STATE_SUSPENDED) {
+ write_unlock_irq(&mhi_chan->lock);
+ return -EINVAL;
+ }
+ mhi_chan->ch_state = MHI_CH_STATE_DISABLED;
write_unlock_irq(&mhi_chan->lock);
- mutex_unlock(&mhi_chan->mutex);
- return;
+
+ cmd = MHI_CMD_RESET_CHAN;
+ break;
+ case MHI_CH_STATE_TYPE_STOP:
+ if (mhi_chan->ch_state != MHI_CH_STATE_ENABLED)
+ return -EINVAL;
+
+ cmd = MHI_CMD_STOP_CHAN;
+ break;
+ case MHI_CH_STATE_TYPE_START:
+ if (mhi_chan->ch_state != MHI_CH_STATE_STOP &&
+ mhi_chan->ch_state != MHI_CH_STATE_DISABLED)
+ return -EINVAL;
+
+ cmd = MHI_CMD_START_CHAN;
+ break;
+ default:
+ dev_err(dev, "%d: Channel state update to %s not allowed\n",
+ mhi_chan->chan, TO_CH_STATE_TYPE_STR(to_state));
+ return -EINVAL;
}
- mhi_chan->ch_state = MHI_CH_STATE_DISABLED;
- write_unlock_irq(&mhi_chan->lock);
+ /* bring host and device out of suspended states */
+ ret = mhi_device_get_sync(mhi_cntrl->mhi_dev);
+ if (ret)
+ return ret;
+ mhi_cntrl->runtime_get(mhi_cntrl);
reinit_completion(&mhi_chan->completion);
- read_lock_bh(&mhi_cntrl->pm_lock);
- if (MHI_PM_IN_ERROR_STATE(mhi_cntrl->pm_state)) {
- read_unlock_bh(&mhi_cntrl->pm_lock);
- goto error_invalid_state;
+ ret = mhi_send_cmd(mhi_cntrl, mhi_chan, cmd);
+ if (ret) {
+ dev_err(dev, "%d: Failed to send %s channel command\n",
+ mhi_chan->chan, TO_CH_STATE_TYPE_STR(to_state));
+ goto exit_channel_update;
}
- mhi_cntrl->wake_toggle(mhi_cntrl);
- read_unlock_bh(&mhi_cntrl->pm_lock);
+ ret = wait_for_completion_timeout(&mhi_chan->completion,
+ msecs_to_jiffies(mhi_cntrl->timeout_ms));
+ if (!ret || mhi_chan->ccs != MHI_EV_CC_SUCCESS) {
+ dev_err(dev,
+ "%d: Failed to receive %s channel command completion\n",
+ mhi_chan->chan, TO_CH_STATE_TYPE_STR(to_state));
+ ret = -EIO;
+ goto exit_channel_update;
+ }
- mhi_cntrl->runtime_get(mhi_cntrl);
+ ret = 0;
+
+ if (to_state != MHI_CH_STATE_TYPE_RESET) {
+ write_lock_irq(&mhi_chan->lock);
+ mhi_chan->ch_state = (to_state == MHI_CH_STATE_TYPE_START) ?
+ MHI_CH_STATE_ENABLED : MHI_CH_STATE_STOP;
+ write_unlock_irq(&mhi_chan->lock);
+ }
+
+ dev_dbg(dev, "%d: Channel state change to %s successful\n",
+ mhi_chan->chan, TO_CH_STATE_TYPE_STR(to_state));
+
+exit_channel_update:
mhi_cntrl->runtime_put(mhi_cntrl);
- ret = mhi_send_cmd(mhi_cntrl, mhi_chan, MHI_CMD_RESET_CHAN);
+ mhi_device_put(mhi_cntrl->mhi_dev);
+
+ return ret;
+}
+
+static void mhi_unprepare_channel(struct mhi_controller *mhi_cntrl,
+ struct mhi_chan *mhi_chan)
+{
+ int ret;
+ struct device *dev = &mhi_chan->mhi_dev->dev;
+
+ mutex_lock(&mhi_chan->mutex);
+
+ if (!(BIT(mhi_cntrl->ee) & mhi_chan->ee_mask)) {
+ dev_dbg(dev, "Current EE: %s Required EE Mask: 0x%x\n",
+ TO_MHI_EXEC_STR(mhi_cntrl->ee), mhi_chan->ee_mask);
+ goto exit_unprepare_channel;
+ }
+
+ /* no more processing events for this channel */
+ ret = mhi_update_channel_state(mhi_cntrl, mhi_chan,
+ MHI_CH_STATE_TYPE_RESET);
if (ret)
- goto error_invalid_state;
+ dev_err(dev, "%d: Failed to reset channel, still resetting\n",
+ mhi_chan->chan);
- /* even if it fails we will still reset */
- ret = wait_for_completion_timeout(&mhi_chan->completion,
- msecs_to_jiffies(mhi_cntrl->timeout_ms));
- if (!ret || mhi_chan->ccs != MHI_EV_CC_SUCCESS)
- dev_err(dev,
- "Failed to receive cmd completion, still resetting\n");
+exit_unprepare_channel:
+ write_lock_irq(&mhi_chan->lock);
+ mhi_chan->ch_state = MHI_CH_STATE_DISABLED;
+ write_unlock_irq(&mhi_chan->lock);
-error_invalid_state:
if (!mhi_chan->offload_ch) {
mhi_reset_chan(mhi_cntrl, mhi_chan);
mhi_deinit_chan_ctxt(mhi_cntrl, mhi_chan);
}
- dev_dbg(dev, "chan:%d successfully resetted\n", mhi_chan->chan);
+ dev_dbg(dev, "%d: successfully reset\n", mhi_chan->chan);
+
mutex_unlock(&mhi_chan->mutex);
}
@@ -1240,28 +1426,16 @@ int mhi_prepare_channel(struct mhi_controller *mhi_cntrl,
struct mhi_chan *mhi_chan)
{
int ret = 0;
- struct device *dev = &mhi_cntrl->mhi_dev->dev;
-
- dev_dbg(dev, "Preparing channel: %d\n", mhi_chan->chan);
+ struct device *dev = &mhi_chan->mhi_dev->dev;
if (!(BIT(mhi_cntrl->ee) & mhi_chan->ee_mask)) {
- dev_err(dev,
- "Current EE: %s Required EE Mask: 0x%x for chan: %s\n",
- TO_MHI_EXEC_STR(mhi_cntrl->ee), mhi_chan->ee_mask,
- mhi_chan->name);
+ dev_err(dev, "Current EE: %s Required EE Mask: 0x%x\n",
+ TO_MHI_EXEC_STR(mhi_cntrl->ee), mhi_chan->ee_mask);
return -ENOTCONN;
}
mutex_lock(&mhi_chan->mutex);
- /* If channel is not in disable state, do not allow it to start */
- if (mhi_chan->ch_state != MHI_CH_STATE_DISABLED) {
- ret = -EIO;
- dev_dbg(dev, "channel: %d is not in disabled state\n",
- mhi_chan->chan);
- goto error_init_chan;
- }
-
/* Check of client manages channel context for offload channels */
if (!mhi_chan->offload_ch) {
ret = mhi_init_chan_ctxt(mhi_cntrl, mhi_chan);
@@ -1269,34 +1443,11 @@ int mhi_prepare_channel(struct mhi_controller *mhi_cntrl,
goto error_init_chan;
}
- reinit_completion(&mhi_chan->completion);
- read_lock_bh(&mhi_cntrl->pm_lock);
- if (MHI_PM_IN_ERROR_STATE(mhi_cntrl->pm_state)) {
- read_unlock_bh(&mhi_cntrl->pm_lock);
- ret = -EIO;
- goto error_pm_state;
- }
-
- mhi_cntrl->wake_toggle(mhi_cntrl);
- read_unlock_bh(&mhi_cntrl->pm_lock);
- mhi_cntrl->runtime_get(mhi_cntrl);
- mhi_cntrl->runtime_put(mhi_cntrl);
-
- ret = mhi_send_cmd(mhi_cntrl, mhi_chan, MHI_CMD_START_CHAN);
+ ret = mhi_update_channel_state(mhi_cntrl, mhi_chan,
+ MHI_CH_STATE_TYPE_START);
if (ret)
goto error_pm_state;
- ret = wait_for_completion_timeout(&mhi_chan->completion,
- msecs_to_jiffies(mhi_cntrl->timeout_ms));
- if (!ret || mhi_chan->ccs != MHI_EV_CC_SUCCESS) {
- ret = -EIO;
- goto error_pm_state;
- }
-
- write_lock_irq(&mhi_chan->lock);
- mhi_chan->ch_state = MHI_CH_STATE_ENABLED;
- write_unlock_irq(&mhi_chan->lock);
-
/* Pre-allocate buffer for xfer ring */
if (mhi_chan->pre_alloc) {
int nr_el = get_nr_avail_ring_elements(mhi_cntrl,
@@ -1334,9 +1485,6 @@ int mhi_prepare_channel(struct mhi_controller *mhi_cntrl,
mutex_unlock(&mhi_chan->mutex);
- dev_dbg(dev, "Chan: %d successfully moved to start state\n",
- mhi_chan->chan);
-
return 0;
error_pm_state:
@@ -1350,7 +1498,7 @@ error_init_chan:
error_pre_alloc:
mutex_unlock(&mhi_chan->mutex);
- __mhi_unprepare_channel(mhi_cntrl, mhi_chan);
+ mhi_unprepare_channel(mhi_cntrl, mhi_chan);
return ret;
}
@@ -1365,6 +1513,7 @@ static void mhi_mark_stale_events(struct mhi_controller *mhi_cntrl,
struct mhi_ring *ev_ring;
struct device *dev = &mhi_cntrl->mhi_dev->dev;
unsigned long flags;
+ dma_addr_t ptr;
dev_dbg(dev, "Marking all events for chan: %d as stale\n", chan);
@@ -1372,7 +1521,15 @@ static void mhi_mark_stale_events(struct mhi_controller *mhi_cntrl,
/* mark all stale events related to channel as STALE event */
spin_lock_irqsave(&mhi_event->lock, flags);
- dev_rp = mhi_to_virtual(ev_ring, er_ctxt->rp);
+
+ ptr = er_ctxt->rp;
+ if (!is_valid_ring_ptr(ev_ring, ptr)) {
+ dev_err(&mhi_cntrl->mhi_dev->dev,
+ "Event ring rp points outside of the event ring\n");
+ dev_rp = ev_ring->rp;
+ } else {
+ dev_rp = mhi_to_virtual(ev_ring, ptr);
+ }
local_rp = ev_ring->rp;
while (dev_rp != local_rp) {
@@ -1403,8 +1560,11 @@ static void mhi_reset_data_chan(struct mhi_controller *mhi_cntrl,
while (tre_ring->rp != tre_ring->wp) {
struct mhi_buf_info *buf_info = buf_ring->rp;
- if (mhi_chan->dir == DMA_TO_DEVICE)
+ if (mhi_chan->dir == DMA_TO_DEVICE) {
atomic_dec(&mhi_cntrl->pending_pkts);
+ /* Release the reference got from mhi_queue() */
+ mhi_cntrl->runtime_put(mhi_cntrl);
+ }
if (!buf_info->pre_mapped)
mhi_cntrl->unmap_single(mhi_cntrl, buf_info);
@@ -1467,7 +1627,7 @@ error_open_chan:
if (!mhi_chan)
continue;
- __mhi_unprepare_channel(mhi_cntrl, mhi_chan);
+ mhi_unprepare_channel(mhi_cntrl, mhi_chan);
}
return ret;
@@ -1485,7 +1645,7 @@ void mhi_unprepare_from_transfer(struct mhi_device *mhi_dev)
if (!mhi_chan)
continue;
- __mhi_unprepare_channel(mhi_cntrl, mhi_chan);
+ mhi_unprepare_channel(mhi_cntrl, mhi_chan);
}
}
EXPORT_SYMBOL_GPL(mhi_unprepare_from_transfer);
diff --git a/drivers/bus/mhi/core/pm.c b/drivers/bus/mhi/core/pm.c
index 681960c72d2a..e2e59a341fef 100644
--- a/drivers/bus/mhi/core/pm.c
+++ b/drivers/bus/mhi/core/pm.c
@@ -153,35 +153,33 @@ static void mhi_toggle_dev_wake(struct mhi_controller *mhi_cntrl)
/* Handle device ready state transition */
int mhi_ready_state_transition(struct mhi_controller *mhi_cntrl)
{
- void __iomem *base = mhi_cntrl->regs;
struct mhi_event *mhi_event;
enum mhi_pm_state cur_state;
struct device *dev = &mhi_cntrl->mhi_dev->dev;
- u32 reset = 1, ready = 0;
+ u32 interval_us = 25000; /* poll register field every 25 milliseconds */
int ret, i;
- /* Wait for RESET to be cleared and READY bit to be set by the device */
- wait_event_timeout(mhi_cntrl->state_event,
- MHI_PM_IN_FATAL_STATE(mhi_cntrl->pm_state) ||
- mhi_read_reg_field(mhi_cntrl, base, MHICTRL,
- MHICTRL_RESET_MASK,
- MHICTRL_RESET_SHIFT, &reset) ||
- mhi_read_reg_field(mhi_cntrl, base, MHISTATUS,
- MHISTATUS_READY_MASK,
- MHISTATUS_READY_SHIFT, &ready) ||
- (!reset && ready),
- msecs_to_jiffies(mhi_cntrl->timeout_ms));
-
/* Check if device entered error state */
if (MHI_PM_IN_FATAL_STATE(mhi_cntrl->pm_state)) {
dev_err(dev, "Device link is not accessible\n");
return -EIO;
}
- /* Timeout if device did not transition to ready state */
- if (reset || !ready) {
- dev_err(dev, "Device Ready timeout\n");
- return -ETIMEDOUT;
+ /* Wait for RESET to be cleared and READY bit to be set by the device */
+ ret = mhi_poll_reg_field(mhi_cntrl, mhi_cntrl->regs, MHICTRL,
+ MHICTRL_RESET_MASK, MHICTRL_RESET_SHIFT, 0,
+ interval_us);
+ if (ret) {
+ dev_err(dev, "Device failed to clear MHI Reset\n");
+ return ret;
+ }
+
+ ret = mhi_poll_reg_field(mhi_cntrl, mhi_cntrl->regs, MHISTATUS,
+ MHISTATUS_READY_MASK, MHISTATUS_READY_SHIFT, 1,
+ interval_us);
+ if (ret) {
+ dev_err(dev, "Device failed to enter MHI Ready\n");
+ return ret;
}
dev_dbg(dev, "Device in READY State\n");
@@ -377,24 +375,28 @@ static int mhi_pm_mission_mode_transition(struct mhi_controller *mhi_cntrl)
{
struct mhi_event *mhi_event;
struct device *dev = &mhi_cntrl->mhi_dev->dev;
+ enum mhi_ee_type ee = MHI_EE_MAX, current_ee = mhi_cntrl->ee;
int i, ret;
dev_dbg(dev, "Processing Mission Mode transition\n");
write_lock_irq(&mhi_cntrl->pm_lock);
if (MHI_REG_ACCESS_VALID(mhi_cntrl->pm_state))
- mhi_cntrl->ee = mhi_get_exec_env(mhi_cntrl);
+ ee = mhi_get_exec_env(mhi_cntrl);
- if (!MHI_IN_MISSION_MODE(mhi_cntrl->ee)) {
+ if (!MHI_IN_MISSION_MODE(ee)) {
mhi_cntrl->pm_state = MHI_PM_LD_ERR_FATAL_DETECT;
write_unlock_irq(&mhi_cntrl->pm_lock);
wake_up_all(&mhi_cntrl->state_event);
return -EIO;
}
+ mhi_cntrl->ee = ee;
write_unlock_irq(&mhi_cntrl->pm_lock);
wake_up_all(&mhi_cntrl->state_event);
+ device_for_each_child(&mhi_cntrl->mhi_dev->dev, &current_ee,
+ mhi_destroy_device);
mhi_cntrl->status_cb(mhi_cntrl, MHI_CB_EE_MISSION_MODE);
/* Force MHI to be in M0 state before continuing */
@@ -560,6 +562,7 @@ static void mhi_pm_disable_transition(struct mhi_controller *mhi_cntrl)
static void mhi_pm_sys_error_transition(struct mhi_controller *mhi_cntrl)
{
enum mhi_pm_state cur_state, prev_state;
+ enum dev_st_transition next_state;
struct mhi_event *mhi_event;
struct mhi_cmd_ctxt *cmd_ctxt;
struct mhi_cmd *mhi_cmd;
@@ -673,7 +676,23 @@ static void mhi_pm_sys_error_transition(struct mhi_controller *mhi_cntrl)
er_ctxt->wp = er_ctxt->rbase;
}
- mhi_ready_state_transition(mhi_cntrl);
+ /* Transition to next state */
+ if (MHI_IN_PBL(mhi_get_exec_env(mhi_cntrl))) {
+ write_lock_irq(&mhi_cntrl->pm_lock);
+ cur_state = mhi_tryset_pm_state(mhi_cntrl, MHI_PM_POR);
+ write_unlock_irq(&mhi_cntrl->pm_lock);
+ if (cur_state != MHI_PM_POR) {
+ dev_err(dev, "Error moving to state %s from %s\n",
+ to_mhi_pm_state_str(MHI_PM_POR),
+ to_mhi_pm_state_str(cur_state));
+ goto exit_sys_error_transition;
+ }
+ next_state = DEV_ST_TRANSITION_PBL;
+ } else {
+ next_state = DEV_ST_TRANSITION_READY;
+ }
+
+ mhi_queue_state_transition(mhi_cntrl, next_state);
exit_sys_error_transition:
dev_dbg(dev, "Exiting with PM state: %s, MHI state: %s\n",
@@ -742,8 +761,7 @@ void mhi_pm_st_worker(struct work_struct *work)
if (MHI_REG_ACCESS_VALID(mhi_cntrl->pm_state))
mhi_cntrl->ee = mhi_get_exec_env(mhi_cntrl);
write_unlock_irq(&mhi_cntrl->pm_lock);
- if (MHI_IN_PBL(mhi_cntrl->ee))
- mhi_fw_load_handler(mhi_cntrl);
+ mhi_fw_load_handler(mhi_cntrl);
break;
case DEV_ST_TRANSITION_SBL:
write_lock_irq(&mhi_cntrl->pm_lock);
@@ -755,10 +773,18 @@ void mhi_pm_st_worker(struct work_struct *work)
* either SBL or AMSS states
*/
mhi_create_devices(mhi_cntrl);
+ if (mhi_cntrl->fbc_download)
+ mhi_download_amss_image(mhi_cntrl);
break;
case DEV_ST_TRANSITION_MISSION_MODE:
mhi_pm_mission_mode_transition(mhi_cntrl);
break;
+ case DEV_ST_TRANSITION_FP:
+ write_lock_irq(&mhi_cntrl->pm_lock);
+ mhi_cntrl->ee = MHI_EE_FP;
+ write_unlock_irq(&mhi_cntrl->pm_lock);
+ mhi_create_devices(mhi_cntrl);
+ break;
case DEV_ST_TRANSITION_READY:
mhi_ready_state_transition(mhi_cntrl);
break;
@@ -822,7 +848,7 @@ int mhi_pm_suspend(struct mhi_controller *mhi_cntrl)
return -EBUSY;
}
- dev_info(dev, "Allowing M3 transition\n");
+ dev_dbg(dev, "Allowing M3 transition\n");
new_state = mhi_tryset_pm_state(mhi_cntrl, MHI_PM_M3_ENTER);
if (new_state != MHI_PM_M3_ENTER) {
write_unlock_irq(&mhi_cntrl->pm_lock);
@@ -836,7 +862,7 @@ int mhi_pm_suspend(struct mhi_controller *mhi_cntrl)
/* Set MHI to M3 and wait for completion */
mhi_set_mhi_state(mhi_cntrl, MHI_STATE_M3);
write_unlock_irq(&mhi_cntrl->pm_lock);
- dev_info(dev, "Wait for M3 completion\n");
+ dev_dbg(dev, "Waiting for M3 completion\n");
ret = wait_event_timeout(mhi_cntrl->state_event,
mhi_cntrl->dev_state == MHI_STATE_M3 ||
@@ -870,9 +896,9 @@ int mhi_pm_resume(struct mhi_controller *mhi_cntrl)
enum mhi_pm_state cur_state;
int ret;
- dev_info(dev, "Entered with PM state: %s, MHI state: %s\n",
- to_mhi_pm_state_str(mhi_cntrl->pm_state),
- TO_MHI_STATE_STR(mhi_cntrl->dev_state));
+ dev_dbg(dev, "Entered with PM state: %s, MHI state: %s\n",
+ to_mhi_pm_state_str(mhi_cntrl->pm_state),
+ TO_MHI_STATE_STR(mhi_cntrl->dev_state));
if (mhi_cntrl->pm_state == MHI_PM_DISABLE)
return 0;
@@ -880,6 +906,9 @@ int mhi_pm_resume(struct mhi_controller *mhi_cntrl)
if (MHI_PM_IN_ERROR_STATE(mhi_cntrl->pm_state))
return -EIO;
+ if (mhi_get_mhi_state(mhi_cntrl) != MHI_STATE_M3)
+ return -EINVAL;
+
/* Notify clients about exiting LPM */
list_for_each_entry_safe(itr, tmp, &mhi_cntrl->lpm_chans, node) {
mutex_lock(&itr->mutex);
@@ -1033,13 +1062,6 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl)
mutex_lock(&mhi_cntrl->pm_mutex);
mhi_cntrl->pm_state = MHI_PM_DISABLE;
- if (!mhi_cntrl->pre_init) {
- /* Setup device context */
- ret = mhi_init_dev_ctxt(mhi_cntrl);
- if (ret)
- goto error_dev_ctxt;
- }
-
ret = mhi_init_irq_setup(mhi_cntrl);
if (ret)
goto error_setup_irq;
@@ -1092,7 +1114,7 @@ int mhi_async_power_up(struct mhi_controller *mhi_cntrl)
&val) ||
!val,
msecs_to_jiffies(mhi_cntrl->timeout_ms));
- if (ret) {
+ if (!ret) {
ret = -EIO;
dev_info(dev, "Failed to reset MHI due to syserr state\n");
goto error_bhi_offset;
@@ -1121,10 +1143,7 @@ error_bhi_offset:
mhi_deinit_free_irq(mhi_cntrl);
error_setup_irq:
- if (!mhi_cntrl->pre_init)
- mhi_deinit_dev_ctxt(mhi_cntrl);
-
-error_dev_ctxt:
+ mhi_cntrl->pm_state = MHI_PM_DISABLE;
mutex_unlock(&mhi_cntrl->pm_mutex);
return ret;
@@ -1136,12 +1155,19 @@ void mhi_power_down(struct mhi_controller *mhi_cntrl, bool graceful)
enum mhi_pm_state cur_state, transition_state;
struct device *dev = &mhi_cntrl->mhi_dev->dev;
+ mutex_lock(&mhi_cntrl->pm_mutex);
+ write_lock_irq(&mhi_cntrl->pm_lock);
+ cur_state = mhi_cntrl->pm_state;
+ if (cur_state == MHI_PM_DISABLE) {
+ write_unlock_irq(&mhi_cntrl->pm_lock);
+ mutex_unlock(&mhi_cntrl->pm_mutex);
+ return; /* Already powered down */
+ }
+
/* If it's not a graceful shutdown, force MHI to linkdown state */
transition_state = (graceful) ? MHI_PM_SHUTDOWN_PROCESS :
MHI_PM_LD_ERR_FATAL_DETECT;
- mutex_lock(&mhi_cntrl->pm_mutex);
- write_lock_irq(&mhi_cntrl->pm_lock);
cur_state = mhi_tryset_pm_state(mhi_cntrl, transition_state);
if (cur_state != transition_state) {
dev_err(dev, "Failed to move to state: %s from: %s\n",
@@ -1166,15 +1192,6 @@ void mhi_power_down(struct mhi_controller *mhi_cntrl, bool graceful)
flush_work(&mhi_cntrl->st_worker);
free_irq(mhi_cntrl->irq[0], mhi_cntrl);
-
- if (!mhi_cntrl->pre_init) {
- /* Free all allocated resources */
- if (mhi_cntrl->fbc_image) {
- mhi_free_bhie_table(mhi_cntrl, mhi_cntrl->fbc_image);
- mhi_cntrl->fbc_image = NULL;
- }
- mhi_deinit_dev_ctxt(mhi_cntrl);
- }
}
EXPORT_SYMBOL_GPL(mhi_power_down);
diff --git a/drivers/bus/mhi/pci_generic.c b/drivers/bus/mhi/pci_generic.c
index 20673a4b4a3c..7c810f02a2ef 100644
--- a/drivers/bus/mhi/pci_generic.c
+++ b/drivers/bus/mhi/pci_generic.c
@@ -14,6 +14,7 @@
#include <linux/mhi.h>
#include <linux/module.h>
#include <linux/pci.h>
+#include <linux/pm_runtime.h>
#include <linux/timer.h>
#include <linux/workqueue.h>
@@ -71,9 +72,9 @@ struct mhi_pci_dev_info {
.doorbell_mode_switch = false, \
}
-#define MHI_EVENT_CONFIG_CTRL(ev_ring) \
+#define MHI_EVENT_CONFIG_CTRL(ev_ring, el_count) \
{ \
- .num_elements = 64, \
+ .num_elements = el_count, \
.irq_moderation_ms = 0, \
.irq = (ev_ring) + 1, \
.priority = 1, \
@@ -114,9 +115,69 @@ struct mhi_pci_dev_info {
.doorbell_mode_switch = true, \
}
-#define MHI_EVENT_CONFIG_DATA(ev_ring) \
+#define MHI_CHANNEL_CONFIG_UL_SBL(ch_num, ch_name, el_count, ev_ring) \
+ { \
+ .num = ch_num, \
+ .name = ch_name, \
+ .num_elements = el_count, \
+ .event_ring = ev_ring, \
+ .dir = DMA_TO_DEVICE, \
+ .ee_mask = BIT(MHI_EE_SBL), \
+ .pollcfg = 0, \
+ .doorbell = MHI_DB_BRST_DISABLE, \
+ .lpm_notify = false, \
+ .offload_channel = false, \
+ .doorbell_mode_switch = false, \
+ } \
+
+#define MHI_CHANNEL_CONFIG_DL_SBL(ch_num, ch_name, el_count, ev_ring) \
+ { \
+ .num = ch_num, \
+ .name = ch_name, \
+ .num_elements = el_count, \
+ .event_ring = ev_ring, \
+ .dir = DMA_FROM_DEVICE, \
+ .ee_mask = BIT(MHI_EE_SBL), \
+ .pollcfg = 0, \
+ .doorbell = MHI_DB_BRST_DISABLE, \
+ .lpm_notify = false, \
+ .offload_channel = false, \
+ .doorbell_mode_switch = false, \
+ }
+
+#define MHI_CHANNEL_CONFIG_UL_FP(ch_num, ch_name, el_count, ev_ring) \
+ { \
+ .num = ch_num, \
+ .name = ch_name, \
+ .num_elements = el_count, \
+ .event_ring = ev_ring, \
+ .dir = DMA_TO_DEVICE, \
+ .ee_mask = BIT(MHI_EE_FP), \
+ .pollcfg = 0, \
+ .doorbell = MHI_DB_BRST_DISABLE, \
+ .lpm_notify = false, \
+ .offload_channel = false, \
+ .doorbell_mode_switch = false, \
+ } \
+
+#define MHI_CHANNEL_CONFIG_DL_FP(ch_num, ch_name, el_count, ev_ring) \
+ { \
+ .num = ch_num, \
+ .name = ch_name, \
+ .num_elements = el_count, \
+ .event_ring = ev_ring, \
+ .dir = DMA_FROM_DEVICE, \
+ .ee_mask = BIT(MHI_EE_FP), \
+ .pollcfg = 0, \
+ .doorbell = MHI_DB_BRST_DISABLE, \
+ .lpm_notify = false, \
+ .offload_channel = false, \
+ .doorbell_mode_switch = false, \
+ }
+
+#define MHI_EVENT_CONFIG_DATA(ev_ring, el_count) \
{ \
- .num_elements = 128, \
+ .num_elements = el_count, \
.irq_moderation_ms = 5, \
.irq = (ev_ring) + 1, \
.priority = 1, \
@@ -127,9 +188,9 @@ struct mhi_pci_dev_info {
.offload_channel = false, \
}
-#define MHI_EVENT_CONFIG_HW_DATA(ev_ring, ch_num) \
+#define MHI_EVENT_CONFIG_HW_DATA(ev_ring, el_count, ch_num) \
{ \
- .num_elements = 2048, \
+ .num_elements = el_count, \
.irq_moderation_ms = 1, \
.irq = (ev_ring) + 1, \
.priority = 1, \
@@ -150,21 +211,23 @@ static const struct mhi_channel_config modem_qcom_v1_mhi_channels[] = {
MHI_CHANNEL_CONFIG_DL(15, "QMI", 4, 0),
MHI_CHANNEL_CONFIG_UL(20, "IPCR", 8, 0),
MHI_CHANNEL_CONFIG_DL(21, "IPCR", 8, 0),
+ MHI_CHANNEL_CONFIG_UL_FP(34, "FIREHOSE", 32, 0),
+ MHI_CHANNEL_CONFIG_DL_FP(35, "FIREHOSE", 32, 0),
MHI_CHANNEL_CONFIG_HW_UL(100, "IP_HW0", 128, 2),
MHI_CHANNEL_CONFIG_HW_DL(101, "IP_HW0", 128, 3),
};
static struct mhi_event_config modem_qcom_v1_mhi_events[] = {
/* first ring is control+data ring */
- MHI_EVENT_CONFIG_CTRL(0),
+ MHI_EVENT_CONFIG_CTRL(0, 64),
/* DIAG dedicated event ring */
- MHI_EVENT_CONFIG_DATA(1),
+ MHI_EVENT_CONFIG_DATA(1, 128),
/* Hardware channels request dedicated hardware event rings */
- MHI_EVENT_CONFIG_HW_DATA(2, 100),
- MHI_EVENT_CONFIG_HW_DATA(3, 101)
+ MHI_EVENT_CONFIG_HW_DATA(2, 1024, 100),
+ MHI_EVENT_CONFIG_HW_DATA(3, 2048, 101)
};
-static struct mhi_controller_config modem_qcom_v1_mhiv_config = {
+static const struct mhi_controller_config modem_qcom_v1_mhiv_config = {
.max_channels = 128,
.timeout_ms = 8000,
.num_channels = ARRAY_SIZE(modem_qcom_v1_mhi_channels),
@@ -173,6 +236,15 @@ static struct mhi_controller_config modem_qcom_v1_mhiv_config = {
.event_cfg = modem_qcom_v1_mhi_events,
};
+static const struct mhi_pci_dev_info mhi_qcom_sdx65_info = {
+ .name = "qcom-sdx65m",
+ .fw = "qcom/sdx65m/xbl.elf",
+ .edl = "qcom/sdx65m/edl.mbn",
+ .config = &modem_qcom_v1_mhiv_config,
+ .bar_num = MHI_PCI_DEFAULT_BAR_NUM,
+ .dma_data_width = 32
+};
+
static const struct mhi_pci_dev_info mhi_qcom_sdx55_info = {
.name = "qcom-sdx55m",
.fw = "qcom/sdx55m/sbl1.mbn",
@@ -182,15 +254,121 @@ static const struct mhi_pci_dev_info mhi_qcom_sdx55_info = {
.dma_data_width = 32
};
+static const struct mhi_pci_dev_info mhi_qcom_sdx24_info = {
+ .name = "qcom-sdx24",
+ .edl = "qcom/prog_firehose_sdx24.mbn",
+ .config = &modem_qcom_v1_mhiv_config,
+ .bar_num = MHI_PCI_DEFAULT_BAR_NUM,
+ .dma_data_width = 32
+};
+
+static const struct mhi_channel_config mhi_quectel_em1xx_channels[] = {
+ MHI_CHANNEL_CONFIG_UL(0, "NMEA", 32, 0),
+ MHI_CHANNEL_CONFIG_DL(1, "NMEA", 32, 0),
+ MHI_CHANNEL_CONFIG_UL_SBL(2, "SAHARA", 32, 0),
+ MHI_CHANNEL_CONFIG_DL_SBL(3, "SAHARA", 32, 0),
+ MHI_CHANNEL_CONFIG_UL(4, "DIAG", 32, 1),
+ MHI_CHANNEL_CONFIG_DL(5, "DIAG", 32, 1),
+ MHI_CHANNEL_CONFIG_UL(12, "MBIM", 32, 0),
+ MHI_CHANNEL_CONFIG_DL(13, "MBIM", 32, 0),
+ MHI_CHANNEL_CONFIG_UL(32, "DUN", 32, 0),
+ MHI_CHANNEL_CONFIG_DL(33, "DUN", 32, 0),
+ /* The EDL firmware is a flash-programmer exposing firehose protocol */
+ MHI_CHANNEL_CONFIG_UL_FP(34, "FIREHOSE", 32, 0),
+ MHI_CHANNEL_CONFIG_DL_FP(35, "FIREHOSE", 32, 0),
+ MHI_CHANNEL_CONFIG_HW_UL(100, "IP_HW0_MBIM", 128, 2),
+ MHI_CHANNEL_CONFIG_HW_DL(101, "IP_HW0_MBIM", 128, 3),
+};
+
+static struct mhi_event_config mhi_quectel_em1xx_events[] = {
+ MHI_EVENT_CONFIG_CTRL(0, 128),
+ MHI_EVENT_CONFIG_DATA(1, 128),
+ MHI_EVENT_CONFIG_HW_DATA(2, 1024, 100),
+ MHI_EVENT_CONFIG_HW_DATA(3, 1024, 101)
+};
+
+static const struct mhi_controller_config modem_quectel_em1xx_config = {
+ .max_channels = 128,
+ .timeout_ms = 20000,
+ .num_channels = ARRAY_SIZE(mhi_quectel_em1xx_channels),
+ .ch_cfg = mhi_quectel_em1xx_channels,
+ .num_events = ARRAY_SIZE(mhi_quectel_em1xx_events),
+ .event_cfg = mhi_quectel_em1xx_events,
+};
+
+static const struct mhi_pci_dev_info mhi_quectel_em1xx_info = {
+ .name = "quectel-em1xx",
+ .edl = "qcom/prog_firehose_sdx24.mbn",
+ .config = &modem_quectel_em1xx_config,
+ .bar_num = MHI_PCI_DEFAULT_BAR_NUM,
+ .dma_data_width = 32
+};
+
+static const struct mhi_channel_config mhi_foxconn_sdx55_channels[] = {
+ MHI_CHANNEL_CONFIG_UL(0, "LOOPBACK", 32, 0),
+ MHI_CHANNEL_CONFIG_DL(1, "LOOPBACK", 32, 0),
+ MHI_CHANNEL_CONFIG_UL(4, "DIAG", 32, 1),
+ MHI_CHANNEL_CONFIG_DL(5, "DIAG", 32, 1),
+ MHI_CHANNEL_CONFIG_UL(12, "MBIM", 32, 0),
+ MHI_CHANNEL_CONFIG_DL(13, "MBIM", 32, 0),
+ MHI_CHANNEL_CONFIG_UL(32, "AT", 32, 0),
+ MHI_CHANNEL_CONFIG_DL(33, "AT", 32, 0),
+ MHI_CHANNEL_CONFIG_HW_UL(100, "IP_HW0_MBIM", 128, 2),
+ MHI_CHANNEL_CONFIG_HW_DL(101, "IP_HW0_MBIM", 128, 3),
+};
+
+static struct mhi_event_config mhi_foxconn_sdx55_events[] = {
+ MHI_EVENT_CONFIG_CTRL(0, 128),
+ MHI_EVENT_CONFIG_DATA(1, 128),
+ MHI_EVENT_CONFIG_HW_DATA(2, 1024, 100),
+ MHI_EVENT_CONFIG_HW_DATA(3, 1024, 101)
+};
+
+static const struct mhi_controller_config modem_foxconn_sdx55_config = {
+ .max_channels = 128,
+ .timeout_ms = 20000,
+ .num_channels = ARRAY_SIZE(mhi_foxconn_sdx55_channels),
+ .ch_cfg = mhi_foxconn_sdx55_channels,
+ .num_events = ARRAY_SIZE(mhi_foxconn_sdx55_events),
+ .event_cfg = mhi_foxconn_sdx55_events,
+};
+
+static const struct mhi_pci_dev_info mhi_foxconn_sdx55_info = {
+ .name = "foxconn-sdx55",
+ .fw = "qcom/sdx55m/sbl1.mbn",
+ .edl = "qcom/sdx55m/edl.mbn",
+ .config = &modem_foxconn_sdx55_config,
+ .bar_num = MHI_PCI_DEFAULT_BAR_NUM,
+ .dma_data_width = 32
+};
+
static const struct pci_device_id mhi_pci_id_table[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_QCOM, 0x0306),
.driver_data = (kernel_ulong_t) &mhi_qcom_sdx55_info },
+ { PCI_DEVICE(PCI_VENDOR_ID_QCOM, 0x0304),
+ .driver_data = (kernel_ulong_t) &mhi_qcom_sdx24_info },
+ { PCI_DEVICE(0x1eac, 0x1001), /* EM120R-GL (sdx24) */
+ .driver_data = (kernel_ulong_t) &mhi_quectel_em1xx_info },
+ { PCI_DEVICE(0x1eac, 0x1002), /* EM160R-GL (sdx24) */
+ .driver_data = (kernel_ulong_t) &mhi_quectel_em1xx_info },
+ { PCI_DEVICE(PCI_VENDOR_ID_QCOM, 0x0308),
+ .driver_data = (kernel_ulong_t) &mhi_qcom_sdx65_info },
+ /* T99W175 (sdx55), Both for eSIM and Non-eSIM */
+ { PCI_DEVICE(PCI_VENDOR_ID_FOXCONN, 0xe0ab),
+ .driver_data = (kernel_ulong_t) &mhi_foxconn_sdx55_info },
+ /* DW5930e (sdx55), With eSIM, It's also T99W175 */
+ { PCI_DEVICE(PCI_VENDOR_ID_FOXCONN, 0xe0b0),
+ .driver_data = (kernel_ulong_t) &mhi_foxconn_sdx55_info },
+ /* DW5930e (sdx55), Non-eSIM, It's also T99W175 */
+ { PCI_DEVICE(PCI_VENDOR_ID_FOXCONN, 0xe0b1),
+ .driver_data = (kernel_ulong_t) &mhi_foxconn_sdx55_info },
{ }
};
MODULE_DEVICE_TABLE(pci, mhi_pci_id_table);
enum mhi_pci_device_status {
MHI_PCI_DEV_STARTED,
+ MHI_PCI_DEV_SUSPENDED,
};
struct mhi_pci_device {
@@ -224,12 +402,31 @@ static void mhi_pci_status_cb(struct mhi_controller *mhi_cntrl,
case MHI_CB_FATAL_ERROR:
case MHI_CB_SYS_ERROR:
dev_warn(&pdev->dev, "firmware crashed (%u)\n", cb);
+ pm_runtime_forbid(&pdev->dev);
+ break;
+ case MHI_CB_EE_MISSION_MODE:
+ pm_runtime_allow(&pdev->dev);
break;
default:
break;
}
}
+static void mhi_pci_wake_get_nop(struct mhi_controller *mhi_cntrl, bool force)
+{
+ /* no-op */
+}
+
+static void mhi_pci_wake_put_nop(struct mhi_controller *mhi_cntrl, bool override)
+{
+ /* no-op */
+}
+
+static void mhi_pci_wake_toggle_nop(struct mhi_controller *mhi_cntrl)
+{
+ /* no-op */
+}
+
static bool mhi_pci_is_alive(struct mhi_controller *mhi_cntrl)
{
struct pci_dev *pdev = to_pci_dev(mhi_cntrl->cntrl_dev);
@@ -330,13 +527,19 @@ static int mhi_pci_get_irqs(struct mhi_controller *mhi_cntrl,
static int mhi_pci_runtime_get(struct mhi_controller *mhi_cntrl)
{
- /* no PM for now */
- return 0;
+ /* The runtime_get() MHI callback means:
+ * Do whatever is requested to leave M3.
+ */
+ return pm_runtime_get(mhi_cntrl->cntrl_dev);
}
static void mhi_pci_runtime_put(struct mhi_controller *mhi_cntrl)
{
- /* no PM for now */
+ /* The runtime_put() MHI callback means:
+ * Device can be moved in M3 state.
+ */
+ pm_runtime_mark_last_busy(mhi_cntrl->cntrl_dev);
+ pm_runtime_put(mhi_cntrl->cntrl_dev);
}
static void mhi_pci_recovery_work(struct work_struct *work)
@@ -350,6 +553,7 @@ static void mhi_pci_recovery_work(struct work_struct *work)
dev_warn(&pdev->dev, "device recovery started\n");
del_timer(&mhi_pdev->health_check_timer);
+ pm_runtime_forbid(&pdev->dev);
/* Clean up MHI state */
if (test_and_clear_bit(MHI_PCI_DEV_STARTED, &mhi_pdev->status)) {
@@ -357,7 +561,6 @@ static void mhi_pci_recovery_work(struct work_struct *work)
mhi_unprepare_after_power_down(mhi_cntrl);
}
- /* Check if we can recover without full reset */
pci_set_power_state(pdev, PCI_D0);
pci_load_saved_state(pdev, mhi_pdev->pci_state);
pci_restore_state(pdev);
@@ -391,6 +594,10 @@ static void health_check(struct timer_list *t)
struct mhi_pci_device *mhi_pdev = from_timer(mhi_pdev, t, health_check_timer);
struct mhi_controller *mhi_cntrl = &mhi_pdev->mhi_cntrl;
+ if (!test_bit(MHI_PCI_DEV_STARTED, &mhi_pdev->status) ||
+ test_bit(MHI_PCI_DEV_SUSPENDED, &mhi_pdev->status))
+ return;
+
if (!mhi_pci_is_alive(mhi_cntrl)) {
dev_err(mhi_cntrl->cntrl_dev, "Device died\n");
queue_work(system_long_wq, &mhi_pdev->recovery_work);
@@ -433,6 +640,9 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
mhi_cntrl->status_cb = mhi_pci_status_cb;
mhi_cntrl->runtime_get = mhi_pci_runtime_get;
mhi_cntrl->runtime_put = mhi_pci_runtime_put;
+ mhi_cntrl->wake_get = mhi_pci_wake_get_nop;
+ mhi_cntrl->wake_put = mhi_pci_wake_put_nop;
+ mhi_cntrl->wake_toggle = mhi_pci_wake_toggle_nop;
err = mhi_pci_claim(mhi_cntrl, info->bar_num, DMA_BIT_MASK(info->dma_data_width));
if (err)
@@ -444,9 +654,12 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
pci_set_drvdata(pdev, mhi_pdev);
- /* Have stored pci confspace at hand for restore in sudden PCI error */
+ /* Have stored pci confspace at hand for restore in sudden PCI error.
+ * cache the state locally and discard the PCI core one.
+ */
pci_save_state(pdev);
mhi_pdev->pci_state = pci_store_saved_state(pdev);
+ pci_load_saved_state(pdev, NULL);
pci_enable_pcie_error_reporting(pdev);
@@ -472,6 +685,14 @@ static int mhi_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
/* start health check */
mod_timer(&mhi_pdev->health_check_timer, jiffies + HEALTH_CHECK_PERIOD);
+ /* Only allow runtime-suspend if PME capable (for wakeup) */
+ if (pci_pme_capable(pdev, PCI_D3hot)) {
+ pm_runtime_set_autosuspend_delay(&pdev->dev, 2000);
+ pm_runtime_use_autosuspend(&pdev->dev);
+ pm_runtime_mark_last_busy(&pdev->dev);
+ pm_runtime_put_noidle(&pdev->dev);
+ }
+
return 0;
err_unprepare:
@@ -495,9 +716,19 @@ static void mhi_pci_remove(struct pci_dev *pdev)
mhi_unprepare_after_power_down(mhi_cntrl);
}
+ /* balancing probe put_noidle */
+ if (pci_pme_capable(pdev, PCI_D3hot))
+ pm_runtime_get_noresume(&pdev->dev);
+
mhi_unregister_controller(mhi_cntrl);
}
+static void mhi_pci_shutdown(struct pci_dev *pdev)
+{
+ mhi_pci_remove(pdev);
+ pci_set_power_state(pdev, PCI_D3hot);
+}
+
static void mhi_pci_reset_prepare(struct pci_dev *pdev)
{
struct mhi_pci_device *mhi_pdev = pci_get_drvdata(pdev);
@@ -605,41 +836,59 @@ static const struct pci_error_handlers mhi_pci_err_handler = {
.reset_done = mhi_pci_reset_done,
};
-static int __maybe_unused mhi_pci_suspend(struct device *dev)
+static int __maybe_unused mhi_pci_runtime_suspend(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
struct mhi_pci_device *mhi_pdev = dev_get_drvdata(dev);
struct mhi_controller *mhi_cntrl = &mhi_pdev->mhi_cntrl;
+ int err;
+
+ if (test_and_set_bit(MHI_PCI_DEV_SUSPENDED, &mhi_pdev->status))
+ return 0;
del_timer(&mhi_pdev->health_check_timer);
cancel_work_sync(&mhi_pdev->recovery_work);
+ if (!test_bit(MHI_PCI_DEV_STARTED, &mhi_pdev->status) ||
+ mhi_cntrl->ee != MHI_EE_AMSS)
+ goto pci_suspend; /* Nothing to do at MHI level */
+
/* Transition to M3 state */
- mhi_pm_suspend(mhi_cntrl);
+ err = mhi_pm_suspend(mhi_cntrl);
+ if (err) {
+ dev_err(&pdev->dev, "failed to suspend device: %d\n", err);
+ clear_bit(MHI_PCI_DEV_SUSPENDED, &mhi_pdev->status);
+ return -EBUSY;
+ }
- pci_save_state(pdev);
+pci_suspend:
pci_disable_device(pdev);
pci_wake_from_d3(pdev, true);
- pci_set_power_state(pdev, PCI_D3hot);
return 0;
}
-static int __maybe_unused mhi_pci_resume(struct device *dev)
+static int __maybe_unused mhi_pci_runtime_resume(struct device *dev)
{
struct pci_dev *pdev = to_pci_dev(dev);
struct mhi_pci_device *mhi_pdev = dev_get_drvdata(dev);
struct mhi_controller *mhi_cntrl = &mhi_pdev->mhi_cntrl;
int err;
- pci_set_power_state(pdev, PCI_D0);
- pci_restore_state(pdev);
- pci_set_master(pdev);
+ if (!test_and_clear_bit(MHI_PCI_DEV_SUSPENDED, &mhi_pdev->status))
+ return 0;
err = pci_enable_device(pdev);
if (err)
goto err_recovery;
+ pci_set_master(pdev);
+ pci_wake_from_d3(pdev, false);
+
+ if (!test_bit(MHI_PCI_DEV_STARTED, &mhi_pdev->status) ||
+ mhi_cntrl->ee != MHI_EE_AMSS)
+ return 0; /* Nothing to do at MHI level */
+
/* Exit M3, transition to M0 state */
err = mhi_pm_resume(mhi_cntrl);
if (err) {
@@ -650,16 +899,44 @@ static int __maybe_unused mhi_pci_resume(struct device *dev)
/* Resume health check */
mod_timer(&mhi_pdev->health_check_timer, jiffies + HEALTH_CHECK_PERIOD);
+ /* It can be a remote wakeup (no mhi runtime_get), update access time */
+ pm_runtime_mark_last_busy(dev);
+
return 0;
err_recovery:
- /* The device may have loose power or crashed, try recovering it */
+ /* Do not fail to not mess up our PCI device state, the device likely
+ * lost power (d3cold) and we simply need to reset it from the recovery
+ * procedure, trigger the recovery asynchronously to prevent system
+ * suspend exit delaying.
+ */
queue_work(system_long_wq, &mhi_pdev->recovery_work);
+ pm_runtime_mark_last_busy(dev);
- return err;
+ return 0;
+}
+
+static int __maybe_unused mhi_pci_suspend(struct device *dev)
+{
+ pm_runtime_disable(dev);
+ return mhi_pci_runtime_suspend(dev);
+}
+
+static int __maybe_unused mhi_pci_resume(struct device *dev)
+{
+ int ret;
+
+ /* Depending the platform, device may have lost power (d3cold), we need
+ * to resume it now to check its state and recover when necessary.
+ */
+ ret = mhi_pci_runtime_resume(dev);
+ pm_runtime_enable(dev);
+
+ return ret;
}
static const struct dev_pm_ops mhi_pci_pm_ops = {
+ SET_RUNTIME_PM_OPS(mhi_pci_runtime_suspend, mhi_pci_runtime_resume, NULL)
SET_SYSTEM_SLEEP_PM_OPS(mhi_pci_suspend, mhi_pci_resume)
};
@@ -668,6 +945,7 @@ static struct pci_driver mhi_pci_driver = {
.id_table = mhi_pci_id_table,
.probe = mhi_pci_probe,
.remove = mhi_pci_remove,
+ .shutdown = mhi_pci_shutdown,
.err_handler = &mhi_pci_err_handler,
.driver.pm = &mhi_pci_pm_ops
};
diff --git a/drivers/bus/moxtet.c b/drivers/bus/moxtet.c
index b20fdcbd035b..fd87a59837fa 100644
--- a/drivers/bus/moxtet.c
+++ b/drivers/bus/moxtet.c
@@ -2,7 +2,7 @@
/*
* Turris Mox module configuration bus driver
*
- * Copyright (C) 2019 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2019 Marek Behún <kabel@kernel.org>
*/
#include <dt-bindings/bus/moxtet.h>
@@ -879,6 +879,6 @@ static void __exit moxtet_exit(void)
}
module_exit(moxtet_exit);
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
MODULE_DESCRIPTION("CZ.NIC's Turris Mox module configuration bus");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/bus/mvebu-mbus.c b/drivers/bus/mvebu-mbus.c
index dd9e7343a5e3..ea0424922de7 100644
--- a/drivers/bus/mvebu-mbus.c
+++ b/drivers/bus/mvebu-mbus.c
@@ -618,7 +618,7 @@ mvebu_mbus_find_bridge_hole(uint64_t *start, uint64_t *end)
* This part of the memory is above 4 GB, so we don't
* care for the MBus bridge hole.
*/
- if (reg_start >= 0x100000000ULL)
+ if ((u64)reg_start >= 0x100000000ULL)
continue;
/*
diff --git a/drivers/bus/omap_l3_noc.c b/drivers/bus/omap_l3_noc.c
index b040447575ad..dcfb32ee5cb6 100644
--- a/drivers/bus/omap_l3_noc.c
+++ b/drivers/bus/omap_l3_noc.c
@@ -285,7 +285,7 @@ static int omap_l3_probe(struct platform_device *pdev)
*/
l3->debug_irq = platform_get_irq(pdev, 0);
ret = devm_request_irq(l3->dev, l3->debug_irq, l3_interrupt_handler,
- 0x0, "l3-dbg-irq", l3);
+ IRQF_NO_THREAD, "l3-dbg-irq", l3);
if (ret) {
dev_err(l3->dev, "request_irq failed for %d\n",
l3->debug_irq);
@@ -294,7 +294,7 @@ static int omap_l3_probe(struct platform_device *pdev)
l3->app_irq = platform_get_irq(pdev, 1);
ret = devm_request_irq(l3->dev, l3->app_irq, l3_interrupt_handler,
- 0x0, "l3-app-irq", l3);
+ IRQF_NO_THREAD, "l3-app-irq", l3);
if (ret)
dev_err(l3->dev, "request_irq failed for %d\n", l3->app_irq);
diff --git a/drivers/bus/qcom-ebi2.c b/drivers/bus/qcom-ebi2.c
index 03ddcf426887..0b8f53a688b8 100644
--- a/drivers/bus/qcom-ebi2.c
+++ b/drivers/bus/qcom-ebi2.c
@@ -353,8 +353,10 @@ static int qcom_ebi2_probe(struct platform_device *pdev)
/* Figure out the chipselect */
ret = of_property_read_u32(child, "reg", &csindex);
- if (ret)
+ if (ret) {
+ of_node_put(child);
return ret;
+ }
if (csindex > 5) {
dev_err(dev,
diff --git a/drivers/bus/ti-sysc.c b/drivers/bus/ti-sysc.c
index a27d751cf219..5fae60f8c135 100644
--- a/drivers/bus/ti-sysc.c
+++ b/drivers/bus/ti-sysc.c
@@ -288,7 +288,7 @@ static int sysc_add_named_clock_from_child(struct sysc *ddata,
* limit for clk_get(). If cl ever needs to be freed, it should be done
* with clkdev_drop().
*/
- cl = kcalloc(1, sizeof(*cl), GFP_KERNEL);
+ cl = kzalloc(sizeof(*cl), GFP_KERNEL);
if (!cl)
return -ENOMEM;
@@ -635,6 +635,51 @@ static int sysc_parse_and_check_child_range(struct sysc *ddata)
return 0;
}
+/* Interconnect instances to probe before l4_per instances */
+static struct resource early_bus_ranges[] = {
+ /* am3/4 l4_wkup */
+ { .start = 0x44c00000, .end = 0x44c00000 + 0x300000, },
+ /* omap4/5 and dra7 l4_cfg */
+ { .start = 0x4a000000, .end = 0x4a000000 + 0x300000, },
+ /* omap4 l4_wkup */
+ { .start = 0x4a300000, .end = 0x4a300000 + 0x30000, },
+ /* omap5 and dra7 l4_wkup without dra7 dcan segment */
+ { .start = 0x4ae00000, .end = 0x4ae00000 + 0x30000, },
+};
+
+static atomic_t sysc_defer = ATOMIC_INIT(10);
+
+/**
+ * sysc_defer_non_critical - defer non_critical interconnect probing
+ * @ddata: device driver data
+ *
+ * We want to probe l4_cfg and l4_wkup interconnect instances before any
+ * l4_per instances as l4_per instances depend on resources on l4_cfg and
+ * l4_wkup interconnects.
+ */
+static int sysc_defer_non_critical(struct sysc *ddata)
+{
+ struct resource *res;
+ int i;
+
+ if (!atomic_read(&sysc_defer))
+ return 0;
+
+ for (i = 0; i < ARRAY_SIZE(early_bus_ranges); i++) {
+ res = &early_bus_ranges[i];
+ if (ddata->module_pa >= res->start &&
+ ddata->module_pa <= res->end) {
+ atomic_set(&sysc_defer, 0);
+
+ return 0;
+ }
+ }
+
+ atomic_dec_if_positive(&sysc_defer);
+
+ return -EPROBE_DEFER;
+}
+
static struct device_node *stdout_path;
static void sysc_init_stdout_path(struct sysc *ddata)
@@ -856,15 +901,19 @@ static int sysc_map_and_check_registers(struct sysc *ddata)
struct device_node *np = ddata->dev->of_node;
int error;
- if (!of_get_property(np, "reg", NULL))
- return 0;
-
error = sysc_parse_and_check_child_range(ddata);
if (error)
return error;
+ error = sysc_defer_non_critical(ddata);
+ if (error)
+ return error;
+
sysc_check_children(ddata);
+ if (!of_get_property(np, "reg", NULL))
+ return 0;
+
error = sysc_parse_registers(ddata);
if (error)
return error;
@@ -1447,12 +1496,16 @@ static const struct sysc_revision_quirk sysc_revision_quirks[] = {
SYSC_QUIRK("dwc3", 0, 0, 0x10, -ENODEV, 0x500a0200, 0xffffffff, 0),
SYSC_QUIRK("d2d", 0x4a0b6000, 0, 0x10, 0x14, 0x00000010, 0xffffffff, 0),
SYSC_QUIRK("d2d", 0x4a0cd000, 0, 0x10, 0x14, 0x00000010, 0xffffffff, 0),
+ SYSC_QUIRK("elm", 0x48080000, 0, 0x10, 0x14, 0x00000020, 0xffffffff, 0),
+ SYSC_QUIRK("emif", 0, 0, -ENODEV, -ENODEV, 0x40441403, 0xffff0fff, 0),
+ SYSC_QUIRK("emif", 0, 0, -ENODEV, -ENODEV, 0x50440500, 0xffffffff, 0),
SYSC_QUIRK("epwmss", 0, 0, 0x4, -ENODEV, 0x47400001, 0xffffffff, 0),
SYSC_QUIRK("gpu", 0, 0x1fc00, 0x1fc10, -ENODEV, 0, 0, 0),
SYSC_QUIRK("gpu", 0, 0xfe00, 0xfe10, -ENODEV, 0x40000000 , 0xffffffff, 0),
SYSC_QUIRK("hdmi", 0, 0, 0x10, -ENODEV, 0x50031d00, 0xffffffff, 0),
SYSC_QUIRK("hsi", 0, 0, 0x10, 0x14, 0x50043101, 0xffffffff, 0),
SYSC_QUIRK("iss", 0, 0, 0x10, -ENODEV, 0x40000101, 0xffffffff, 0),
+ SYSC_QUIRK("keypad", 0x4a31c000, 0, 0x10, 0x14, 0x00000020, 0xffffffff, 0),
SYSC_QUIRK("mcasp", 0, 0, 0x4, -ENODEV, 0x44306302, 0xffffffff, 0),
SYSC_QUIRK("mcasp", 0, 0, 0x4, -ENODEV, 0x44307b02, 0xffffffff, 0),
SYSC_QUIRK("mcbsp", 0, -ENODEV, 0x8c, -ENODEV, 0, 0, 0),
@@ -1464,11 +1517,14 @@ static const struct sysc_revision_quirk sysc_revision_quirks[] = {
SYSC_QUIRK("ocp2scp", 0, 0, -ENODEV, -ENODEV, 0x50060007, 0xffffffff, 0),
SYSC_QUIRK("padconf", 0, 0, 0x10, -ENODEV, 0x4fff0800, 0xffffffff, 0),
SYSC_QUIRK("padconf", 0, 0, -ENODEV, -ENODEV, 0x40001100, 0xffffffff, 0),
+ SYSC_QUIRK("pcie", 0x51000000, -ENODEV, -ENODEV, -ENODEV, 0, 0, 0),
+ SYSC_QUIRK("pcie", 0x51800000, -ENODEV, -ENODEV, -ENODEV, 0, 0, 0),
SYSC_QUIRK("prcm", 0, 0, -ENODEV, -ENODEV, 0x40000100, 0xffffffff, 0),
SYSC_QUIRK("prcm", 0, 0, -ENODEV, -ENODEV, 0x00004102, 0xffffffff, 0),
SYSC_QUIRK("prcm", 0, 0, -ENODEV, -ENODEV, 0x40000400, 0xffffffff, 0),
SYSC_QUIRK("rfbi", 0x4832a800, 0, 0x10, 0x14, 0x00000010, 0xffffffff, 0),
SYSC_QUIRK("rfbi", 0x58002000, 0, 0x10, 0x14, 0x00000010, 0xffffffff, 0),
+ SYSC_QUIRK("sata", 0, 0xfc, 0x1100, -ENODEV, 0x5e412000, 0xffffffff, 0),
SYSC_QUIRK("scm", 0, 0, 0x10, -ENODEV, 0x40000900, 0xffffffff, 0),
SYSC_QUIRK("scm", 0, 0, -ENODEV, -ENODEV, 0x4e8b0100, 0xffffffff, 0),
SYSC_QUIRK("scm", 0, 0, -ENODEV, -ENODEV, 0x4f000100, 0xffffffff, 0),
@@ -1592,7 +1648,7 @@ static u32 sysc_quirk_dispc(struct sysc *ddata, int dispc_offset,
case SOC_UNKNOWN:
default:
return 0;
- };
+ }
/* Remap the whole module range to be able to reset dispc outputs */
devm_iounmap(ddata->dev, ddata->module_va);
@@ -2802,6 +2858,7 @@ static int sysc_init_soc(struct sysc *ddata)
const struct soc_device_attribute *match;
struct ti_sysc_platform_data *pdata;
unsigned long features = 0;
+ struct device_node *np;
if (sysc_soc)
return 0;
@@ -2822,6 +2879,24 @@ static int sysc_init_soc(struct sysc *ddata)
if (match && match->data)
sysc_soc->soc = (int)match->data;
+ /*
+ * Check and warn about possible old incomplete dtb. We now want to see
+ * simple-pm-bus instead of simple-bus in the dtb for genpd using SoCs.
+ */
+ switch (sysc_soc->soc) {
+ case SOC_AM3:
+ case SOC_AM4:
+ case SOC_4430 ... SOC_4470:
+ case SOC_5430:
+ case SOC_DRA7:
+ np = of_find_node_by_path("/ocp");
+ WARN_ONCE(np && of_device_is_compatible(np, "simple-bus"),
+ "ti-sysc: Incomplete old dtb, please update\n");
+ break;
+ default:
+ break;
+ }
+
/* Ignore devices that are not available on HS and EMU SoCs */
if (!sysc_soc->general_purpose) {
switch (sysc_soc->soc) {
@@ -2830,7 +2905,7 @@ static int sysc_init_soc(struct sysc *ddata)
break;
default:
break;
- };
+ }
}
match = soc_device_match(sysc_soc_feat_match);
@@ -3053,7 +3128,9 @@ static int sysc_remove(struct platform_device *pdev)
pm_runtime_put_sync(&pdev->dev);
pm_runtime_disable(&pdev->dev);
- reset_control_assert(ddata->rsts);
+
+ if (!reset_control_status(ddata->rsts))
+ reset_control_assert(ddata->rsts);
unprepare:
sysc_unprepare(ddata);
diff --git a/drivers/cdrom/gdrom.c b/drivers/cdrom/gdrom.c
index 9874fc1c815b..742b4a0932e3 100644
--- a/drivers/cdrom/gdrom.c
+++ b/drivers/cdrom/gdrom.c
@@ -583,7 +583,8 @@ static blk_status_t gdrom_readdisk_dma(struct request *req)
read_command->cmd[1] = 0x20;
block = blk_rq_pos(req)/GD_TO_BLK + GD_SESSION_OFFSET;
block_cnt = blk_rq_sectors(req)/GD_TO_BLK;
- __raw_writel(virt_to_phys(bio_data(req->bio)), GDROM_DMA_STARTADDR_REG);
+ __raw_writel(page_to_phys(bio_page(req->bio)) + bio_offset(req->bio),
+ GDROM_DMA_STARTADDR_REG);
__raw_writel(block_cnt * GDROM_HARD_SECTOR, GDROM_DMA_LENGTH_REG);
__raw_writel(1, GDROM_DMA_DIRECTION_REG);
__raw_writel(1, GDROM_DMA_ENABLE_REG);
@@ -789,8 +790,6 @@ static int probe_gdrom(struct platform_device *devptr)
goto probe_fail_requestq;
}
- blk_queue_bounce_limit(gd.gdrom_rq, BLK_BOUNCE_HIGH);
-
err = probe_gdrom_setupqueue();
if (err)
goto probe_fail_toc;
diff --git a/drivers/char/Kconfig b/drivers/char/Kconfig
index d229a2d0c017..b151e0fcdeb5 100644
--- a/drivers/char/Kconfig
+++ b/drivers/char/Kconfig
@@ -334,16 +334,6 @@ config DEVMEM
memory.
When in doubt, say "Y".
-config DEVKMEM
- bool "/dev/kmem virtual device support"
- # On arm64, VMALLOC_START < PAGE_OFFSET, which confuses kmem read/write
- depends on !ARM64
- help
- Say Y here if you want to support the /dev/kmem device. The
- /dev/kmem device is rarely used, but can be used for certain
- kind of kernel debugging operations.
- When in doubt, say "N".
-
config NVRAM
tristate "/dev/nvram support"
depends on X86 || HAVE_ARCH_NVRAM_OPS
diff --git a/drivers/char/agp/Kconfig b/drivers/char/agp/Kconfig
index a086dd34f932..4f501e4842ab 100644
--- a/drivers/char/agp/Kconfig
+++ b/drivers/char/agp/Kconfig
@@ -125,7 +125,7 @@ config AGP_HP_ZX1
config AGP_PARISC
tristate "HP Quicksilver AGP support"
- depends on AGP && PARISC && 64BIT
+ depends on AGP && PARISC && 64BIT && IOMMU_SBA
help
This option gives you AGP GART support for the HP Quicksilver
AGP bus adapter on HP PA-RISC machines (Ok, just on the C8000
diff --git a/drivers/char/applicom.c b/drivers/char/applicom.c
index 14b2d8034c51..deb85a334c93 100644
--- a/drivers/char/applicom.c
+++ b/drivers/char/applicom.c
@@ -81,9 +81,6 @@ MODULE_DESCRIPTION("Driver for Applicom Profibus card");
MODULE_LICENSE("GPL");
MODULE_ALIAS_MISCDEV(AC_MINOR);
-MODULE_SUPPORTED_DEVICE("ac");
-
-
static struct applicom_board {
unsigned long PhysIO;
void __iomem *RamIO;
@@ -839,7 +836,7 @@ static long ac_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
Dummy = readb(apbs[IndexCard].RamIO + VERS);
kfree(adgl);
mutex_unlock(&ac_mutex);
- return 0;
+ return ret;
err:
if (warncount) {
diff --git a/drivers/char/hw_random/ba431-rng.c b/drivers/char/hw_random/ba431-rng.c
index 410b50b05e21..5b7ca0416490 100644
--- a/drivers/char/hw_random/ba431-rng.c
+++ b/drivers/char/hw_random/ba431-rng.c
@@ -170,7 +170,6 @@ static int ba431_trng_init(struct hwrng *rng)
static int ba431_trng_probe(struct platform_device *pdev)
{
struct ba431_trng *ba431;
- struct resource *res;
int ret;
ba431 = devm_kzalloc(&pdev->dev, sizeof(*ba431), GFP_KERNEL);
@@ -179,8 +178,7 @@ static int ba431_trng_probe(struct platform_device *pdev)
ba431->dev = &pdev->dev;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- ba431->base = devm_ioremap_resource(&pdev->dev, res);
+ ba431->base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(ba431->base))
return PTR_ERR(ba431->base);
@@ -193,7 +191,7 @@ static int ba431_trng_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, ba431);
- ret = hwrng_register(&ba431->rng);
+ ret = devm_hwrng_register(&pdev->dev, &ba431->rng);
if (ret) {
dev_err(&pdev->dev, "BA431 registration failed (%d)\n", ret);
return ret;
@@ -204,15 +202,6 @@ static int ba431_trng_probe(struct platform_device *pdev)
return 0;
}
-static int ba431_trng_remove(struct platform_device *pdev)
-{
- struct ba431_trng *ba431 = platform_get_drvdata(pdev);
-
- hwrng_unregister(&ba431->rng);
-
- return 0;
-}
-
static const struct of_device_id ba431_trng_dt_ids[] = {
{ .compatible = "silex-insight,ba431-rng", .data = NULL },
{ /* sentinel */ }
@@ -225,7 +214,6 @@ static struct platform_driver ba431_trng_driver = {
.of_match_table = ba431_trng_dt_ids,
},
.probe = ba431_trng_probe,
- .remove = ba431_trng_remove,
};
module_platform_driver(ba431_trng_driver);
diff --git a/drivers/char/hw_random/bcm2835-rng.c b/drivers/char/hw_random/bcm2835-rng.c
index 1a7c43b43c6b..e7dd457e9b22 100644
--- a/drivers/char/hw_random/bcm2835-rng.c
+++ b/drivers/char/hw_random/bcm2835-rng.c
@@ -13,6 +13,7 @@
#include <linux/platform_device.h>
#include <linux/printk.h>
#include <linux/clk.h>
+#include <linux/reset.h>
#define RNG_CTRL 0x0
#define RNG_STATUS 0x4
@@ -32,6 +33,7 @@ struct bcm2835_rng_priv {
void __iomem *base;
bool mask_interrupts;
struct clk *clk;
+ struct reset_control *reset;
};
static inline struct bcm2835_rng_priv *to_rng_priv(struct hwrng *rng)
@@ -88,11 +90,13 @@ static int bcm2835_rng_init(struct hwrng *rng)
int ret = 0;
u32 val;
- if (!IS_ERR(priv->clk)) {
- ret = clk_prepare_enable(priv->clk);
- if (ret)
- return ret;
- }
+ ret = clk_prepare_enable(priv->clk);
+ if (ret)
+ return ret;
+
+ ret = reset_control_reset(priv->reset);
+ if (ret)
+ return ret;
if (priv->mask_interrupts) {
/* mask the interrupt */
@@ -115,8 +119,7 @@ static void bcm2835_rng_cleanup(struct hwrng *rng)
/* disable rng hardware */
rng_writel(priv, 0, RNG_CTRL);
- if (!IS_ERR(priv->clk))
- clk_disable_unprepare(priv->clk);
+ clk_disable_unprepare(priv->clk);
}
struct bcm2835_rng_of_data {
@@ -155,9 +158,13 @@ static int bcm2835_rng_probe(struct platform_device *pdev)
return PTR_ERR(priv->base);
/* Clock is optional on most platforms */
- priv->clk = devm_clk_get(dev, NULL);
- if (PTR_ERR(priv->clk) == -EPROBE_DEFER)
- return -EPROBE_DEFER;
+ priv->clk = devm_clk_get_optional(dev, NULL);
+ if (IS_ERR(priv->clk))
+ return PTR_ERR(priv->clk);
+
+ priv->reset = devm_reset_control_get_optional_exclusive(dev, NULL);
+ if (IS_ERR(priv->reset))
+ return PTR_ERR(priv->reset);
priv->rng.name = pdev->name;
priv->rng.init = bcm2835_rng_init;
diff --git a/drivers/char/hw_random/cctrng.c b/drivers/char/hw_random/cctrng.c
index 7a293f2147a0..302ffa354c2f 100644
--- a/drivers/char/hw_random/cctrng.c
+++ b/drivers/char/hw_random/cctrng.c
@@ -486,7 +486,6 @@ static void cc_trng_clk_fini(struct cctrng_drvdata *drvdata)
static int cctrng_probe(struct platform_device *pdev)
{
- struct resource *req_mem_cc_regs = NULL;
struct cctrng_drvdata *drvdata;
struct device *dev = &pdev->dev;
int rc = 0;
@@ -510,27 +509,16 @@ static int cctrng_probe(struct platform_device *pdev)
drvdata->circ.buf = (char *)drvdata->data_buf;
- /* Get device resources */
- /* First CC registers space */
- req_mem_cc_regs = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- /* Map registers space */
- drvdata->cc_base = devm_ioremap_resource(dev, req_mem_cc_regs);
+ drvdata->cc_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(drvdata->cc_base)) {
dev_err(dev, "Failed to ioremap registers");
return PTR_ERR(drvdata->cc_base);
}
- dev_dbg(dev, "Got MEM resource (%s): %pR\n", req_mem_cc_regs->name,
- req_mem_cc_regs);
- dev_dbg(dev, "CC registers mapped from %pa to 0x%p\n",
- &req_mem_cc_regs->start, drvdata->cc_base);
-
/* Then IRQ */
irq = platform_get_irq(pdev, 0);
- if (irq < 0) {
- dev_err(dev, "Failed getting IRQ resource\n");
+ if (irq < 0)
return irq;
- }
/* parse sampling rate from device tree */
rc = cc_trng_parse_sampling_ratio(drvdata);
@@ -585,7 +573,7 @@ static int cctrng_probe(struct platform_device *pdev)
atomic_set(&drvdata->pending_hw, 1);
/* registration of the hwrng device */
- rc = hwrng_register(&drvdata->rng);
+ rc = devm_hwrng_register(dev, &drvdata->rng);
if (rc) {
dev_err(dev, "Could not register hwrng device.\n");
goto post_pm_err;
@@ -618,8 +606,6 @@ static int cctrng_remove(struct platform_device *pdev)
dev_dbg(dev, "Releasing cctrng resources...\n");
- hwrng_unregister(&drvdata->rng);
-
cc_trng_pm_fini(drvdata);
cc_trng_clk_fini(drvdata);
diff --git a/drivers/char/hw_random/core.c b/drivers/char/hw_random/core.c
index 8c1c47dd9f46..adb3c2bd7783 100644
--- a/drivers/char/hw_random/core.c
+++ b/drivers/char/hw_random/core.c
@@ -396,7 +396,7 @@ static ssize_t hwrng_attr_selected_show(struct device *dev,
struct device_attribute *attr,
char *buf)
{
- return snprintf(buf, PAGE_SIZE, "%d\n", cur_rng_set_by_user);
+ return sysfs_emit(buf, "%d\n", cur_rng_set_by_user);
}
static DEVICE_ATTR(rng_current, S_IRUGO | S_IWUSR,
diff --git a/drivers/char/hw_random/intel-rng.c b/drivers/char/hw_random/intel-rng.c
index eb7db27f9f19..d740b8814bf3 100644
--- a/drivers/char/hw_random/intel-rng.c
+++ b/drivers/char/hw_random/intel-rng.c
@@ -25,13 +25,13 @@
*/
#include <linux/hw_random.h>
+#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci.h>
#include <linux/stop_machine.h>
#include <linux/delay.h>
#include <linux/slab.h>
-#include <asm/io.h>
#define PFX KBUILD_MODNAME ": "
diff --git a/drivers/char/hw_random/omap-rng.c b/drivers/char/hw_random/omap-rng.c
index 5cc5fc504968..cede9f159102 100644
--- a/drivers/char/hw_random/omap-rng.c
+++ b/drivers/char/hw_random/omap-rng.c
@@ -30,8 +30,7 @@
#include <linux/of_address.h>
#include <linux/interrupt.h>
#include <linux/clk.h>
-
-#include <asm/io.h>
+#include <linux/io.h>
#define RNG_REG_STATUS_RDY (1 << 0)
@@ -378,16 +377,13 @@ MODULE_DEVICE_TABLE(of, omap_rng_of_match);
static int of_get_omap_rng_device_details(struct omap_rng_dev *priv,
struct platform_device *pdev)
{
- const struct of_device_id *match;
struct device *dev = &pdev->dev;
int irq, err;
- match = of_match_device(of_match_ptr(omap_rng_of_match), dev);
- if (!match) {
- dev_err(dev, "no compatible OF match\n");
- return -EINVAL;
- }
- priv->pdata = match->data;
+ priv->pdata = of_device_get_match_data(dev);
+ if (!priv->pdata)
+ return -ENODEV;
+
if (of_device_is_compatible(dev->of_node, "ti,omap4-rng") ||
of_device_is_compatible(dev->of_node, "inside-secure,safexcel-eip76")) {
diff --git a/drivers/char/hw_random/pic32-rng.c b/drivers/char/hw_random/pic32-rng.c
index e8210c1715cf..99c8bd0859a1 100644
--- a/drivers/char/hw_random/pic32-rng.c
+++ b/drivers/char/hw_random/pic32-rng.c
@@ -96,7 +96,7 @@ static int pic32_rng_probe(struct platform_device *pdev)
priv->rng.name = pdev->name;
priv->rng.read = pic32_rng_read;
- ret = hwrng_register(&priv->rng);
+ ret = devm_hwrng_register(&pdev->dev, &priv->rng);
if (ret)
goto err_register;
@@ -113,7 +113,6 @@ static int pic32_rng_remove(struct platform_device *pdev)
{
struct pic32_rng *rng = platform_get_drvdata(pdev);
- hwrng_unregister(&rng->rng);
writel(0, rng->base + RNGCON);
clk_disable_unprepare(rng->clk);
return 0;
diff --git a/drivers/char/hw_random/pseries-rng.c b/drivers/char/hw_random/pseries-rng.c
index 8038a8a9fb58..f4949b689bd5 100644
--- a/drivers/char/hw_random/pseries-rng.c
+++ b/drivers/char/hw_random/pseries-rng.c
@@ -54,10 +54,9 @@ static int pseries_rng_probe(struct vio_dev *dev,
return hwrng_register(&pseries_rng);
}
-static int pseries_rng_remove(struct vio_dev *dev)
+static void pseries_rng_remove(struct vio_dev *dev)
{
hwrng_unregister(&pseries_rng);
- return 0;
}
static const struct vio_device_id pseries_rng_driver_ids[] = {
diff --git a/drivers/char/hw_random/xiphera-trng.c b/drivers/char/hw_random/xiphera-trng.c
index 7bdab8c8a6a8..2a9fea72b2e0 100644
--- a/drivers/char/hw_random/xiphera-trng.c
+++ b/drivers/char/hw_random/xiphera-trng.c
@@ -63,14 +63,12 @@ static int xiphera_trng_probe(struct platform_device *pdev)
int ret;
struct xiphera_trng *trng;
struct device *dev = &pdev->dev;
- struct resource *res;
trng = devm_kzalloc(dev, sizeof(*trng), GFP_KERNEL);
if (!trng)
return -ENOMEM;
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- trng->mem = devm_ioremap_resource(dev, res);
+ trng->mem = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(trng->mem))
return PTR_ERR(trng->mem);
diff --git a/drivers/char/ipmi/ipmi_msghandler.c b/drivers/char/ipmi/ipmi_msghandler.c
index c44ad18464f1..8a0e97b33cae 100644
--- a/drivers/char/ipmi/ipmi_msghandler.c
+++ b/drivers/char/ipmi/ipmi_msghandler.c
@@ -49,11 +49,17 @@ static int handle_one_recv_msg(struct ipmi_smi *intf,
static bool initialized;
static bool drvregistered;
+/* Numbers in this enumerator should be mapped to ipmi_panic_event_str */
enum ipmi_panic_event_op {
IPMI_SEND_PANIC_EVENT_NONE,
IPMI_SEND_PANIC_EVENT,
- IPMI_SEND_PANIC_EVENT_STRING
+ IPMI_SEND_PANIC_EVENT_STRING,
+ IPMI_SEND_PANIC_EVENT_MAX
};
+
+/* Indices in this array should be mapped to enum ipmi_panic_event_op */
+static const char *const ipmi_panic_event_str[] = { "none", "event", "string", NULL };
+
#ifdef CONFIG_IPMI_PANIC_STRING
#define IPMI_PANIC_DEFAULT IPMI_SEND_PANIC_EVENT_STRING
#elif defined(CONFIG_IPMI_PANIC_EVENT)
@@ -68,46 +74,27 @@ static int panic_op_write_handler(const char *val,
const struct kernel_param *kp)
{
char valcp[16];
- char *s;
-
- strncpy(valcp, val, 15);
- valcp[15] = '\0';
+ int e;
- s = strstrip(valcp);
-
- if (strcmp(s, "none") == 0)
- ipmi_send_panic_event = IPMI_SEND_PANIC_EVENT_NONE;
- else if (strcmp(s, "event") == 0)
- ipmi_send_panic_event = IPMI_SEND_PANIC_EVENT;
- else if (strcmp(s, "string") == 0)
- ipmi_send_panic_event = IPMI_SEND_PANIC_EVENT_STRING;
- else
- return -EINVAL;
+ strscpy(valcp, val, sizeof(valcp));
+ e = match_string(ipmi_panic_event_str, -1, strstrip(valcp));
+ if (e < 0)
+ return e;
+ ipmi_send_panic_event = e;
return 0;
}
static int panic_op_read_handler(char *buffer, const struct kernel_param *kp)
{
- switch (ipmi_send_panic_event) {
- case IPMI_SEND_PANIC_EVENT_NONE:
- strcpy(buffer, "none\n");
- break;
-
- case IPMI_SEND_PANIC_EVENT:
- strcpy(buffer, "event\n");
- break;
-
- case IPMI_SEND_PANIC_EVENT_STRING:
- strcpy(buffer, "string\n");
- break;
+ const char *event_str;
- default:
- strcpy(buffer, "???\n");
- break;
- }
+ if (ipmi_send_panic_event >= IPMI_SEND_PANIC_EVENT_MAX)
+ event_str = "???";
+ else
+ event_str = ipmi_panic_event_str[ipmi_send_panic_event];
- return strlen(buffer);
+ return sprintf(buffer, "%s\n", event_str);
}
static const struct kernel_param_ops panic_op_ops = {
@@ -2447,10 +2434,8 @@ retry:
wait_event(intf->waitq, bmc->dyn_id_set != 2);
if (!bmc->dyn_id_set) {
- if ((bmc->cc == IPMI_DEVICE_IN_FW_UPDATE_ERR
- || bmc->cc == IPMI_DEVICE_IN_INIT_ERR
- || bmc->cc == IPMI_NOT_IN_MY_STATE_ERR)
- && ++retry_count <= GET_DEVICE_ID_MAX_RETRY) {
+ if (bmc->cc != IPMI_CC_NO_ERROR &&
+ ++retry_count <= GET_DEVICE_ID_MAX_RETRY) {
msleep(500);
dev_warn(intf->si_dev,
"BMC returned 0x%2.2x, retry get bmc device id\n",
@@ -5224,7 +5209,6 @@ module_exit(cleanup_ipmi);
module_init(ipmi_init_msghandler_mod);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Corey Minyard <minyard@mvista.com>");
-MODULE_DESCRIPTION("Incoming and outgoing message routing for an IPMI"
- " interface.");
+MODULE_DESCRIPTION("Incoming and outgoing message routing for an IPMI interface.");
MODULE_VERSION(IPMI_DRIVER_VERSION);
MODULE_SOFTDEP("post: ipmi_devintf");
diff --git a/drivers/char/ipmi/ipmi_plat_data.c b/drivers/char/ipmi/ipmi_plat_data.c
index 28471ff2a3a3..747b51ae01a8 100644
--- a/drivers/char/ipmi/ipmi_plat_data.c
+++ b/drivers/char/ipmi/ipmi_plat_data.c
@@ -102,7 +102,7 @@ struct platform_device *ipmi_platform_add(const char *name, unsigned int inst,
goto err;
}
add_properties:
- rv = platform_device_add_properties(pdev, pr);
+ rv = device_create_managed_software_node(&pdev->dev, pr, NULL);
if (rv) {
dev_err(&pdev->dev,
"Unable to add hard-code properties: %d\n", rv);
diff --git a/drivers/char/ipmi/ipmi_si.h b/drivers/char/ipmi/ipmi_si.h
index bac0ff86e48e..0a4c69539f24 100644
--- a/drivers/char/ipmi/ipmi_si.h
+++ b/drivers/char/ipmi/ipmi_si.h
@@ -18,10 +18,14 @@
#define DEFAULT_REGSPACING 1
#define DEFAULT_REGSIZE 1
+/* Numbers in this enumerator should be mapped to si_to_str[] */
enum si_type {
- SI_TYPE_INVALID, SI_KCS, SI_SMIC, SI_BT
+ SI_TYPE_INVALID, SI_KCS, SI_SMIC, SI_BT, SI_TYPE_MAX
};
+/* Array is defined in the ipmi_si_intf.c */
+extern const char *const si_to_str[];
+
enum ipmi_addr_space {
IPMI_IO_ADDR_SPACE, IPMI_MEM_ADDR_SPACE
};
@@ -48,8 +52,6 @@ struct si_sm_io {
enum ipmi_addr_space addr_space;
unsigned long addr_data;
enum ipmi_addr_src addr_source; /* ACPI, PCI, SMBIOS, hardcode, etc. */
- void (*addr_source_cleanup)(struct si_sm_io *io);
- void *addr_source_data;
union ipmi_smi_info_union addr_info;
int (*io_setup)(struct si_sm_io *info);
diff --git a/drivers/char/ipmi/ipmi_si_hardcode.c b/drivers/char/ipmi/ipmi_si_hardcode.c
index f6ece7569504..ed5e91b1e040 100644
--- a/drivers/char/ipmi/ipmi_si_hardcode.c
+++ b/drivers/char/ipmi/ipmi_si_hardcode.c
@@ -32,47 +32,29 @@ static int slave_addrs[SI_MAX_PARMS] __initdata;
static unsigned int num_slave_addrs __initdata;
module_param_string(type, si_type_str, MAX_SI_TYPE_STR, 0);
-MODULE_PARM_DESC(type, "Defines the type of each interface, each"
- " interface separated by commas. The types are 'kcs',"
- " 'smic', and 'bt'. For example si_type=kcs,bt will set"
- " the first interface to kcs and the second to bt");
+MODULE_PARM_DESC(type,
+ "Defines the type of each interface, each interface separated by commas. The types are 'kcs', 'smic', and 'bt'. For example si_type=kcs,bt will set the first interface to kcs and the second to bt");
module_param_hw_array(addrs, ulong, iomem, &num_addrs, 0);
-MODULE_PARM_DESC(addrs, "Sets the memory address of each interface, the"
- " addresses separated by commas. Only use if an interface"
- " is in memory. Otherwise, set it to zero or leave"
- " it blank.");
+MODULE_PARM_DESC(addrs,
+ "Sets the memory address of each interface, the addresses separated by commas. Only use if an interface is in memory. Otherwise, set it to zero or leave it blank.");
module_param_hw_array(ports, uint, ioport, &num_ports, 0);
-MODULE_PARM_DESC(ports, "Sets the port address of each interface, the"
- " addresses separated by commas. Only use if an interface"
- " is a port. Otherwise, set it to zero or leave"
- " it blank.");
+MODULE_PARM_DESC(ports,
+ "Sets the port address of each interface, the addresses separated by commas. Only use if an interface is a port. Otherwise, set it to zero or leave it blank.");
module_param_hw_array(irqs, int, irq, &num_irqs, 0);
-MODULE_PARM_DESC(irqs, "Sets the interrupt of each interface, the"
- " addresses separated by commas. Only use if an interface"
- " has an interrupt. Otherwise, set it to zero or leave"
- " it blank.");
+MODULE_PARM_DESC(irqs,
+ "Sets the interrupt of each interface, the addresses separated by commas. Only use if an interface has an interrupt. Otherwise, set it to zero or leave it blank.");
module_param_hw_array(regspacings, int, other, &num_regspacings, 0);
-MODULE_PARM_DESC(regspacings, "The number of bytes between the start address"
- " and each successive register used by the interface. For"
- " instance, if the start address is 0xca2 and the spacing"
- " is 2, then the second address is at 0xca4. Defaults"
- " to 1.");
+MODULE_PARM_DESC(regspacings,
+ "The number of bytes between the start address and each successive register used by the interface. For instance, if the start address is 0xca2 and the spacing is 2, then the second address is at 0xca4. Defaults to 1.");
module_param_hw_array(regsizes, int, other, &num_regsizes, 0);
-MODULE_PARM_DESC(regsizes, "The size of the specific IPMI register in bytes."
- " This should generally be 1, 2, 4, or 8 for an 8-bit,"
- " 16-bit, 32-bit, or 64-bit register. Use this if you"
- " the 8-bit IPMI register has to be read from a larger"
- " register.");
+MODULE_PARM_DESC(regsizes,
+ "The size of the specific IPMI register in bytes. This should generally be 1, 2, 4, or 8 for an 8-bit, 16-bit, 32-bit, or 64-bit register. Use this if you the 8-bit IPMI register has to be read from a larger register.");
module_param_hw_array(regshifts, int, other, &num_regshifts, 0);
-MODULE_PARM_DESC(regshifts, "The amount to shift the data read from the."
- " IPMI register, in bits. For instance, if the data"
- " is read from a 32-bit word and the IPMI data is in"
- " bit 8-15, then the shift would be 8");
+MODULE_PARM_DESC(regshifts,
+ "The amount to shift the data read from the. IPMI register, in bits. For instance, if the data is read from a 32-bit word and the IPMI data is in bit 8-15, then the shift would be 8");
module_param_hw_array(slave_addrs, int, other, &num_slave_addrs, 0);
-MODULE_PARM_DESC(slave_addrs, "Set the default IPMB slave address for"
- " the controller. Normally this is 0x20, but can be"
- " overridden by this parm. This is an array indexed"
- " by interface number.");
+MODULE_PARM_DESC(slave_addrs,
+ "Set the default IPMB slave address for the controller. Normally this is 0x20, but can be overridden by this parm. This is an array indexed by interface number.");
static void __init ipmi_hardcode_init_one(const char *si_type_str,
unsigned int i,
@@ -80,26 +62,21 @@ static void __init ipmi_hardcode_init_one(const char *si_type_str,
enum ipmi_addr_space addr_space)
{
struct ipmi_plat_data p;
+ int t;
memset(&p, 0, sizeof(p));
p.iftype = IPMI_PLAT_IF_SI;
- if (!si_type_str || !*si_type_str || strcmp(si_type_str, "kcs") == 0) {
+ if (!si_type_str || !*si_type_str) {
p.type = SI_KCS;
- } else if (strcmp(si_type_str, "smic") == 0) {
- p.type = SI_SMIC;
- } else if (strcmp(si_type_str, "bt") == 0) {
- p.type = SI_BT;
- } else if (strcmp(si_type_str, "invalid") == 0) {
- /*
- * Allow a firmware-specified interface to be
- * disabled.
- */
- p.type = SI_TYPE_INVALID;
} else {
- pr_warn("Interface type specified for interface %d, was invalid: %s\n",
- i, si_type_str);
- return;
+ t = match_string(si_to_str, -1, si_type_str);
+ if (t < 0) {
+ pr_warn("Interface type specified for interface %d, was invalid: %s\n",
+ i, si_type_str);
+ return;
+ }
+ p.type = t;
}
p.regsize = regsizes[i];
diff --git a/drivers/char/ipmi/ipmi_si_hotmod.c b/drivers/char/ipmi/ipmi_si_hotmod.c
index 4fbb4e18bae2..6b12a83ccd4c 100644
--- a/drivers/char/ipmi/ipmi_si_hotmod.c
+++ b/drivers/char/ipmi/ipmi_si_hotmod.c
@@ -17,9 +17,8 @@
static int hotmod_handler(const char *val, const struct kernel_param *kp);
module_param_call(hotmod, hotmod_handler, NULL, NULL, 0200);
-MODULE_PARM_DESC(hotmod, "Add and remove interfaces. See"
- " Documentation/driver-api/ipmi.rst in the kernel sources for the"
- " gory details.");
+MODULE_PARM_DESC(hotmod,
+ "Add and remove interfaces. See Documentation/driver-api/ipmi.rst in the kernel sources for the gory details.");
/*
* Parms come in as <op1>[:op2[:op3...]]. ops are:
@@ -185,24 +184,16 @@ static atomic_t hotmod_nr;
static int hotmod_handler(const char *val, const struct kernel_param *kp)
{
- char *str = kstrdup(val, GFP_KERNEL), *curr, *next;
int rv;
struct ipmi_plat_data h;
- unsigned int len;
- int ival;
+ char *str, *curr, *next;
+ str = kstrdup(val, GFP_KERNEL);
if (!str)
return -ENOMEM;
/* Kill any trailing spaces, as we can get a "\n" from echo. */
- len = strlen(str);
- ival = len - 1;
- while ((ival >= 0) && isspace(str[ival])) {
- str[ival] = '\0';
- ival--;
- }
-
- for (curr = str; curr; curr = next) {
+ for (curr = strstrip(str); curr; curr = next) {
enum hotmod_op op;
next = strchr(curr, ':');
@@ -231,11 +222,10 @@ static int hotmod_handler(const char *val, const struct kernel_param *kp)
if (strcmp(pdev->name, "hotmod-ipmi-si") == 0)
platform_device_unregister(pdev);
}
- if (dev)
- put_device(dev);
+ put_device(dev);
}
}
- rv = len;
+ rv = strlen(val);
out:
kfree(str);
return rv;
diff --git a/drivers/char/ipmi/ipmi_si_intf.c b/drivers/char/ipmi/ipmi_si_intf.c
index 5eac94cf4ff8..62929a3e397e 100644
--- a/drivers/char/ipmi/ipmi_si_intf.c
+++ b/drivers/char/ipmi/ipmi_si_intf.c
@@ -70,7 +70,8 @@ enum si_intf_state {
#define IPMI_BT_INTMASK_CLEAR_IRQ_BIT 2
#define IPMI_BT_INTMASK_ENABLE_IRQ_BIT 1
-static const char * const si_to_str[] = { "invalid", "kcs", "smic", "bt" };
+/* 'invalid' to allow a firmware-specified interface to be disabled */
+const char *const si_to_str[] = { "invalid", "kcs", "smic", "bt", NULL };
static bool initialized;
@@ -1169,9 +1170,8 @@ static int smi_start_processing(void *send_info,
new_smi->thread = kthread_run(ipmi_thread, new_smi,
"kipmi%d", new_smi->si_num);
if (IS_ERR(new_smi->thread)) {
- dev_notice(new_smi->io.dev, "Could not start"
- " kernel thread due to error %ld, only using"
- " timers to drive the interface\n",
+ dev_notice(new_smi->io.dev,
+ "Could not start kernel thread due to error %ld, only using timers to drive the interface\n",
PTR_ERR(new_smi->thread));
new_smi->thread = NULL;
}
@@ -1223,18 +1223,14 @@ static int smi_num; /* Used to sequence the SMIs */
static const char * const addr_space_to_str[] = { "i/o", "mem" };
module_param_array(force_kipmid, int, &num_force_kipmid, 0);
-MODULE_PARM_DESC(force_kipmid, "Force the kipmi daemon to be enabled (1) or"
- " disabled(0). Normally the IPMI driver auto-detects"
- " this, but the value may be overridden by this parm.");
+MODULE_PARM_DESC(force_kipmid,
+ "Force the kipmi daemon to be enabled (1) or disabled(0). Normally the IPMI driver auto-detects this, but the value may be overridden by this parm.");
module_param(unload_when_empty, bool, 0);
-MODULE_PARM_DESC(unload_when_empty, "Unload the module if no interfaces are"
- " specified or found, default is 1. Setting to 0"
- " is useful for hot add of devices using hotmod.");
+MODULE_PARM_DESC(unload_when_empty,
+ "Unload the module if no interfaces are specified or found, default is 1. Setting to 0 is useful for hot add of devices using hotmod.");
module_param_array(kipmid_max_busy_us, uint, &num_max_busy_us, 0644);
MODULE_PARM_DESC(kipmid_max_busy_us,
- "Max time (in microseconds) to busy-wait for IPMI data before"
- " sleeping. 0 (default) means to wait forever. Set to 100-500"
- " if kipmid is using up a lot of CPU time.");
+ "Max time (in microseconds) to busy-wait for IPMI data before sleeping. 0 (default) means to wait forever. Set to 100-500 if kipmid is using up a lot of CPU time.");
void ipmi_irq_finish_setup(struct si_sm_io *io)
{
@@ -1270,8 +1266,7 @@ int ipmi_std_irq_setup(struct si_sm_io *io)
SI_DEVICE_NAME,
io->irq_handler_data);
if (rv) {
- dev_warn(io->dev, "%s unable to claim interrupt %d,"
- " running polled\n",
+ dev_warn(io->dev, "%s unable to claim interrupt %d, running polled\n",
SI_DEVICE_NAME, io->irq);
io->irq = 0;
} else {
@@ -1346,10 +1341,8 @@ retry:
/* record completion code */
unsigned char cc = *(resp + 2);
- if ((cc == IPMI_DEVICE_IN_FW_UPDATE_ERR
- || cc == IPMI_DEVICE_IN_INIT_ERR
- || cc == IPMI_NOT_IN_MY_STATE_ERR)
- && ++retry_count <= GET_DEVICE_ID_MAX_RETRY) {
+ if (cc != IPMI_CC_NO_ERROR &&
+ ++retry_count <= GET_DEVICE_ID_MAX_RETRY) {
dev_warn(smi_info->io.dev,
"BMC returned 0x%2.2x, retry get bmc device id\n",
cc);
@@ -2207,10 +2200,6 @@ static void shutdown_smi(void *send_info)
if (smi_info->handlers)
smi_info->handlers->cleanup(smi_info->si_sm);
- if (smi_info->io.addr_source_cleanup) {
- smi_info->io.addr_source_cleanup(&smi_info->io);
- smi_info->io.addr_source_cleanup = NULL;
- }
if (smi_info->io.io_cleanup) {
smi_info->io.io_cleanup(&smi_info->io);
smi_info->io.io_cleanup = NULL;
@@ -2306,5 +2295,4 @@ module_exit(cleanup_ipmi_si);
MODULE_ALIAS("platform:dmi-ipmi-si");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Corey Minyard <minyard@mvista.com>");
-MODULE_DESCRIPTION("Interface to the IPMI driver for the KCS, SMIC, and BT"
- " system interfaces.");
+MODULE_DESCRIPTION("Interface to the IPMI driver for the KCS, SMIC, and BT system interfaces.");
diff --git a/drivers/char/ipmi/ipmi_si_pci.c b/drivers/char/ipmi/ipmi_si_pci.c
index 95bbcfba5408..74fa2055868b 100644
--- a/drivers/char/ipmi/ipmi_si_pci.c
+++ b/drivers/char/ipmi/ipmi_si_pci.c
@@ -16,18 +16,11 @@ static bool pci_registered;
static bool si_trypci = true;
module_param_named(trypci, si_trypci, bool, 0);
-MODULE_PARM_DESC(trypci, "Setting this to zero will disable the"
- " default scan of the interfaces identified via pci");
+MODULE_PARM_DESC(trypci,
+ "Setting this to zero will disable the default scan of the interfaces identified via pci");
#define PCI_DEVICE_ID_HP_MMC 0x121A
-static void ipmi_pci_cleanup(struct si_sm_io *io)
-{
- struct pci_dev *pdev = io->addr_source_data;
-
- pci_disable_device(pdev);
-}
-
static int ipmi_pci_probe_regspacing(struct si_sm_io *io)
{
if (io->si_type == SI_KCS) {
@@ -97,15 +90,12 @@ static int ipmi_pci_probe(struct pci_dev *pdev,
return -ENOMEM;
}
- rv = pci_enable_device(pdev);
+ rv = pcim_enable_device(pdev);
if (rv) {
dev_err(&pdev->dev, "couldn't enable PCI device\n");
return rv;
}
- io.addr_source_cleanup = ipmi_pci_cleanup;
- io.addr_source_data = pdev;
-
if (pci_resource_flags(pdev, 0) & IORESOURCE_IO) {
io.addr_space = IPMI_IO_ADDR_SPACE;
io.io_setup = ipmi_si_port_setup;
@@ -128,11 +118,7 @@ static int ipmi_pci_probe(struct pci_dev *pdev,
dev_info(&pdev->dev, "%pR regsize %d spacing %d irq %d\n",
&pdev->resource[0], io.regsize, io.regspacing, io.irq);
- rv = ipmi_si_add_smi(&io);
- if (rv)
- pci_disable_device(pdev);
-
- return rv;
+ return ipmi_si_add_smi(&io);
}
static void ipmi_pci_remove(struct pci_dev *pdev)
diff --git a/drivers/char/ipmi/ipmi_si_platform.c b/drivers/char/ipmi/ipmi_si_platform.c
index 129b5713f187..380a6a542890 100644
--- a/drivers/char/ipmi/ipmi_si_platform.c
+++ b/drivers/char/ipmi/ipmi_si_platform.c
@@ -34,23 +34,22 @@ static bool si_trydmi = false;
#endif
module_param_named(tryplatform, si_tryplatform, bool, 0);
-MODULE_PARM_DESC(tryplatform, "Setting this to zero will disable the"
- " default scan of the interfaces identified via platform"
- " interfaces besides ACPI, OpenFirmware, and DMI");
+MODULE_PARM_DESC(tryplatform,
+ "Setting this to zero will disable the default scan of the interfaces identified via platform interfaces besides ACPI, OpenFirmware, and DMI");
#ifdef CONFIG_ACPI
module_param_named(tryacpi, si_tryacpi, bool, 0);
-MODULE_PARM_DESC(tryacpi, "Setting this to zero will disable the"
- " default scan of the interfaces identified via ACPI");
+MODULE_PARM_DESC(tryacpi,
+ "Setting this to zero will disable the default scan of the interfaces identified via ACPI");
#endif
#ifdef CONFIG_OF
module_param_named(tryopenfirmware, si_tryopenfirmware, bool, 0);
-MODULE_PARM_DESC(tryopenfirmware, "Setting this to zero will disable the"
- " default scan of the interfaces identified via OpenFirmware");
+MODULE_PARM_DESC(tryopenfirmware,
+ "Setting this to zero will disable the default scan of the interfaces identified via OpenFirmware");
#endif
#ifdef CONFIG_DMI
module_param_named(trydmi, si_trydmi, bool, 0);
-MODULE_PARM_DESC(trydmi, "Setting this to zero will disable the"
- " default scan of the interfaces identified via DMI");
+MODULE_PARM_DESC(trydmi,
+ "Setting this to zero will disable the default scan of the interfaces identified via DMI");
#endif
#ifdef CONFIG_ACPI
@@ -85,47 +84,46 @@ static int acpi_gpe_irq_setup(struct si_sm_io *io)
ACPI_GPE_LEVEL_TRIGGERED,
&ipmi_acpi_gpe,
io);
- if (status != AE_OK) {
+ if (ACPI_FAILURE(status)) {
dev_warn(io->dev,
"Unable to claim ACPI GPE %d, running polled\n",
io->irq);
io->irq = 0;
return -EINVAL;
- } else {
- io->irq_cleanup = acpi_gpe_irq_cleanup;
- ipmi_irq_finish_setup(io);
- dev_info(io->dev, "Using ACPI GPE %d\n", io->irq);
- return 0;
}
+
+ io->irq_cleanup = acpi_gpe_irq_cleanup;
+ ipmi_irq_finish_setup(io);
+ dev_info(io->dev, "Using ACPI GPE %d\n", io->irq);
+ return 0;
}
#endif
+static void ipmi_set_addr_data_and_space(struct resource *r, struct si_sm_io *io)
+{
+ if (resource_type(r) == IORESOURCE_IO)
+ io->addr_space = IPMI_IO_ADDR_SPACE;
+ else
+ io->addr_space = IPMI_MEM_ADDR_SPACE;
+ io->addr_data = r->start;
+}
+
static struct resource *
ipmi_get_info_from_resources(struct platform_device *pdev,
struct si_sm_io *io)
{
struct resource *res, *res_second;
- res = platform_get_resource(pdev, IORESOURCE_IO, 0);
- if (res) {
- io->addr_space = IPMI_IO_ADDR_SPACE;
- } else {
- res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (res)
- io->addr_space = IPMI_MEM_ADDR_SPACE;
- }
+ res = platform_get_mem_or_io(pdev, 0);
if (!res) {
dev_err(&pdev->dev, "no I/O or memory address\n");
return NULL;
}
- io->addr_data = res->start;
+ ipmi_set_addr_data_and_space(res, io);
io->regspacing = DEFAULT_REGSPACING;
- res_second = platform_get_resource(pdev,
- (io->addr_space == IPMI_IO_ADDR_SPACE) ?
- IORESOURCE_IO : IORESOURCE_MEM,
- 1);
- if (res_second) {
+ res_second = platform_get_mem_or_io(pdev, 1);
+ if (res_second && resource_type(res_second) == resource_type(res)) {
if (res_second->start > io->addr_data)
io->regspacing = res_second->start - io->addr_data;
}
@@ -275,12 +273,7 @@ static int of_ipmi_probe(struct platform_device *pdev)
io.addr_source = SI_DEVICETREE;
io.irq_setup = ipmi_std_irq_setup;
- if (resource.flags & IORESOURCE_IO)
- io.addr_space = IPMI_IO_ADDR_SPACE;
- else
- io.addr_space = IPMI_MEM_ADDR_SPACE;
-
- io.addr_data = resource.start;
+ ipmi_set_addr_data_and_space(&resource, &io);
io.regsize = regsize ? be32_to_cpup(regsize) : DEFAULT_REGSIZE;
io.regspacing = regspacing ? be32_to_cpup(regspacing) : DEFAULT_REGSPACING;
@@ -317,32 +310,31 @@ static int find_slave_address(struct si_sm_io *io, int slave_addr)
static int acpi_ipmi_probe(struct platform_device *pdev)
{
+ struct device *dev = &pdev->dev;
struct si_sm_io io;
acpi_handle handle;
acpi_status status;
unsigned long long tmp;
struct resource *res;
- int rv = -EINVAL;
if (!si_tryacpi)
return -ENODEV;
- handle = ACPI_HANDLE(&pdev->dev);
+ handle = ACPI_HANDLE(dev);
if (!handle)
return -ENODEV;
memset(&io, 0, sizeof(io));
io.addr_source = SI_ACPI;
- dev_info(&pdev->dev, "probing via ACPI\n");
+ dev_info(dev, "probing via ACPI\n");
io.addr_info.acpi_info.acpi_handle = handle;
/* _IFT tells us the interface type: KCS, BT, etc */
status = acpi_evaluate_integer(handle, "_IFT", NULL, &tmp);
if (ACPI_FAILURE(status)) {
- dev_err(&pdev->dev,
- "Could not find ACPI IPMI interface type\n");
- goto err_free;
+ dev_err(dev, "Could not find ACPI IPMI interface type\n");
+ return -EINVAL;
}
switch (tmp) {
@@ -356,21 +348,19 @@ static int acpi_ipmi_probe(struct platform_device *pdev)
io.si_type = SI_BT;
break;
case 4: /* SSIF, just ignore */
- rv = -ENODEV;
- goto err_free;
+ return -ENODEV;
default:
- dev_info(&pdev->dev, "unknown IPMI type %lld\n", tmp);
- goto err_free;
+ dev_info(dev, "unknown IPMI type %lld\n", tmp);
+ return -EINVAL;
}
+ io.dev = dev;
io.regsize = DEFAULT_REGSIZE;
io.regshift = 0;
res = ipmi_get_info_from_resources(pdev, &io);
- if (!res) {
- rv = -EINVAL;
- goto err_free;
- }
+ if (!res)
+ return -EINVAL;
/* If _GPE exists, use it; otherwise use standard interrupts */
status = acpi_evaluate_integer(handle, "_GPE", NULL, &tmp);
@@ -388,17 +378,12 @@ static int acpi_ipmi_probe(struct platform_device *pdev)
io.slave_addr = find_slave_address(&io, io.slave_addr);
- io.dev = &pdev->dev;
-
- dev_info(io.dev, "%pR regsize %d spacing %d irq %d\n",
+ dev_info(dev, "%pR regsize %d spacing %d irq %d\n",
res, io.regsize, io.regspacing, io.irq);
request_module("acpi_ipmi");
return ipmi_si_add_smi(&io);
-
-err_free:
- return rv;
}
static const struct acpi_device_id acpi_ipmi_match[] = {
diff --git a/drivers/char/ipmi/ipmi_ssif.c b/drivers/char/ipmi/ipmi_ssif.c
index 0416b9c9d410..20d5af92966d 100644
--- a/drivers/char/ipmi/ipmi_ssif.c
+++ b/drivers/char/ipmi/ipmi_ssif.c
@@ -510,7 +510,7 @@ static int ipmi_ssif_thread(void *data)
return 0;
}
-static int ssif_i2c_send(struct ssif_info *ssif_info,
+static void ssif_i2c_send(struct ssif_info *ssif_info,
ssif_i2c_done handler,
int read_write, int command,
unsigned char *data, unsigned int size)
@@ -522,7 +522,6 @@ static int ssif_i2c_send(struct ssif_info *ssif_info,
ssif_info->i2c_data = data;
ssif_info->i2c_size = size;
complete(&ssif_info->wake_thread);
- return 0;
}
@@ -531,22 +530,12 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result,
static void start_get(struct ssif_info *ssif_info)
{
- int rv;
-
ssif_info->rtc_us_timer = 0;
ssif_info->multi_pos = 0;
- rv = ssif_i2c_send(ssif_info, msg_done_handler, I2C_SMBUS_READ,
- SSIF_IPMI_RESPONSE,
- ssif_info->recv, I2C_SMBUS_BLOCK_DATA);
- if (rv < 0) {
- /* request failed, just return the error. */
- if (ssif_info->ssif_debug & SSIF_DEBUG_MSG)
- dev_dbg(&ssif_info->client->dev,
- "Error from i2c_non_blocking_op(5)\n");
-
- msg_done_handler(ssif_info, -EIO, NULL, 0);
- }
+ ssif_i2c_send(ssif_info, msg_done_handler, I2C_SMBUS_READ,
+ SSIF_IPMI_RESPONSE,
+ ssif_info->recv, I2C_SMBUS_BLOCK_DATA);
}
static void retry_timeout(struct timer_list *t)
@@ -620,7 +609,6 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result,
{
struct ipmi_smi_msg *msg;
unsigned long oflags, *flags;
- int rv;
/*
* We are single-threaded here, so no need for a lock until we
@@ -666,17 +654,10 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result,
ssif_info->multi_len = len;
ssif_info->multi_pos = 1;
- rv = ssif_i2c_send(ssif_info, msg_done_handler, I2C_SMBUS_READ,
- SSIF_IPMI_MULTI_PART_RESPONSE_MIDDLE,
- ssif_info->recv, I2C_SMBUS_BLOCK_DATA);
- if (rv < 0) {
- if (ssif_info->ssif_debug & SSIF_DEBUG_MSG)
- dev_dbg(&ssif_info->client->dev,
- "Error from i2c_non_blocking_op(1)\n");
-
- result = -EIO;
- } else
- return;
+ ssif_i2c_send(ssif_info, msg_done_handler, I2C_SMBUS_READ,
+ SSIF_IPMI_MULTI_PART_RESPONSE_MIDDLE,
+ ssif_info->recv, I2C_SMBUS_BLOCK_DATA);
+ return;
} else if (ssif_info->multi_pos) {
/* Middle of multi-part read. Start the next transaction. */
int i;
@@ -738,19 +719,12 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result,
ssif_info->multi_pos++;
- rv = ssif_i2c_send(ssif_info, msg_done_handler,
- I2C_SMBUS_READ,
- SSIF_IPMI_MULTI_PART_RESPONSE_MIDDLE,
- ssif_info->recv,
- I2C_SMBUS_BLOCK_DATA);
- if (rv < 0) {
- if (ssif_info->ssif_debug & SSIF_DEBUG_MSG)
- dev_dbg(&ssif_info->client->dev,
- "Error from ssif_i2c_send\n");
-
- result = -EIO;
- } else
- return;
+ ssif_i2c_send(ssif_info, msg_done_handler,
+ I2C_SMBUS_READ,
+ SSIF_IPMI_MULTI_PART_RESPONSE_MIDDLE,
+ ssif_info->recv,
+ I2C_SMBUS_BLOCK_DATA);
+ return;
}
}
@@ -908,8 +882,6 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result,
static void msg_written_handler(struct ssif_info *ssif_info, int result,
unsigned char *data, unsigned int len)
{
- int rv;
-
/* We are single-threaded here, so no need for a lock. */
if (result < 0) {
ssif_info->retries_left--;
@@ -972,18 +944,9 @@ static void msg_written_handler(struct ssif_info *ssif_info, int result,
ssif_info->multi_data = NULL;
}
- rv = ssif_i2c_send(ssif_info, msg_written_handler,
- I2C_SMBUS_WRITE, cmd,
- data_to_send, I2C_SMBUS_BLOCK_DATA);
- if (rv < 0) {
- /* request failed, just return the error. */
- ssif_inc_stat(ssif_info, send_errors);
-
- if (ssif_info->ssif_debug & SSIF_DEBUG_MSG)
- dev_dbg(&ssif_info->client->dev,
- "Error from i2c_non_blocking_op(3)\n");
- msg_done_handler(ssif_info, -EIO, NULL, 0);
- }
+ ssif_i2c_send(ssif_info, msg_written_handler,
+ I2C_SMBUS_WRITE, cmd,
+ data_to_send, I2C_SMBUS_BLOCK_DATA);
} else {
/* Ready to request the result. */
unsigned long oflags, *flags;
@@ -1012,7 +975,6 @@ static void msg_written_handler(struct ssif_info *ssif_info, int result,
static int start_resend(struct ssif_info *ssif_info)
{
- int rv;
int command;
ssif_info->got_alert = false;
@@ -1034,12 +996,9 @@ static int start_resend(struct ssif_info *ssif_info)
ssif_info->data[0] = ssif_info->data_len;
}
- rv = ssif_i2c_send(ssif_info, msg_written_handler, I2C_SMBUS_WRITE,
- command, ssif_info->data, I2C_SMBUS_BLOCK_DATA);
- if (rv && (ssif_info->ssif_debug & SSIF_DEBUG_MSG))
- dev_dbg(&ssif_info->client->dev,
- "Error from i2c_non_blocking_op(4)\n");
- return rv;
+ ssif_i2c_send(ssif_info, msg_written_handler, I2C_SMBUS_WRITE,
+ command, ssif_info->data, I2C_SMBUS_BLOCK_DATA);
+ return 0;
}
static int start_send(struct ssif_info *ssif_info,
diff --git a/drivers/char/ipmi/kcs_bmc_aspeed.c b/drivers/char/ipmi/kcs_bmc_aspeed.c
index a140203c079b..eefe362f65f0 100644
--- a/drivers/char/ipmi/kcs_bmc_aspeed.c
+++ b/drivers/char/ipmi/kcs_bmc_aspeed.c
@@ -27,7 +27,6 @@
#define KCS_CHANNEL_MAX 4
-/* mapped to lpc-bmc@0 IO space */
#define LPC_HICR0 0x000
#define LPC_HICR0_LPC3E BIT(7)
#define LPC_HICR0_LPC2E BIT(6)
@@ -52,15 +51,13 @@
#define LPC_STR1 0x03C
#define LPC_STR2 0x040
#define LPC_STR3 0x044
-
-/* mapped to lpc-host@80 IO space */
-#define LPC_HICRB 0x080
+#define LPC_HICRB 0x100
#define LPC_HICRB_IBFIF4 BIT(1)
#define LPC_HICRB_LPC4E BIT(0)
-#define LPC_LADR4 0x090
-#define LPC_IDR4 0x094
-#define LPC_ODR4 0x098
-#define LPC_STR4 0x09C
+#define LPC_LADR4 0x110
+#define LPC_IDR4 0x114
+#define LPC_ODR4 0x118
+#define LPC_STR4 0x11C
struct aspeed_kcs_bmc {
struct regmap *map;
@@ -348,12 +345,20 @@ static int aspeed_kcs_probe(struct platform_device *pdev)
struct device_node *np;
int rc;
- np = pdev->dev.of_node;
+ np = dev->of_node->parent;
+ if (!of_device_is_compatible(np, "aspeed,ast2400-lpc-v2") &&
+ !of_device_is_compatible(np, "aspeed,ast2500-lpc-v2") &&
+ !of_device_is_compatible(np, "aspeed,ast2600-lpc-v2")) {
+ dev_err(dev, "unsupported LPC device binding\n");
+ return -ENODEV;
+ }
+
+ np = dev->of_node;
if (of_device_is_compatible(np, "aspeed,ast2400-kcs-bmc") ||
- of_device_is_compatible(np, "aspeed,ast2500-kcs-bmc"))
+ of_device_is_compatible(np, "aspeed,ast2500-kcs-bmc"))
kcs_bmc = aspeed_kcs_probe_of_v1(pdev);
else if (of_device_is_compatible(np, "aspeed,ast2400-kcs-bmc-v2") ||
- of_device_is_compatible(np, "aspeed,ast2500-kcs-bmc-v2"))
+ of_device_is_compatible(np, "aspeed,ast2500-kcs-bmc-v2"))
kcs_bmc = aspeed_kcs_probe_of_v2(pdev);
else
return -EINVAL;
diff --git a/drivers/char/lp.c b/drivers/char/lp.c
index 862c2fd933c7..0e22e3b0a04e 100644
--- a/drivers/char/lp.c
+++ b/drivers/char/lp.c
@@ -546,7 +546,7 @@ static int lp_open(struct inode *inode, struct file *file)
}
/* Determine if the peripheral supports ECP mode */
lp_claim_parport_or_block(&lp_table[minor]);
- if ( (lp_table[minor].dev->port->modes & PARPORT_MODE_ECP) &&
+ if ((lp_table[minor].dev->port->modes & PARPORT_MODE_ECP) &&
!parport_negotiate(lp_table[minor].dev->port,
IEEE1284_MODE_ECP)) {
printk(KERN_INFO "lp%d: ECP mode\n", minor);
@@ -590,7 +590,7 @@ static int lp_do_ioctl(unsigned int minor, unsigned int cmd,
return -ENODEV;
if ((LP_F(minor) & LP_EXIST) == 0)
return -ENODEV;
- switch ( cmd ) {
+ switch (cmd) {
case LPTIME:
if (arg > UINT_MAX / HZ)
return -EINVAL;
diff --git a/drivers/char/mem.c b/drivers/char/mem.c
index 869b9f5e8e03..15dc54fa1d47 100644
--- a/drivers/char/mem.c
+++ b/drivers/char/mem.c
@@ -403,221 +403,6 @@ static int mmap_mem(struct file *file, struct vm_area_struct *vma)
return 0;
}
-static int mmap_kmem(struct file *file, struct vm_area_struct *vma)
-{
- unsigned long pfn;
-
- /* Turn a kernel-virtual address into a physical page frame */
- pfn = __pa((u64)vma->vm_pgoff << PAGE_SHIFT) >> PAGE_SHIFT;
-
- /*
- * RED-PEN: on some architectures there is more mapped memory than
- * available in mem_map which pfn_valid checks for. Perhaps should add a
- * new macro here.
- *
- * RED-PEN: vmalloc is not supported right now.
- */
- if (!pfn_valid(pfn))
- return -EIO;
-
- vma->vm_pgoff = pfn;
- return mmap_mem(file, vma);
-}
-
-/*
- * This function reads the *virtual* memory as seen by the kernel.
- */
-static ssize_t read_kmem(struct file *file, char __user *buf,
- size_t count, loff_t *ppos)
-{
- unsigned long p = *ppos;
- ssize_t low_count, read, sz;
- char *kbuf; /* k-addr because vread() takes vmlist_lock rwlock */
- int err = 0;
-
- read = 0;
- if (p < (unsigned long) high_memory) {
- low_count = count;
- if (count > (unsigned long)high_memory - p)
- low_count = (unsigned long)high_memory - p;
-
-#ifdef __ARCH_HAS_NO_PAGE_ZERO_MAPPED
- /* we don't have page 0 mapped on sparc and m68k.. */
- if (p < PAGE_SIZE && low_count > 0) {
- sz = size_inside_page(p, low_count);
- if (clear_user(buf, sz))
- return -EFAULT;
- buf += sz;
- p += sz;
- read += sz;
- low_count -= sz;
- count -= sz;
- }
-#endif
- while (low_count > 0) {
- sz = size_inside_page(p, low_count);
-
- /*
- * On ia64 if a page has been mapped somewhere as
- * uncached, then it must also be accessed uncached
- * by the kernel or data corruption may occur
- */
- kbuf = xlate_dev_kmem_ptr((void *)p);
- if (!virt_addr_valid(kbuf))
- return -ENXIO;
-
- if (copy_to_user(buf, kbuf, sz))
- return -EFAULT;
- buf += sz;
- p += sz;
- read += sz;
- low_count -= sz;
- count -= sz;
- if (should_stop_iteration()) {
- count = 0;
- break;
- }
- }
- }
-
- if (count > 0) {
- kbuf = (char *)__get_free_page(GFP_KERNEL);
- if (!kbuf)
- return -ENOMEM;
- while (count > 0) {
- sz = size_inside_page(p, count);
- if (!is_vmalloc_or_module_addr((void *)p)) {
- err = -ENXIO;
- break;
- }
- sz = vread(kbuf, (char *)p, sz);
- if (!sz)
- break;
- if (copy_to_user(buf, kbuf, sz)) {
- err = -EFAULT;
- break;
- }
- count -= sz;
- buf += sz;
- read += sz;
- p += sz;
- if (should_stop_iteration())
- break;
- }
- free_page((unsigned long)kbuf);
- }
- *ppos = p;
- return read ? read : err;
-}
-
-
-static ssize_t do_write_kmem(unsigned long p, const char __user *buf,
- size_t count, loff_t *ppos)
-{
- ssize_t written, sz;
- unsigned long copied;
-
- written = 0;
-#ifdef __ARCH_HAS_NO_PAGE_ZERO_MAPPED
- /* we don't have page 0 mapped on sparc and m68k.. */
- if (p < PAGE_SIZE) {
- sz = size_inside_page(p, count);
- /* Hmm. Do something? */
- buf += sz;
- p += sz;
- count -= sz;
- written += sz;
- }
-#endif
-
- while (count > 0) {
- void *ptr;
-
- sz = size_inside_page(p, count);
-
- /*
- * On ia64 if a page has been mapped somewhere as uncached, then
- * it must also be accessed uncached by the kernel or data
- * corruption may occur.
- */
- ptr = xlate_dev_kmem_ptr((void *)p);
- if (!virt_addr_valid(ptr))
- return -ENXIO;
-
- copied = copy_from_user(ptr, buf, sz);
- if (copied) {
- written += sz - copied;
- if (written)
- break;
- return -EFAULT;
- }
- buf += sz;
- p += sz;
- count -= sz;
- written += sz;
- if (should_stop_iteration())
- break;
- }
-
- *ppos += written;
- return written;
-}
-
-/*
- * This function writes to the *virtual* memory as seen by the kernel.
- */
-static ssize_t write_kmem(struct file *file, const char __user *buf,
- size_t count, loff_t *ppos)
-{
- unsigned long p = *ppos;
- ssize_t wrote = 0;
- ssize_t virtr = 0;
- char *kbuf; /* k-addr because vwrite() takes vmlist_lock rwlock */
- int err = 0;
-
- if (p < (unsigned long) high_memory) {
- unsigned long to_write = min_t(unsigned long, count,
- (unsigned long)high_memory - p);
- wrote = do_write_kmem(p, buf, to_write, ppos);
- if (wrote != to_write)
- return wrote;
- p += wrote;
- buf += wrote;
- count -= wrote;
- }
-
- if (count > 0) {
- kbuf = (char *)__get_free_page(GFP_KERNEL);
- if (!kbuf)
- return wrote ? wrote : -ENOMEM;
- while (count > 0) {
- unsigned long sz = size_inside_page(p, count);
- unsigned long n;
-
- if (!is_vmalloc_or_module_addr((void *)p)) {
- err = -ENXIO;
- break;
- }
- n = copy_from_user(kbuf, buf, sz);
- if (n) {
- err = -EFAULT;
- break;
- }
- vwrite(kbuf, (char *)p, sz);
- count -= sz;
- buf += sz;
- virtr += sz;
- p += sz;
- if (should_stop_iteration())
- break;
- }
- free_page((unsigned long)kbuf);
- }
-
- *ppos = p;
- return virtr + wrote ? : err;
-}
-
static ssize_t read_port(struct file *file, char __user *buf,
size_t count, loff_t *ppos)
{
@@ -855,7 +640,6 @@ static int open_port(struct inode *inode, struct file *filp)
#define write_zero write_null
#define write_iter_zero write_iter_null
#define open_mem open_port
-#define open_kmem open_mem
static const struct file_operations __maybe_unused mem_fops = {
.llseek = memory_lseek,
@@ -869,18 +653,6 @@ static const struct file_operations __maybe_unused mem_fops = {
#endif
};
-static const struct file_operations __maybe_unused kmem_fops = {
- .llseek = memory_lseek,
- .read = read_kmem,
- .write = write_kmem,
- .mmap = mmap_kmem,
- .open = open_kmem,
-#ifndef CONFIG_MMU
- .get_unmapped_area = get_unmapped_area_mem,
- .mmap_capabilities = memory_mmap_capabilities,
-#endif
-};
-
static const struct file_operations null_fops = {
.llseek = null_lseek,
.read = read_null,
@@ -925,9 +697,6 @@ static const struct memdev {
#ifdef CONFIG_DEVMEM
[DEVMEM_MINOR] = { "mem", 0, &mem_fops, FMODE_UNSIGNED_OFFSET },
#endif
-#ifdef CONFIG_DEVKMEM
- [2] = { "kmem", 0, &kmem_fops, FMODE_UNSIGNED_OFFSET },
-#endif
[3] = { "null", 0666, &null_fops, 0 },
#ifdef CONFIG_DEVPORT
[4] = { "port", 0, &port_fops, 0 },
diff --git a/drivers/char/mwave/tp3780i.c b/drivers/char/mwave/tp3780i.c
index 5e1618a76b2a..8588b51202e5 100644
--- a/drivers/char/mwave/tp3780i.c
+++ b/drivers/char/mwave/tp3780i.c
@@ -177,14 +177,10 @@ int tp3780I_InitializeBoardData(THINKPAD_BD_DATA * pBDData)
return retval;
}
-int tp3780I_Cleanup(THINKPAD_BD_DATA * pBDData)
+void tp3780I_Cleanup(THINKPAD_BD_DATA *pBDData)
{
- int retval = 0;
-
PRINTK_2(TRACE_TP3780I,
"tp3780i::tp3780I_Cleanup entry and exit pBDData %p\n", pBDData);
-
- return retval;
}
int tp3780I_CalcResources(THINKPAD_BD_DATA * pBDData)
diff --git a/drivers/char/mwave/tp3780i.h b/drivers/char/mwave/tp3780i.h
index 07685b68538f..8bd976d42fae 100644
--- a/drivers/char/mwave/tp3780i.h
+++ b/drivers/char/mwave/tp3780i.h
@@ -91,7 +91,7 @@ int tp3780I_DisableDSP(THINKPAD_BD_DATA * pBDData);
int tp3780I_ResetDSP(THINKPAD_BD_DATA * pBDData);
int tp3780I_StartDSP(THINKPAD_BD_DATA * pBDData);
int tp3780I_QueryAbilities(THINKPAD_BD_DATA * pBDData, MW_ABILITIES * pAbilities);
-int tp3780I_Cleanup(THINKPAD_BD_DATA * pBDData);
+void tp3780I_Cleanup(THINKPAD_BD_DATA *pBDData);
int tp3780I_ReadWriteDspDStore(THINKPAD_BD_DATA * pBDData, unsigned int uOpcode,
void __user *pvBuffer, unsigned int uCount,
unsigned long ulDSPAddr);
diff --git a/drivers/char/pcmcia/synclink_cs.c b/drivers/char/pcmcia/synclink_cs.c
index 2be8d9a8eec5..3287a7627ed0 100644
--- a/drivers/char/pcmcia/synclink_cs.c
+++ b/drivers/char/pcmcia/synclink_cs.c
@@ -530,8 +530,6 @@ static int mgslpc_probe(struct pcmcia_device *link)
info->port.ops = &mgslpc_port_ops;
INIT_WORK(&info->task, bh_handler);
info->max_frame_size = 4096;
- info->port.close_delay = 5*HZ/10;
- info->port.closing_wait = 30*HZ;
init_waitqueue_head(&info->status_event_wait_q);
init_waitqueue_head(&info->event_wait_q);
spin_lock_init(&info->lock);
diff --git a/drivers/char/random.c b/drivers/char/random.c
index 0fe9e200e4c8..605969ed0f96 100644
--- a/drivers/char/random.c
+++ b/drivers/char/random.c
@@ -500,7 +500,6 @@ struct entropy_store {
unsigned short add_ptr;
unsigned short input_rotate;
int entropy_count;
- unsigned int initialized:1;
unsigned int last_data_init:1;
__u8 last_data[EXTRACT_SIZE];
};
@@ -660,7 +659,7 @@ static void process_random_ready_list(void)
*/
static void credit_entropy_bits(struct entropy_store *r, int nbits)
{
- int entropy_count, orig, has_initialized = 0;
+ int entropy_count, orig;
const int pool_size = r->poolinfo->poolfracbits;
int nfrac = nbits << ENTROPY_SHIFT;
@@ -717,23 +716,14 @@ retry:
if (cmpxchg(&r->entropy_count, orig, entropy_count) != orig)
goto retry;
- if (has_initialized) {
- r->initialized = 1;
- kill_fasync(&fasync, SIGIO, POLL_IN);
- }
-
trace_credit_entropy_bits(r->name, nbits,
entropy_count >> ENTROPY_SHIFT, _RET_IP_);
if (r == &input_pool) {
int entropy_bits = entropy_count >> ENTROPY_SHIFT;
- if (crng_init < 2) {
- if (entropy_bits < 128)
- return;
+ if (crng_init < 2 && entropy_bits >= 128)
crng_reseed(&primary_crng, r);
- entropy_bits = ENTROPY_BITS(r);
- }
}
}
@@ -819,7 +809,7 @@ static bool __init crng_init_try_arch_early(struct crng_state *crng)
static void __maybe_unused crng_initialize_secondary(struct crng_state *crng)
{
- memcpy(&crng->state[0], "expand 32-byte k", 16);
+ chacha_init_consts(crng->state);
_get_random_bytes(&crng->state[4], sizeof(__u32) * 12);
crng_init_try_arch(crng);
crng->init_time = jiffies - CRNG_RESEED_INTERVAL - 1;
@@ -827,7 +817,7 @@ static void __maybe_unused crng_initialize_secondary(struct crng_state *crng)
static void __init crng_initialize_primary(struct crng_state *crng)
{
- memcpy(&crng->state[0], "expand 32-byte k", 16);
+ chacha_init_consts(crng->state);
_extract_entropy(&input_pool, &crng->state[4], sizeof(__u32) * 12, 0);
if (crng_init_try_arch_early(crng) && trust_cpu) {
invalidate_batched_entropy();
@@ -1372,8 +1362,7 @@ retry:
}
/*
- * This function does the actual extraction for extract_entropy and
- * extract_entropy_user.
+ * This function does the actual extraction for extract_entropy.
*
* Note: we assume that .poolwords is a multiple of 16 words.
*/
diff --git a/drivers/char/toshiba.c b/drivers/char/toshiba.c
index aff0a8e44fff..776abbfd85d6 100644
--- a/drivers/char/toshiba.c
+++ b/drivers/char/toshiba.c
@@ -64,7 +64,6 @@
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Jonathan Buzzard <jonathan@buzzard.org.uk>");
MODULE_DESCRIPTION("Toshiba laptop SMM driver");
-MODULE_SUPPORTED_DEVICE("toshiba");
static DEFINE_MUTEX(tosh_mutex);
static int tosh_fn;
diff --git a/drivers/char/tpm/eventlog/acpi.c b/drivers/char/tpm/eventlog/acpi.c
index 3633ed70f48f..1b18ce5ebab1 100644
--- a/drivers/char/tpm/eventlog/acpi.c
+++ b/drivers/char/tpm/eventlog/acpi.c
@@ -41,6 +41,27 @@ struct acpi_tcpa {
};
};
+/* Check that the given log is indeed a TPM2 log. */
+static bool tpm_is_tpm2_log(void *bios_event_log, u64 len)
+{
+ struct tcg_efi_specid_event_head *efispecid;
+ struct tcg_pcr_event *event_header;
+ int n;
+
+ if (len < sizeof(*event_header))
+ return false;
+ len -= sizeof(*event_header);
+ event_header = bios_event_log;
+
+ if (len < sizeof(*efispecid))
+ return false;
+ efispecid = (struct tcg_efi_specid_event_head *)event_header->event;
+
+ n = memcmp(efispecid->signature, TCG_SPECID_SIG,
+ sizeof(TCG_SPECID_SIG));
+ return n == 0;
+}
+
/* read binary bios log */
int tpm_read_log_acpi(struct tpm_chip *chip)
{
@@ -52,6 +73,7 @@ int tpm_read_log_acpi(struct tpm_chip *chip)
struct acpi_table_tpm2 *tbl;
struct acpi_tpm2_phy *tpm2_phy;
int format;
+ int ret;
log = &chip->log;
@@ -112,6 +134,7 @@ int tpm_read_log_acpi(struct tpm_chip *chip)
log->bios_event_log_end = log->bios_event_log + len;
+ ret = -EIO;
virt = acpi_os_map_iomem(start, len);
if (!virt)
goto err;
@@ -119,11 +142,19 @@ int tpm_read_log_acpi(struct tpm_chip *chip)
memcpy_fromio(log->bios_event_log, virt, len);
acpi_os_unmap_iomem(virt, len);
+
+ if (chip->flags & TPM_CHIP_FLAG_TPM2 &&
+ !tpm_is_tpm2_log(log->bios_event_log, len)) {
+ /* try EFI log next */
+ ret = -ENODEV;
+ goto err;
+ }
+
return format;
err:
kfree(log->bios_event_log);
log->bios_event_log = NULL;
- return -EIO;
+ return ret;
}
diff --git a/drivers/char/tpm/eventlog/common.c b/drivers/char/tpm/eventlog/common.c
index 7460f230bae4..8512ec76d526 100644
--- a/drivers/char/tpm/eventlog/common.c
+++ b/drivers/char/tpm/eventlog/common.c
@@ -107,6 +107,9 @@ void tpm_bios_log_setup(struct tpm_chip *chip)
int log_version;
int rc = 0;
+ if (chip->flags & TPM_CHIP_FLAG_VIRTUAL)
+ return;
+
rc = tpm_read_log(chip);
if (rc < 0)
return;
diff --git a/drivers/char/tpm/eventlog/efi.c b/drivers/char/tpm/eventlog/efi.c
index 35229e5143ca..e6cb9d525e30 100644
--- a/drivers/char/tpm/eventlog/efi.c
+++ b/drivers/char/tpm/eventlog/efi.c
@@ -17,6 +17,7 @@ int tpm_read_log_efi(struct tpm_chip *chip)
{
struct efi_tcg2_final_events_table *final_tbl = NULL;
+ int final_events_log_size = efi_tpm_final_log_size;
struct linux_efi_tpm_eventlog *log_tbl;
struct tpm_bios_log *log;
u32 log_size;
@@ -66,12 +67,12 @@ int tpm_read_log_efi(struct tpm_chip *chip)
ret = tpm_log_version;
if (efi.tpm_final_log == EFI_INVALID_TABLE_ADDR ||
- efi_tpm_final_log_size == 0 ||
+ final_events_log_size == 0 ||
tpm_log_version != EFI_TCG2_EVENT_LOG_FORMAT_TCG_2)
goto out;
final_tbl = memremap(efi.tpm_final_log,
- sizeof(*final_tbl) + efi_tpm_final_log_size,
+ sizeof(*final_tbl) + final_events_log_size,
MEMREMAP_WB);
if (!final_tbl) {
pr_err("Could not map UEFI TPM final log\n");
@@ -80,10 +81,18 @@ int tpm_read_log_efi(struct tpm_chip *chip)
goto out;
}
- efi_tpm_final_log_size -= log_tbl->final_events_preboot_size;
+ /*
+ * The 'final events log' size excludes the 'final events preboot log'
+ * at its beginning.
+ */
+ final_events_log_size -= log_tbl->final_events_preboot_size;
+ /*
+ * Allocate memory for the 'combined log' where we will append the
+ * 'final events log' to.
+ */
tmp = krealloc(log->bios_event_log,
- log_size + efi_tpm_final_log_size,
+ log_size + final_events_log_size,
GFP_KERNEL);
if (!tmp) {
kfree(log->bios_event_log);
@@ -94,15 +103,19 @@ int tpm_read_log_efi(struct tpm_chip *chip)
log->bios_event_log = tmp;
/*
- * Copy any of the final events log that didn't also end up in the
- * main log. Events can be logged in both if events are generated
+ * Append any of the 'final events log' that didn't also end up in the
+ * 'main log'. Events can be logged in both if events are generated
* between GetEventLog() and ExitBootServices().
*/
memcpy((void *)log->bios_event_log + log_size,
final_tbl->events + log_tbl->final_events_preboot_size,
- efi_tpm_final_log_size);
+ final_events_log_size);
+ /*
+ * The size of the 'combined log' is the size of the 'main log' plus
+ * the size of the 'final events log'.
+ */
log->bios_event_log_end = log->bios_event_log +
- log_size + efi_tpm_final_log_size;
+ log_size + final_events_log_size;
out:
memunmap(final_tbl);
diff --git a/drivers/char/tpm/tpm2-cmd.c b/drivers/char/tpm/tpm2-cmd.c
index eff1f12d981a..c84d23951219 100644
--- a/drivers/char/tpm/tpm2-cmd.c
+++ b/drivers/char/tpm/tpm2-cmd.c
@@ -656,6 +656,7 @@ int tpm2_get_cc_attrs_tbl(struct tpm_chip *chip)
if (nr_commands !=
be32_to_cpup((__be32 *)&buf.data[TPM_HEADER_SIZE + 5])) {
+ rc = -EFAULT;
tpm_buf_destroy(&buf);
goto out;
}
diff --git a/drivers/char/tpm/tpm_ibmvtpm.c b/drivers/char/tpm/tpm_ibmvtpm.c
index 994385bf37c0..903604769de9 100644
--- a/drivers/char/tpm/tpm_ibmvtpm.c
+++ b/drivers/char/tpm/tpm_ibmvtpm.c
@@ -343,7 +343,7 @@ static int ibmvtpm_crq_send_init_complete(struct ibmvtpm_dev *ibmvtpm)
*
* Return: Always 0.
*/
-static int tpm_ibmvtpm_remove(struct vio_dev *vdev)
+static void tpm_ibmvtpm_remove(struct vio_dev *vdev)
{
struct tpm_chip *chip = dev_get_drvdata(&vdev->dev);
struct ibmvtpm_dev *ibmvtpm = dev_get_drvdata(&chip->dev);
@@ -372,8 +372,6 @@ static int tpm_ibmvtpm_remove(struct vio_dev *vdev)
kfree(ibmvtpm);
/* For tpm_ibmvtpm_get_desired_dma */
dev_set_drvdata(&vdev->dev, NULL);
-
- return 0;
}
/**
diff --git a/drivers/char/tpm/tpm_tis_core.c b/drivers/char/tpm/tpm_tis_core.c
index a2e0395cbe61..55b9d3965ae1 100644
--- a/drivers/char/tpm/tpm_tis_core.c
+++ b/drivers/char/tpm/tpm_tis_core.c
@@ -709,16 +709,14 @@ static int tpm_tis_gen_interrupt(struct tpm_chip *chip)
cap_t cap;
int ret;
- /* TPM 2.0 */
- if (chip->flags & TPM_CHIP_FLAG_TPM2)
- return tpm2_get_tpm_pt(chip, 0x100, &cap2, desc);
-
- /* TPM 1.2 */
ret = request_locality(chip, 0);
if (ret < 0)
return ret;
- ret = tpm1_getcap(chip, TPM_CAP_PROP_TIS_TIMEOUT, &cap, desc, 0);
+ if (chip->flags & TPM_CHIP_FLAG_TPM2)
+ ret = tpm2_get_tpm_pt(chip, 0x100, &cap2, desc);
+ else
+ ret = tpm1_getcap(chip, TPM_CAP_PROP_TIS_TIMEOUT, &cap, desc, 0);
release_locality(chip, 0);
@@ -1127,12 +1125,20 @@ int tpm_tis_resume(struct device *dev)
if (ret)
return ret;
- /* TPM 1.2 requires self-test on resume. This function actually returns
+ /*
+ * TPM 1.2 requires self-test on resume. This function actually returns
* an error code but for unknown reason it isn't handled.
*/
- if (!(chip->flags & TPM_CHIP_FLAG_TPM2))
+ if (!(chip->flags & TPM_CHIP_FLAG_TPM2)) {
+ ret = request_locality(chip, 0);
+ if (ret < 0)
+ return ret;
+
tpm1_do_selftest(chip);
+ release_locality(chip, 0);
+ }
+
return 0;
}
EXPORT_SYMBOL_GPL(tpm_tis_resume);
diff --git a/drivers/char/tpm/tpm_tis_i2c_cr50.c b/drivers/char/tpm/tpm_tis_i2c_cr50.c
index ec9a65e7887d..f19c227d20f4 100644
--- a/drivers/char/tpm/tpm_tis_i2c_cr50.c
+++ b/drivers/char/tpm/tpm_tis_i2c_cr50.c
@@ -483,6 +483,7 @@ static int tpm_cr50_i2c_tis_recv(struct tpm_chip *chip, u8 *buf, size_t buf_len)
expected = be32_to_cpup((__be32 *)(buf + 2));
if (expected > buf_len) {
dev_err(&chip->dev, "Buffer too small to receive i2c data\n");
+ rc = -E2BIG;
goto out_err;
}
diff --git a/drivers/char/ttyprintk.c b/drivers/char/ttyprintk.c
index 6a0059e508e3..93f5d11c830b 100644
--- a/drivers/char/ttyprintk.c
+++ b/drivers/char/ttyprintk.c
@@ -158,12 +158,23 @@ static int tpk_ioctl(struct tty_struct *tty,
return 0;
}
+/*
+ * TTY operations hangup function.
+ */
+static void tpk_hangup(struct tty_struct *tty)
+{
+ struct ttyprintk_port *tpkp = tty->driver_data;
+
+ tty_port_hangup(&tpkp->port);
+}
+
static const struct tty_operations ttyprintk_ops = {
.open = tpk_open,
.close = tpk_close,
.write = tpk_write,
.write_room = tpk_write_room,
.ioctl = tpk_ioctl,
+ .hangup = tpk_hangup,
};
static const struct tty_port_operations null_ops = { };
diff --git a/drivers/char/virtio_console.c b/drivers/char/virtio_console.c
index 1836cc56e357..59dfd9c421a1 100644
--- a/drivers/char/virtio_console.c
+++ b/drivers/char/virtio_console.c
@@ -1456,18 +1456,15 @@ static int add_port(struct ports_device *portdev, u32 id)
*/
send_control_msg(port, VIRTIO_CONSOLE_PORT_READY, 1);
- if (pdrvdata.debugfs_dir) {
- /*
- * Finally, create the debugfs file that we can use to
- * inspect a port's state at any time
- */
- snprintf(debugfs_name, sizeof(debugfs_name), "vport%up%u",
- port->portdev->vdev->index, id);
- port->debugfs_file = debugfs_create_file(debugfs_name, 0444,
- pdrvdata.debugfs_dir,
- port,
- &port_debugfs_fops);
- }
+ /*
+ * Finally, create the debugfs file that we can use to
+ * inspect a port's state at any time
+ */
+ snprintf(debugfs_name, sizeof(debugfs_name), "vport%up%u",
+ port->portdev->vdev->index, id);
+ port->debugfs_file = debugfs_create_file(debugfs_name, 0444,
+ pdrvdata.debugfs_dir,
+ port, &port_debugfs_fops);
return 0;
free_inbufs:
@@ -2244,8 +2241,6 @@ static int __init init(void)
}
pdrvdata.debugfs_dir = debugfs_create_dir("virtio-ports", NULL);
- if (!pdrvdata.debugfs_dir)
- pr_warn("Error creating debugfs dir for virtio-ports\n");
INIT_LIST_HEAD(&pdrvdata.consoles);
INIT_LIST_HEAD(&pdrvdata.portdevs);
diff --git a/drivers/clk/Kconfig b/drivers/clk/Kconfig
index a588d56502d4..e80918be8e9c 100644
--- a/drivers/clk/Kconfig
+++ b/drivers/clk/Kconfig
@@ -390,10 +390,12 @@ source "drivers/clk/meson/Kconfig"
source "drivers/clk/mstar/Kconfig"
source "drivers/clk/mvebu/Kconfig"
source "drivers/clk/qcom/Kconfig"
+source "drivers/clk/ralink/Kconfig"
source "drivers/clk/renesas/Kconfig"
source "drivers/clk/rockchip/Kconfig"
source "drivers/clk/samsung/Kconfig"
source "drivers/clk/sifive/Kconfig"
+source "drivers/clk/socfpga/Kconfig"
source "drivers/clk/sprd/Kconfig"
source "drivers/clk/sunxi/Kconfig"
source "drivers/clk/sunxi-ng/Kconfig"
diff --git a/drivers/clk/Makefile b/drivers/clk/Makefile
index b22ae4f81e0b..5f06879d7fe9 100644
--- a/drivers/clk/Makefile
+++ b/drivers/clk/Makefile
@@ -100,13 +100,12 @@ obj-$(CONFIG_COMMON_CLK_NXP) += nxp/
obj-$(CONFIG_MACH_PISTACHIO) += pistachio/
obj-$(CONFIG_COMMON_CLK_PXA) += pxa/
obj-$(CONFIG_COMMON_CLK_QCOM) += qcom/
+obj-y += ralink/
obj-y += renesas/
obj-$(CONFIG_ARCH_ROCKCHIP) += rockchip/
obj-$(CONFIG_COMMON_CLK_SAMSUNG) += samsung/
obj-$(CONFIG_CLK_SIFIVE) += sifive/
-obj-$(CONFIG_ARCH_SOCFPGA) += socfpga/
-obj-$(CONFIG_ARCH_AGILEX) += socfpga/
-obj-$(CONFIG_ARCH_STRATIX10) += socfpga/
+obj-y += socfpga/
obj-$(CONFIG_PLAT_SPEAR) += spear/
obj-y += sprd/
obj-$(CONFIG_ARCH_STI) += st/
diff --git a/drivers/clk/at91/sama7g5.c b/drivers/clk/at91/sama7g5.c
index a6e20b35960e..9e1ec48c4474 100644
--- a/drivers/clk/at91/sama7g5.c
+++ b/drivers/clk/at91/sama7g5.c
@@ -166,7 +166,7 @@ static const struct {
.c = &pll_characteristics,
.t = PLL_TYPE_FRAC,
/*
- * This feeds syspll_divpmcck which may feed critial parts
+ * This feeds syspll_divpmcck which may feed critical parts
* of the systems like timers. Therefore it should not be
* disabled.
*/
@@ -178,7 +178,7 @@ static const struct {
.c = &pll_characteristics,
.t = PLL_TYPE_DIV,
/*
- * This may feed critial parts of the systems like timers.
+ * This may feed critical parts of the systems like timers.
* Therefore it should not be disabled.
*/
.f = CLK_IS_CRITICAL | CLK_SET_RATE_GATE,
@@ -455,7 +455,7 @@ static const struct {
* @pp: PLL parents
* @pp_mux_table: PLL parents mux table
* @r: clock output range
- * @pp_chg_id: id in parrent array of changeable PLL parent
+ * @pp_chg_id: id in parent array of changeable PLL parent
* @pp_count: PLL parents count
* @id: clock id
*/
diff --git a/drivers/clk/bcm/clk-raspberrypi.c b/drivers/clk/bcm/clk-raspberrypi.c
index f89b9cfc4309..dd3b71eafabf 100644
--- a/drivers/clk/bcm/clk-raspberrypi.c
+++ b/drivers/clk/bcm/clk-raspberrypi.c
@@ -314,7 +314,7 @@ static int raspberrypi_clk_probe(struct platform_device *pdev)
return -ENOENT;
}
- firmware = rpi_firmware_get(firmware_node);
+ firmware = devm_rpi_firmware_get(&pdev->dev, firmware_node);
of_node_put(firmware_node);
if (!firmware)
return -EPROBE_DEFER;
diff --git a/drivers/clk/clk-ast2600.c b/drivers/clk/clk-ast2600.c
index a55b37fc2c8b..bc3be5f3eae1 100644
--- a/drivers/clk/clk-ast2600.c
+++ b/drivers/clk/clk-ast2600.c
@@ -61,10 +61,10 @@ static void __iomem *scu_g6_base;
static const struct aspeed_gate_data aspeed_g6_gates[] = {
/* clk rst name parent flags */
[ASPEED_CLK_GATE_MCLK] = { 0, -1, "mclk-gate", "mpll", CLK_IS_CRITICAL }, /* SDRAM */
- [ASPEED_CLK_GATE_ECLK] = { 1, -1, "eclk-gate", "eclk", 0 }, /* Video Engine */
+ [ASPEED_CLK_GATE_ECLK] = { 1, 6, "eclk-gate", "eclk", 0 }, /* Video Engine */
[ASPEED_CLK_GATE_GCLK] = { 2, 7, "gclk-gate", NULL, 0 }, /* 2D engine */
/* vclk parent - dclk/d1clk/hclk/mclk */
- [ASPEED_CLK_GATE_VCLK] = { 3, 6, "vclk-gate", NULL, 0 }, /* Video Capture */
+ [ASPEED_CLK_GATE_VCLK] = { 3, -1, "vclk-gate", NULL, 0 }, /* Video Capture */
[ASPEED_CLK_GATE_BCLK] = { 4, 8, "bclk-gate", "bclk", 0 }, /* PCIe/PCI */
/* From dpll */
[ASPEED_CLK_GATE_DCLK] = { 5, -1, "dclk-gate", NULL, CLK_IS_CRITICAL }, /* DAC */
diff --git a/drivers/clk/clk-bd718x7.c b/drivers/clk/clk-bd718x7.c
index 17d90e09f1c0..d9e70e506d18 100644
--- a/drivers/clk/clk-bd718x7.c
+++ b/drivers/clk/clk-bd718x7.c
@@ -13,6 +13,8 @@
#include <linux/regmap.h>
/* clk control registers */
+/* BD71815 */
+#define BD71815_REG_OUT32K 0x1d
/* BD70528 */
#define BD70528_REG_OUT32K 0x2c
/* BD71828 */
@@ -118,6 +120,10 @@ static int bd71837_clk_probe(struct platform_device *pdev)
c->reg = BD70528_REG_OUT32K;
c->mask = CLK_OUT_EN_MASK;
break;
+ case ROHM_CHIP_TYPE_BD71815:
+ c->reg = BD71815_REG_OUT32K;
+ c->mask = CLK_OUT_EN_MASK;
+ break;
default:
dev_err(&pdev->dev, "Unknown clk chip\n");
return -EINVAL;
@@ -146,6 +152,7 @@ static const struct platform_device_id bd718x7_clk_id[] = {
{ "bd71847-clk", ROHM_CHIP_TYPE_BD71847 },
{ "bd70528-clk", ROHM_CHIP_TYPE_BD70528 },
{ "bd71828-clk", ROHM_CHIP_TYPE_BD71828 },
+ { "bd71815-clk", ROHM_CHIP_TYPE_BD71815 },
{ },
};
MODULE_DEVICE_TABLE(platform, bd718x7_clk_id);
@@ -161,6 +168,6 @@ static struct platform_driver bd71837_clk = {
module_platform_driver(bd71837_clk);
MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
-MODULE_DESCRIPTION("BD71837/BD71847/BD70528 chip clk driver");
+MODULE_DESCRIPTION("BD718(15/18/28/37/47/50) and BD70528 chip clk driver");
MODULE_LICENSE("GPL");
MODULE_ALIAS("platform:bd718xx-clk");
diff --git a/drivers/clk/clk-fixed-factor.c b/drivers/clk/clk-fixed-factor.c
index 4f7bf3929d6d..4e4b6d367612 100644
--- a/drivers/clk/clk-fixed-factor.c
+++ b/drivers/clk/clk-fixed-factor.c
@@ -66,7 +66,14 @@ EXPORT_SYMBOL_GPL(clk_fixed_factor_ops);
static void devm_clk_hw_register_fixed_factor_release(struct device *dev, void *res)
{
- clk_hw_unregister_fixed_factor(&((struct clk_fixed_factor *)res)->hw);
+ struct clk_fixed_factor *fix = res;
+
+ /*
+ * We can not use clk_hw_unregister_fixed_factor, since it will kfree()
+ * the hw, resulting in double free. Just unregister the hw and let
+ * devres code kfree() it.
+ */
+ clk_hw_unregister(&fix->hw);
}
static struct clk_hw *
diff --git a/drivers/clk/clk-mux.c b/drivers/clk/clk-mux.c
index e54e79714818..20582aae7a35 100644
--- a/drivers/clk/clk-mux.c
+++ b/drivers/clk/clk-mux.c
@@ -8,6 +8,7 @@
*/
#include <linux/clk-provider.h>
+#include <linux/device.h>
#include <linux/module.h>
#include <linux/slab.h>
#include <linux/io.h>
@@ -206,6 +207,40 @@ struct clk_hw *__clk_hw_register_mux(struct device *dev, struct device_node *np,
}
EXPORT_SYMBOL_GPL(__clk_hw_register_mux);
+static void devm_clk_hw_release_mux(struct device *dev, void *res)
+{
+ clk_hw_unregister_mux(*(struct clk_hw **)res);
+}
+
+struct clk_hw *__devm_clk_hw_register_mux(struct device *dev, struct device_node *np,
+ const char *name, u8 num_parents,
+ const char * const *parent_names,
+ const struct clk_hw **parent_hws,
+ const struct clk_parent_data *parent_data,
+ unsigned long flags, void __iomem *reg, u8 shift, u32 mask,
+ u8 clk_mux_flags, u32 *table, spinlock_t *lock)
+{
+ struct clk_hw **ptr, *hw;
+
+ ptr = devres_alloc(devm_clk_hw_release_mux, sizeof(*ptr), GFP_KERNEL);
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+
+ hw = __clk_hw_register_mux(dev, np, name, num_parents, parent_names, parent_hws,
+ parent_data, flags, reg, shift, mask,
+ clk_mux_flags, table, lock);
+
+ if (!IS_ERR(hw)) {
+ *ptr = hw;
+ devres_add(dev, ptr);
+ } else {
+ devres_free(ptr);
+ }
+
+ return hw;
+}
+EXPORT_SYMBOL_GPL(__devm_clk_hw_register_mux);
+
struct clk *clk_register_mux_table(struct device *dev, const char *name,
const char * const *parent_names, u8 num_parents,
unsigned long flags, void __iomem *reg, u8 shift, u32 mask,
diff --git a/drivers/clk/clk-scmi.c b/drivers/clk/clk-scmi.c
index c754dfbb73fd..1e357d364ca2 100644
--- a/drivers/clk/clk-scmi.c
+++ b/drivers/clk/clk-scmi.c
@@ -2,7 +2,7 @@
/*
* System Control and Power Interface (SCMI) Protocol based clock driver
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#include <linux/clk-provider.h>
@@ -13,11 +13,13 @@
#include <linux/scmi_protocol.h>
#include <asm/div64.h>
+static const struct scmi_clk_proto_ops *scmi_proto_clk_ops;
+
struct scmi_clk {
u32 id;
struct clk_hw hw;
const struct scmi_clock_info *info;
- const struct scmi_handle *handle;
+ const struct scmi_protocol_handle *ph;
};
#define to_scmi_clk(clk) container_of(clk, struct scmi_clk, hw)
@@ -29,7 +31,7 @@ static unsigned long scmi_clk_recalc_rate(struct clk_hw *hw,
u64 rate;
struct scmi_clk *clk = to_scmi_clk(hw);
- ret = clk->handle->clk_ops->rate_get(clk->handle, clk->id, &rate);
+ ret = scmi_proto_clk_ops->rate_get(clk->ph, clk->id, &rate);
if (ret)
return 0;
return rate;
@@ -69,21 +71,21 @@ static int scmi_clk_set_rate(struct clk_hw *hw, unsigned long rate,
{
struct scmi_clk *clk = to_scmi_clk(hw);
- return clk->handle->clk_ops->rate_set(clk->handle, clk->id, rate);
+ return scmi_proto_clk_ops->rate_set(clk->ph, clk->id, rate);
}
static int scmi_clk_enable(struct clk_hw *hw)
{
struct scmi_clk *clk = to_scmi_clk(hw);
- return clk->handle->clk_ops->enable(clk->handle, clk->id);
+ return scmi_proto_clk_ops->enable(clk->ph, clk->id);
}
static void scmi_clk_disable(struct clk_hw *hw)
{
struct scmi_clk *clk = to_scmi_clk(hw);
- clk->handle->clk_ops->disable(clk->handle, clk->id);
+ scmi_proto_clk_ops->disable(clk->ph, clk->id);
}
static const struct clk_ops scmi_clk_ops = {
@@ -142,11 +144,17 @@ static int scmi_clocks_probe(struct scmi_device *sdev)
struct device *dev = &sdev->dev;
struct device_node *np = dev->of_node;
const struct scmi_handle *handle = sdev->handle;
+ struct scmi_protocol_handle *ph;
- if (!handle || !handle->clk_ops)
+ if (!handle)
return -ENODEV;
- count = handle->clk_ops->count_get(handle);
+ scmi_proto_clk_ops =
+ handle->devm_protocol_get(sdev, SCMI_PROTOCOL_CLOCK, &ph);
+ if (IS_ERR(scmi_proto_clk_ops))
+ return PTR_ERR(scmi_proto_clk_ops);
+
+ count = scmi_proto_clk_ops->count_get(ph);
if (count < 0) {
dev_err(dev, "%pOFn: invalid clock output count\n", np);
return -EINVAL;
@@ -167,14 +175,14 @@ static int scmi_clocks_probe(struct scmi_device *sdev)
if (!sclk)
return -ENOMEM;
- sclk->info = handle->clk_ops->info_get(handle, idx);
+ sclk->info = scmi_proto_clk_ops->info_get(ph, idx);
if (!sclk->info) {
dev_dbg(dev, "invalid clock info for idx %d\n", idx);
continue;
}
sclk->id = idx;
- sclk->handle = handle;
+ sclk->ph = ph;
err = scmi_clk_ops_init(dev, sclk);
if (err) {
diff --git a/drivers/clk/clk.c b/drivers/clk/clk.c
index 5052541a0986..65508eb89ec9 100644
--- a/drivers/clk/clk.c
+++ b/drivers/clk/clk.c
@@ -1330,7 +1330,7 @@ static int clk_core_determine_round_nolock(struct clk_core *core,
return 0;
/*
- * At this point, core protection will be disabled if
+ * At this point, core protection will be disabled
* - if the provider is not protected at all
* - if the calling consumer is the only one which has exclusivity
* over the provider
@@ -2078,12 +2078,8 @@ static void clk_change_rate(struct clk_core *core)
return;
if (core->flags & CLK_SET_RATE_UNGATE) {
- unsigned long flags;
-
clk_core_prepare(core);
- flags = clk_enable_lock();
- clk_core_enable(core);
- clk_enable_unlock(flags);
+ clk_core_enable_lock(core);
}
if (core->new_parent && core->new_parent != core->parent) {
@@ -2116,11 +2112,7 @@ static void clk_change_rate(struct clk_core *core)
core->rate = clk_recalc(core, best_parent_rate);
if (core->flags & CLK_SET_RATE_UNGATE) {
- unsigned long flags;
-
- flags = clk_enable_lock();
- clk_core_disable(core);
- clk_enable_unlock(flags);
+ clk_core_disable_lock(core);
clk_core_unprepare(core);
}
@@ -3564,8 +3556,6 @@ static int __clk_core_init(struct clk_core *core)
* reparenting clocks
*/
if (core->flags & CLK_IS_CRITICAL) {
- unsigned long flags;
-
ret = clk_core_prepare(core);
if (ret) {
pr_warn("%s: critical clk '%s' failed to prepare\n",
@@ -3573,9 +3563,7 @@ static int __clk_core_init(struct clk_core *core)
goto out;
}
- flags = clk_enable_lock();
- ret = clk_core_enable(core);
- clk_enable_unlock(flags);
+ ret = clk_core_enable_lock(core);
if (ret) {
pr_warn("%s: critical clk '%s' failed to enable\n",
__func__, core->name);
@@ -4357,20 +4345,19 @@ int clk_notifier_register(struct clk *clk, struct notifier_block *nb)
/* search the list of notifiers for this clk */
list_for_each_entry(cn, &clk_notifier_list, node)
if (cn->clk == clk)
- break;
+ goto found;
/* if clk wasn't in the notifier list, allocate new clk_notifier */
- if (cn->clk != clk) {
- cn = kzalloc(sizeof(*cn), GFP_KERNEL);
- if (!cn)
- goto out;
+ cn = kzalloc(sizeof(*cn), GFP_KERNEL);
+ if (!cn)
+ goto out;
- cn->clk = clk;
- srcu_init_notifier_head(&cn->notifier_head);
+ cn->clk = clk;
+ srcu_init_notifier_head(&cn->notifier_head);
- list_add(&cn->node, &clk_notifier_list);
- }
+ list_add(&cn->node, &clk_notifier_list);
+found:
ret = srcu_notifier_chain_register(&cn->notifier_head, nb);
clk->core->notifier_count++;
@@ -4395,32 +4382,28 @@ EXPORT_SYMBOL_GPL(clk_notifier_register);
*/
int clk_notifier_unregister(struct clk *clk, struct notifier_block *nb)
{
- struct clk_notifier *cn = NULL;
- int ret = -EINVAL;
+ struct clk_notifier *cn;
+ int ret = -ENOENT;
if (!clk || !nb)
return -EINVAL;
clk_prepare_lock();
- list_for_each_entry(cn, &clk_notifier_list, node)
- if (cn->clk == clk)
- break;
-
- if (cn->clk == clk) {
- ret = srcu_notifier_chain_unregister(&cn->notifier_head, nb);
+ list_for_each_entry(cn, &clk_notifier_list, node) {
+ if (cn->clk == clk) {
+ ret = srcu_notifier_chain_unregister(&cn->notifier_head, nb);
- clk->core->notifier_count--;
+ clk->core->notifier_count--;
- /* XXX the notifier code should handle this better */
- if (!cn->notifier_head.head) {
- srcu_cleanup_notifier_head(&cn->notifier_head);
- list_del(&cn->node);
- kfree(cn);
+ /* XXX the notifier code should handle this better */
+ if (!cn->notifier_head.head) {
+ srcu_cleanup_notifier_head(&cn->notifier_head);
+ list_del(&cn->node);
+ kfree(cn);
+ }
+ break;
}
-
- } else {
- ret = -ENOENT;
}
clk_prepare_unlock();
@@ -4557,6 +4540,9 @@ int of_clk_add_provider(struct device_node *np,
struct of_clk_provider *cp;
int ret;
+ if (!np)
+ return 0;
+
cp = kzalloc(sizeof(*cp), GFP_KERNEL);
if (!cp)
return -ENOMEM;
@@ -4596,6 +4582,9 @@ int of_clk_add_hw_provider(struct device_node *np,
struct of_clk_provider *cp;
int ret;
+ if (!np)
+ return 0;
+
cp = kzalloc(sizeof(*cp), GFP_KERNEL);
if (!cp)
return -ENOMEM;
@@ -4615,6 +4604,8 @@ int of_clk_add_hw_provider(struct device_node *np,
if (ret < 0)
of_clk_del_provider(np);
+ fwnode_dev_initialized(&np->fwnode, true);
+
return ret;
}
EXPORT_SYMBOL_GPL(of_clk_add_hw_provider);
@@ -4691,6 +4682,9 @@ void of_clk_del_provider(struct device_node *np)
{
struct of_clk_provider *cp;
+ if (!np)
+ return;
+
mutex_lock(&of_clk_mutex);
list_for_each_entry(cp, &of_clk_providers, link) {
if (cp->node == np) {
diff --git a/drivers/clk/imx/clk-imx25.c b/drivers/clk/imx/clk-imx25.c
index a66cabfbf94f..66192fe0a898 100644
--- a/drivers/clk/imx/clk-imx25.c
+++ b/drivers/clk/imx/clk-imx25.c
@@ -73,16 +73,6 @@ enum mx25_clks {
static struct clk *clk[clk_max];
-static struct clk ** const uart_clks[] __initconst = {
- &clk[uart_ipg_per],
- &clk[uart1_ipg],
- &clk[uart2_ipg],
- &clk[uart3_ipg],
- &clk[uart4_ipg],
- &clk[uart5_ipg],
- NULL
-};
-
static int __init __mx25_clocks_init(void __iomem *ccm_base)
{
BUG_ON(!ccm_base);
@@ -228,7 +218,7 @@ static int __init __mx25_clocks_init(void __iomem *ccm_base)
*/
clk_set_parent(clk[cko_sel], clk[ipg]);
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(6);
return 0;
}
diff --git a/drivers/clk/imx/clk-imx27.c b/drivers/clk/imx/clk-imx27.c
index 5585ded8b8c6..56a5fc402b10 100644
--- a/drivers/clk/imx/clk-imx27.c
+++ b/drivers/clk/imx/clk-imx27.c
@@ -49,17 +49,6 @@ static const char *ssi_sel_clks[] = { "spll_gate", "mpll", };
static struct clk *clk[IMX27_CLK_MAX];
static struct clk_onecell_data clk_data;
-static struct clk ** const uart_clks[] __initconst = {
- &clk[IMX27_CLK_PER1_GATE],
- &clk[IMX27_CLK_UART1_IPG_GATE],
- &clk[IMX27_CLK_UART2_IPG_GATE],
- &clk[IMX27_CLK_UART3_IPG_GATE],
- &clk[IMX27_CLK_UART4_IPG_GATE],
- &clk[IMX27_CLK_UART5_IPG_GATE],
- &clk[IMX27_CLK_UART6_IPG_GATE],
- NULL
-};
-
static void __init _mx27_clocks_init(unsigned long fref)
{
BUG_ON(!ccm);
@@ -176,7 +165,7 @@ static void __init _mx27_clocks_init(unsigned long fref)
clk_prepare_enable(clk[IMX27_CLK_EMI_AHB_GATE]);
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(7);
imx_print_silicon_rev("i.MX27", mx27_revision());
}
diff --git a/drivers/clk/imx/clk-imx35.c b/drivers/clk/imx/clk-imx35.c
index c1df03665c09..0fe5ac210156 100644
--- a/drivers/clk/imx/clk-imx35.c
+++ b/drivers/clk/imx/clk-imx35.c
@@ -82,14 +82,6 @@ enum mx35_clks {
static struct clk *clk[clk_max];
-static struct clk ** const uart_clks[] __initconst = {
- &clk[ipg],
- &clk[uart1_gate],
- &clk[uart2_gate],
- &clk[uart3_gate],
- NULL
-};
-
static void __init _mx35_clocks_init(void)
{
void __iomem *base;
@@ -243,7 +235,7 @@ static void __init _mx35_clocks_init(void)
*/
clk_prepare_enable(clk[scc_gate]);
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(4);
imx_print_silicon_rev("i.MX35", mx35_revision());
}
diff --git a/drivers/clk/imx/clk-imx5.c b/drivers/clk/imx/clk-imx5.c
index 01e079b81026..e4493846454d 100644
--- a/drivers/clk/imx/clk-imx5.c
+++ b/drivers/clk/imx/clk-imx5.c
@@ -128,30 +128,6 @@ static const char *ieee1588_sels[] = { "pll3_sw", "pll4_sw", "dummy" /* usbphy2_
static struct clk *clk[IMX5_CLK_END];
static struct clk_onecell_data clk_data;
-static struct clk ** const uart_clks_mx51[] __initconst = {
- &clk[IMX5_CLK_UART1_IPG_GATE],
- &clk[IMX5_CLK_UART1_PER_GATE],
- &clk[IMX5_CLK_UART2_IPG_GATE],
- &clk[IMX5_CLK_UART2_PER_GATE],
- &clk[IMX5_CLK_UART3_IPG_GATE],
- &clk[IMX5_CLK_UART3_PER_GATE],
- NULL
-};
-
-static struct clk ** const uart_clks_mx50_mx53[] __initconst = {
- &clk[IMX5_CLK_UART1_IPG_GATE],
- &clk[IMX5_CLK_UART1_PER_GATE],
- &clk[IMX5_CLK_UART2_IPG_GATE],
- &clk[IMX5_CLK_UART2_PER_GATE],
- &clk[IMX5_CLK_UART3_IPG_GATE],
- &clk[IMX5_CLK_UART3_PER_GATE],
- &clk[IMX5_CLK_UART4_IPG_GATE],
- &clk[IMX5_CLK_UART4_PER_GATE],
- &clk[IMX5_CLK_UART5_IPG_GATE],
- &clk[IMX5_CLK_UART5_PER_GATE],
- NULL
-};
-
static void __init mx5_clocks_common_init(void __iomem *ccm_base)
{
clk[IMX5_CLK_DUMMY] = imx_clk_fixed("dummy", 0);
@@ -382,7 +358,7 @@ static void __init mx50_clocks_init(struct device_node *np)
r = clk_round_rate(clk[IMX5_CLK_USBOH3_PER_GATE], 54000000);
clk_set_rate(clk[IMX5_CLK_USBOH3_PER_GATE], r);
- imx_register_uart_clocks(uart_clks_mx50_mx53);
+ imx_register_uart_clocks(5);
}
CLK_OF_DECLARE(imx50_ccm, "fsl,imx50-ccm", mx50_clocks_init);
@@ -488,7 +464,7 @@ static void __init mx51_clocks_init(struct device_node *np)
val |= 1 << 23;
writel(val, MXC_CCM_CLPCR);
- imx_register_uart_clocks(uart_clks_mx51);
+ imx_register_uart_clocks(3);
}
CLK_OF_DECLARE(imx51_ccm, "fsl,imx51-ccm", mx51_clocks_init);
@@ -633,6 +609,6 @@ static void __init mx53_clocks_init(struct device_node *np)
r = clk_round_rate(clk[IMX5_CLK_USBOH3_PER_GATE], 54000000);
clk_set_rate(clk[IMX5_CLK_USBOH3_PER_GATE], r);
- imx_register_uart_clocks(uart_clks_mx50_mx53);
+ imx_register_uart_clocks(5);
}
CLK_OF_DECLARE(imx53_ccm, "fsl,imx53-ccm", mx53_clocks_init);
diff --git a/drivers/clk/imx/clk-imx6q.c b/drivers/clk/imx/clk-imx6q.c
index 521d6136d22c..496900de0b0b 100644
--- a/drivers/clk/imx/clk-imx6q.c
+++ b/drivers/clk/imx/clk-imx6q.c
@@ -140,13 +140,6 @@ static inline int clk_on_imx6dl(void)
return of_machine_is_compatible("fsl,imx6dl");
}
-static const int uart_clk_ids[] __initconst = {
- IMX6QDL_CLK_UART_IPG,
- IMX6QDL_CLK_UART_SERIAL,
-};
-
-static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
-
static int ldb_di_sel_by_clock_id(int clock_id)
{
switch (clock_id) {
@@ -440,7 +433,6 @@ static void __init imx6q_clocks_init(struct device_node *ccm_node)
struct device_node *np;
void __iomem *anatop_base, *base;
int ret;
- int i;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX6QDL_CLK_END), GFP_KERNEL);
@@ -982,12 +974,6 @@ static void __init imx6q_clocks_init(struct device_node *ccm_node)
hws[IMX6QDL_CLK_PLL3_USB_OTG]->clk);
}
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_clks[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(1);
}
CLK_OF_DECLARE(imx6q, "fsl,imx6q-ccm", imx6q_clocks_init);
diff --git a/drivers/clk/imx/clk-imx6sl.c b/drivers/clk/imx/clk-imx6sl.c
index 29eab05c9068..277365970320 100644
--- a/drivers/clk/imx/clk-imx6sl.c
+++ b/drivers/clk/imx/clk-imx6sl.c
@@ -179,19 +179,11 @@ void imx6sl_set_wait_clk(bool enter)
imx6sl_enable_pll_arm(false);
}
-static const int uart_clk_ids[] __initconst = {
- IMX6SL_CLK_UART,
- IMX6SL_CLK_UART_SERIAL,
-};
-
-static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
-
static void __init imx6sl_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
int ret;
- int i;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX6SL_CLK_END), GFP_KERNEL);
@@ -448,12 +440,6 @@ static void __init imx6sl_clocks_init(struct device_node *ccm_node)
clk_set_parent(hws[IMX6SL_CLK_LCDIF_AXI_SEL]->clk,
hws[IMX6SL_CLK_PLL2_PFD2]->clk);
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_clks[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(2);
}
CLK_OF_DECLARE(imx6sl, "fsl,imx6sl-ccm", imx6sl_clocks_init);
diff --git a/drivers/clk/imx/clk-imx6sll.c b/drivers/clk/imx/clk-imx6sll.c
index 8e8288bda4d0..31d777f30039 100644
--- a/drivers/clk/imx/clk-imx6sll.c
+++ b/drivers/clk/imx/clk-imx6sll.c
@@ -76,26 +76,10 @@ static u32 share_count_ssi1;
static u32 share_count_ssi2;
static u32 share_count_ssi3;
-static const int uart_clk_ids[] __initconst = {
- IMX6SLL_CLK_UART1_IPG,
- IMX6SLL_CLK_UART1_SERIAL,
- IMX6SLL_CLK_UART2_IPG,
- IMX6SLL_CLK_UART2_SERIAL,
- IMX6SLL_CLK_UART3_IPG,
- IMX6SLL_CLK_UART3_SERIAL,
- IMX6SLL_CLK_UART4_IPG,
- IMX6SLL_CLK_UART4_SERIAL,
- IMX6SLL_CLK_UART5_IPG,
- IMX6SLL_CLK_UART5_SERIAL,
-};
-
-static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
-
static void __init imx6sll_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
- int i;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX6SLL_CLK_END), GFP_KERNEL);
@@ -356,13 +340,7 @@ static void __init imx6sll_clocks_init(struct device_node *ccm_node)
of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_clks[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(5);
/* Lower the AHB clock rate before changing the clock source. */
clk_set_rate(hws[IMX6SLL_CLK_AHB]->clk, 99000000);
diff --git a/drivers/clk/imx/clk-imx6sx.c b/drivers/clk/imx/clk-imx6sx.c
index 20dcce526d07..fc1bd23d4583 100644
--- a/drivers/clk/imx/clk-imx6sx.c
+++ b/drivers/clk/imx/clk-imx6sx.c
@@ -117,18 +117,10 @@ static u32 share_count_ssi3;
static u32 share_count_sai1;
static u32 share_count_sai2;
-static const int uart_clk_ids[] __initconst = {
- IMX6SX_CLK_UART_IPG,
- IMX6SX_CLK_UART_SERIAL,
-};
-
-static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
-
static void __init imx6sx_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
- int i;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX6SX_CLK_CLK_END), GFP_KERNEL);
@@ -556,12 +548,6 @@ static void __init imx6sx_clocks_init(struct device_node *ccm_node)
clk_set_parent(hws[IMX6SX_CLK_QSPI1_SEL]->clk, hws[IMX6SX_CLK_PLL2_BUS]->clk);
clk_set_parent(hws[IMX6SX_CLK_QSPI2_SEL]->clk, hws[IMX6SX_CLK_PLL2_BUS]->clk);
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_clks[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(2);
}
CLK_OF_DECLARE(imx6sx, "fsl,imx6sx-ccm", imx6sx_clocks_init);
diff --git a/drivers/clk/imx/clk-imx7d.c b/drivers/clk/imx/clk-imx7d.c
index 22d24a6a05e7..c4e0f1c07192 100644
--- a/drivers/clk/imx/clk-imx7d.c
+++ b/drivers/clk/imx/clk-imx7d.c
@@ -377,23 +377,10 @@ static const char *pll_video_bypass_sel[] = { "pll_video_main", "pll_video_main_
static struct clk_hw **hws;
static struct clk_hw_onecell_data *clk_hw_data;
-static const int uart_clk_ids[] __initconst = {
- IMX7D_UART1_ROOT_CLK,
- IMX7D_UART2_ROOT_CLK,
- IMX7D_UART3_ROOT_CLK,
- IMX7D_UART4_ROOT_CLK,
- IMX7D_UART5_ROOT_CLK,
- IMX7D_UART6_ROOT_CLK,
- IMX7D_UART7_ROOT_CLK,
-};
-
-static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1] __initdata;
-
static void __init imx7d_clocks_init(struct device_node *ccm_node)
{
struct device_node *np;
void __iomem *base;
- int i;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX7D_CLK_END), GFP_KERNEL);
@@ -897,14 +884,7 @@ static void __init imx7d_clocks_init(struct device_node *ccm_node)
hws[IMX7D_USB1_MAIN_480M_CLK] = imx_clk_hw_fixed_factor("pll_usb1_main_clk", "osc", 20, 1);
hws[IMX7D_USB_MAIN_480M_CLK] = imx_clk_hw_fixed_factor("pll_usb_main_clk", "osc", 20, 1);
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_clks[i] = &hws[index]->clk;
- }
-
-
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(7);
}
CLK_OF_DECLARE(imx7d, "fsl,imx7d-ccm", imx7d_clocks_init);
diff --git a/drivers/clk/imx/clk-imx7ulp.c b/drivers/clk/imx/clk-imx7ulp.c
index 634c0b6636b0..779e09105da7 100644
--- a/drivers/clk/imx/clk-imx7ulp.c
+++ b/drivers/clk/imx/clk-imx7ulp.c
@@ -43,19 +43,6 @@ static const struct clk_div_table ulp_div_table[] = {
{ /* sentinel */ },
};
-static const int pcc2_uart_clk_ids[] __initconst = {
- IMX7ULP_CLK_LPUART4,
- IMX7ULP_CLK_LPUART5,
-};
-
-static const int pcc3_uart_clk_ids[] __initconst = {
- IMX7ULP_CLK_LPUART6,
- IMX7ULP_CLK_LPUART7,
-};
-
-static struct clk **pcc2_uart_clks[ARRAY_SIZE(pcc2_uart_clk_ids) + 1] __initdata;
-static struct clk **pcc3_uart_clks[ARRAY_SIZE(pcc3_uart_clk_ids) + 1] __initdata;
-
static void __init imx7ulp_clk_scg1_init(struct device_node *np)
{
struct clk_hw_onecell_data *clk_data;
@@ -150,7 +137,6 @@ static void __init imx7ulp_clk_pcc2_init(struct device_node *np)
struct clk_hw_onecell_data *clk_data;
struct clk_hw **hws;
void __iomem *base;
- int i;
clk_data = kzalloc(struct_size(clk_data, hws, IMX7ULP_CLK_PCC2_END),
GFP_KERNEL);
@@ -190,13 +176,7 @@ static void __init imx7ulp_clk_pcc2_init(struct device_node *np)
of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_data);
- for (i = 0; i < ARRAY_SIZE(pcc2_uart_clk_ids); i++) {
- int index = pcc2_uart_clk_ids[i];
-
- pcc2_uart_clks[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(pcc2_uart_clks);
+ imx_register_uart_clocks(2);
}
CLK_OF_DECLARE(imx7ulp_clk_pcc2, "fsl,imx7ulp-pcc2", imx7ulp_clk_pcc2_init);
@@ -205,7 +185,6 @@ static void __init imx7ulp_clk_pcc3_init(struct device_node *np)
struct clk_hw_onecell_data *clk_data;
struct clk_hw **hws;
void __iomem *base;
- int i;
clk_data = kzalloc(struct_size(clk_data, hws, IMX7ULP_CLK_PCC3_END),
GFP_KERNEL);
@@ -244,13 +223,7 @@ static void __init imx7ulp_clk_pcc3_init(struct device_node *np)
of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_data);
- for (i = 0; i < ARRAY_SIZE(pcc3_uart_clk_ids); i++) {
- int index = pcc3_uart_clk_ids[i];
-
- pcc3_uart_clks[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(pcc3_uart_clks);
+ imx_register_uart_clocks(7);
}
CLK_OF_DECLARE(imx7ulp_clk_pcc3, "fsl,imx7ulp-pcc3", imx7ulp_clk_pcc3_init);
diff --git a/drivers/clk/imx/clk-imx8mm.c b/drivers/clk/imx/clk-imx8mm.c
index 6a01eec36dd0..f1919fafb124 100644
--- a/drivers/clk/imx/clk-imx8mm.c
+++ b/drivers/clk/imx/clk-imx8mm.c
@@ -296,20 +296,12 @@ static const char * const clkout_sels[] = {"audio_pll1_out", "audio_pll2_out", "
static struct clk_hw_onecell_data *clk_hw_data;
static struct clk_hw **hws;
-static const int uart_clk_ids[] = {
- IMX8MM_CLK_UART1_ROOT,
- IMX8MM_CLK_UART2_ROOT,
- IMX8MM_CLK_UART3_ROOT,
- IMX8MM_CLK_UART4_ROOT,
-};
-static struct clk **uart_hws[ARRAY_SIZE(uart_clk_ids) + 1];
-
static int imx8mm_clocks_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
void __iomem *base;
- int ret, i;
+ int ret;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX8MM_CLK_END), GFP_KERNEL);
@@ -634,13 +626,7 @@ static int imx8mm_clocks_probe(struct platform_device *pdev)
goto unregister_hws;
}
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_hws[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_hws);
+ imx_register_uart_clocks(4);
return 0;
diff --git a/drivers/clk/imx/clk-imx8mn.c b/drivers/clk/imx/clk-imx8mn.c
index 324c5fd0aa04..88f6630cd472 100644
--- a/drivers/clk/imx/clk-imx8mn.c
+++ b/drivers/clk/imx/clk-imx8mn.c
@@ -289,20 +289,12 @@ static const char * const clkout_sels[] = {"audio_pll1_out", "audio_pll2_out", "
static struct clk_hw_onecell_data *clk_hw_data;
static struct clk_hw **hws;
-static const int uart_clk_ids[] = {
- IMX8MN_CLK_UART1_ROOT,
- IMX8MN_CLK_UART2_ROOT,
- IMX8MN_CLK_UART3_ROOT,
- IMX8MN_CLK_UART4_ROOT,
-};
-static struct clk **uart_hws[ARRAY_SIZE(uart_clk_ids) + 1];
-
static int imx8mn_clocks_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
void __iomem *base;
- int ret, i;
+ int ret;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX8MN_CLK_END), GFP_KERNEL);
@@ -585,13 +577,7 @@ static int imx8mn_clocks_probe(struct platform_device *pdev)
goto unregister_hws;
}
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_hws[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_hws);
+ imx_register_uart_clocks(4);
return 0;
diff --git a/drivers/clk/imx/clk-imx8mp.c b/drivers/clk/imx/clk-imx8mp.c
index 2f4e1d674e1c..e39c9c907c38 100644
--- a/drivers/clk/imx/clk-imx8mp.c
+++ b/drivers/clk/imx/clk-imx8mp.c
@@ -152,10 +152,6 @@ static const char * const imx8mp_can2_sels[] = {"osc_24m", "sys_pll2_200m", "sys
"sys_pll1_160m", "sys_pll1_800m", "sys_pll3_out",
"sys_pll2_250m", "audio_pll2_out", };
-static const char * const imx8mp_pcie_phy_sels[] = {"osc_24m", "sys_pll2_100m", "sys_pll2_500m",
- "clk_ext1", "clk_ext2", "clk_ext3",
- "clk_ext4", "sys_pll1_400m", };
-
static const char * const imx8mp_pcie_aux_sels[] = {"osc_24m", "sys_pll2_200m", "sys_pll2_50m",
"sys_pll3_out", "sys_pll2_100m", "sys_pll1_80m",
"sys_pll1_160m", "sys_pll1_200m", };
@@ -380,14 +376,6 @@ static const char * const imx8mp_memrepair_sels[] = {"osc_24m", "sys_pll2_100m",
"sys_pll1_800m", "sys_pll2_1000m", "sys_pll3_out",
"clk_ext3", "audio_pll2_out", };
-static const char * const imx8mp_pcie2_ctrl_sels[] = {"osc_24m", "sys_pll2_250m", "sys_pll2_200m",
- "sys_pll1_266m", "sys_pll1_800m", "sys_pll2_500m",
- "sys_pll2_333m", "sys_pll3_out", };
-
-static const char * const imx8mp_pcie2_phy_sels[] = {"osc_24m", "sys_pll2_100m", "sys_pll2_500m",
- "clk_ext1", "clk_ext2", "clk_ext3",
- "clk_ext4", "sys_pll1_400m", };
-
static const char * const imx8mp_media_mipi_test_byte_sels[] = {"osc_24m", "sys_pll2_200m", "sys_pll2_50m",
"sys_pll3_out", "sys_pll2_100m",
"sys_pll1_80m", "sys_pll1_160m",
@@ -414,20 +402,11 @@ static const char * const imx8mp_dram_core_sels[] = {"dram_pll_out", "dram_alt_r
static struct clk_hw **hws;
static struct clk_hw_onecell_data *clk_hw_data;
-static const int uart_clk_ids[] = {
- IMX8MP_CLK_UART1_ROOT,
- IMX8MP_CLK_UART2_ROOT,
- IMX8MP_CLK_UART3_ROOT,
- IMX8MP_CLK_UART4_ROOT,
-};
-static struct clk **uart_clks[ARRAY_SIZE(uart_clk_ids) + 1];
-
static int imx8mp_clocks_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct device_node *np;
void __iomem *anatop_base, *ccm_base;
- int i;
np = of_find_compatible_node(NULL, NULL, "fsl,imx8mp-anatop");
anatop_base = of_iomap(np, 0);
@@ -585,7 +564,6 @@ static int imx8mp_clocks_probe(struct platform_device *pdev)
hws[IMX8MP_CLK_VPU_G2] = imx8m_clk_hw_composite("vpu_g2", imx8mp_vpu_g2_sels, ccm_base + 0xa180);
hws[IMX8MP_CLK_CAN1] = imx8m_clk_hw_composite("can1", imx8mp_can1_sels, ccm_base + 0xa200);
hws[IMX8MP_CLK_CAN2] = imx8m_clk_hw_composite("can2", imx8mp_can2_sels, ccm_base + 0xa280);
- hws[IMX8MP_CLK_PCIE_PHY] = imx8m_clk_hw_composite("pcie_phy", imx8mp_pcie_phy_sels, ccm_base + 0xa380);
hws[IMX8MP_CLK_PCIE_AUX] = imx8m_clk_hw_composite("pcie_aux", imx8mp_pcie_aux_sels, ccm_base + 0xa400);
hws[IMX8MP_CLK_I2C5] = imx8m_clk_hw_composite("i2c5", imx8mp_i2c5_sels, ccm_base + 0xa480);
hws[IMX8MP_CLK_I2C6] = imx8m_clk_hw_composite("i2c6", imx8mp_i2c6_sels, ccm_base + 0xa500);
@@ -643,8 +621,6 @@ static int imx8mp_clocks_probe(struct platform_device *pdev)
hws[IMX8MP_CLK_MEDIA_CAM2_PIX] = imx8m_clk_hw_composite("media_cam2_pix", imx8mp_media_cam2_pix_sels, ccm_base + 0xbe80);
hws[IMX8MP_CLK_MEDIA_LDB] = imx8m_clk_hw_composite("media_ldb", imx8mp_media_ldb_sels, ccm_base + 0xbf00);
hws[IMX8MP_CLK_MEMREPAIR] = imx8m_clk_hw_composite_critical("mem_repair", imx8mp_memrepair_sels, ccm_base + 0xbf80);
- hws[IMX8MP_CLK_PCIE2_CTRL] = imx8m_clk_hw_composite("pcie2_ctrl", imx8mp_pcie2_ctrl_sels, ccm_base + 0xc000);
- hws[IMX8MP_CLK_PCIE2_PHY] = imx8m_clk_hw_composite("pcie2_phy", imx8mp_pcie2_phy_sels, ccm_base + 0xc080);
hws[IMX8MP_CLK_MEDIA_MIPI_TEST_BYTE] = imx8m_clk_hw_composite("media_mipi_test_byte", imx8mp_media_mipi_test_byte_sels, ccm_base + 0xc100);
hws[IMX8MP_CLK_ECSPI3] = imx8m_clk_hw_composite("ecspi3", imx8mp_ecspi3_sels, ccm_base + 0xc180);
hws[IMX8MP_CLK_PDM] = imx8m_clk_hw_composite("pdm", imx8mp_pdm_sels, ccm_base + 0xc200);
@@ -737,13 +713,7 @@ static int imx8mp_clocks_probe(struct platform_device *pdev)
of_clk_add_hw_provider(np, of_clk_hw_onecell_get, clk_hw_data);
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_clks[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_clks);
+ imx_register_uart_clocks(4);
return 0;
}
diff --git a/drivers/clk/imx/clk-imx8mq.c b/drivers/clk/imx/clk-imx8mq.c
index 4dd4ae9d022b..b08019e1faf9 100644
--- a/drivers/clk/imx/clk-imx8mq.c
+++ b/drivers/clk/imx/clk-imx8mq.c
@@ -113,12 +113,12 @@ static const char * const imx8mq_disp_dtrc_sels[] = {"osc_25m", "vpu_pll_out", "
static const char * const imx8mq_disp_dc8000_sels[] = {"osc_25m", "vpu_pll_out", "sys1_pll_800m", "sys2_pll_1000m", "sys1_pll_160m", "sys2_pll_100m", "sys3_pll_out", "audio_pll2_out", };
static const char * const imx8mq_pcie1_ctrl_sels[] = {"osc_25m", "sys2_pll_250m", "sys2_pll_200m", "sys1_pll_266m",
- "sys1_pll_800m", "sys2_pll_500m", "sys2_pll_250m", "sys3_pll_out", };
+ "sys1_pll_800m", "sys2_pll_500m", "sys2_pll_333m", "sys3_pll_out", };
static const char * const imx8mq_pcie1_phy_sels[] = {"osc_25m", "sys2_pll_100m", "sys2_pll_500m", "clk_ext1", "clk_ext2",
"clk_ext3", "clk_ext4", };
-static const char * const imx8mq_pcie1_aux_sels[] = {"osc_25m", "sys2_pll_200m", "sys2_pll_500m", "sys3_pll_out",
+static const char * const imx8mq_pcie1_aux_sels[] = {"osc_25m", "sys2_pll_200m", "sys2_pll_50m", "sys3_pll_out",
"sys2_pll_100m", "sys1_pll_80m", "sys1_pll_160m", "sys1_pll_200m", };
static const char * const imx8mq_dc_pixel_sels[] = {"osc_25m", "video_pll1_out", "audio_pll2_out", "audio_pll1_out", "sys1_pll_800m", "sys2_pll_1000m", "sys3_pll_out", "clk_ext4", };
@@ -281,20 +281,12 @@ static const char * const pllout_monitor_sels[] = {"osc_25m", "osc_27m", "dummy"
static struct clk_hw_onecell_data *clk_hw_data;
static struct clk_hw **hws;
-static const int uart_clk_ids[] = {
- IMX8MQ_CLK_UART1_ROOT,
- IMX8MQ_CLK_UART2_ROOT,
- IMX8MQ_CLK_UART3_ROOT,
- IMX8MQ_CLK_UART4_ROOT,
-};
-static struct clk **uart_hws[ARRAY_SIZE(uart_clk_ids) + 1];
-
static int imx8mq_clocks_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct device_node *np = dev->of_node;
void __iomem *base;
- int err, i;
+ int err;
clk_hw_data = kzalloc(struct_size(clk_hw_data, hws,
IMX8MQ_CLK_END), GFP_KERNEL);
@@ -629,13 +621,7 @@ static int imx8mq_clocks_probe(struct platform_device *pdev)
goto unregister_hws;
}
- for (i = 0; i < ARRAY_SIZE(uart_clk_ids); i++) {
- int index = uart_clk_ids[i];
-
- uart_hws[i] = &hws[index]->clk;
- }
-
- imx_register_uart_clocks(uart_hws);
+ imx_register_uart_clocks(4);
return 0;
diff --git a/drivers/clk/imx/clk-lpcg-scu.c b/drivers/clk/imx/clk-lpcg-scu.c
index 77be7632866d..dd5abd09f3e2 100644
--- a/drivers/clk/imx/clk-lpcg-scu.c
+++ b/drivers/clk/imx/clk-lpcg-scu.c
@@ -114,6 +114,7 @@ struct clk_hw *__imx_clk_lpcg_scu(struct device *dev, const char *name,
if (ret) {
kfree(clk);
hw = ERR_PTR(ret);
+ return hw;
}
if (dev)
diff --git a/drivers/clk/imx/clk-scu.c b/drivers/clk/imx/clk-scu.c
index 1f5518b7ab39..f89b4da10e80 100644
--- a/drivers/clk/imx/clk-scu.c
+++ b/drivers/clk/imx/clk-scu.c
@@ -426,6 +426,7 @@ struct clk_hw *__imx_clk_scu(struct device *dev, const char *name,
if (ret) {
kfree(clk);
hw = ERR_PTR(ret);
+ return hw;
}
if (dev)
diff --git a/drivers/clk/imx/clk.c b/drivers/clk/imx/clk.c
index 47882c51cb85..7cc669934253 100644
--- a/drivers/clk/imx/clk.c
+++ b/drivers/clk/imx/clk.c
@@ -147,8 +147,10 @@ void imx_cscmr1_fixup(u32 *val)
}
#ifndef MODULE
-static int imx_keep_uart_clocks;
-static struct clk ** const *imx_uart_clocks;
+
+static bool imx_keep_uart_clocks;
+static int imx_enabled_uart_clocks;
+static struct clk **imx_uart_clocks;
static int __init imx_keep_uart_clocks_param(char *str)
{
@@ -161,24 +163,45 @@ __setup_param("earlycon", imx_keep_uart_earlycon,
__setup_param("earlyprintk", imx_keep_uart_earlyprintk,
imx_keep_uart_clocks_param, 0);
-void imx_register_uart_clocks(struct clk ** const clks[])
+void imx_register_uart_clocks(unsigned int clk_count)
{
+ imx_enabled_uart_clocks = 0;
+
+/* i.MX boards use device trees now. For build tests without CONFIG_OF, do nothing */
+#ifdef CONFIG_OF
if (imx_keep_uart_clocks) {
int i;
- imx_uart_clocks = clks;
- for (i = 0; imx_uart_clocks[i]; i++)
- clk_prepare_enable(*imx_uart_clocks[i]);
+ imx_uart_clocks = kcalloc(clk_count, sizeof(struct clk *), GFP_KERNEL);
+
+ if (!of_stdout)
+ return;
+
+ for (i = 0; i < clk_count; i++) {
+ imx_uart_clocks[imx_enabled_uart_clocks] = of_clk_get(of_stdout, i);
+
+ /* Stop if there are no more of_stdout references */
+ if (IS_ERR(imx_uart_clocks[imx_enabled_uart_clocks]))
+ return;
+
+ /* Only enable the clock if it's not NULL */
+ if (imx_uart_clocks[imx_enabled_uart_clocks])
+ clk_prepare_enable(imx_uart_clocks[imx_enabled_uart_clocks++]);
+ }
}
+#endif
}
static int __init imx_clk_disable_uart(void)
{
- if (imx_keep_uart_clocks && imx_uart_clocks) {
+ if (imx_keep_uart_clocks && imx_enabled_uart_clocks) {
int i;
- for (i = 0; imx_uart_clocks[i]; i++)
- clk_disable_unprepare(*imx_uart_clocks[i]);
+ for (i = 0; i < imx_enabled_uart_clocks; i++) {
+ clk_disable_unprepare(imx_uart_clocks[i]);
+ clk_put(imx_uart_clocks[i]);
+ }
+ kfree(imx_uart_clocks);
}
return 0;
diff --git a/drivers/clk/imx/clk.h b/drivers/clk/imx/clk.h
index 4f04c8287286..7571603bee23 100644
--- a/drivers/clk/imx/clk.h
+++ b/drivers/clk/imx/clk.h
@@ -11,9 +11,9 @@ extern spinlock_t imx_ccm_lock;
void imx_check_clocks(struct clk *clks[], unsigned int count);
void imx_check_clk_hws(struct clk_hw *clks[], unsigned int count);
#ifndef MODULE
-void imx_register_uart_clocks(struct clk ** const clks[]);
+void imx_register_uart_clocks(unsigned int clk_count);
#else
-static inline void imx_register_uart_clocks(struct clk ** const clks[])
+static inline void imx_register_uart_clocks(unsigned int clk_count)
{
}
#endif
diff --git a/drivers/clk/keystone/sci-clk.c b/drivers/clk/keystone/sci-clk.c
index aaf31abe1c8f..7e1b136e71ae 100644
--- a/drivers/clk/keystone/sci-clk.c
+++ b/drivers/clk/keystone/sci-clk.c
@@ -503,8 +503,8 @@ static int ti_sci_scan_clocks_from_fw(struct sci_clk_provider *provider)
#else
-static int _cmp_sci_clk_list(void *priv, struct list_head *a,
- struct list_head *b)
+static int _cmp_sci_clk_list(void *priv, const struct list_head *a,
+ const struct list_head *b)
{
struct sci_clk *ca = container_of(a, struct sci_clk, node);
struct sci_clk *cb = container_of(b, struct sci_clk, node);
diff --git a/drivers/clk/mvebu/armada-37xx-periph.c b/drivers/clk/mvebu/armada-37xx-periph.c
index f5746f9ea929..32ac6b6b7530 100644
--- a/drivers/clk/mvebu/armada-37xx-periph.c
+++ b/drivers/clk/mvebu/armada-37xx-periph.c
@@ -84,6 +84,7 @@ struct clk_pm_cpu {
void __iomem *reg_div;
u8 shift_div;
struct regmap *nb_pm_base;
+ unsigned long l1_expiration;
};
#define to_clk_double_div(_hw) container_of(_hw, struct clk_double_div, hw)
@@ -440,33 +441,6 @@ static u8 clk_pm_cpu_get_parent(struct clk_hw *hw)
return val;
}
-static int clk_pm_cpu_set_parent(struct clk_hw *hw, u8 index)
-{
- struct clk_pm_cpu *pm_cpu = to_clk_pm_cpu(hw);
- struct regmap *base = pm_cpu->nb_pm_base;
- int load_level;
-
- /*
- * We set the clock parent only if the DVFS is available but
- * not enabled.
- */
- if (IS_ERR(base) || armada_3700_pm_dvfs_is_enabled(base))
- return -EINVAL;
-
- /* Set the parent clock for all the load level */
- for (load_level = 0; load_level < LOAD_LEVEL_NR; load_level++) {
- unsigned int reg, mask, val,
- offset = ARMADA_37XX_NB_TBG_SEL_OFF;
-
- armada_3700_pm_dvfs_update_regs(load_level, &reg, &offset);
-
- val = index << offset;
- mask = ARMADA_37XX_NB_TBG_SEL_MASK << offset;
- regmap_update_bits(base, reg, mask, val);
- }
- return 0;
-}
-
static unsigned long clk_pm_cpu_recalc_rate(struct clk_hw *hw,
unsigned long parent_rate)
{
@@ -514,8 +488,10 @@ static long clk_pm_cpu_round_rate(struct clk_hw *hw, unsigned long rate,
}
/*
- * Switching the CPU from the L2 or L3 frequencies (300 and 200 Mhz
- * respectively) to L0 frequency (1.2 Ghz) requires a significant
+ * Workaround when base CPU frequnecy is 1000 or 1200 MHz
+ *
+ * Switching the CPU from the L2 or L3 frequencies (250/300 or 200 MHz
+ * respectively) to L0 frequency (1/1.2 GHz) requires a significant
* amount of time to let VDD stabilize to the appropriate
* voltage. This amount of time is large enough that it cannot be
* covered by the hardware countdown register. Due to this, the CPU
@@ -525,26 +501,56 @@ static long clk_pm_cpu_round_rate(struct clk_hw *hw, unsigned long rate,
* To work around this problem, we prevent switching directly from the
* L2/L3 frequencies to the L0 frequency, and instead switch to the L1
* frequency in-between. The sequence therefore becomes:
- * 1. First switch from L2/L3(200/300MHz) to L1(600MHZ)
+ * 1. First switch from L2/L3 (200/250/300 MHz) to L1 (500/600 MHz)
* 2. Sleep 20ms for stabling VDD voltage
- * 3. Then switch from L1(600MHZ) to L0(1200Mhz).
+ * 3. Then switch from L1 (500/600 MHz) to L0 (1000/1200 MHz).
*/
-static void clk_pm_cpu_set_rate_wa(unsigned long rate, struct regmap *base)
+static void clk_pm_cpu_set_rate_wa(struct clk_pm_cpu *pm_cpu,
+ unsigned int new_level, unsigned long rate,
+ struct regmap *base)
{
unsigned int cur_level;
- if (rate != 1200 * 1000 * 1000)
- return;
-
regmap_read(base, ARMADA_37XX_NB_CPU_LOAD, &cur_level);
cur_level &= ARMADA_37XX_NB_CPU_LOAD_MASK;
- if (cur_level <= ARMADA_37XX_DVFS_LOAD_1)
+
+ if (cur_level == new_level)
+ return;
+
+ /*
+ * System wants to go to L1 on its own. If we are going from L2/L3,
+ * remember when 20ms will expire. If from L0, set the value so that
+ * next switch to L0 won't have to wait.
+ */
+ if (new_level == ARMADA_37XX_DVFS_LOAD_1) {
+ if (cur_level == ARMADA_37XX_DVFS_LOAD_0)
+ pm_cpu->l1_expiration = jiffies;
+ else
+ pm_cpu->l1_expiration = jiffies + msecs_to_jiffies(20);
return;
+ }
+
+ /*
+ * If we are setting to L2/L3, just invalidate L1 expiration time,
+ * sleeping is not needed.
+ */
+ if (rate < 1000*1000*1000)
+ goto invalidate_l1_exp;
+
+ /*
+ * We are going to L0 with rate >= 1GHz. Check whether we have been at
+ * L1 for long enough time. If not, go to L1 for 20ms.
+ */
+ if (pm_cpu->l1_expiration && jiffies >= pm_cpu->l1_expiration)
+ goto invalidate_l1_exp;
regmap_update_bits(base, ARMADA_37XX_NB_CPU_LOAD,
ARMADA_37XX_NB_CPU_LOAD_MASK,
ARMADA_37XX_DVFS_LOAD_1);
msleep(20);
+
+invalidate_l1_exp:
+ pm_cpu->l1_expiration = 0;
}
static int clk_pm_cpu_set_rate(struct clk_hw *hw, unsigned long rate,
@@ -578,7 +584,9 @@ static int clk_pm_cpu_set_rate(struct clk_hw *hw, unsigned long rate,
reg = ARMADA_37XX_NB_CPU_LOAD;
mask = ARMADA_37XX_NB_CPU_LOAD_MASK;
- clk_pm_cpu_set_rate_wa(rate, base);
+ /* Apply workaround when base CPU frequency is 1000 or 1200 MHz */
+ if (parent_rate >= 1000*1000*1000)
+ clk_pm_cpu_set_rate_wa(pm_cpu, load_level, rate, base);
regmap_update_bits(base, reg, mask, load_level);
@@ -592,7 +600,6 @@ static int clk_pm_cpu_set_rate(struct clk_hw *hw, unsigned long rate,
static const struct clk_ops clk_pm_cpu_ops = {
.get_parent = clk_pm_cpu_get_parent,
- .set_parent = clk_pm_cpu_set_parent,
.round_rate = clk_pm_cpu_round_rate,
.set_rate = clk_pm_cpu_set_rate,
.recalc_rate = clk_pm_cpu_recalc_rate,
diff --git a/drivers/clk/qcom/a53-pll.c b/drivers/clk/qcom/a53-pll.c
index 45cfc57bff92..af6ac17c7dae 100644
--- a/drivers/clk/qcom/a53-pll.c
+++ b/drivers/clk/qcom/a53-pll.c
@@ -93,6 +93,7 @@ static const struct of_device_id qcom_a53pll_match_table[] = {
{ .compatible = "qcom,msm8916-a53pll" },
{ }
};
+MODULE_DEVICE_TABLE(of, qcom_a53pll_match_table);
static struct platform_driver qcom_a53pll_driver = {
.probe = qcom_a53pll_probe,
diff --git a/drivers/clk/qcom/a7-pll.c b/drivers/clk/qcom/a7-pll.c
index e171d3caf2cf..c4a53e5db229 100644
--- a/drivers/clk/qcom/a7-pll.c
+++ b/drivers/clk/qcom/a7-pll.c
@@ -86,6 +86,7 @@ static const struct of_device_id qcom_a7pll_match_table[] = {
{ .compatible = "qcom,sdx55-a7pll" },
{ }
};
+MODULE_DEVICE_TABLE(of, qcom_a7pll_match_table);
static struct platform_driver qcom_a7pll_driver = {
.probe = qcom_a7pll_probe,
diff --git a/drivers/clk/qcom/apss-ipq-pll.c b/drivers/clk/qcom/apss-ipq-pll.c
index 30be87fb222a..bef7899ad0d6 100644
--- a/drivers/clk/qcom/apss-ipq-pll.c
+++ b/drivers/clk/qcom/apss-ipq-pll.c
@@ -81,6 +81,7 @@ static const struct of_device_id apss_ipq_pll_match_table[] = {
{ .compatible = "qcom,ipq6018-a53pll" },
{ }
};
+MODULE_DEVICE_TABLE(of, apss_ipq_pll_match_table);
static struct platform_driver apss_ipq_pll_driver = {
.probe = apss_ipq_pll_probe,
diff --git a/drivers/clk/qcom/camcc-sc7180.c b/drivers/clk/qcom/camcc-sc7180.c
index dbac5651ab85..9bcf2f8ed4de 100644
--- a/drivers/clk/qcom/camcc-sc7180.c
+++ b/drivers/clk/qcom/camcc-sc7180.c
@@ -304,7 +304,7 @@ static struct clk_rcg2 cam_cc_bps_clk_src = {
.name = "cam_cc_bps_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -325,7 +325,7 @@ static struct clk_rcg2 cam_cc_cci_0_clk_src = {
.name = "cam_cc_cci_0_clk_src",
.parent_data = cam_cc_parent_data_5,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -339,7 +339,7 @@ static struct clk_rcg2 cam_cc_cci_1_clk_src = {
.name = "cam_cc_cci_1_clk_src",
.parent_data = cam_cc_parent_data_5,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -360,7 +360,7 @@ static struct clk_rcg2 cam_cc_cphy_rx_clk_src = {
.name = "cam_cc_cphy_rx_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -379,7 +379,7 @@ static struct clk_rcg2 cam_cc_csi0phytimer_clk_src = {
.name = "cam_cc_csi0phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -393,7 +393,7 @@ static struct clk_rcg2 cam_cc_csi1phytimer_clk_src = {
.name = "cam_cc_csi1phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -407,7 +407,7 @@ static struct clk_rcg2 cam_cc_csi2phytimer_clk_src = {
.name = "cam_cc_csi2phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -421,7 +421,7 @@ static struct clk_rcg2 cam_cc_csi3phytimer_clk_src = {
.name = "cam_cc_csi3phytimer_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -443,7 +443,7 @@ static struct clk_rcg2 cam_cc_fast_ahb_clk_src = {
.name = "cam_cc_fast_ahb_clk_src",
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -466,7 +466,7 @@ static struct clk_rcg2 cam_cc_icp_clk_src = {
.name = "cam_cc_icp_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -488,7 +488,7 @@ static struct clk_rcg2 cam_cc_ife_0_clk_src = {
.name = "cam_cc_ife_0_clk_src",
.parent_data = cam_cc_parent_data_4,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -510,7 +510,7 @@ static struct clk_rcg2 cam_cc_ife_0_csid_clk_src = {
.name = "cam_cc_ife_0_csid_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -524,7 +524,7 @@ static struct clk_rcg2 cam_cc_ife_1_clk_src = {
.name = "cam_cc_ife_1_clk_src",
.parent_data = cam_cc_parent_data_4,
.num_parents = 4,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -538,7 +538,7 @@ static struct clk_rcg2 cam_cc_ife_1_csid_clk_src = {
.name = "cam_cc_ife_1_csid_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -553,7 +553,7 @@ static struct clk_rcg2 cam_cc_ife_lite_clk_src = {
.parent_data = cam_cc_parent_data_4,
.num_parents = 4,
.flags = CLK_SET_RATE_PARENT,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -567,7 +567,7 @@ static struct clk_rcg2 cam_cc_ife_lite_csid_clk_src = {
.name = "cam_cc_ife_lite_csid_clk_src",
.parent_data = cam_cc_parent_data_3,
.num_parents = 6,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -590,7 +590,7 @@ static struct clk_rcg2 cam_cc_ipe_0_clk_src = {
.name = "cam_cc_ipe_0_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -613,7 +613,7 @@ static struct clk_rcg2 cam_cc_jpeg_clk_src = {
.name = "cam_cc_jpeg_clk_src",
.parent_data = cam_cc_parent_data_2,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -635,7 +635,7 @@ static struct clk_rcg2 cam_cc_lrme_clk_src = {
.name = "cam_cc_lrme_clk_src",
.parent_data = cam_cc_parent_data_6,
.num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -656,7 +656,7 @@ static struct clk_rcg2 cam_cc_mclk0_clk_src = {
.name = "cam_cc_mclk0_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -670,7 +670,7 @@ static struct clk_rcg2 cam_cc_mclk1_clk_src = {
.name = "cam_cc_mclk1_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -684,7 +684,7 @@ static struct clk_rcg2 cam_cc_mclk2_clk_src = {
.name = "cam_cc_mclk2_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -698,7 +698,7 @@ static struct clk_rcg2 cam_cc_mclk3_clk_src = {
.name = "cam_cc_mclk3_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -712,7 +712,7 @@ static struct clk_rcg2 cam_cc_mclk4_clk_src = {
.name = "cam_cc_mclk4_clk_src",
.parent_data = cam_cc_parent_data_1,
.num_parents = 3,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
@@ -732,7 +732,7 @@ static struct clk_rcg2 cam_cc_slow_ahb_clk_src = {
.parent_data = cam_cc_parent_data_0,
.num_parents = 4,
.flags = CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
- .ops = &clk_rcg2_ops,
+ .ops = &clk_rcg2_shared_ops,
},
};
diff --git a/drivers/clk/qcom/clk-rcg2.c b/drivers/clk/qcom/clk-rcg2.c
index 42f13a2d1cc1..05ff3b0d233e 100644
--- a/drivers/clk/qcom/clk-rcg2.c
+++ b/drivers/clk/qcom/clk-rcg2.c
@@ -730,7 +730,8 @@ static int clk_gfx3d_determine_rate(struct clk_hw *hw,
struct clk_rate_request parent_req = { };
struct clk_rcg2_gfx3d *cgfx = to_clk_rcg2_gfx3d(hw);
struct clk_hw *xo, *p0, *p1, *p2;
- unsigned long request, p0_rate;
+ unsigned long p0_rate;
+ u8 mux_div = cgfx->div;
int ret;
p0 = cgfx->hws[0];
@@ -750,14 +751,15 @@ static int clk_gfx3d_determine_rate(struct clk_hw *hw,
return 0;
}
- request = req->rate;
- if (cgfx->div > 1)
- parent_req.rate = request = request * cgfx->div;
+ if (mux_div == 0)
+ mux_div = 1;
+
+ parent_req.rate = req->rate * mux_div;
/* This has to be a fixed rate PLL */
p0_rate = clk_hw_get_rate(p0);
- if (request == p0_rate) {
+ if (parent_req.rate == p0_rate) {
req->rate = req->best_parent_rate = p0_rate;
req->best_parent_hw = p0;
return 0;
@@ -765,7 +767,7 @@ static int clk_gfx3d_determine_rate(struct clk_hw *hw,
if (req->best_parent_hw == p0) {
/* Are we going back to a previously used rate? */
- if (clk_hw_get_rate(p2) == request)
+ if (clk_hw_get_rate(p2) == parent_req.rate)
req->best_parent_hw = p2;
else
req->best_parent_hw = p1;
@@ -780,8 +782,7 @@ static int clk_gfx3d_determine_rate(struct clk_hw *hw,
return ret;
req->rate = req->best_parent_rate = parent_req.rate;
- if (cgfx->div > 1)
- req->rate /= cgfx->div;
+ req->rate /= mux_div;
return 0;
}
diff --git a/drivers/clk/qcom/clk-rpmh.c b/drivers/clk/qcom/clk-rpmh.c
index 91dc390a583b..552d1cbfea4c 100644
--- a/drivers/clk/qcom/clk-rpmh.c
+++ b/drivers/clk/qcom/clk-rpmh.c
@@ -380,6 +380,7 @@ static const struct clk_rpmh_desc clk_rpmh_sdm845 = {
DEFINE_CLK_RPMH_VRM(sdx55, rf_clk1, rf_clk1_ao, "rfclkd1", 1);
DEFINE_CLK_RPMH_VRM(sdx55, rf_clk2, rf_clk2_ao, "rfclkd2", 1);
DEFINE_CLK_RPMH_BCM(sdx55, qpic_clk, "QP0");
+DEFINE_CLK_RPMH_BCM(sdx55, ipa, "IP0");
static struct clk_hw *sdx55_rpmh_clocks[] = {
[RPMH_CXO_CLK] = &sdm845_bi_tcxo.hw,
@@ -389,6 +390,7 @@ static struct clk_hw *sdx55_rpmh_clocks[] = {
[RPMH_RF_CLK2] = &sdx55_rf_clk2.hw,
[RPMH_RF_CLK2_A] = &sdx55_rf_clk2_ao.hw,
[RPMH_QPIC_CLK] = &sdx55_qpic_clk.hw,
+ [RPMH_IPA_CLK] = &sdx55_ipa.hw,
};
static const struct clk_rpmh_desc clk_rpmh_sdx55 = {
@@ -510,9 +512,12 @@ static const struct clk_rpmh_desc clk_rpmh_sm8350 = {
.num_clks = ARRAY_SIZE(sm8350_rpmh_clocks),
};
+/* Resource name must match resource id present in cmd-db */
+DEFINE_CLK_RPMH_ARC(sc7280, bi_tcxo, bi_tcxo_ao, "xo.lvl", 0x3, 4);
+
static struct clk_hw *sc7280_rpmh_clocks[] = {
- [RPMH_CXO_CLK] = &sdm845_bi_tcxo.hw,
- [RPMH_CXO_CLK_A] = &sdm845_bi_tcxo_ao.hw,
+ [RPMH_CXO_CLK] = &sc7280_bi_tcxo.hw,
+ [RPMH_CXO_CLK_A] = &sc7280_bi_tcxo_ao.hw,
[RPMH_LN_BB_CLK2] = &sdm845_ln_bb_clk2.hw,
[RPMH_LN_BB_CLK2_A] = &sdm845_ln_bb_clk2_ao.hw,
[RPMH_RF_CLK1] = &sdm845_rf_clk1.hw,
diff --git a/drivers/clk/qcom/dispcc-sc7180.c b/drivers/clk/qcom/dispcc-sc7180.c
index f487515701e3..538e4963c915 100644
--- a/drivers/clk/qcom/dispcc-sc7180.c
+++ b/drivers/clk/qcom/dispcc-sc7180.c
@@ -19,8 +19,6 @@
enum {
P_BI_TCXO,
- P_CHIP_SLEEP_CLK,
- P_CORE_BI_PLL_TEST_SE,
P_DISP_CC_PLL0_OUT_EVEN,
P_DISP_CC_PLL0_OUT_MAIN,
P_DP_PHY_PLL_LINK_CLK,
@@ -65,8 +63,8 @@ static struct clk_alpha_pll_postdiv disp_cc_pll0_out_even = {
.regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_pll0_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_pll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_pll0.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -319,8 +317,8 @@ static struct clk_branch disp_cc_mdss_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -337,8 +335,8 @@ static struct clk_branch disp_cc_mdss_byte0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -383,8 +381,8 @@ static struct clk_branch disp_cc_mdss_byte0_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_intf_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte0_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -401,8 +399,8 @@ static struct clk_branch disp_cc_mdss_dp_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -419,8 +417,8 @@ static struct clk_branch disp_cc_mdss_dp_crypto_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_crypto_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_crypto_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_crypto_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -437,8 +435,8 @@ static struct clk_branch disp_cc_mdss_dp_link_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -455,8 +453,8 @@ static struct clk_branch disp_cc_mdss_dp_link_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link_intf_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link_div_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -472,8 +470,8 @@ static struct clk_branch disp_cc_mdss_dp_pixel_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_pixel_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_pixel_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -490,8 +488,8 @@ static struct clk_branch disp_cc_mdss_esc0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_esc0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_esc0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_esc0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -508,8 +506,8 @@ static struct clk_branch disp_cc_mdss_mdp_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_mdp_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_mdp_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_mdp_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -526,8 +524,8 @@ static struct clk_branch disp_cc_mdss_mdp_lut_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_mdp_lut_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_mdp_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_mdp_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -543,8 +541,8 @@ static struct clk_branch disp_cc_mdss_non_gdsc_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_non_gdsc_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -561,8 +559,8 @@ static struct clk_branch disp_cc_mdss_pclk0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_pclk0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_pclk0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_pclk0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -579,8 +577,8 @@ static struct clk_branch disp_cc_mdss_rot_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rot_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_rot_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_rot_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -597,8 +595,8 @@ static struct clk_branch disp_cc_mdss_rscc_vsync_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rscc_vsync_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_vsync_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_vsync_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -615,8 +613,8 @@ static struct clk_branch disp_cc_mdss_vsync_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_vsync_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_vsync_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_vsync_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/dispcc-sdm845.c b/drivers/clk/qcom/dispcc-sdm845.c
index 5c932cd17b14..735adfefc379 100644
--- a/drivers/clk/qcom/dispcc-sdm845.c
+++ b/drivers/clk/qcom/dispcc-sdm845.c
@@ -21,7 +21,6 @@
enum {
P_BI_TCXO,
- P_CORE_BI_PLL_TEST_SE,
P_DISP_CC_PLL0_OUT_MAIN,
P_DSI0_PHY_PLL_OUT_BYTECLK,
P_DSI0_PHY_PLL_OUT_DSICLK,
@@ -33,42 +32,51 @@ enum {
P_DP_PHY_PLL_VCO_DIV_CLK,
};
+static struct clk_alpha_pll disp_cc_pll0 = {
+ .offset = 0x0,
+ .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
+ .clkr = {
+ .hw.init = &(struct clk_init_data){
+ .name = "disp_cc_pll0",
+ .parent_data = &(const struct clk_parent_data){
+ .fw_name = "bi_tcxo", .name = "bi_tcxo",
+ },
+ .num_parents = 1,
+ .ops = &clk_alpha_pll_fabia_ops,
+ },
+ },
+};
+
static const struct parent_map disp_cc_parent_map_0[] = {
{ P_BI_TCXO, 0 },
{ P_DSI0_PHY_PLL_OUT_BYTECLK, 1 },
{ P_DSI1_PHY_PLL_OUT_BYTECLK, 2 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const disp_cc_parent_names_0[] = {
- "bi_tcxo",
- "dsi0_phy_pll_out_byteclk",
- "dsi1_phy_pll_out_byteclk",
- "core_bi_pll_test_se",
+static const struct clk_parent_data disp_cc_parent_data_0[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .fw_name = "dsi0_phy_pll_out_byteclk", .name = "dsi0_phy_pll_out_byteclk" },
+ { .fw_name = "dsi1_phy_pll_out_byteclk", .name = "dsi1_phy_pll_out_byteclk" },
};
static const struct parent_map disp_cc_parent_map_1[] = {
{ P_BI_TCXO, 0 },
{ P_DP_PHY_PLL_LINK_CLK, 1 },
{ P_DP_PHY_PLL_VCO_DIV_CLK, 2 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const disp_cc_parent_names_1[] = {
- "bi_tcxo",
- "dp_link_clk_divsel_ten",
- "dp_vco_divided_clk_src_mux",
- "core_bi_pll_test_se",
+static const struct clk_parent_data disp_cc_parent_data_1[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .fw_name = "dp_link_clk_divsel_ten", .name = "dp_link_clk_divsel_ten" },
+ { .fw_name = "dp_vco_divided_clk_src_mux", .name = "dp_vco_divided_clk_src_mux" },
};
static const struct parent_map disp_cc_parent_map_2[] = {
{ P_BI_TCXO, 0 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const disp_cc_parent_names_2[] = {
- "bi_tcxo",
- "core_bi_pll_test_se",
+static const struct clk_parent_data disp_cc_parent_data_2[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
};
static const struct parent_map disp_cc_parent_map_3[] = {
@@ -76,42 +84,25 @@ static const struct parent_map disp_cc_parent_map_3[] = {
{ P_DISP_CC_PLL0_OUT_MAIN, 1 },
{ P_GPLL0_OUT_MAIN, 4 },
{ P_GPLL0_OUT_MAIN_DIV, 5 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const disp_cc_parent_names_3[] = {
- "bi_tcxo",
- "disp_cc_pll0",
- "gcc_disp_gpll0_clk_src",
- "gcc_disp_gpll0_div_clk_src",
- "core_bi_pll_test_se",
+static const struct clk_parent_data disp_cc_parent_data_3[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &disp_cc_pll0.clkr.hw },
+ { .fw_name = "gcc_disp_gpll0_clk_src", .name = "gcc_disp_gpll0_clk_src" },
+ { .fw_name = "gcc_disp_gpll0_div_clk_src", .name = "gcc_disp_gpll0_div_clk_src" },
};
static const struct parent_map disp_cc_parent_map_4[] = {
{ P_BI_TCXO, 0 },
{ P_DSI0_PHY_PLL_OUT_DSICLK, 1 },
{ P_DSI1_PHY_PLL_OUT_DSICLK, 2 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const disp_cc_parent_names_4[] = {
- "bi_tcxo",
- "dsi0_phy_pll_out_dsiclk",
- "dsi1_phy_pll_out_dsiclk",
- "core_bi_pll_test_se",
-};
-
-static struct clk_alpha_pll disp_cc_pll0 = {
- .offset = 0x0,
- .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
- .clkr = {
- .hw.init = &(struct clk_init_data){
- .name = "disp_cc_pll0",
- .parent_names = (const char *[]){ "bi_tcxo" },
- .num_parents = 1,
- .ops = &clk_alpha_pll_fabia_ops,
- },
- },
+static const struct clk_parent_data disp_cc_parent_data_4[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .fw_name = "dsi0_phy_pll_out_dsiclk", .name = "dsi0_phy_pll_out_dsiclk" },
+ { .fw_name = "dsi1_phy_pll_out_dsiclk", .name = "dsi1_phy_pll_out_dsiclk" },
};
/* Return the HW recalc rate for idle use case */
@@ -122,8 +113,8 @@ static struct clk_rcg2 disp_cc_mdss_byte0_clk_src = {
.parent_map = disp_cc_parent_map_0,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_clk_src",
- .parent_names = disp_cc_parent_names_0,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_0,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_byte2_ops,
},
@@ -137,8 +128,8 @@ static struct clk_rcg2 disp_cc_mdss_byte1_clk_src = {
.parent_map = disp_cc_parent_map_0,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte1_clk_src",
- .parent_names = disp_cc_parent_names_0,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_0,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_byte2_ops,
},
@@ -157,8 +148,8 @@ static struct clk_rcg2 disp_cc_mdss_dp_aux_clk_src = {
.freq_tbl = ftbl_disp_cc_mdss_dp_aux_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_aux_clk_src",
- .parent_names = disp_cc_parent_names_2,
- .num_parents = 2,
+ .parent_data = disp_cc_parent_data_2,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -171,8 +162,8 @@ static struct clk_rcg2 disp_cc_mdss_dp_crypto_clk_src = {
.parent_map = disp_cc_parent_map_1,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_crypto_clk_src",
- .parent_names = disp_cc_parent_names_1,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_1,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_1),
.ops = &clk_byte2_ops,
},
};
@@ -184,8 +175,8 @@ static struct clk_rcg2 disp_cc_mdss_dp_link_clk_src = {
.parent_map = disp_cc_parent_map_1,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link_clk_src",
- .parent_names = disp_cc_parent_names_1,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_1,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_byte2_ops,
},
@@ -198,8 +189,8 @@ static struct clk_rcg2 disp_cc_mdss_dp_pixel1_clk_src = {
.parent_map = disp_cc_parent_map_1,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel1_clk_src",
- .parent_names = disp_cc_parent_names_1,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_1,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_dp_ops,
},
@@ -212,8 +203,8 @@ static struct clk_rcg2 disp_cc_mdss_dp_pixel_clk_src = {
.parent_map = disp_cc_parent_map_1,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel_clk_src",
- .parent_names = disp_cc_parent_names_1,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_1,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_dp_ops,
},
@@ -232,8 +223,8 @@ static struct clk_rcg2 disp_cc_mdss_esc0_clk_src = {
.freq_tbl = ftbl_disp_cc_mdss_esc0_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_esc0_clk_src",
- .parent_names = disp_cc_parent_names_0,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_0,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -246,8 +237,8 @@ static struct clk_rcg2 disp_cc_mdss_esc1_clk_src = {
.freq_tbl = ftbl_disp_cc_mdss_esc0_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_esc1_clk_src",
- .parent_names = disp_cc_parent_names_0,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_0,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -273,8 +264,8 @@ static struct clk_rcg2 disp_cc_mdss_mdp_clk_src = {
.freq_tbl = ftbl_disp_cc_mdss_mdp_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_mdp_clk_src",
- .parent_names = disp_cc_parent_names_3,
- .num_parents = 5,
+ .parent_data = disp_cc_parent_data_3,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_3),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -287,8 +278,8 @@ static struct clk_rcg2 disp_cc_mdss_pclk0_clk_src = {
.parent_map = disp_cc_parent_map_4,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_pclk0_clk_src",
- .parent_names = disp_cc_parent_names_4,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_4,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_4),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_pixel_ops,
},
@@ -302,8 +293,8 @@ static struct clk_rcg2 disp_cc_mdss_pclk1_clk_src = {
.parent_map = disp_cc_parent_map_4,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_pclk1_clk_src",
- .parent_names = disp_cc_parent_names_4,
- .num_parents = 4,
+ .parent_data = disp_cc_parent_data_4,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_4),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_pixel_ops,
},
@@ -326,8 +317,8 @@ static struct clk_rcg2 disp_cc_mdss_rot_clk_src = {
.freq_tbl = ftbl_disp_cc_mdss_rot_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rot_clk_src",
- .parent_names = disp_cc_parent_names_3,
- .num_parents = 5,
+ .parent_data = disp_cc_parent_data_3,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_3),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -340,8 +331,8 @@ static struct clk_rcg2 disp_cc_mdss_vsync_clk_src = {
.freq_tbl = ftbl_disp_cc_mdss_esc0_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_vsync_clk_src",
- .parent_names = disp_cc_parent_names_2,
- .num_parents = 2,
+ .parent_data = disp_cc_parent_data_2,
+ .num_parents = ARRAY_SIZE(disp_cc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -381,8 +372,8 @@ static struct clk_branch disp_cc_mdss_byte0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_byte0_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -399,8 +390,8 @@ static struct clk_regmap_div disp_cc_mdss_byte0_div_clk_src = {
.clkr = {
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_div_clk_src",
- .parent_names = (const char *[]){
- "disp_cc_mdss_byte0_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_regmap_div_ops,
@@ -417,8 +408,8 @@ static struct clk_branch disp_cc_mdss_byte0_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_intf_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_byte0_div_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -436,8 +427,8 @@ static struct clk_branch disp_cc_mdss_byte1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte1_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_byte1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -454,8 +445,8 @@ static struct clk_regmap_div disp_cc_mdss_byte1_div_clk_src = {
.clkr = {
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte1_div_clk_src",
- .parent_names = (const char *[]){
- "disp_cc_mdss_byte1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte1_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_regmap_div_ops,
@@ -472,8 +463,8 @@ static struct clk_branch disp_cc_mdss_byte1_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte1_intf_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_byte1_div_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte1_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -490,8 +481,8 @@ static struct clk_branch disp_cc_mdss_dp_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_aux_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_dp_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -508,8 +499,8 @@ static struct clk_branch disp_cc_mdss_dp_crypto_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_crypto_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_dp_crypto_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_crypto_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -526,8 +517,8 @@ static struct clk_branch disp_cc_mdss_dp_link_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_dp_link_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -545,8 +536,8 @@ static struct clk_branch disp_cc_mdss_dp_link_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link_intf_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_dp_link_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -562,8 +553,8 @@ static struct clk_branch disp_cc_mdss_dp_pixel1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel1_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_dp_pixel1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_pixel1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -580,8 +571,8 @@ static struct clk_branch disp_cc_mdss_dp_pixel_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_dp_pixel_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_pixel_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -598,8 +589,8 @@ static struct clk_branch disp_cc_mdss_esc0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_esc0_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_esc0_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_esc0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -616,8 +607,8 @@ static struct clk_branch disp_cc_mdss_esc1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_esc1_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_esc1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_esc1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -634,8 +625,8 @@ static struct clk_branch disp_cc_mdss_mdp_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_mdp_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_mdp_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_mdp_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -652,8 +643,8 @@ static struct clk_branch disp_cc_mdss_mdp_lut_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_mdp_lut_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_mdp_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_mdp_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -670,8 +661,8 @@ static struct clk_branch disp_cc_mdss_pclk0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_pclk0_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_pclk0_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_pclk0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -689,8 +680,8 @@ static struct clk_branch disp_cc_mdss_pclk1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_pclk1_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_pclk1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_pclk1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -707,8 +698,8 @@ static struct clk_branch disp_cc_mdss_rot_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rot_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_rot_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_rot_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -738,8 +729,8 @@ static struct clk_branch disp_cc_mdss_rscc_vsync_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rscc_vsync_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_vsync_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_vsync_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -756,8 +747,8 @@ static struct clk_branch disp_cc_mdss_vsync_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_vsync_clk",
- .parent_names = (const char *[]){
- "disp_cc_mdss_vsync_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_vsync_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/dispcc-sm8250.c b/drivers/clk/qcom/dispcc-sm8250.c
index 588575e1169d..de09cd5c209f 100644
--- a/drivers/clk/qcom/dispcc-sm8250.c
+++ b/drivers/clk/qcom/dispcc-sm8250.c
@@ -21,23 +21,15 @@
enum {
P_BI_TCXO,
- P_CHIP_SLEEP_CLK,
- P_CORE_BI_PLL_TEST_SE,
P_DISP_CC_PLL0_OUT_MAIN,
P_DISP_CC_PLL1_OUT_EVEN,
P_DISP_CC_PLL1_OUT_MAIN,
P_DP_PHY_PLL_LINK_CLK,
P_DP_PHY_PLL_VCO_DIV_CLK,
- P_DPTX1_PHY_PLL_LINK_CLK,
- P_DPTX1_PHY_PLL_VCO_DIV_CLK,
- P_DPTX2_PHY_PLL_LINK_CLK,
- P_DPTX2_PHY_PLL_VCO_DIV_CLK,
P_DSI0_PHY_PLL_OUT_BYTECLK,
P_DSI0_PHY_PLL_OUT_DSICLK,
P_DSI1_PHY_PLL_OUT_BYTECLK,
P_DSI1_PHY_PLL_OUT_DSICLK,
- P_EDP_PHY_PLL_LINK_CLK,
- P_EDP_PHY_PLL_VCO_DIV_CLK,
};
static struct pll_vco vco_table[] = {
@@ -456,8 +448,8 @@ static struct clk_regmap_div disp_cc_mdss_byte0_div_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "disp_cc_mdss_byte0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_regmap_div_ops,
@@ -471,8 +463,8 @@ static struct clk_regmap_div disp_cc_mdss_byte1_div_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "disp_cc_mdss_byte1_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte1_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_regmap_div_ops,
@@ -486,8 +478,8 @@ static struct clk_regmap_div disp_cc_mdss_dp_link1_div_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "disp_cc_mdss_dp_link1_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link1_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_regmap_div_ro_ops,
@@ -501,8 +493,8 @@ static struct clk_regmap_div disp_cc_mdss_dp_link_div_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "disp_cc_mdss_dp_link_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_regmap_div_ro_ops,
@@ -517,8 +509,8 @@ static struct clk_branch disp_cc_mdss_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -535,8 +527,8 @@ static struct clk_branch disp_cc_mdss_byte0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -553,8 +545,8 @@ static struct clk_branch disp_cc_mdss_byte0_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte0_intf_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte0_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte0_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -571,8 +563,8 @@ static struct clk_branch disp_cc_mdss_byte1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -589,8 +581,8 @@ static struct clk_branch disp_cc_mdss_byte1_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_byte1_intf_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_byte1_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_byte1_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -607,8 +599,8 @@ static struct clk_branch disp_cc_mdss_dp_aux1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_aux1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_aux1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_aux1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -625,8 +617,8 @@ static struct clk_branch disp_cc_mdss_dp_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -643,8 +635,8 @@ static struct clk_branch disp_cc_mdss_dp_link1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -661,8 +653,8 @@ static struct clk_branch disp_cc_mdss_dp_link1_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link1_intf_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link1_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link1_div_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -678,8 +670,8 @@ static struct clk_branch disp_cc_mdss_dp_link_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -696,8 +688,8 @@ static struct clk_branch disp_cc_mdss_dp_link_intf_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_link_intf_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_link_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_link_div_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -713,8 +705,8 @@ static struct clk_branch disp_cc_mdss_dp_pixel1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_pixel1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_pixel1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -731,8 +723,8 @@ static struct clk_branch disp_cc_mdss_dp_pixel2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_pixel2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_pixel2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -749,8 +741,8 @@ static struct clk_branch disp_cc_mdss_dp_pixel_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_dp_pixel_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_dp_pixel_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_dp_pixel_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -767,8 +759,8 @@ static struct clk_branch disp_cc_mdss_esc0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_esc0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_esc0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_esc0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -785,8 +777,8 @@ static struct clk_branch disp_cc_mdss_esc1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_esc1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_esc1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_esc1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -803,8 +795,8 @@ static struct clk_branch disp_cc_mdss_mdp_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_mdp_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_mdp_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_mdp_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -821,8 +813,8 @@ static struct clk_branch disp_cc_mdss_mdp_lut_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_mdp_lut_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_mdp_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_mdp_clk_src.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -838,8 +830,8 @@ static struct clk_branch disp_cc_mdss_non_gdsc_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_non_gdsc_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -856,8 +848,8 @@ static struct clk_branch disp_cc_mdss_pclk0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_pclk0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_pclk0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_pclk0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -874,8 +866,8 @@ static struct clk_branch disp_cc_mdss_pclk1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_pclk1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_pclk1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_pclk1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -892,8 +884,8 @@ static struct clk_branch disp_cc_mdss_rot_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rot_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_rot_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_rot_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -910,8 +902,8 @@ static struct clk_branch disp_cc_mdss_rscc_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rscc_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -928,8 +920,8 @@ static struct clk_branch disp_cc_mdss_rscc_vsync_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_rscc_vsync_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_vsync_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_vsync_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -946,8 +938,8 @@ static struct clk_branch disp_cc_mdss_vsync_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "disp_cc_mdss_vsync_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &disp_cc_mdss_vsync_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &disp_cc_mdss_vsync_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gcc-sc7180.c b/drivers/clk/qcom/gcc-sc7180.c
index 88e896abb663..c2ea09945c47 100644
--- a/drivers/clk/qcom/gcc-sc7180.c
+++ b/drivers/clk/qcom/gcc-sc7180.c
@@ -65,8 +65,8 @@ static struct clk_alpha_pll_postdiv gpll0_out_even = {
.regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
.clkr.hw.init = &(struct clk_init_data){
.name = "gpll0_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_fabia_ops,
@@ -78,8 +78,8 @@ static struct clk_fixed_factor gcc_pll0_main_div_cdiv = {
.div = 2,
.hw.init = &(struct clk_init_data){
.name = "gcc_pll0_main_div_cdiv",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_fixed_factor_ops,
@@ -285,7 +285,7 @@ static struct clk_rcg2 gcc_cpuss_ahb_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk_src",
.parent_data = gcc_parent_data_0_ao,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0_ao),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -309,7 +309,7 @@ static struct clk_rcg2 gcc_gp1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk_src",
.parent_data = gcc_parent_data_4,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_4),
.ops = &clk_rcg2_ops,
},
};
@@ -323,7 +323,7 @@ static struct clk_rcg2 gcc_gp2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk_src",
.parent_data = gcc_parent_data_4,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_4),
.ops = &clk_rcg2_ops,
},
};
@@ -337,7 +337,7 @@ static struct clk_rcg2 gcc_gp3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk_src",
.parent_data = gcc_parent_data_4,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_4),
.ops = &clk_rcg2_ops,
},
};
@@ -357,7 +357,7 @@ static struct clk_rcg2 gcc_pdm2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -378,7 +378,7 @@ static struct clk_rcg2 gcc_qspi_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_core_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 6,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -619,8 +619,8 @@ static struct clk_rcg2 gcc_sdcc1_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc1_apps_clk_src",
.parent_data = gcc_parent_data_1,
- .num_parents = 5,
- .ops = &clk_rcg2_ops,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
+ .ops = &clk_rcg2_floor_ops,
},
};
@@ -641,8 +641,8 @@ static struct clk_rcg2 gcc_sdcc1_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc1_ice_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
- .ops = &clk_rcg2_floor_ops,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
+ .ops = &clk_rcg2_ops,
},
};
@@ -666,7 +666,7 @@ static struct clk_rcg2 gcc_sdcc2_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk_src",
.parent_data = gcc_parent_data_5,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_5),
.ops = &clk_rcg2_floor_ops,
},
};
@@ -689,7 +689,7 @@ static struct clk_rcg2 gcc_ufs_phy_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -711,7 +711,7 @@ static struct clk_rcg2 gcc_ufs_phy_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -731,7 +731,7 @@ static struct clk_rcg2 gcc_ufs_phy_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk_src",
.parent_data = gcc_parent_data_3,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_3),
.ops = &clk_rcg2_ops,
},
};
@@ -752,7 +752,7 @@ static struct clk_rcg2 gcc_ufs_phy_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -774,7 +774,7 @@ static struct clk_rcg2 gcc_usb30_prim_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -794,7 +794,7 @@ static struct clk_rcg2 gcc_usb30_prim_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -813,7 +813,7 @@ static struct clk_rcg2 gcc_usb3_prim_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk_src",
.parent_data = gcc_parent_data_6,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_6),
.ops = &clk_rcg2_ops,
},
};
@@ -848,8 +848,8 @@ static struct clk_branch gcc_aggre_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -866,8 +866,8 @@ static struct clk_branch gcc_aggre_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -968,8 +968,8 @@ static struct clk_branch gcc_cfg_noc_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -987,8 +987,8 @@ static struct clk_branch gcc_cpuss_ahb_clk = {
.enable_mask = BIT(21),
.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_cpuss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_IS_CRITICAL | CLK_SET_RATE_PARENT,
@@ -1030,8 +1030,8 @@ static struct clk_branch gcc_disp_gpll0_clk_src = {
.enable_mask = BIT(18),
.hw.init = &(struct clk_init_data){
.name = "gcc_disp_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_aon_ops,
@@ -1046,8 +1046,8 @@ static struct clk_branch gcc_disp_gpll0_div_clk_src = {
.enable_mask = BIT(19),
.hw.init = &(struct clk_init_data){
.name = "gcc_disp_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pll0_main_div_cdiv.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pll0_main_div_cdiv.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -1091,8 +1091,8 @@ static struct clk_branch gcc_gp1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1109,8 +1109,8 @@ static struct clk_branch gcc_gp2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1127,8 +1127,8 @@ static struct clk_branch gcc_gp3_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1144,8 +1144,8 @@ static struct clk_branch gcc_gpu_gpll0_clk_src = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -1160,8 +1160,8 @@ static struct clk_branch gcc_gpu_gpll0_div_clk_src = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pll0_main_div_cdiv.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pll0_main_div_cdiv.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -1284,8 +1284,8 @@ static struct clk_branch gcc_npu_gpll0_clk_src = {
.enable_mask = BIT(25),
.hw.init = &(struct clk_init_data){
.name = "gcc_npu_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -1300,8 +1300,8 @@ static struct clk_branch gcc_npu_gpll0_div_clk_src = {
.enable_mask = BIT(26),
.hw.init = &(struct clk_init_data){
.name = "gcc_npu_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pll0_main_div_cdiv.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pll0_main_div_cdiv.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1318,8 +1318,8 @@ static struct clk_branch gcc_pdm2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pdm2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pdm2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1394,8 +1394,8 @@ static struct clk_branch gcc_qspi_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qspi_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qspi_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1438,8 +1438,8 @@ static struct clk_branch gcc_qupv3_wrap0_s0_clk = {
.enable_mask = BIT(10),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1456,8 +1456,8 @@ static struct clk_branch gcc_qupv3_wrap0_s1_clk = {
.enable_mask = BIT(11),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1474,8 +1474,8 @@ static struct clk_branch gcc_qupv3_wrap0_s2_clk = {
.enable_mask = BIT(12),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1492,8 +1492,8 @@ static struct clk_branch gcc_qupv3_wrap0_s3_clk = {
.enable_mask = BIT(13),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1510,8 +1510,8 @@ static struct clk_branch gcc_qupv3_wrap0_s4_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1528,8 +1528,8 @@ static struct clk_branch gcc_qupv3_wrap0_s5_clk = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1572,8 +1572,8 @@ static struct clk_branch gcc_qupv3_wrap1_s0_clk = {
.enable_mask = BIT(22),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1590,8 +1590,8 @@ static struct clk_branch gcc_qupv3_wrap1_s1_clk = {
.enable_mask = BIT(23),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1608,8 +1608,8 @@ static struct clk_branch gcc_qupv3_wrap1_s2_clk = {
.enable_mask = BIT(24),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1626,8 +1626,8 @@ static struct clk_branch gcc_qupv3_wrap1_s3_clk = {
.enable_mask = BIT(25),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1644,8 +1644,8 @@ static struct clk_branch gcc_qupv3_wrap1_s4_clk = {
.enable_mask = BIT(26),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1662,8 +1662,8 @@ static struct clk_branch gcc_qupv3_wrap1_s5_clk = {
.enable_mask = BIT(27),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1749,8 +1749,8 @@ static struct clk_branch gcc_sdcc1_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc1_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc1_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc1_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1767,8 +1767,8 @@ static struct clk_branch gcc_sdcc1_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc1_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc1_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc1_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1798,8 +1798,8 @@ static struct clk_branch gcc_sdcc2_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc2_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc2_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1817,8 +1817,8 @@ static struct clk_branch gcc_sys_noc_cpuss_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sys_noc_cpuss_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_cpuss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_IS_CRITICAL | CLK_SET_RATE_PARENT,
@@ -1865,8 +1865,8 @@ static struct clk_branch gcc_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1885,8 +1885,8 @@ static struct clk_branch gcc_ufs_phy_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1905,8 +1905,8 @@ static struct clk_branch gcc_ufs_phy_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1951,8 +1951,8 @@ static struct clk_branch gcc_ufs_phy_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1969,8 +1969,8 @@ static struct clk_branch gcc_usb30_prim_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2032,8 +2032,8 @@ static struct clk_branch gcc_usb3_prim_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2050,8 +2050,8 @@ static struct clk_branch gcc_usb3_prim_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_com_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2108,8 +2108,8 @@ static struct clk_branch gcc_video_gpll0_div_clk_src = {
.enable_mask = BIT(20),
.hw.init = &(struct clk_init_data){
.name = "gcc_video_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pll0_main_div_cdiv.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pll0_main_div_cdiv.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gcc-sc7280.c b/drivers/clk/qcom/gcc-sc7280.c
index 22736c16ed16..ef734db316df 100644
--- a/drivers/clk/qcom/gcc-sc7280.c
+++ b/drivers/clk/qcom/gcc-sc7280.c
@@ -70,8 +70,8 @@ static struct clk_alpha_pll_postdiv gcc_gpll0_out_even = {
.regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_LUCID],
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gpll0_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_lucid_ops,
@@ -92,8 +92,8 @@ static struct clk_alpha_pll_postdiv gcc_gpll0_out_odd = {
.regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_LUCID],
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gpll0_out_odd",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_lucid_ops,
@@ -175,8 +175,8 @@ static struct clk_branch gcc_mss_gpll0_main_div_clk_src = {
.enable_mask = BIT(17),
.hw.init = &(struct clk_init_data){
.name = "gcc_mss_gpll0_main_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0_out_even.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0_out_even.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1244,8 +1244,8 @@ static struct clk_regmap_div gcc_cpuss_ahb_postdiv_clk_src = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_cpuss_ahb_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_cpuss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1259,8 +1259,8 @@ static struct clk_regmap_div gcc_usb30_prim_mock_utmi_postdiv_clk_src = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_usb30_prim_mock_utmi_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_mock_utmi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1274,8 +1274,8 @@ static struct clk_regmap_div gcc_usb30_sec_mock_utmi_postdiv_clk_src = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_usb30_sec_mock_utmi_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_mock_utmi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1379,8 +1379,8 @@ static struct clk_branch gcc_aggre_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1399,8 +1399,8 @@ static struct clk_branch gcc_aggre_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1419,8 +1419,8 @@ static struct clk_branch gcc_aggre_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_sec_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1469,8 +1469,8 @@ static struct clk_branch gcc_cfg_noc_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1489,8 +1489,8 @@ static struct clk_branch gcc_cfg_noc_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_sec_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1510,8 +1510,8 @@ static struct clk_branch gcc_cpuss_ahb_clk = {
.enable_mask = BIT(21),
.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_cpuss_ahb_postdiv_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_postdiv_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_IS_CRITICAL | CLK_SET_RATE_PARENT,
@@ -1557,8 +1557,8 @@ static struct clk_branch gcc_disp_gpll0_clk_src = {
.enable_mask = BIT(7),
.hw.init = &(struct clk_init_data){
.name = "gcc_disp_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1605,8 +1605,8 @@ static struct clk_branch gcc_gp1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1623,8 +1623,8 @@ static struct clk_branch gcc_gp2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1641,8 +1641,8 @@ static struct clk_branch gcc_gp3_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1658,8 +1658,8 @@ static struct clk_branch gcc_gpu_gpll0_clk_src = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1675,8 +1675,8 @@ static struct clk_branch gcc_gpu_gpll0_div_clk_src = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0_out_even.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0_out_even.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1734,8 +1734,8 @@ static struct clk_branch gcc_pcie0_phy_rchng_clk = {
.enable_mask = BIT(22),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie0_phy_rchng_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_phy_rchng_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_phy_rchng_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1752,8 +1752,8 @@ static struct clk_branch gcc_pcie1_phy_rchng_clk = {
.enable_mask = BIT(23),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie1_phy_rchng_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_1_phy_rchng_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_phy_rchng_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1770,8 +1770,8 @@ static struct clk_branch gcc_pcie_0_aux_clk = {
.enable_mask = BIT(3),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1816,8 +1816,8 @@ static struct clk_branch gcc_pcie_0_pipe_clk = {
.enable_mask = BIT(4),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1860,8 +1860,8 @@ static struct clk_branch gcc_pcie_1_aux_clk = {
.enable_mask = BIT(29),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_1_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1906,8 +1906,8 @@ static struct clk_branch gcc_pcie_1_pipe_clk = {
.enable_mask = BIT(30),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_1_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1965,8 +1965,8 @@ static struct clk_branch gcc_pdm2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pdm2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pdm2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2084,8 +2084,8 @@ static struct clk_branch gcc_qspi_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qspi_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qspi_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2128,8 +2128,8 @@ static struct clk_branch gcc_qupv3_wrap0_s0_clk = {
.enable_mask = BIT(10),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2146,8 +2146,8 @@ static struct clk_branch gcc_qupv3_wrap0_s1_clk = {
.enable_mask = BIT(11),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2164,8 +2164,8 @@ static struct clk_branch gcc_qupv3_wrap0_s2_clk = {
.enable_mask = BIT(12),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2182,8 +2182,8 @@ static struct clk_branch gcc_qupv3_wrap0_s3_clk = {
.enable_mask = BIT(13),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2200,8 +2200,8 @@ static struct clk_branch gcc_qupv3_wrap0_s4_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2218,8 +2218,8 @@ static struct clk_branch gcc_qupv3_wrap0_s5_clk = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2236,8 +2236,8 @@ static struct clk_branch gcc_qupv3_wrap0_s6_clk = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s6_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s6_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s6_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2254,8 +2254,8 @@ static struct clk_branch gcc_qupv3_wrap0_s7_clk = {
.enable_mask = BIT(17),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s7_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s7_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s7_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2298,8 +2298,8 @@ static struct clk_branch gcc_qupv3_wrap1_s0_clk = {
.enable_mask = BIT(22),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2316,8 +2316,8 @@ static struct clk_branch gcc_qupv3_wrap1_s1_clk = {
.enable_mask = BIT(23),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2334,8 +2334,8 @@ static struct clk_branch gcc_qupv3_wrap1_s2_clk = {
.enable_mask = BIT(24),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2352,8 +2352,8 @@ static struct clk_branch gcc_qupv3_wrap1_s3_clk = {
.enable_mask = BIT(25),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2370,8 +2370,8 @@ static struct clk_branch gcc_qupv3_wrap1_s4_clk = {
.enable_mask = BIT(26),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2388,8 +2388,8 @@ static struct clk_branch gcc_qupv3_wrap1_s5_clk = {
.enable_mask = BIT(27),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2406,8 +2406,8 @@ static struct clk_branch gcc_qupv3_wrap1_s6_clk = {
.enable_mask = BIT(13),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s6_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s6_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s6_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2424,8 +2424,8 @@ static struct clk_branch gcc_qupv3_wrap1_s7_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s7_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s7_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s7_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2515,8 +2515,8 @@ static struct clk_branch gcc_sdcc1_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc1_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc1_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc1_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2535,8 +2535,8 @@ static struct clk_branch gcc_sdcc1_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc1_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc1_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc1_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2566,8 +2566,8 @@ static struct clk_branch gcc_sdcc2_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc2_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc2_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2597,8 +2597,8 @@ static struct clk_branch gcc_sdcc4_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc4_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc4_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2618,8 +2618,8 @@ static struct clk_branch gcc_sys_noc_cpuss_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sys_noc_cpuss_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_cpuss_ahb_postdiv_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_postdiv_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_IS_CRITICAL | CLK_SET_RATE_PARENT,
@@ -2709,8 +2709,8 @@ static struct clk_branch gcc_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2729,8 +2729,8 @@ static struct clk_branch gcc_ufs_phy_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2749,8 +2749,8 @@ static struct clk_branch gcc_ufs_phy_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2767,8 +2767,8 @@ static struct clk_branch gcc_ufs_phy_rx_symbol_0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_rx_symbol_0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_rx_symbol_0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_rx_symbol_0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2785,8 +2785,8 @@ static struct clk_branch gcc_ufs_phy_rx_symbol_1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_rx_symbol_1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_rx_symbol_1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_rx_symbol_1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2803,8 +2803,8 @@ static struct clk_branch gcc_ufs_phy_tx_symbol_0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_tx_symbol_0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_tx_symbol_0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_tx_symbol_0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2823,8 +2823,8 @@ static struct clk_branch gcc_ufs_phy_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2841,8 +2841,8 @@ static struct clk_branch gcc_usb30_prim_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2891,8 +2891,8 @@ static struct clk_branch gcc_usb30_sec_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2941,8 +2941,8 @@ static struct clk_branch gcc_usb3_prim_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2959,8 +2959,8 @@ static struct clk_branch gcc_usb3_prim_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_com_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2979,8 +2979,8 @@ static struct clk_branch gcc_usb3_prim_phy_pipe_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3076,8 +3076,8 @@ static struct clk_branch gcc_usb3_sec_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3094,8 +3094,8 @@ static struct clk_branch gcc_usb3_sec_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_com_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3114,8 +3114,8 @@ static struct clk_branch gcc_usb3_sec_phy_pipe_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gcc-sc8180x.c b/drivers/clk/qcom/gcc-sc8180x.c
index 90525ae1bb3a..c41b9f010585 100644
--- a/drivers/clk/qcom/gcc-sc8180x.c
+++ b/drivers/clk/qcom/gcc-sc8180x.c
@@ -277,7 +277,7 @@ static struct clk_rcg2 gcc_cpuss_ahb_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -300,7 +300,7 @@ static struct clk_rcg2 gcc_emac_ptp_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_emac_ptp_clk_src",
.parent_data = gcc_parents_6,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_6),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -326,7 +326,7 @@ static struct clk_rcg2 gcc_emac_rgmii_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_emac_rgmii_clk_src",
.parent_data = gcc_parents_6,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_6),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -350,7 +350,7 @@ static struct clk_rcg2 gcc_gp1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -365,7 +365,7 @@ static struct clk_rcg2 gcc_gp2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -380,7 +380,7 @@ static struct clk_rcg2 gcc_gp3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -395,7 +395,7 @@ static struct clk_rcg2 gcc_gp4_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp4_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -410,7 +410,7 @@ static struct clk_rcg2 gcc_gp5_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp5_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -436,7 +436,7 @@ static struct clk_rcg2 gcc_npu_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_npu_axi_clk_src",
.parent_data = gcc_parents_3,
- .num_parents = 7,
+ .num_parents = ARRAY_SIZE(gcc_parents_3),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -457,7 +457,7 @@ static struct clk_rcg2 gcc_pcie_0_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -472,7 +472,7 @@ static struct clk_rcg2 gcc_pcie_1_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -487,7 +487,7 @@ static struct clk_rcg2 gcc_pcie_2_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_2_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -502,7 +502,7 @@ static struct clk_rcg2 gcc_pcie_3_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_3_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -523,7 +523,7 @@ static struct clk_rcg2 gcc_pcie_phy_refgen_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_phy_refgen_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -545,7 +545,7 @@ static struct clk_rcg2 gcc_pdm2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -568,7 +568,7 @@ static struct clk_rcg2 gcc_qspi_1_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_1_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -583,7 +583,7 @@ static struct clk_rcg2 gcc_qspi_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -619,7 +619,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s0_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s0_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -634,7 +634,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s1_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -649,7 +649,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -664,7 +664,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s3_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -679,7 +679,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s4_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s4_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -694,7 +694,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s5_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s5_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -709,7 +709,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s6_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s6_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -724,7 +724,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s7_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s7_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -739,7 +739,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s0_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s0_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -754,7 +754,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s1_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -769,7 +769,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -784,7 +784,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s3_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -799,7 +799,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s4_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s4_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -814,7 +814,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s5_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s5_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -829,7 +829,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s0_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s0_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -844,7 +844,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s1_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -859,7 +859,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -874,7 +874,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s3_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -889,7 +889,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s4_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s4_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -904,7 +904,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s5_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s5_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -930,7 +930,7 @@ static struct clk_rcg2 gcc_sdcc2_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk_src",
.parent_data = gcc_parents_7,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_7),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_floor_ops,
},
@@ -955,7 +955,7 @@ static struct clk_rcg2 gcc_sdcc4_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk_src",
.parent_data = gcc_parents_5,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_5),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_floor_ops,
},
@@ -975,7 +975,7 @@ static struct clk_rcg2 gcc_tsif_ref_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_tsif_ref_clk_src",
.parent_data = gcc_parents_8,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_8),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -998,7 +998,7 @@ static struct clk_rcg2 gcc_ufs_card_2_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_2_axi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1013,7 +1013,7 @@ static struct clk_rcg2 gcc_ufs_card_2_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_2_ice_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1033,7 +1033,7 @@ static struct clk_rcg2 gcc_ufs_card_2_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_2_phy_aux_clk_src",
.parent_data = gcc_parents_4,
- .num_parents = 1,
+ .num_parents = ARRAY_SIZE(gcc_parents_4),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1048,7 +1048,7 @@ static struct clk_rcg2 gcc_ufs_card_2_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_2_unipro_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1072,7 +1072,7 @@ static struct clk_rcg2 gcc_ufs_card_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1094,7 +1094,7 @@ static struct clk_rcg2 gcc_ufs_card_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1109,7 +1109,7 @@ static struct clk_rcg2 gcc_ufs_card_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk_src",
.parent_data = gcc_parents_4,
- .num_parents = 1,
+ .num_parents = ARRAY_SIZE(gcc_parents_4),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1131,7 +1131,7 @@ static struct clk_rcg2 gcc_ufs_card_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1155,7 +1155,7 @@ static struct clk_rcg2 gcc_ufs_phy_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1170,7 +1170,7 @@ static struct clk_rcg2 gcc_ufs_phy_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1185,7 +1185,7 @@ static struct clk_rcg2 gcc_ufs_phy_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk_src",
.parent_data = gcc_parents_4,
- .num_parents = 1,
+ .num_parents = ARRAY_SIZE(gcc_parents_4),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1200,7 +1200,7 @@ static struct clk_rcg2 gcc_ufs_phy_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1224,7 +1224,7 @@ static struct clk_rcg2 gcc_usb30_mp_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_mp_master_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1247,7 +1247,7 @@ static struct clk_rcg2 gcc_usb30_mp_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_mp_mock_utmi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1262,7 +1262,7 @@ static struct clk_rcg2 gcc_usb30_prim_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1277,7 +1277,7 @@ static struct clk_rcg2 gcc_usb30_prim_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1292,7 +1292,7 @@ static struct clk_rcg2 gcc_usb30_sec_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1307,7 +1307,7 @@ static struct clk_rcg2 gcc_usb30_sec_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_mock_utmi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1322,7 +1322,7 @@ static struct clk_rcg2 gcc_usb3_mp_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_mp_phy_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1337,7 +1337,7 @@ static struct clk_rcg2 gcc_usb3_prim_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1352,7 +1352,7 @@ static struct clk_rcg2 gcc_usb3_sec_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
diff --git a/drivers/clk/qcom/gcc-sdm845.c b/drivers/clk/qcom/gcc-sdm845.c
index 90f7febaf528..58aa3ec9a7fc 100644
--- a/drivers/clk/qcom/gcc-sdm845.c
+++ b/drivers/clk/qcom/gcc-sdm845.c
@@ -28,25 +28,81 @@
enum {
P_BI_TCXO,
P_AUD_REF_CLK,
- P_CORE_BI_PLL_TEST_SE,
P_GPLL0_OUT_EVEN,
P_GPLL0_OUT_MAIN,
P_GPLL4_OUT_MAIN,
P_SLEEP_CLK,
};
+static struct clk_alpha_pll gpll0 = {
+ .offset = 0x0,
+ .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
+ .clkr = {
+ .enable_reg = 0x52000,
+ .enable_mask = BIT(0),
+ .hw.init = &(struct clk_init_data){
+ .name = "gpll0",
+ .parent_data = &(const struct clk_parent_data){
+ .fw_name = "bi_tcxo", .name = "bi_tcxo",
+ },
+ .num_parents = 1,
+ .ops = &clk_alpha_pll_fixed_fabia_ops,
+ },
+ },
+};
+
+static struct clk_alpha_pll gpll4 = {
+ .offset = 0x76000,
+ .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
+ .clkr = {
+ .enable_reg = 0x52000,
+ .enable_mask = BIT(4),
+ .hw.init = &(struct clk_init_data){
+ .name = "gpll4",
+ .parent_data = &(const struct clk_parent_data){
+ .fw_name = "bi_tcxo", .name = "bi_tcxo",
+ },
+ .num_parents = 1,
+ .ops = &clk_alpha_pll_fixed_fabia_ops,
+ },
+ },
+};
+
+static const struct clk_div_table post_div_table_fabia_even[] = {
+ { 0x0, 1 },
+ { 0x1, 2 },
+ { 0x3, 4 },
+ { 0x7, 8 },
+ { }
+};
+
+static struct clk_alpha_pll_postdiv gpll0_out_even = {
+ .offset = 0x0,
+ .post_div_shift = 8,
+ .post_div_table = post_div_table_fabia_even,
+ .num_post_div = ARRAY_SIZE(post_div_table_fabia_even),
+ .width = 4,
+ .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
+ .clkr.hw.init = &(struct clk_init_data){
+ .name = "gpll0_out_even",
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
+ },
+ .num_parents = 1,
+ .ops = &clk_alpha_pll_postdiv_fabia_ops,
+ },
+};
+
static const struct parent_map gcc_parent_map_0[] = {
{ P_BI_TCXO, 0 },
{ P_GPLL0_OUT_MAIN, 1 },
{ P_GPLL0_OUT_EVEN, 6 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const gcc_parent_names_0[] = {
- "bi_tcxo",
- "gpll0",
- "gpll0_out_even",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_0[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &gpll0.clkr.hw },
+ { .hw = &gpll0_out_even.clkr.hw },
};
static const struct parent_map gcc_parent_map_1[] = {
@@ -54,49 +110,41 @@ static const struct parent_map gcc_parent_map_1[] = {
{ P_GPLL0_OUT_MAIN, 1 },
{ P_SLEEP_CLK, 5 },
{ P_GPLL0_OUT_EVEN, 6 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const gcc_parent_names_1[] = {
- "bi_tcxo",
- "gpll0",
- "core_pi_sleep_clk",
- "gpll0_out_even",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_1[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &gpll0.clkr.hw },
+ { .fw_name = "sleep_clk", .name = "core_pi_sleep_clk" },
+ { .hw = &gpll0_out_even.clkr.hw },
};
static const struct parent_map gcc_parent_map_2[] = {
{ P_BI_TCXO, 0 },
{ P_SLEEP_CLK, 5 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const gcc_parent_names_2[] = {
- "bi_tcxo",
- "core_pi_sleep_clk",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_2[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .fw_name = "sleep_clk", .name = "core_pi_sleep_clk" },
};
static const struct parent_map gcc_parent_map_3[] = {
{ P_BI_TCXO, 0 },
{ P_GPLL0_OUT_MAIN, 1 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const gcc_parent_names_3[] = {
- "bi_tcxo",
- "gpll0",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_3[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &gpll0.clkr.hw },
};
static const struct parent_map gcc_parent_map_4[] = {
{ P_BI_TCXO, 0 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const gcc_parent_names_4[] = {
- "bi_tcxo",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_4[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
};
static const struct parent_map gcc_parent_map_6[] = {
@@ -104,34 +152,32 @@ static const struct parent_map gcc_parent_map_6[] = {
{ P_GPLL0_OUT_MAIN, 1 },
{ P_AUD_REF_CLK, 2 },
{ P_GPLL0_OUT_EVEN, 6 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
};
-static const char * const gcc_parent_names_6[] = {
- "bi_tcxo",
- "gpll0",
- "aud_ref_clk",
- "gpll0_out_even",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_6[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &gpll0.clkr.hw },
+ { .fw_name = "aud_ref_clk", .name = "aud_ref_clk" },
+ { .hw = &gpll0_out_even.clkr.hw },
};
-static const char * const gcc_parent_names_7_ao[] = {
- "bi_tcxo_ao",
- "gpll0",
- "gpll0_out_even",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_7_ao[] = {
+ { .fw_name = "bi_tcxo_ao", .name = "bi_tcxo_ao" },
+ { .hw = &gpll0.clkr.hw },
+ { .hw = &gpll0_out_even.clkr.hw },
+ { .fw_name = "core_bi_pll_test_se", .name = "core_bi_pll_test_se" },
};
-static const char * const gcc_parent_names_8[] = {
- "bi_tcxo",
- "gpll0",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_8[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &gpll0.clkr.hw },
+ { .fw_name = "core_bi_pll_test_se", .name = "core_bi_pll_test_se" },
};
-static const char * const gcc_parent_names_8_ao[] = {
- "bi_tcxo_ao",
- "gpll0",
- "core_bi_pll_test_se",
+static const struct clk_parent_data gcc_parent_data_8_ao[] = {
+ { .fw_name = "bi_tcxo_ao", .name = "bi_tcxo_ao" },
+ { .hw = &gpll0.clkr.hw },
+ { .fw_name = "core_bi_pll_test_se", .name = "core_bi_pll_test_se" },
};
static const struct parent_map gcc_parent_map_10[] = {
@@ -139,69 +185,15 @@ static const struct parent_map gcc_parent_map_10[] = {
{ P_GPLL0_OUT_MAIN, 1 },
{ P_GPLL4_OUT_MAIN, 5 },
{ P_GPLL0_OUT_EVEN, 6 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
-};
-
-static const char * const gcc_parent_names_10[] = {
- "bi_tcxo",
- "gpll0",
- "gpll4",
- "gpll0_out_even",
- "core_bi_pll_test_se",
-};
-
-static struct clk_alpha_pll gpll0 = {
- .offset = 0x0,
- .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
- .clkr = {
- .enable_reg = 0x52000,
- .enable_mask = BIT(0),
- .hw.init = &(struct clk_init_data){
- .name = "gpll0",
- .parent_names = (const char *[]){ "bi_tcxo" },
- .num_parents = 1,
- .ops = &clk_alpha_pll_fixed_fabia_ops,
- },
- },
-};
-
-static struct clk_alpha_pll gpll4 = {
- .offset = 0x76000,
- .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
- .clkr = {
- .enable_reg = 0x52000,
- .enable_mask = BIT(4),
- .hw.init = &(struct clk_init_data){
- .name = "gpll4",
- .parent_names = (const char *[]){ "bi_tcxo" },
- .num_parents = 1,
- .ops = &clk_alpha_pll_fixed_fabia_ops,
- },
- },
};
-static const struct clk_div_table post_div_table_fabia_even[] = {
- { 0x0, 1 },
- { 0x1, 2 },
- { 0x3, 4 },
- { 0x7, 8 },
- { }
+static const struct clk_parent_data gcc_parent_data_10[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &gpll0.clkr.hw },
+ { .hw = &gpll4.clkr.hw },
+ { .hw = &gpll0_out_even.clkr.hw },
};
-static struct clk_alpha_pll_postdiv gpll0_out_even = {
- .offset = 0x0,
- .post_div_shift = 8,
- .post_div_table = post_div_table_fabia_even,
- .num_post_div = ARRAY_SIZE(post_div_table_fabia_even),
- .width = 4,
- .regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_FABIA],
- .clkr.hw.init = &(struct clk_init_data){
- .name = "gpll0_out_even",
- .parent_names = (const char *[]){ "gpll0" },
- .num_parents = 1,
- .ops = &clk_alpha_pll_postdiv_fabia_ops,
- },
-};
static const struct freq_tbl ftbl_gcc_cpuss_ahb_clk_src[] = {
F(19200000, P_BI_TCXO, 1, 0, 0),
@@ -216,8 +208,8 @@ static struct clk_rcg2 gcc_cpuss_ahb_clk_src = {
.freq_tbl = ftbl_gcc_cpuss_ahb_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk_src",
- .parent_names = gcc_parent_names_7_ao,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_7_ao,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_7_ao),
.ops = &clk_rcg2_ops,
},
};
@@ -235,8 +227,8 @@ static struct clk_rcg2 gcc_cpuss_rbcpr_clk_src = {
.freq_tbl = ftbl_gcc_cpuss_rbcpr_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_rbcpr_clk_src",
- .parent_names = gcc_parent_names_8_ao,
- .num_parents = 3,
+ .parent_data = gcc_parent_data_8_ao,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_8_ao),
.ops = &clk_rcg2_ops,
},
};
@@ -258,8 +250,8 @@ static struct clk_rcg2 gcc_gp1_clk_src = {
.freq_tbl = ftbl_gcc_gp1_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk_src",
- .parent_names = gcc_parent_names_1,
- .num_parents = 5,
+ .parent_data = gcc_parent_data_1,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.ops = &clk_rcg2_ops,
},
};
@@ -272,8 +264,8 @@ static struct clk_rcg2 gcc_gp2_clk_src = {
.freq_tbl = ftbl_gcc_gp1_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk_src",
- .parent_names = gcc_parent_names_1,
- .num_parents = 5,
+ .parent_data = gcc_parent_data_1,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.ops = &clk_rcg2_ops,
},
};
@@ -286,8 +278,8 @@ static struct clk_rcg2 gcc_gp3_clk_src = {
.freq_tbl = ftbl_gcc_gp1_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk_src",
- .parent_names = gcc_parent_names_1,
- .num_parents = 5,
+ .parent_data = gcc_parent_data_1,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.ops = &clk_rcg2_ops,
},
};
@@ -306,8 +298,8 @@ static struct clk_rcg2 gcc_pcie_0_aux_clk_src = {
.freq_tbl = ftbl_gcc_pcie_0_aux_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk_src",
- .parent_names = gcc_parent_names_2,
- .num_parents = 3,
+ .parent_data = gcc_parent_data_2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -320,8 +312,8 @@ static struct clk_rcg2 gcc_pcie_1_aux_clk_src = {
.freq_tbl = ftbl_gcc_pcie_0_aux_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk_src",
- .parent_names = gcc_parent_names_2,
- .num_parents = 3,
+ .parent_data = gcc_parent_data_2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -340,8 +332,8 @@ static struct clk_rcg2 gcc_pcie_phy_refgen_clk_src = {
.freq_tbl = ftbl_gcc_pcie_phy_refgen_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_phy_refgen_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -362,8 +354,8 @@ static struct clk_rcg2 gcc_qspi_core_clk_src = {
.freq_tbl = ftbl_gcc_qspi_core_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_core_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_floor_ops,
},
};
@@ -383,8 +375,8 @@ static struct clk_rcg2 gcc_pdm2_clk_src = {
.freq_tbl = ftbl_gcc_pdm2_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -410,8 +402,8 @@ static const struct freq_tbl ftbl_gcc_qupv3_wrap0_s0_clk_src[] = {
static struct clk_init_data gcc_qupv3_wrap0_s0_clk_src_init = {
.name = "gcc_qupv3_wrap0_s0_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -426,8 +418,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s1_clk_src_init = {
.name = "gcc_qupv3_wrap0_s1_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -442,8 +434,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s1_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s2_clk_src_init = {
.name = "gcc_qupv3_wrap0_s2_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -458,8 +450,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s3_clk_src_init = {
.name = "gcc_qupv3_wrap0_s3_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -474,8 +466,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s4_clk_src_init = {
.name = "gcc_qupv3_wrap0_s4_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -490,8 +482,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s5_clk_src_init = {
.name = "gcc_qupv3_wrap0_s5_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -506,8 +498,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s5_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s6_clk_src_init = {
.name = "gcc_qupv3_wrap0_s6_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -522,8 +514,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s6_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s7_clk_src_init = {
.name = "gcc_qupv3_wrap0_s7_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -538,8 +530,8 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s7_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s0_clk_src_init = {
.name = "gcc_qupv3_wrap1_s0_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -554,8 +546,8 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s1_clk_src_init = {
.name = "gcc_qupv3_wrap1_s1_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -570,8 +562,8 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s1_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s2_clk_src_init = {
.name = "gcc_qupv3_wrap1_s2_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -586,8 +578,8 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s3_clk_src_init = {
.name = "gcc_qupv3_wrap1_s3_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -602,8 +594,8 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s4_clk_src_init = {
.name = "gcc_qupv3_wrap1_s4_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -618,8 +610,8 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s5_clk_src_init = {
.name = "gcc_qupv3_wrap1_s5_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -634,8 +626,8 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s5_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s6_clk_src_init = {
.name = "gcc_qupv3_wrap1_s6_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -650,8 +642,8 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s6_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s7_clk_src_init = {
.name = "gcc_qupv3_wrap1_s7_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
};
@@ -683,8 +675,8 @@ static struct clk_rcg2 gcc_sdcc2_apps_clk_src = {
.freq_tbl = ftbl_gcc_sdcc2_apps_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk_src",
- .parent_names = gcc_parent_names_10,
- .num_parents = 5,
+ .parent_data = gcc_parent_data_10,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_10),
.ops = &clk_rcg2_floor_ops,
},
};
@@ -707,8 +699,8 @@ static struct clk_rcg2 gcc_sdcc4_apps_clk_src = {
.freq_tbl = ftbl_gcc_sdcc4_apps_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_floor_ops,
},
};
@@ -726,8 +718,8 @@ static struct clk_rcg2 gcc_tsif_ref_clk_src = {
.freq_tbl = ftbl_gcc_tsif_ref_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_tsif_ref_clk_src",
- .parent_names = gcc_parent_names_6,
- .num_parents = 5,
+ .parent_data = gcc_parent_data_6,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_6),
.ops = &clk_rcg2_ops,
},
};
@@ -749,8 +741,8 @@ static struct clk_rcg2 gcc_ufs_card_axi_clk_src = {
.freq_tbl = ftbl_gcc_ufs_card_axi_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -771,8 +763,8 @@ static struct clk_rcg2 gcc_ufs_card_ice_core_clk_src = {
.freq_tbl = ftbl_gcc_ufs_card_ice_core_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -785,8 +777,8 @@ static struct clk_rcg2 gcc_ufs_card_phy_aux_clk_src = {
.freq_tbl = ftbl_gcc_cpuss_rbcpr_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk_src",
- .parent_names = gcc_parent_names_4,
- .num_parents = 2,
+ .parent_data = gcc_parent_data_4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_4),
.ops = &clk_rcg2_ops,
},
};
@@ -806,8 +798,8 @@ static struct clk_rcg2 gcc_ufs_card_unipro_core_clk_src = {
.freq_tbl = ftbl_gcc_ufs_card_unipro_core_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -829,8 +821,8 @@ static struct clk_rcg2 gcc_ufs_phy_axi_clk_src = {
.freq_tbl = ftbl_gcc_ufs_phy_axi_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -843,8 +835,8 @@ static struct clk_rcg2 gcc_ufs_phy_ice_core_clk_src = {
.freq_tbl = ftbl_gcc_ufs_card_ice_core_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -857,8 +849,8 @@ static struct clk_rcg2 gcc_ufs_phy_phy_aux_clk_src = {
.freq_tbl = ftbl_gcc_pcie_0_aux_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk_src",
- .parent_names = gcc_parent_names_4,
- .num_parents = 2,
+ .parent_data = gcc_parent_data_4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_4),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -871,8 +863,8 @@ static struct clk_rcg2 gcc_ufs_phy_unipro_core_clk_src = {
.freq_tbl = ftbl_gcc_ufs_card_unipro_core_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -894,8 +886,8 @@ static struct clk_rcg2 gcc_usb30_prim_master_clk_src = {
.freq_tbl = ftbl_gcc_usb30_prim_master_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -916,8 +908,8 @@ static struct clk_rcg2 gcc_usb30_prim_mock_utmi_clk_src = {
.freq_tbl = ftbl_gcc_usb30_prim_mock_utmi_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -930,8 +922,8 @@ static struct clk_rcg2 gcc_usb30_sec_master_clk_src = {
.freq_tbl = ftbl_gcc_usb30_prim_master_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -944,8 +936,8 @@ static struct clk_rcg2 gcc_usb30_sec_mock_utmi_clk_src = {
.freq_tbl = ftbl_gcc_usb30_prim_mock_utmi_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_mock_utmi_clk_src",
- .parent_names = gcc_parent_names_0,
- .num_parents = 4,
+ .parent_data = gcc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -958,8 +950,8 @@ static struct clk_rcg2 gcc_usb3_prim_phy_aux_clk_src = {
.freq_tbl = ftbl_gcc_cpuss_rbcpr_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk_src",
- .parent_names = gcc_parent_names_2,
- .num_parents = 3,
+ .parent_data = gcc_parent_data_2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -972,8 +964,8 @@ static struct clk_rcg2 gcc_usb3_sec_phy_aux_clk_src = {
.freq_tbl = ftbl_gcc_cpuss_rbcpr_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk_src",
- .parent_names = gcc_parent_names_2,
- .num_parents = 3,
+ .parent_data = gcc_parent_data_2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -986,8 +978,8 @@ static struct clk_rcg2 gcc_vs_ctrl_clk_src = {
.freq_tbl = ftbl_gcc_cpuss_rbcpr_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_vs_ctrl_clk_src",
- .parent_names = gcc_parent_names_3,
- .num_parents = 3,
+ .parent_data = gcc_parent_data_3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_3),
.ops = &clk_rcg2_ops,
},
};
@@ -1007,8 +999,8 @@ static struct clk_rcg2 gcc_vsensor_clk_src = {
.freq_tbl = ftbl_gcc_vsensor_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_vsensor_clk_src",
- .parent_names = gcc_parent_names_8,
- .num_parents = 3,
+ .parent_data = gcc_parent_data_8,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_8),
.ops = &clk_rcg2_ops,
},
};
@@ -1036,8 +1028,8 @@ static struct clk_branch gcc_aggre_ufs_card_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_card_axi_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_card_axi_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1056,8 +1048,8 @@ static struct clk_branch gcc_aggre_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_phy_axi_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_phy_axi_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1074,8 +1066,8 @@ static struct clk_branch gcc_aggre_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_prim_axi_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_prim_master_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1092,8 +1084,8 @@ static struct clk_branch gcc_aggre_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_sec_axi_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_sec_master_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1110,8 +1102,8 @@ static struct clk_branch gcc_apc_vs_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_apc_vs_clk",
- .parent_names = (const char *[]){
- "gcc_vsensor_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_vsensor_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1227,8 +1219,8 @@ static struct clk_branch gcc_cfg_noc_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_prim_axi_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_prim_master_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1245,8 +1237,8 @@ static struct clk_branch gcc_cfg_noc_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_sec_axi_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_sec_master_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1263,8 +1255,8 @@ static struct clk_branch gcc_cpuss_ahb_clk = {
.enable_mask = BIT(21),
.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk",
- .parent_names = (const char *[]){
- "gcc_cpuss_ahb_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT | CLK_IS_CRITICAL,
@@ -1281,8 +1273,8 @@ static struct clk_branch gcc_cpuss_rbcpr_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_rbcpr_clk",
- .parent_names = (const char *[]){
- "gcc_cpuss_rbcpr_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_rbcpr_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1340,8 +1332,8 @@ static struct clk_branch gcc_disp_gpll0_clk_src = {
.enable_mask = BIT(18),
.hw.init = &(struct clk_init_data){
.name = "gcc_disp_gpll0_clk_src",
- .parent_names = (const char *[]){
- "gpll0",
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_aon_ops,
@@ -1356,8 +1348,8 @@ static struct clk_branch gcc_disp_gpll0_div_clk_src = {
.enable_mask = BIT(19),
.hw.init = &(struct clk_init_data){
.name = "gcc_disp_gpll0_div_clk_src",
- .parent_names = (const char *[]){
- "gpll0_out_even",
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0_out_even.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -1387,8 +1379,8 @@ static struct clk_branch gcc_gp1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk",
- .parent_names = (const char *[]){
- "gcc_gp1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1405,8 +1397,8 @@ static struct clk_branch gcc_gp2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk",
- .parent_names = (const char *[]){
- "gcc_gp2_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1423,8 +1415,8 @@ static struct clk_branch gcc_gp3_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk",
- .parent_names = (const char *[]){
- "gcc_gp3_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1456,8 +1448,8 @@ static struct clk_branch gcc_gpu_gpll0_clk_src = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_clk_src",
- .parent_names = (const char *[]){
- "gpll0",
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -1472,8 +1464,8 @@ static struct clk_branch gcc_gpu_gpll0_div_clk_src = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_div_clk_src",
- .parent_names = (const char *[]){
- "gpll0_out_even",
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0_out_even.clkr.hw,
},
.num_parents = 1,
.ops = &clk_branch2_ops,
@@ -1528,8 +1520,8 @@ static struct clk_branch gcc_gpu_vs_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_vs_clk",
- .parent_names = (const char *[]){
- "gcc_vsensor_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_vsensor_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1627,8 +1619,8 @@ static struct clk_branch gcc_mss_vs_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_mss_vs_clk",
- .parent_names = (const char *[]){
- "gcc_vsensor_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_vsensor_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1645,8 +1637,8 @@ static struct clk_branch gcc_pcie_0_aux_clk = {
.enable_mask = BIT(3),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk",
- .parent_names = (const char *[]){
- "gcc_pcie_0_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1703,7 +1695,9 @@ static struct clk_branch gcc_pcie_0_pipe_clk = {
.enable_mask = BIT(4),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_pipe_clk",
- .parent_names = (const char *[]){ "pcie_0_pipe_clk" },
+ .parent_data = &(const struct clk_parent_data){
+ .fw_name = "pcie_0_pipe_clk", .name = "pcie_0_pipe_clk",
+ },
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_branch2_ops,
@@ -1747,8 +1741,8 @@ static struct clk_branch gcc_pcie_1_aux_clk = {
.enable_mask = BIT(29),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk",
- .parent_names = (const char *[]){
- "gcc_pcie_1_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1805,7 +1799,9 @@ static struct clk_branch gcc_pcie_1_pipe_clk = {
.enable_mask = BIT(30),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_pipe_clk",
- .parent_names = (const char *[]){ "pcie_1_pipe_clk" },
+ .parent_data = &(const struct clk_parent_data){
+ .fw_name = "pcie_1_pipe_clk", .name = "pcie_1_pipe_clk",
+ },
.num_parents = 1,
.ops = &clk_branch2_ops,
},
@@ -1848,8 +1844,8 @@ static struct clk_branch gcc_pcie_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_phy_aux_clk",
- .parent_names = (const char *[]){
- "gcc_pcie_0_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1866,8 +1862,8 @@ static struct clk_branch gcc_pcie_phy_refgen_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_phy_refgen_clk",
- .parent_names = (const char *[]){
- "gcc_pcie_phy_refgen_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_phy_refgen_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1884,8 +1880,8 @@ static struct clk_branch gcc_pdm2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk",
- .parent_names = (const char *[]){
- "gcc_pdm2_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pdm2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2003,8 +1999,8 @@ static struct clk_branch gcc_qspi_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_core_clk",
- .parent_names = (const char *[]){
- "gcc_qspi_core_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qspi_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2021,8 +2017,8 @@ static struct clk_branch gcc_qupv3_wrap0_s0_clk = {
.enable_mask = BIT(10),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s0_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s0_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2039,8 +2035,8 @@ static struct clk_branch gcc_qupv3_wrap0_s1_clk = {
.enable_mask = BIT(11),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s1_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2057,8 +2053,8 @@ static struct clk_branch gcc_qupv3_wrap0_s2_clk = {
.enable_mask = BIT(12),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s2_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s2_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2075,8 +2071,8 @@ static struct clk_branch gcc_qupv3_wrap0_s3_clk = {
.enable_mask = BIT(13),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s3_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s3_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2093,8 +2089,8 @@ static struct clk_branch gcc_qupv3_wrap0_s4_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s4_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s4_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2111,8 +2107,8 @@ static struct clk_branch gcc_qupv3_wrap0_s5_clk = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s5_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s5_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2129,8 +2125,8 @@ static struct clk_branch gcc_qupv3_wrap0_s6_clk = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s6_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s6_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s6_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2147,8 +2143,8 @@ static struct clk_branch gcc_qupv3_wrap0_s7_clk = {
.enable_mask = BIT(17),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s7_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap0_s7_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s7_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2165,8 +2161,8 @@ static struct clk_branch gcc_qupv3_wrap1_s0_clk = {
.enable_mask = BIT(22),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s0_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s0_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2183,8 +2179,8 @@ static struct clk_branch gcc_qupv3_wrap1_s1_clk = {
.enable_mask = BIT(23),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s1_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s1_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2201,8 +2197,8 @@ static struct clk_branch gcc_qupv3_wrap1_s2_clk = {
.enable_mask = BIT(24),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s2_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s2_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2219,8 +2215,8 @@ static struct clk_branch gcc_qupv3_wrap1_s3_clk = {
.enable_mask = BIT(25),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s3_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s3_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2237,8 +2233,8 @@ static struct clk_branch gcc_qupv3_wrap1_s4_clk = {
.enable_mask = BIT(26),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s4_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s4_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2255,8 +2251,8 @@ static struct clk_branch gcc_qupv3_wrap1_s5_clk = {
.enable_mask = BIT(27),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s5_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s5_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2273,8 +2269,8 @@ static struct clk_branch gcc_qupv3_wrap1_s6_clk = {
.enable_mask = BIT(28),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s6_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s6_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s6_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2291,8 +2287,8 @@ static struct clk_branch gcc_qupv3_wrap1_s7_clk = {
.enable_mask = BIT(29),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s7_clk",
- .parent_names = (const char *[]){
- "gcc_qupv3_wrap1_s7_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s7_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2378,8 +2374,8 @@ static struct clk_branch gcc_sdcc2_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk",
- .parent_names = (const char *[]){
- "gcc_sdcc2_apps_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc2_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2409,8 +2405,8 @@ static struct clk_branch gcc_sdcc4_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk",
- .parent_names = (const char *[]){
- "gcc_sdcc4_apps_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc4_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2427,8 +2423,8 @@ static struct clk_branch gcc_sys_noc_cpuss_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sys_noc_cpuss_ahb_clk",
- .parent_names = (const char *[]){
- "gcc_cpuss_ahb_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT | CLK_IS_CRITICAL,
@@ -2471,8 +2467,8 @@ static struct clk_branch gcc_tsif_ref_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_tsif_ref_clk",
- .parent_names = (const char *[]){
- "gcc_tsif_ref_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_tsif_ref_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2506,8 +2502,8 @@ static struct clk_branch gcc_ufs_card_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_card_axi_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2539,8 +2535,8 @@ static struct clk_branch gcc_ufs_card_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_card_ice_core_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2559,8 +2555,8 @@ static struct clk_branch gcc_ufs_card_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_card_phy_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2615,8 +2611,8 @@ static struct clk_branch gcc_ufs_card_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_card_unipro_core_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2663,8 +2659,8 @@ static struct clk_branch gcc_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_phy_axi_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2683,8 +2679,8 @@ static struct clk_branch gcc_ufs_phy_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_phy_ice_core_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2703,8 +2699,8 @@ static struct clk_branch gcc_ufs_phy_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_phy_phy_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2759,8 +2755,8 @@ static struct clk_branch gcc_ufs_phy_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk",
- .parent_names = (const char *[]){
- "gcc_ufs_phy_unipro_core_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2777,8 +2773,8 @@ static struct clk_branch gcc_usb30_prim_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_prim_master_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2795,8 +2791,8 @@ static struct clk_branch gcc_usb30_prim_mock_utmi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_prim_mock_utmi_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2826,8 +2822,8 @@ static struct clk_branch gcc_usb30_sec_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_sec_master_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2844,8 +2840,8 @@ static struct clk_branch gcc_usb30_sec_mock_utmi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_mock_utmi_clk",
- .parent_names = (const char *[]){
- "gcc_usb30_sec_mock_utmi_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2888,8 +2884,8 @@ static struct clk_branch gcc_usb3_prim_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk",
- .parent_names = (const char *[]){
- "gcc_usb3_prim_phy_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2906,8 +2902,8 @@ static struct clk_branch gcc_usb3_prim_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_com_aux_clk",
- .parent_names = (const char *[]){
- "gcc_usb3_prim_phy_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2949,8 +2945,8 @@ static struct clk_branch gcc_usb3_sec_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk",
- .parent_names = (const char *[]){
- "gcc_usb3_sec_phy_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2967,8 +2963,8 @@ static struct clk_branch gcc_usb3_sec_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_com_aux_clk",
- .parent_names = (const char *[]){
- "gcc_usb3_sec_phy_aux_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3012,8 +3008,8 @@ static struct clk_branch gcc_vdda_vs_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_vdda_vs_clk",
- .parent_names = (const char *[]){
- "gcc_vsensor_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_vsensor_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3030,8 +3026,8 @@ static struct clk_branch gcc_vddcx_vs_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_vddcx_vs_clk",
- .parent_names = (const char *[]){
- "gcc_vsensor_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_vsensor_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3048,8 +3044,8 @@ static struct clk_branch gcc_vddmx_vs_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_vddmx_vs_clk",
- .parent_names = (const char *[]){
- "gcc_vsensor_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_vsensor_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3124,8 +3120,8 @@ static struct clk_branch gcc_vs_ctrl_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_vs_ctrl_clk",
- .parent_names = (const char *[]){
- "gcc_vs_ctrl_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_vs_ctrl_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gcc-sdx55.c b/drivers/clk/qcom/gcc-sdx55.c
index e3b9030b2bae..4fca19006a77 100644
--- a/drivers/clk/qcom/gcc-sdx55.c
+++ b/drivers/clk/qcom/gcc-sdx55.c
@@ -70,8 +70,8 @@ static struct clk_alpha_pll_postdiv gpll0_out_even = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data){
.name = "gpll0_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_lucid_ops,
@@ -106,8 +106,8 @@ static struct clk_alpha_pll_postdiv gpll4_out_even = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data){
.name = "gpll4_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll4.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll4.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_lucid_ops,
diff --git a/drivers/clk/qcom/gcc-sm8150.c b/drivers/clk/qcom/gcc-sm8150.c
index 8e9b5b3cceaf..245794485719 100644
--- a/drivers/clk/qcom/gcc-sm8150.c
+++ b/drivers/clk/qcom/gcc-sm8150.c
@@ -69,8 +69,8 @@ static struct clk_alpha_pll_postdiv gpll0_out_even = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data){
.name = "gpll0_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_trion_ops,
@@ -241,7 +241,7 @@ static struct clk_rcg2 gcc_cpuss_ahb_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -264,7 +264,7 @@ static struct clk_rcg2 gcc_emac_ptp_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_emac_ptp_clk_src",
.parent_data = gcc_parents_5,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_5),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -290,7 +290,7 @@ static struct clk_rcg2 gcc_emac_rgmii_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_emac_rgmii_clk_src",
.parent_data = gcc_parents_5,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_5),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -314,7 +314,7 @@ static struct clk_rcg2 gcc_gp1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -329,7 +329,7 @@ static struct clk_rcg2 gcc_gp2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -344,7 +344,7 @@ static struct clk_rcg2 gcc_gp3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk_src",
.parent_data = gcc_parents_1,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -365,7 +365,7 @@ static struct clk_rcg2 gcc_pcie_0_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -380,7 +380,7 @@ static struct clk_rcg2 gcc_pcie_1_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -401,7 +401,7 @@ static struct clk_rcg2 gcc_pcie_phy_refgen_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_phy_refgen_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -423,7 +423,7 @@ static struct clk_rcg2 gcc_pdm2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -446,7 +446,7 @@ static struct clk_rcg2 gcc_qspi_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qspi_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -480,7 +480,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s0_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s0_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -495,7 +495,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s1_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -510,7 +510,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -525,7 +525,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s3_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -540,7 +540,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s4_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s4_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -555,7 +555,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s5_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s5_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -570,7 +570,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s6_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s6_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -585,7 +585,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s7_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s7_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -600,7 +600,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s0_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s0_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -615,7 +615,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s1_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -630,7 +630,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -645,7 +645,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s3_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -660,7 +660,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s4_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s4_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -675,7 +675,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s5_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s5_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -690,7 +690,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s0_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s0_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -705,7 +705,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s1_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -720,7 +720,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s2_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -735,7 +735,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s3_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -750,7 +750,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s4_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s4_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -765,7 +765,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s5_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s5_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -791,7 +791,7 @@ static struct clk_rcg2 gcc_sdcc2_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk_src",
.parent_data = gcc_parents_6,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_6),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_floor_ops,
},
@@ -816,7 +816,7 @@ static struct clk_rcg2 gcc_sdcc4_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk_src",
.parent_data = gcc_parents_3,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_3),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_floor_ops,
},
@@ -836,7 +836,7 @@ static struct clk_rcg2 gcc_tsif_ref_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_tsif_ref_clk_src",
.parent_data = gcc_parents_7,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parents_7),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -860,7 +860,7 @@ static struct clk_rcg2 gcc_ufs_card_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -883,7 +883,7 @@ static struct clk_rcg2 gcc_ufs_card_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -903,7 +903,7 @@ static struct clk_rcg2 gcc_ufs_card_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk_src",
.parent_data = gcc_parents_4,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_4),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -925,7 +925,7 @@ static struct clk_rcg2 gcc_ufs_card_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -949,7 +949,7 @@ static struct clk_rcg2 gcc_ufs_phy_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -964,7 +964,7 @@ static struct clk_rcg2 gcc_ufs_phy_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -979,7 +979,7 @@ static struct clk_rcg2 gcc_ufs_phy_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk_src",
.parent_data = gcc_parents_4,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parents_4),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -994,7 +994,7 @@ static struct clk_rcg2 gcc_ufs_phy_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1018,7 +1018,7 @@ static struct clk_rcg2 gcc_usb30_prim_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1040,7 +1040,7 @@ static struct clk_rcg2 gcc_usb30_prim_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1055,7 +1055,7 @@ static struct clk_rcg2 gcc_usb30_sec_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1070,7 +1070,7 @@ static struct clk_rcg2 gcc_usb30_sec_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_mock_utmi_clk_src",
.parent_data = gcc_parents_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parents_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1085,7 +1085,7 @@ static struct clk_rcg2 gcc_usb3_prim_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1100,7 +1100,7 @@ static struct clk_rcg2 gcc_usb3_sec_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk_src",
.parent_data = gcc_parents_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parents_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
diff --git a/drivers/clk/qcom/gcc-sm8250.c b/drivers/clk/qcom/gcc-sm8250.c
index ab594a0f0c40..9755ef4888c1 100644
--- a/drivers/clk/qcom/gcc-sm8250.c
+++ b/drivers/clk/qcom/gcc-sm8250.c
@@ -25,7 +25,6 @@
enum {
P_BI_TCXO,
P_AUD_REF_CLK,
- P_CORE_BI_PLL_TEST_SE,
P_GPLL0_OUT_EVEN,
P_GPLL0_OUT_MAIN,
P_GPLL4_OUT_MAIN,
@@ -64,8 +63,8 @@ static struct clk_alpha_pll_postdiv gpll0_out_even = {
.regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_LUCID],
.clkr.hw.init = &(struct clk_init_data){
.name = "gpll0_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_lucid_ops,
@@ -200,7 +199,7 @@ static struct clk_rcg2 gcc_cpuss_ahb_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk_src",
.parent_data = gcc_parent_data_0_ao,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0_ao),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -224,7 +223,7 @@ static struct clk_rcg2 gcc_gp1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk_src",
.parent_data = gcc_parent_data_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.ops = &clk_rcg2_ops,
},
};
@@ -238,7 +237,7 @@ static struct clk_rcg2 gcc_gp2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk_src",
.parent_data = gcc_parent_data_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.ops = &clk_rcg2_ops,
},
};
@@ -252,7 +251,7 @@ static struct clk_rcg2 gcc_gp3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk_src",
.parent_data = gcc_parent_data_1,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.ops = &clk_rcg2_ops,
},
};
@@ -272,7 +271,7 @@ static struct clk_rcg2 gcc_pcie_0_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -286,7 +285,7 @@ static struct clk_rcg2 gcc_pcie_1_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -300,7 +299,7 @@ static struct clk_rcg2 gcc_pcie_2_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_2_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -320,7 +319,7 @@ static struct clk_rcg2 gcc_pcie_phy_refgen_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_phy_refgen_clk_src",
.parent_data = gcc_parent_data_0_ao,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0_ao),
.ops = &clk_rcg2_ops,
},
};
@@ -341,7 +340,7 @@ static struct clk_rcg2 gcc_pdm2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -369,7 +368,7 @@ static const struct freq_tbl ftbl_gcc_qupv3_wrap0_s0_clk_src[] = {
static struct clk_init_data gcc_qupv3_wrap0_s0_clk_src_init = {
.name = "gcc_qupv3_wrap0_s0_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -385,7 +384,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s1_clk_src_init = {
.name = "gcc_qupv3_wrap0_s1_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -417,7 +416,7 @@ static const struct freq_tbl ftbl_gcc_qupv3_wrap0_s2_clk_src[] = {
static struct clk_init_data gcc_qupv3_wrap0_s2_clk_src_init = {
.name = "gcc_qupv3_wrap0_s2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -433,7 +432,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s3_clk_src_init = {
.name = "gcc_qupv3_wrap0_s3_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -449,7 +448,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s4_clk_src_init = {
.name = "gcc_qupv3_wrap0_s4_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -465,7 +464,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s5_clk_src_init = {
.name = "gcc_qupv3_wrap0_s5_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -481,7 +480,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s5_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s6_clk_src_init = {
.name = "gcc_qupv3_wrap0_s6_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -497,7 +496,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s6_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s7_clk_src_init = {
.name = "gcc_qupv3_wrap0_s7_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -513,7 +512,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s7_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s0_clk_src_init = {
.name = "gcc_qupv3_wrap1_s0_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -529,7 +528,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s1_clk_src_init = {
.name = "gcc_qupv3_wrap1_s1_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -545,7 +544,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s1_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s2_clk_src_init = {
.name = "gcc_qupv3_wrap1_s2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -561,7 +560,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s3_clk_src_init = {
.name = "gcc_qupv3_wrap1_s3_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -577,7 +576,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s4_clk_src_init = {
.name = "gcc_qupv3_wrap1_s4_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -593,7 +592,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s5_clk_src_init = {
.name = "gcc_qupv3_wrap1_s5_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -609,7 +608,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s5_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s0_clk_src_init = {
.name = "gcc_qupv3_wrap2_s0_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -625,7 +624,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s1_clk_src_init = {
.name = "gcc_qupv3_wrap2_s1_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -641,7 +640,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s1_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s2_clk_src_init = {
.name = "gcc_qupv3_wrap2_s2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -657,7 +656,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s3_clk_src_init = {
.name = "gcc_qupv3_wrap2_s3_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -673,7 +672,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s4_clk_src_init = {
.name = "gcc_qupv3_wrap2_s4_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -689,7 +688,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s5_clk_src_init = {
.name = "gcc_qupv3_wrap2_s5_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
};
@@ -721,7 +720,7 @@ static struct clk_rcg2 gcc_sdcc2_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk_src",
.parent_data = gcc_parent_data_4,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_4),
.ops = &clk_rcg2_floor_ops,
},
};
@@ -744,7 +743,7 @@ static struct clk_rcg2 gcc_sdcc4_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_floor_ops,
},
};
@@ -763,7 +762,7 @@ static struct clk_rcg2 gcc_tsif_ref_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_tsif_ref_clk_src",
.parent_data = gcc_parent_data_5,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_5),
.ops = &clk_rcg2_ops,
},
};
@@ -785,7 +784,7 @@ static struct clk_rcg2 gcc_ufs_card_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -807,7 +806,7 @@ static struct clk_rcg2 gcc_ufs_card_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -826,7 +825,7 @@ static struct clk_rcg2 gcc_ufs_card_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk_src",
.parent_data = gcc_parent_data_3,
- .num_parents = 1,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_3),
.ops = &clk_rcg2_ops,
},
};
@@ -847,7 +846,7 @@ static struct clk_rcg2 gcc_ufs_card_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -870,7 +869,7 @@ static struct clk_rcg2 gcc_ufs_phy_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -884,7 +883,7 @@ static struct clk_rcg2 gcc_ufs_phy_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -898,7 +897,7 @@ static struct clk_rcg2 gcc_ufs_phy_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk_src",
.parent_data = gcc_parent_data_3,
- .num_parents = 1,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_3),
.ops = &clk_rcg2_ops,
},
};
@@ -912,7 +911,7 @@ static struct clk_rcg2 gcc_ufs_phy_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -935,7 +934,7 @@ static struct clk_rcg2 gcc_usb30_prim_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -949,7 +948,7 @@ static struct clk_rcg2 gcc_usb30_prim_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -963,7 +962,7 @@ static struct clk_rcg2 gcc_usb30_sec_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -977,7 +976,7 @@ static struct clk_rcg2 gcc_usb30_sec_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_mock_utmi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.ops = &clk_rcg2_ops,
},
};
@@ -991,7 +990,7 @@ static struct clk_rcg2 gcc_usb3_prim_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -1005,7 +1004,7 @@ static struct clk_rcg2 gcc_usb3_sec_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.ops = &clk_rcg2_ops,
},
};
@@ -1016,8 +1015,8 @@ static struct clk_regmap_div gcc_cpuss_ahb_postdiv_clk_src = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_cpuss_ahb_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_cpuss_ahb_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1031,8 +1030,8 @@ static struct clk_regmap_div gcc_usb30_prim_mock_utmi_postdiv_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_usb30_prim_mock_utmi_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_mock_utmi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1046,8 +1045,8 @@ static struct clk_regmap_div gcc_usb30_sec_mock_utmi_postdiv_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_usb30_sec_mock_utmi_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_mock_utmi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1078,8 +1077,8 @@ static struct clk_branch gcc_aggre_ufs_card_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_card_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1098,8 +1097,8 @@ static struct clk_branch gcc_aggre_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1116,8 +1115,8 @@ static struct clk_branch gcc_aggre_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1134,8 +1133,8 @@ static struct clk_branch gcc_aggre_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_sec_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1206,8 +1205,8 @@ static struct clk_branch gcc_cfg_noc_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1224,8 +1223,8 @@ static struct clk_branch gcc_cfg_noc_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_sec_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1242,8 +1241,8 @@ static struct clk_branch gcc_cpuss_ahb_clk = {
.enable_mask = BIT(21),
.hw.init = &(struct clk_init_data){
.name = "gcc_cpuss_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_cpuss_ahb_postdiv_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_cpuss_ahb_postdiv_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_IS_CRITICAL | CLK_SET_RATE_PARENT,
@@ -1338,8 +1337,8 @@ static struct clk_branch gcc_gp1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1356,8 +1355,8 @@ static struct clk_branch gcc_gp2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1374,8 +1373,8 @@ static struct clk_branch gcc_gp3_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1391,8 +1390,8 @@ static struct clk_branch gcc_gpu_gpll0_clk_src = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1408,8 +1407,8 @@ static struct clk_branch gcc_gpu_gpll0_div_clk_src = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0_out_even.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0_out_even.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1531,8 +1530,8 @@ static struct clk_branch gcc_npu_gpll0_clk_src = {
.enable_mask = BIT(18),
.hw.init = &(struct clk_init_data){
.name = "gcc_npu_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1548,8 +1547,8 @@ static struct clk_branch gcc_npu_gpll0_div_clk_src = {
.enable_mask = BIT(19),
.hw.init = &(struct clk_init_data){
.name = "gcc_npu_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpll0_out_even.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpll0_out_even.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1566,8 +1565,8 @@ static struct clk_branch gcc_pcie0_phy_refgen_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie0_phy_refgen_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_phy_refgen_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_phy_refgen_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1584,8 +1583,8 @@ static struct clk_branch gcc_pcie1_phy_refgen_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie1_phy_refgen_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_phy_refgen_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_phy_refgen_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1602,8 +1601,8 @@ static struct clk_branch gcc_pcie2_phy_refgen_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie2_phy_refgen_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_phy_refgen_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_phy_refgen_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1620,8 +1619,8 @@ static struct clk_branch gcc_pcie_0_aux_clk = {
.enable_mask = BIT(3),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1707,8 +1706,8 @@ static struct clk_branch gcc_pcie_1_aux_clk = {
.enable_mask = BIT(29),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_1_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1794,8 +1793,8 @@ static struct clk_branch gcc_pcie_2_aux_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_2_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_2_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_2_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1894,8 +1893,8 @@ static struct clk_branch gcc_pcie_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1938,8 +1937,8 @@ static struct clk_branch gcc_pdm2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pdm2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pdm2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2098,8 +2097,8 @@ static struct clk_branch gcc_qupv3_wrap0_s0_clk = {
.enable_mask = BIT(10),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2116,8 +2115,8 @@ static struct clk_branch gcc_qupv3_wrap0_s1_clk = {
.enable_mask = BIT(11),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2134,8 +2133,8 @@ static struct clk_branch gcc_qupv3_wrap0_s2_clk = {
.enable_mask = BIT(12),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2152,8 +2151,8 @@ static struct clk_branch gcc_qupv3_wrap0_s3_clk = {
.enable_mask = BIT(13),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2170,8 +2169,8 @@ static struct clk_branch gcc_qupv3_wrap0_s4_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2188,8 +2187,8 @@ static struct clk_branch gcc_qupv3_wrap0_s5_clk = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2206,8 +2205,8 @@ static struct clk_branch gcc_qupv3_wrap0_s6_clk = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s6_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s6_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s6_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2224,8 +2223,8 @@ static struct clk_branch gcc_qupv3_wrap0_s7_clk = {
.enable_mask = BIT(17),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s7_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s7_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s7_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2268,8 +2267,8 @@ static struct clk_branch gcc_qupv3_wrap1_s0_clk = {
.enable_mask = BIT(22),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2286,8 +2285,8 @@ static struct clk_branch gcc_qupv3_wrap1_s1_clk = {
.enable_mask = BIT(23),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2304,8 +2303,8 @@ static struct clk_branch gcc_qupv3_wrap1_s2_clk = {
.enable_mask = BIT(24),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2322,8 +2321,8 @@ static struct clk_branch gcc_qupv3_wrap1_s3_clk = {
.enable_mask = BIT(25),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2340,8 +2339,8 @@ static struct clk_branch gcc_qupv3_wrap1_s4_clk = {
.enable_mask = BIT(26),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2358,8 +2357,8 @@ static struct clk_branch gcc_qupv3_wrap1_s5_clk = {
.enable_mask = BIT(27),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2402,8 +2401,8 @@ static struct clk_branch gcc_qupv3_wrap2_s0_clk = {
.enable_mask = BIT(4),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2420,8 +2419,8 @@ static struct clk_branch gcc_qupv3_wrap2_s1_clk = {
.enable_mask = BIT(5),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2438,8 +2437,8 @@ static struct clk_branch gcc_qupv3_wrap2_s2_clk = {
.enable_mask = BIT(6),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2456,8 +2455,8 @@ static struct clk_branch gcc_qupv3_wrap2_s3_clk = {
.enable_mask = BIT(7),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2474,8 +2473,8 @@ static struct clk_branch gcc_qupv3_wrap2_s4_clk = {
.enable_mask = BIT(8),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2492,8 +2491,8 @@ static struct clk_branch gcc_qupv3_wrap2_s5_clk = {
.enable_mask = BIT(9),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2607,8 +2606,8 @@ static struct clk_branch gcc_sdcc2_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc2_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc2_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2638,8 +2637,8 @@ static struct clk_branch gcc_sdcc4_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc4_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc4_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2682,8 +2681,8 @@ static struct clk_branch gcc_tsif_ref_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_tsif_ref_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_tsif_ref_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_tsif_ref_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2730,8 +2729,8 @@ static struct clk_branch gcc_ufs_card_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2750,8 +2749,8 @@ static struct clk_branch gcc_ufs_card_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2770,8 +2769,8 @@ static struct clk_branch gcc_ufs_card_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2829,8 +2828,8 @@ static struct clk_branch gcc_ufs_card_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2864,8 +2863,8 @@ static struct clk_branch gcc_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2884,8 +2883,8 @@ static struct clk_branch gcc_ufs_phy_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2904,8 +2903,8 @@ static struct clk_branch gcc_ufs_phy_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2963,8 +2962,8 @@ static struct clk_branch gcc_ufs_phy_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2981,8 +2980,8 @@ static struct clk_branch gcc_usb30_prim_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3031,8 +3030,8 @@ static struct clk_branch gcc_usb30_sec_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3081,8 +3080,8 @@ static struct clk_branch gcc_usb3_prim_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3099,8 +3098,8 @@ static struct clk_branch gcc_usb3_prim_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_com_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3143,8 +3142,8 @@ static struct clk_branch gcc_usb3_sec_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3161,8 +3160,8 @@ static struct clk_branch gcc_usb3_sec_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_com_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gcc-sm8350.c b/drivers/clk/qcom/gcc-sm8350.c
index 1c23b9f84900..6d0a9e2d5104 100644
--- a/drivers/clk/qcom/gcc-sm8350.c
+++ b/drivers/clk/qcom/gcc-sm8350.c
@@ -70,8 +70,8 @@ static struct clk_alpha_pll_postdiv gcc_gpll0_out_even = {
.regs = clk_alpha_pll_regs[CLK_ALPHA_PLL_TYPE_LUCID],
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gpll0_out_even",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0.clkr.hw,
},
.num_parents = 1,
.ops = &clk_alpha_pll_postdiv_lucid_5lpe_ops,
@@ -297,7 +297,7 @@ static struct clk_regmap_mux gcc_pcie_0_pipe_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_pipe_clk_src",
.parent_data = gcc_parent_data_4,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_4),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -312,7 +312,7 @@ static struct clk_regmap_mux gcc_pcie_1_pipe_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_pipe_clk_src",
.parent_data = gcc_parent_data_5,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_5),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -327,7 +327,7 @@ static struct clk_regmap_mux gcc_ufs_card_rx_symbol_0_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_rx_symbol_0_clk_src",
.parent_data = gcc_parent_data_7,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_7),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -342,7 +342,7 @@ static struct clk_regmap_mux gcc_ufs_card_rx_symbol_1_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_rx_symbol_1_clk_src",
.parent_data = gcc_parent_data_8,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_8),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -357,7 +357,7 @@ static struct clk_regmap_mux gcc_ufs_card_tx_symbol_0_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_tx_symbol_0_clk_src",
.parent_data = gcc_parent_data_9,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_9),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -372,7 +372,7 @@ static struct clk_regmap_mux gcc_ufs_phy_rx_symbol_0_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_rx_symbol_0_clk_src",
.parent_data = gcc_parent_data_10,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_10),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -387,7 +387,7 @@ static struct clk_regmap_mux gcc_ufs_phy_rx_symbol_1_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_rx_symbol_1_clk_src",
.parent_data = gcc_parent_data_11,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_11),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -402,7 +402,7 @@ static struct clk_regmap_mux gcc_ufs_phy_tx_symbol_0_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_tx_symbol_0_clk_src",
.parent_data = gcc_parent_data_12,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_12),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -417,7 +417,7 @@ static struct clk_regmap_mux gcc_usb3_prim_phy_pipe_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_pipe_clk_src",
.parent_data = gcc_parent_data_13,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_13),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -432,7 +432,7 @@ static struct clk_regmap_mux gcc_usb3_sec_phy_pipe_clk_src = {
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_pipe_clk_src",
.parent_data = gcc_parent_data_14,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_14),
.ops = &clk_regmap_mux_closest_ops,
},
},
@@ -454,7 +454,7 @@ static struct clk_rcg2 gcc_gp1_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk_src",
.parent_data = gcc_parent_data_1,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -469,7 +469,7 @@ static struct clk_rcg2 gcc_gp2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk_src",
.parent_data = gcc_parent_data_1,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -484,7 +484,7 @@ static struct clk_rcg2 gcc_gp3_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk_src",
.parent_data = gcc_parent_data_1,
- .num_parents = 5,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_1),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -505,7 +505,7 @@ static struct clk_rcg2 gcc_pcie_0_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -526,7 +526,7 @@ static struct clk_rcg2 gcc_pcie_0_phy_rchng_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_phy_rchng_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -541,7 +541,7 @@ static struct clk_rcg2 gcc_pcie_1_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -556,7 +556,7 @@ static struct clk_rcg2 gcc_pcie_1_phy_rchng_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_phy_rchng_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -576,7 +576,7 @@ static struct clk_rcg2 gcc_pdm2_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -600,7 +600,7 @@ static const struct freq_tbl ftbl_gcc_qupv3_wrap0_s0_clk_src[] = {
static struct clk_init_data gcc_qupv3_wrap0_s0_clk_src_init = {
.name = "gcc_qupv3_wrap0_s0_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -617,7 +617,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s1_clk_src_init = {
.name = "gcc_qupv3_wrap0_s1_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -634,7 +634,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s1_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s2_clk_src_init = {
.name = "gcc_qupv3_wrap0_s2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -651,7 +651,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s3_clk_src_init = {
.name = "gcc_qupv3_wrap0_s3_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -668,7 +668,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s4_clk_src_init = {
.name = "gcc_qupv3_wrap0_s4_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -685,7 +685,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s5_clk_src_init = {
.name = "gcc_qupv3_wrap0_s5_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -702,7 +702,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s5_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s6_clk_src_init = {
.name = "gcc_qupv3_wrap0_s6_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -719,7 +719,7 @@ static struct clk_rcg2 gcc_qupv3_wrap0_s6_clk_src = {
static struct clk_init_data gcc_qupv3_wrap0_s7_clk_src_init = {
.name = "gcc_qupv3_wrap0_s7_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -755,7 +755,7 @@ static const struct freq_tbl ftbl_gcc_qupv3_wrap1_s0_clk_src[] = {
static struct clk_init_data gcc_qupv3_wrap1_s0_clk_src_init = {
.name = "gcc_qupv3_wrap1_s0_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -772,7 +772,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s1_clk_src_init = {
.name = "gcc_qupv3_wrap1_s1_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -789,7 +789,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s1_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s2_clk_src_init = {
.name = "gcc_qupv3_wrap1_s2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -806,7 +806,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s3_clk_src_init = {
.name = "gcc_qupv3_wrap1_s3_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -823,7 +823,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s4_clk_src_init = {
.name = "gcc_qupv3_wrap1_s4_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -840,7 +840,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap1_s5_clk_src_init = {
.name = "gcc_qupv3_wrap1_s5_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -857,7 +857,7 @@ static struct clk_rcg2 gcc_qupv3_wrap1_s5_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s0_clk_src_init = {
.name = "gcc_qupv3_wrap2_s0_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -874,7 +874,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s0_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s1_clk_src_init = {
.name = "gcc_qupv3_wrap2_s1_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -891,7 +891,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s1_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s2_clk_src_init = {
.name = "gcc_qupv3_wrap2_s2_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -908,7 +908,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s2_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s3_clk_src_init = {
.name = "gcc_qupv3_wrap2_s3_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -925,7 +925,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s3_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s4_clk_src_init = {
.name = "gcc_qupv3_wrap2_s4_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -942,7 +942,7 @@ static struct clk_rcg2 gcc_qupv3_wrap2_s4_clk_src = {
static struct clk_init_data gcc_qupv3_wrap2_s5_clk_src_init = {
.name = "gcc_qupv3_wrap2_s5_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
};
@@ -974,7 +974,7 @@ static struct clk_rcg2 gcc_sdcc2_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk_src",
.parent_data = gcc_parent_data_6,
- .num_parents = 6,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_6),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_floor_ops,
},
@@ -996,7 +996,7 @@ static struct clk_rcg2 gcc_sdcc4_apps_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_floor_ops,
},
@@ -1019,7 +1019,7 @@ static struct clk_rcg2 gcc_ufs_card_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1041,7 +1041,7 @@ static struct clk_rcg2 gcc_ufs_card_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1061,7 +1061,7 @@ static struct clk_rcg2 gcc_ufs_card_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk_src",
.parent_data = gcc_parent_data_3,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_3),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1076,7 +1076,7 @@ static struct clk_rcg2 gcc_ufs_card_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1091,7 +1091,7 @@ static struct clk_rcg2 gcc_ufs_phy_axi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1106,7 +1106,7 @@ static struct clk_rcg2 gcc_ufs_phy_ice_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1121,7 +1121,7 @@ static struct clk_rcg2 gcc_ufs_phy_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk_src",
.parent_data = gcc_parent_data_3,
- .num_parents = 2,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_3),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1136,7 +1136,7 @@ static struct clk_rcg2 gcc_ufs_phy_unipro_core_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1159,7 +1159,7 @@ static struct clk_rcg2 gcc_usb30_prim_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1174,7 +1174,7 @@ static struct clk_rcg2 gcc_usb30_prim_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1189,7 +1189,7 @@ static struct clk_rcg2 gcc_usb30_sec_master_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1204,7 +1204,7 @@ static struct clk_rcg2 gcc_usb30_sec_mock_utmi_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_mock_utmi_clk_src",
.parent_data = gcc_parent_data_0,
- .num_parents = 4,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1219,7 +1219,7 @@ static struct clk_rcg2 gcc_usb3_prim_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1234,7 +1234,7 @@ static struct clk_rcg2 gcc_usb3_sec_phy_aux_clk_src = {
.clkr.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk_src",
.parent_data = gcc_parent_data_2,
- .num_parents = 3,
+ .num_parents = ARRAY_SIZE(gcc_parent_data_2),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_ops,
},
@@ -1246,8 +1246,8 @@ static struct clk_regmap_div gcc_usb30_prim_mock_utmi_postdiv_clk_src = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_usb30_prim_mock_utmi_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_mock_utmi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1261,8 +1261,8 @@ static struct clk_regmap_div gcc_usb30_sec_mock_utmi_postdiv_clk_src = {
.width = 4,
.clkr.hw.init = &(struct clk_init_data) {
.name = "gcc_usb30_sec_mock_utmi_postdiv_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_mock_utmi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_mock_utmi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1323,8 +1323,8 @@ static struct clk_branch gcc_aggre_ufs_card_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_card_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1343,8 +1343,8 @@ static struct clk_branch gcc_aggre_ufs_card_axi_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_card_axi_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1363,8 +1363,8 @@ static struct clk_branch gcc_aggre_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1383,8 +1383,8 @@ static struct clk_branch gcc_aggre_ufs_phy_axi_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_ufs_phy_axi_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1403,8 +1403,8 @@ static struct clk_branch gcc_aggre_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1423,8 +1423,8 @@ static struct clk_branch gcc_aggre_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_aggre_usb3_sec_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1490,8 +1490,8 @@ static struct clk_branch gcc_cfg_noc_usb3_prim_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_prim_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1510,8 +1510,8 @@ static struct clk_branch gcc_cfg_noc_usb3_sec_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_cfg_noc_usb3_sec_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1592,8 +1592,8 @@ static struct clk_branch gcc_gp1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1610,8 +1610,8 @@ static struct clk_branch gcc_gp2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1628,8 +1628,8 @@ static struct clk_branch gcc_gp3_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_gp3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gp3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gp3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1646,8 +1646,8 @@ static struct clk_branch gcc_gpu_gpll0_clk_src = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1664,8 +1664,8 @@ static struct clk_branch gcc_gpu_gpll0_div_clk_src = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_gpu_gpll0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_gpll0_out_even.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_gpll0_out_even.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1723,8 +1723,8 @@ static struct clk_branch gcc_pcie0_phy_rchng_clk = {
.enable_mask = BIT(22),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie0_phy_rchng_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_phy_rchng_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_phy_rchng_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1741,8 +1741,8 @@ static struct clk_branch gcc_pcie1_phy_rchng_clk = {
.enable_mask = BIT(23),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie1_phy_rchng_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_1_phy_rchng_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_phy_rchng_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1759,8 +1759,8 @@ static struct clk_branch gcc_pcie_0_aux_clk = {
.enable_mask = BIT(3),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1822,8 +1822,8 @@ static struct clk_branch gcc_pcie_0_pipe_clk = {
.enable_mask = BIT(4),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_0_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_0_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_0_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1868,8 +1868,8 @@ static struct clk_branch gcc_pcie_1_aux_clk = {
.enable_mask = BIT(29),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_1_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1931,8 +1931,8 @@ static struct clk_branch gcc_pcie_1_pipe_clk = {
.enable_mask = BIT(30),
.hw.init = &(struct clk_init_data){
.name = "gcc_pcie_1_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pcie_1_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pcie_1_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -1977,8 +1977,8 @@ static struct clk_branch gcc_pdm2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_pdm2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_pdm2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_pdm2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2124,8 +2124,8 @@ static struct clk_branch gcc_qupv3_wrap0_s0_clk = {
.enable_mask = BIT(10),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2142,8 +2142,8 @@ static struct clk_branch gcc_qupv3_wrap0_s1_clk = {
.enable_mask = BIT(11),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2160,8 +2160,8 @@ static struct clk_branch gcc_qupv3_wrap0_s2_clk = {
.enable_mask = BIT(12),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2178,8 +2178,8 @@ static struct clk_branch gcc_qupv3_wrap0_s3_clk = {
.enable_mask = BIT(13),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2196,8 +2196,8 @@ static struct clk_branch gcc_qupv3_wrap0_s4_clk = {
.enable_mask = BIT(14),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2214,8 +2214,8 @@ static struct clk_branch gcc_qupv3_wrap0_s5_clk = {
.enable_mask = BIT(15),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2232,8 +2232,8 @@ static struct clk_branch gcc_qupv3_wrap0_s6_clk = {
.enable_mask = BIT(16),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s6_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s6_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s6_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2250,8 +2250,8 @@ static struct clk_branch gcc_qupv3_wrap0_s7_clk = {
.enable_mask = BIT(17),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap0_s7_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap0_s7_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap0_s7_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2324,8 +2324,8 @@ static struct clk_branch gcc_qupv3_wrap1_s0_clk = {
.enable_mask = BIT(22),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2342,8 +2342,8 @@ static struct clk_branch gcc_qupv3_wrap1_s1_clk = {
.enable_mask = BIT(23),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2360,8 +2360,8 @@ static struct clk_branch gcc_qupv3_wrap1_s2_clk = {
.enable_mask = BIT(24),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2378,8 +2378,8 @@ static struct clk_branch gcc_qupv3_wrap1_s3_clk = {
.enable_mask = BIT(25),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2396,8 +2396,8 @@ static struct clk_branch gcc_qupv3_wrap1_s4_clk = {
.enable_mask = BIT(26),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2414,8 +2414,8 @@ static struct clk_branch gcc_qupv3_wrap1_s5_clk = {
.enable_mask = BIT(27),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap1_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap1_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2458,8 +2458,8 @@ static struct clk_branch gcc_qupv3_wrap2_s0_clk = {
.enable_mask = BIT(4),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2476,8 +2476,8 @@ static struct clk_branch gcc_qupv3_wrap2_s1_clk = {
.enable_mask = BIT(5),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2494,8 +2494,8 @@ static struct clk_branch gcc_qupv3_wrap2_s2_clk = {
.enable_mask = BIT(6),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s2_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s2_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2512,8 +2512,8 @@ static struct clk_branch gcc_qupv3_wrap2_s3_clk = {
.enable_mask = BIT(7),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s3_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s3_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s3_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2530,8 +2530,8 @@ static struct clk_branch gcc_qupv3_wrap2_s4_clk = {
.enable_mask = BIT(8),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s4_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s4_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s4_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2548,8 +2548,8 @@ static struct clk_branch gcc_qupv3_wrap2_s5_clk = {
.enable_mask = BIT(9),
.hw.init = &(struct clk_init_data){
.name = "gcc_qupv3_wrap2_s5_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_qupv3_wrap2_s5_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_qupv3_wrap2_s5_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2639,8 +2639,8 @@ static struct clk_branch gcc_sdcc2_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc2_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc2_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc2_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2670,8 +2670,8 @@ static struct clk_branch gcc_sdcc4_apps_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_sdcc4_apps_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_sdcc4_apps_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_sdcc4_apps_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2731,8 +2731,8 @@ static struct clk_branch gcc_ufs_card_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2751,8 +2751,8 @@ static struct clk_branch gcc_ufs_card_axi_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_axi_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2771,8 +2771,8 @@ static struct clk_branch gcc_ufs_card_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2791,8 +2791,8 @@ static struct clk_branch gcc_ufs_card_ice_core_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_ice_core_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2811,8 +2811,8 @@ static struct clk_branch gcc_ufs_card_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2831,8 +2831,8 @@ static struct clk_branch gcc_ufs_card_phy_aux_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_phy_aux_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2850,8 +2850,8 @@ static struct clk_branch gcc_ufs_card_rx_symbol_0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_rx_symbol_0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_rx_symbol_0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_rx_symbol_0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2869,8 +2869,8 @@ static struct clk_branch gcc_ufs_card_rx_symbol_1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_rx_symbol_1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_rx_symbol_1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_rx_symbol_1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2888,8 +2888,8 @@ static struct clk_branch gcc_ufs_card_tx_symbol_0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_tx_symbol_0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_tx_symbol_0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_tx_symbol_0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2908,8 +2908,8 @@ static struct clk_branch gcc_ufs_card_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2928,8 +2928,8 @@ static struct clk_branch gcc_ufs_card_unipro_core_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_card_unipro_core_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_card_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_card_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2963,8 +2963,8 @@ static struct clk_branch gcc_ufs_phy_axi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -2983,8 +2983,8 @@ static struct clk_branch gcc_ufs_phy_axi_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_axi_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_axi_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_axi_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3003,8 +3003,8 @@ static struct clk_branch gcc_ufs_phy_ice_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3023,8 +3023,8 @@ static struct clk_branch gcc_ufs_phy_ice_core_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_ice_core_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_ice_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3043,8 +3043,8 @@ static struct clk_branch gcc_ufs_phy_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3063,8 +3063,8 @@ static struct clk_branch gcc_ufs_phy_phy_aux_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_phy_aux_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3082,8 +3082,8 @@ static struct clk_branch gcc_ufs_phy_rx_symbol_0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_rx_symbol_0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_rx_symbol_0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_rx_symbol_0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3101,8 +3101,8 @@ static struct clk_branch gcc_ufs_phy_rx_symbol_1_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_rx_symbol_1_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_rx_symbol_1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_rx_symbol_1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3120,8 +3120,8 @@ static struct clk_branch gcc_ufs_phy_tx_symbol_0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_tx_symbol_0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_tx_symbol_0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_tx_symbol_0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3140,8 +3140,8 @@ static struct clk_branch gcc_ufs_phy_unipro_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3160,8 +3160,8 @@ static struct clk_branch gcc_ufs_phy_unipro_core_hw_ctl_clk = {
.enable_mask = BIT(1),
.hw.init = &(struct clk_init_data){
.name = "gcc_ufs_phy_unipro_core_hw_ctl_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_ufs_phy_unipro_core_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3178,8 +3178,8 @@ static struct clk_branch gcc_usb30_prim_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_master_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_prim_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3209,9 +3209,8 @@ static struct clk_branch gcc_usb30_prim_mock_utmi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_prim_mock_utmi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw =
- &gcc_usb30_prim_mock_utmi_postdiv_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_prim_mock_utmi_postdiv_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3241,8 +3240,8 @@ static struct clk_branch gcc_usb30_sec_master_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_master_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb30_sec_master_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_master_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3272,9 +3271,8 @@ static struct clk_branch gcc_usb30_sec_mock_utmi_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb30_sec_mock_utmi_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw =
- &gcc_usb30_sec_mock_utmi_postdiv_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb30_sec_mock_utmi_postdiv_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3304,8 +3302,8 @@ static struct clk_branch gcc_usb3_prim_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3322,8 +3320,8 @@ static struct clk_branch gcc_usb3_prim_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_com_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3343,8 +3341,8 @@ static struct clk_branch gcc_usb3_prim_phy_pipe_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_prim_phy_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_prim_phy_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_prim_phy_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3374,8 +3372,8 @@ static struct clk_branch gcc_usb3_sec_phy_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3392,8 +3390,8 @@ static struct clk_branch gcc_usb3_sec_phy_com_aux_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_com_aux_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_aux_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -3411,8 +3409,8 @@ static struct clk_branch gcc_usb3_sec_phy_pipe_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gcc_usb3_sec_phy_pipe_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gcc_usb3_sec_phy_pipe_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gcc_usb3_sec_phy_pipe_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gpucc-sc7180.c b/drivers/clk/qcom/gpucc-sc7180.c
index 88a739b6fec3..d738251cba17 100644
--- a/drivers/clk/qcom/gpucc-sc7180.c
+++ b/drivers/clk/qcom/gpucc-sc7180.c
@@ -26,12 +26,9 @@
enum {
P_BI_TCXO,
- P_CORE_BI_PLL_TEST_SE,
P_GPLL0_OUT_MAIN,
P_GPLL0_OUT_MAIN_DIV,
- P_GPU_CC_PLL1_OUT_EVEN,
P_GPU_CC_PLL1_OUT_MAIN,
- P_GPU_CC_PLL1_OUT_ODD,
};
static const struct pll_vco fabia_vco[] = {
diff --git a/drivers/clk/qcom/gpucc-sdm845.c b/drivers/clk/qcom/gpucc-sdm845.c
index 5663698b306b..110b54401bc6 100644
--- a/drivers/clk/qcom/gpucc-sdm845.c
+++ b/drivers/clk/qcom/gpucc-sdm845.c
@@ -27,28 +27,9 @@
enum {
P_BI_TCXO,
- P_CORE_BI_PLL_TEST_SE,
P_GPLL0_OUT_MAIN,
P_GPLL0_OUT_MAIN_DIV,
- P_GPU_CC_PLL1_OUT_EVEN,
P_GPU_CC_PLL1_OUT_MAIN,
- P_GPU_CC_PLL1_OUT_ODD,
-};
-
-static const struct parent_map gpu_cc_parent_map_0[] = {
- { P_BI_TCXO, 0 },
- { P_GPU_CC_PLL1_OUT_MAIN, 3 },
- { P_GPLL0_OUT_MAIN, 5 },
- { P_GPLL0_OUT_MAIN_DIV, 6 },
- { P_CORE_BI_PLL_TEST_SE, 7 },
-};
-
-static const char * const gpu_cc_parent_names_0[] = {
- "bi_tcxo",
- "gpu_cc_pll1",
- "gcc_gpu_gpll0_clk_src",
- "gcc_gpu_gpll0_div_clk_src",
- "core_bi_pll_test_se",
};
static const struct alpha_pll_config gpu_cc_pll1_config = {
@@ -62,13 +43,29 @@ static struct clk_alpha_pll gpu_cc_pll1 = {
.clkr = {
.hw.init = &(struct clk_init_data){
.name = "gpu_cc_pll1",
- .parent_names = (const char *[]){ "bi_tcxo" },
+ .parent_data = &(const struct clk_parent_data){
+ .fw_name = "bi_tcxo", .name = "bi_tcxo",
+ },
.num_parents = 1,
.ops = &clk_alpha_pll_fabia_ops,
},
},
};
+static const struct parent_map gpu_cc_parent_map_0[] = {
+ { P_BI_TCXO, 0 },
+ { P_GPU_CC_PLL1_OUT_MAIN, 3 },
+ { P_GPLL0_OUT_MAIN, 5 },
+ { P_GPLL0_OUT_MAIN_DIV, 6 },
+};
+
+static const struct clk_parent_data gpu_cc_parent_data_0[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &gpu_cc_pll1.clkr.hw },
+ { .fw_name = "gcc_gpu_gpll0_clk_src", .name = "gcc_gpu_gpll0_clk_src" },
+ { .fw_name = "gcc_gpu_gpll0_div_clk_src", .name = "gcc_gpu_gpll0_div_clk_src" },
+};
+
static const struct freq_tbl ftbl_gpu_cc_gmu_clk_src[] = {
F(19200000, P_BI_TCXO, 1, 0, 0),
F(200000000, P_GPLL0_OUT_MAIN_DIV, 1.5, 0, 0),
@@ -84,8 +81,8 @@ static struct clk_rcg2 gpu_cc_gmu_clk_src = {
.freq_tbl = ftbl_gpu_cc_gmu_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "gpu_cc_gmu_clk_src",
- .parent_names = gpu_cc_parent_names_0,
- .num_parents = 5,
+ .parent_data = gpu_cc_parent_data_0,
+ .num_parents = ARRAY_SIZE(gpu_cc_parent_data_0),
.ops = &clk_rcg2_shared_ops,
},
};
@@ -98,8 +95,8 @@ static struct clk_branch gpu_cc_cx_gmu_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gpu_cc_cx_gmu_clk",
- .parent_names = (const char *[]){
- "gpu_cc_gmu_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &gpu_cc_gmu_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gpucc-sm8150.c b/drivers/clk/qcom/gpucc-sm8150.c
index 27c40754b2c7..80fb6f73601d 100644
--- a/drivers/clk/qcom/gpucc-sm8150.c
+++ b/drivers/clk/qcom/gpucc-sm8150.c
@@ -21,7 +21,6 @@
enum {
P_BI_TCXO,
- P_CORE_BI_PLL_TEST_SE,
P_GPLL0_OUT_MAIN,
P_GPLL0_OUT_MAIN_DIV,
P_GPU_CC_PLL1_OUT_MAIN,
@@ -145,8 +144,8 @@ static struct clk_branch gpu_cc_cx_gmu_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gpu_cc_cx_gmu_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpu_cc_gmu_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpu_cc_gmu_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -202,8 +201,8 @@ static struct clk_branch gpu_cc_gx_gmu_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gpu_cc_gx_gmu_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpu_cc_gmu_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpu_cc_gmu_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/gpucc-sm8250.c b/drivers/clk/qcom/gpucc-sm8250.c
index 3fa7d1f9ff98..9c1f8ce32da4 100644
--- a/drivers/clk/qcom/gpucc-sm8250.c
+++ b/drivers/clk/qcom/gpucc-sm8250.c
@@ -26,7 +26,6 @@
enum {
P_BI_TCXO,
- P_CORE_BI_PLL_TEST_SE,
P_GPLL0_OUT_MAIN,
P_GPLL0_OUT_MAIN_DIV,
P_GPU_CC_PLL0_OUT_MAIN,
@@ -148,8 +147,8 @@ static struct clk_branch gpu_cc_cx_gmu_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gpu_cc_cx_gmu_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpu_cc_gmu_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpu_cc_gmu_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -205,8 +204,8 @@ static struct clk_branch gpu_cc_gx_gmu_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "gpu_cc_gx_gmu_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &gpu_cc_gmu_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &gpu_cc_gmu_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/videocc-sc7180.c b/drivers/clk/qcom/videocc-sc7180.c
index 276e5ecd4840..ed57bbb19f88 100644
--- a/drivers/clk/qcom/videocc-sc7180.c
+++ b/drivers/clk/qcom/videocc-sc7180.c
@@ -19,11 +19,7 @@
enum {
P_BI_TCXO,
- P_CHIP_SLEEP_CLK,
- P_CORE_BI_PLL_TEST_SE,
- P_VIDEO_PLL0_OUT_EVEN,
P_VIDEO_PLL0_OUT_MAIN,
- P_VIDEO_PLL0_OUT_ODD,
};
static const struct pll_vco fabia_vco[] = {
diff --git a/drivers/clk/qcom/videocc-sdm845.c b/drivers/clk/qcom/videocc-sdm845.c
index 5d6a7724a194..c77a4dd5d39c 100644
--- a/drivers/clk/qcom/videocc-sdm845.c
+++ b/drivers/clk/qcom/videocc-sdm845.c
@@ -20,26 +20,9 @@
enum {
P_BI_TCXO,
- P_CORE_BI_PLL_TEST_SE,
- P_VIDEO_PLL0_OUT_EVEN,
P_VIDEO_PLL0_OUT_MAIN,
- P_VIDEO_PLL0_OUT_ODD,
-};
-
-static const struct parent_map video_cc_parent_map_0[] = {
- { P_BI_TCXO, 0 },
- { P_VIDEO_PLL0_OUT_MAIN, 1 },
- { P_VIDEO_PLL0_OUT_EVEN, 2 },
- { P_VIDEO_PLL0_OUT_ODD, 3 },
- { P_CORE_BI_PLL_TEST_SE, 4 },
-};
-
-static const char * const video_cc_parent_names_0[] = {
- "bi_tcxo",
- "video_pll0",
- "video_pll0_out_even",
- "video_pll0_out_odd",
- "core_bi_pll_test_se",
+ /* P_VIDEO_PLL0_OUT_EVEN, */
+ /* P_VIDEO_PLL0_OUT_ODD, */
};
static const struct alpha_pll_config video_pll0_config = {
@@ -53,13 +36,29 @@ static struct clk_alpha_pll video_pll0 = {
.clkr = {
.hw.init = &(struct clk_init_data){
.name = "video_pll0",
- .parent_names = (const char *[]){ "bi_tcxo" },
+ .parent_data = &(const struct clk_parent_data){
+ .fw_name = "bi_tcxo", .name = "bi_tcxo",
+ },
.num_parents = 1,
.ops = &clk_alpha_pll_fabia_ops,
},
},
};
+static const struct parent_map video_cc_parent_map_0[] = {
+ { P_BI_TCXO, 0 },
+ { P_VIDEO_PLL0_OUT_MAIN, 1 },
+ /* { P_VIDEO_PLL0_OUT_EVEN, 2 }, */
+ /* { P_VIDEO_PLL0_OUT_ODD, 3 }, */
+};
+
+static const struct clk_parent_data video_cc_parent_data_0[] = {
+ { .fw_name = "bi_tcxo", .name = "bi_tcxo" },
+ { .hw = &video_pll0.clkr.hw },
+ /* { .name = "video_pll0_out_even" }, */
+ /* { .name = "video_pll0_out_odd" }, */
+};
+
static const struct freq_tbl ftbl_video_cc_venus_clk_src[] = {
F(100000000, P_VIDEO_PLL0_OUT_MAIN, 4, 0, 0),
F(200000000, P_VIDEO_PLL0_OUT_MAIN, 2, 0, 0),
@@ -78,8 +77,8 @@ static struct clk_rcg2 video_cc_venus_clk_src = {
.freq_tbl = ftbl_video_cc_venus_clk_src,
.clkr.hw.init = &(struct clk_init_data){
.name = "video_cc_venus_clk_src",
- .parent_names = video_cc_parent_names_0,
- .num_parents = 5,
+ .parent_data = video_cc_parent_data_0,
+ .num_parents = ARRAY_SIZE(video_cc_parent_data_0),
.flags = CLK_SET_RATE_PARENT,
.ops = &clk_rcg2_shared_ops,
},
@@ -158,8 +157,8 @@ static struct clk_branch video_cc_vcodec0_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_vcodec0_core_clk",
- .parent_names = (const char *[]){
- "video_cc_venus_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_venus_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -189,8 +188,8 @@ static struct clk_branch video_cc_vcodec1_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_vcodec1_core_clk",
- .parent_names = (const char *[]){
- "video_cc_venus_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_venus_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -233,8 +232,8 @@ static struct clk_branch video_cc_venus_ctl_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_venus_ctl_core_clk",
- .parent_names = (const char *[]){
- "video_cc_venus_clk_src",
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_venus_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/videocc-sm8150.c b/drivers/clk/qcom/videocc-sm8150.c
index 3087e2ec8fd4..1afdbe4a249d 100644
--- a/drivers/clk/qcom/videocc-sm8150.c
+++ b/drivers/clk/qcom/videocc-sm8150.c
@@ -20,11 +20,7 @@
enum {
P_BI_TCXO,
- P_CHIP_SLEEP_CLK,
- P_CORE_BI_PLL_TEST_SE,
- P_VIDEO_PLL0_OUT_EVEN,
P_VIDEO_PLL0_OUT_MAIN,
- P_VIDEO_PLL0_OUT_ODD,
};
static struct pll_vco trion_vco[] = {
@@ -103,8 +99,8 @@ static struct clk_branch video_cc_iris_ahb_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_iris_ahb_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_iris_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_iris_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -121,8 +117,8 @@ static struct clk_branch video_cc_mvs0_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_mvs0_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_iris_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_iris_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -139,8 +135,8 @@ static struct clk_branch video_cc_mvs1_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_mvs1_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_iris_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_iris_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -157,8 +153,8 @@ static struct clk_branch video_cc_mvsc_core_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_mvsc_core_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_iris_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_iris_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/qcom/videocc-sm8250.c b/drivers/clk/qcom/videocc-sm8250.c
index b0efadc19634..7b435a1c2c4b 100644
--- a/drivers/clk/qcom/videocc-sm8250.c
+++ b/drivers/clk/qcom/videocc-sm8250.c
@@ -21,8 +21,6 @@
enum {
P_BI_TCXO,
- P_CHIP_SLEEP_CLK,
- P_CORE_BI_PLL_TEST_SE,
P_VIDEO_PLL0_OUT_MAIN,
P_VIDEO_PLL1_OUT_MAIN,
};
@@ -160,8 +158,8 @@ static struct clk_regmap_div video_cc_mvs0c_div2_div_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "video_cc_mvs0c_div2_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_mvs0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_mvs0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -175,8 +173,8 @@ static struct clk_regmap_div video_cc_mvs0_div_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "video_cc_mvs0_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_mvs0_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_mvs0_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -190,8 +188,8 @@ static struct clk_regmap_div video_cc_mvs1c_div2_div_clk_src = {
.width = 2,
.clkr.hw.init = &(struct clk_init_data) {
.name = "video_cc_mvs1c_div2_div_clk_src",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_mvs1_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_mvs1_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -207,8 +205,8 @@ static struct clk_branch video_cc_mvs0c_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_mvs0c_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_mvs0c_div2_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_mvs0c_div2_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -225,8 +223,8 @@ static struct clk_branch video_cc_mvs0_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_mvs0_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_mvs0_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_mvs0_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -243,8 +241,8 @@ static struct clk_branch video_cc_mvs1_div2_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_mvs1_div2_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_mvs1c_div2_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_mvs1c_div2_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
@@ -261,8 +259,8 @@ static struct clk_branch video_cc_mvs1c_clk = {
.enable_mask = BIT(0),
.hw.init = &(struct clk_init_data){
.name = "video_cc_mvs1c_clk",
- .parent_data = &(const struct clk_parent_data){
- .hw = &video_cc_mvs1c_div2_div_clk_src.clkr.hw,
+ .parent_hws = (const struct clk_hw*[]){
+ &video_cc_mvs1c_div2_div_clk_src.clkr.hw,
},
.num_parents = 1,
.flags = CLK_SET_RATE_PARENT,
diff --git a/drivers/clk/ralink/Kconfig b/drivers/clk/ralink/Kconfig
new file mode 100644
index 000000000000..6580d5edc676
--- /dev/null
+++ b/drivers/clk/ralink/Kconfig
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# MediaTek Mt7621 Clock Driver
+#
+config CLK_MT7621
+ bool "Clock driver for MediaTek MT7621"
+ depends on SOC_MT7621 || COMPILE_TEST
+ default SOC_MT7621
+ select MFD_SYSCON
+ help
+ This driver supports MediaTek MT7621 basic clocks.
diff --git a/drivers/clk/ralink/Makefile b/drivers/clk/ralink/Makefile
new file mode 100644
index 000000000000..cf6f9216379d
--- /dev/null
+++ b/drivers/clk/ralink/Makefile
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0
+obj-$(CONFIG_CLK_MT7621) += clk-mt7621.o
diff --git a/drivers/clk/ralink/clk-mt7621.c b/drivers/clk/ralink/clk-mt7621.c
new file mode 100644
index 000000000000..857da1e274be
--- /dev/null
+++ b/drivers/clk/ralink/clk-mt7621.c
@@ -0,0 +1,495 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Mediatek MT7621 Clock Driver
+ * Author: Sergio Paracuellos <sergio.paracuellos@gmail.com>
+ */
+
+#include <linux/bitfield.h>
+#include <linux/bitops.h>
+#include <linux/clk-provider.h>
+#include <linux/clk.h>
+#include <linux/mfd/syscon.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+#include <dt-bindings/clock/mt7621-clk.h>
+
+/* Configuration registers */
+#define SYSC_REG_SYSTEM_CONFIG0 0x10
+#define SYSC_REG_SYSTEM_CONFIG1 0x14
+#define SYSC_REG_CLKCFG0 0x2c
+#define SYSC_REG_CLKCFG1 0x30
+#define SYSC_REG_CUR_CLK_STS 0x44
+#define MEMC_REG_CPU_PLL 0x648
+
+#define XTAL_MODE_SEL_MASK GENMASK(8, 6)
+#define CPU_CLK_SEL_MASK GENMASK(31, 30)
+#define CUR_CPU_FDIV_MASK GENMASK(12, 8)
+#define CUR_CPU_FFRAC_MASK GENMASK(4, 0)
+#define CPU_PLL_PREDIV_MASK GENMASK(13, 12)
+#define CPU_PLL_FBDIV_MASK GENMASK(10, 4)
+
+struct mt7621_clk_priv {
+ struct regmap *sysc;
+ struct regmap *memc;
+};
+
+struct mt7621_clk {
+ struct clk_hw hw;
+ struct mt7621_clk_priv *priv;
+};
+
+struct mt7621_fixed_clk {
+ u8 idx;
+ const char *name;
+ const char *parent_name;
+ unsigned long rate;
+ struct clk_hw *hw;
+};
+
+struct mt7621_gate {
+ u8 idx;
+ const char *name;
+ const char *parent_name;
+ struct mt7621_clk_priv *priv;
+ u32 bit_idx;
+ struct clk_hw hw;
+};
+
+#define GATE(_id, _name, _pname, _shift) \
+ { \
+ .idx = _id, \
+ .name = _name, \
+ .parent_name = _pname, \
+ .bit_idx = _shift \
+ }
+
+static struct mt7621_gate mt7621_gates[] = {
+ GATE(MT7621_CLK_HSDMA, "hsdma", "150m", BIT(5)),
+ GATE(MT7621_CLK_FE, "fe", "250m", BIT(6)),
+ GATE(MT7621_CLK_SP_DIVTX, "sp_divtx", "270m", BIT(7)),
+ GATE(MT7621_CLK_TIMER, "timer", "50m", BIT(8)),
+ GATE(MT7621_CLK_PCM, "pcm", "270m", BIT(11)),
+ GATE(MT7621_CLK_PIO, "pio", "50m", BIT(13)),
+ GATE(MT7621_CLK_GDMA, "gdma", "bus", BIT(14)),
+ GATE(MT7621_CLK_NAND, "nand", "125m", BIT(15)),
+ GATE(MT7621_CLK_I2C, "i2c", "50m", BIT(16)),
+ GATE(MT7621_CLK_I2S, "i2s", "270m", BIT(17)),
+ GATE(MT7621_CLK_SPI, "spi", "bus", BIT(18)),
+ GATE(MT7621_CLK_UART1, "uart1", "50m", BIT(19)),
+ GATE(MT7621_CLK_UART2, "uart2", "50m", BIT(20)),
+ GATE(MT7621_CLK_UART3, "uart3", "50m", BIT(21)),
+ GATE(MT7621_CLK_ETH, "eth", "50m", BIT(23)),
+ GATE(MT7621_CLK_PCIE0, "pcie0", "125m", BIT(24)),
+ GATE(MT7621_CLK_PCIE1, "pcie1", "125m", BIT(25)),
+ GATE(MT7621_CLK_PCIE2, "pcie2", "125m", BIT(26)),
+ GATE(MT7621_CLK_CRYPTO, "crypto", "250m", BIT(29)),
+ GATE(MT7621_CLK_SHXC, "shxc", "50m", BIT(30))
+};
+
+static inline struct mt7621_gate *to_mt7621_gate(struct clk_hw *hw)
+{
+ return container_of(hw, struct mt7621_gate, hw);
+}
+
+static int mt7621_gate_enable(struct clk_hw *hw)
+{
+ struct mt7621_gate *clk_gate = to_mt7621_gate(hw);
+ struct regmap *sysc = clk_gate->priv->sysc;
+
+ return regmap_update_bits(sysc, SYSC_REG_CLKCFG1,
+ clk_gate->bit_idx, clk_gate->bit_idx);
+}
+
+static void mt7621_gate_disable(struct clk_hw *hw)
+{
+ struct mt7621_gate *clk_gate = to_mt7621_gate(hw);
+ struct regmap *sysc = clk_gate->priv->sysc;
+
+ regmap_update_bits(sysc, SYSC_REG_CLKCFG1, clk_gate->bit_idx, 0);
+}
+
+static int mt7621_gate_is_enabled(struct clk_hw *hw)
+{
+ struct mt7621_gate *clk_gate = to_mt7621_gate(hw);
+ struct regmap *sysc = clk_gate->priv->sysc;
+ u32 val;
+
+ if (regmap_read(sysc, SYSC_REG_CLKCFG1, &val))
+ return 0;
+
+ return val & BIT(clk_gate->bit_idx);
+}
+
+static const struct clk_ops mt7621_gate_ops = {
+ .enable = mt7621_gate_enable,
+ .disable = mt7621_gate_disable,
+ .is_enabled = mt7621_gate_is_enabled,
+};
+
+static int mt7621_gate_ops_init(struct device *dev,
+ struct mt7621_gate *sclk)
+{
+ struct clk_init_data init = {
+ /*
+ * Until now no clock driver existed so
+ * these SoC drivers are not prepared
+ * yet for the clock. We don't want kernel to
+ * disable anything so we add CLK_IS_CRITICAL
+ * flag here.
+ */
+ .flags = CLK_SET_RATE_PARENT | CLK_IS_CRITICAL,
+ .num_parents = 1,
+ .parent_names = &sclk->parent_name,
+ .ops = &mt7621_gate_ops,
+ .name = sclk->name,
+ };
+
+ sclk->hw.init = &init;
+ return devm_clk_hw_register(dev, &sclk->hw);
+}
+
+static int mt7621_register_gates(struct device *dev,
+ struct clk_hw_onecell_data *clk_data,
+ struct mt7621_clk_priv *priv)
+{
+ struct clk_hw **hws = clk_data->hws;
+ struct mt7621_gate *sclk;
+ int ret, i;
+
+ for (i = 0; i < ARRAY_SIZE(mt7621_gates); i++) {
+ sclk = &mt7621_gates[i];
+ sclk->priv = priv;
+ ret = mt7621_gate_ops_init(dev, sclk);
+ if (ret) {
+ dev_err(dev, "Couldn't register clock %s\n", sclk->name);
+ goto err_clk_unreg;
+ }
+
+ hws[sclk->idx] = &sclk->hw;
+ }
+
+ return 0;
+
+err_clk_unreg:
+ while (--i >= 0) {
+ sclk = &mt7621_gates[i];
+ clk_hw_unregister(&sclk->hw);
+ }
+ return ret;
+}
+
+#define FIXED(_id, _name, _rate) \
+ { \
+ .idx = _id, \
+ .name = _name, \
+ .parent_name = "xtal", \
+ .rate = _rate \
+ }
+
+static struct mt7621_fixed_clk mt7621_fixed_clks[] = {
+ FIXED(MT7621_CLK_50M, "50m", 50000000),
+ FIXED(MT7621_CLK_125M, "125m", 125000000),
+ FIXED(MT7621_CLK_150M, "150m", 150000000),
+ FIXED(MT7621_CLK_250M, "250m", 250000000),
+ FIXED(MT7621_CLK_270M, "270m", 270000000),
+};
+
+static int mt7621_register_fixed_clocks(struct device *dev,
+ struct clk_hw_onecell_data *clk_data)
+{
+ struct clk_hw **hws = clk_data->hws;
+ struct mt7621_fixed_clk *sclk;
+ int ret, i;
+
+ for (i = 0; i < ARRAY_SIZE(mt7621_fixed_clks); i++) {
+ sclk = &mt7621_fixed_clks[i];
+ sclk->hw = clk_hw_register_fixed_rate(dev, sclk->name,
+ sclk->parent_name, 0,
+ sclk->rate);
+ if (IS_ERR(sclk->hw)) {
+ dev_err(dev, "Couldn't register clock %s\n", sclk->name);
+ ret = PTR_ERR(sclk->hw);
+ goto err_clk_unreg;
+ }
+
+ hws[sclk->idx] = sclk->hw;
+ }
+
+ return 0;
+
+err_clk_unreg:
+ while (--i >= 0) {
+ sclk = &mt7621_fixed_clks[i];
+ clk_hw_unregister_fixed_rate(sclk->hw);
+ }
+ return ret;
+}
+
+static inline struct mt7621_clk *to_mt7621_clk(struct clk_hw *hw)
+{
+ return container_of(hw, struct mt7621_clk, hw);
+}
+
+static unsigned long mt7621_xtal_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ struct mt7621_clk *clk = to_mt7621_clk(hw);
+ struct regmap *sysc = clk->priv->sysc;
+ u32 val;
+
+ regmap_read(sysc, SYSC_REG_SYSTEM_CONFIG0, &val);
+ val = FIELD_GET(XTAL_MODE_SEL_MASK, val);
+
+ if (val <= 2)
+ return 20000000;
+ if (val <= 5)
+ return 40000000;
+
+ return 25000000;
+}
+
+static unsigned long mt7621_cpu_recalc_rate(struct clk_hw *hw,
+ unsigned long xtal_clk)
+{
+ static const u32 prediv_tbl[] = { 0, 1, 2, 2 };
+ struct mt7621_clk *clk = to_mt7621_clk(hw);
+ struct regmap *sysc = clk->priv->sysc;
+ struct regmap *memc = clk->priv->memc;
+ u32 clkcfg, clk_sel, curclk, ffiv, ffrac;
+ u32 pll, prediv, fbdiv;
+ unsigned long cpu_clk;
+
+ regmap_read(sysc, SYSC_REG_CLKCFG0, &clkcfg);
+ clk_sel = FIELD_GET(CPU_CLK_SEL_MASK, clkcfg);
+
+ regmap_read(sysc, SYSC_REG_CUR_CLK_STS, &curclk);
+ ffiv = FIELD_GET(CUR_CPU_FDIV_MASK, curclk);
+ ffrac = FIELD_GET(CUR_CPU_FFRAC_MASK, curclk);
+
+ switch (clk_sel) {
+ case 0:
+ cpu_clk = 500000000;
+ break;
+ case 1:
+ regmap_read(memc, MEMC_REG_CPU_PLL, &pll);
+ fbdiv = FIELD_GET(CPU_PLL_FBDIV_MASK, pll);
+ prediv = FIELD_GET(CPU_PLL_PREDIV_MASK, pll);
+ cpu_clk = ((fbdiv + 1) * xtal_clk) >> prediv_tbl[prediv];
+ break;
+ default:
+ cpu_clk = xtal_clk;
+ }
+
+ return cpu_clk / ffiv * ffrac;
+}
+
+static unsigned long mt7621_bus_recalc_rate(struct clk_hw *hw,
+ unsigned long parent_rate)
+{
+ return parent_rate / 4;
+}
+
+#define CLK_BASE(_name, _parent, _recalc) { \
+ .init = &(struct clk_init_data) { \
+ .name = _name, \
+ .ops = &(const struct clk_ops) { \
+ .recalc_rate = _recalc, \
+ }, \
+ .parent_data = &(const struct clk_parent_data) { \
+ .name = _parent, \
+ .fw_name = _parent \
+ }, \
+ .num_parents = _parent ? 1 : 0 \
+ }, \
+}
+
+static struct mt7621_clk mt7621_clks_base[] = {
+ { CLK_BASE("xtal", NULL, mt7621_xtal_recalc_rate) },
+ { CLK_BASE("cpu", "xtal", mt7621_cpu_recalc_rate) },
+ { CLK_BASE("bus", "cpu", mt7621_bus_recalc_rate) },
+};
+
+static struct clk_hw *mt7621_clk_early[MT7621_CLK_MAX];
+
+static int mt7621_register_early_clocks(struct device_node *np,
+ struct clk_hw_onecell_data *clk_data,
+ struct mt7621_clk_priv *priv)
+{
+ struct clk_hw **hws = clk_data->hws;
+ struct mt7621_clk *sclk;
+ int ret, i, j;
+
+ for (i = 0; i < ARRAY_SIZE(mt7621_clks_base); i++) {
+ sclk = &mt7621_clks_base[i];
+ sclk->priv = priv;
+ ret = of_clk_hw_register(np, &sclk->hw);
+ if (ret) {
+ pr_err("Couldn't register top clock %i\n", i);
+ goto err_clk_unreg;
+ }
+
+ hws[i] = &sclk->hw;
+ mt7621_clk_early[i] = &sclk->hw;
+ }
+
+ for (j = i; j < MT7621_CLK_MAX; j++)
+ mt7621_clk_early[j] = ERR_PTR(-EPROBE_DEFER);
+
+ return 0;
+
+err_clk_unreg:
+ while (--i >= 0) {
+ sclk = &mt7621_clks_base[i];
+ clk_hw_unregister(&sclk->hw);
+ }
+ return ret;
+}
+
+static void __init mt7621_clk_init(struct device_node *node)
+{
+ struct mt7621_clk_priv *priv;
+ struct clk_hw_onecell_data *clk_data;
+ int ret, i, count;
+
+ priv = kzalloc(sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return;
+
+ priv->sysc = syscon_node_to_regmap(node);
+ if (IS_ERR(priv->sysc)) {
+ pr_err("Could not get sysc syscon regmap\n");
+ goto free_clk_priv;
+ }
+
+ priv->memc = syscon_regmap_lookup_by_phandle(node, "ralink,memctl");
+ if (IS_ERR(priv->memc)) {
+ pr_err("Could not get memc syscon regmap\n");
+ goto free_clk_priv;
+ }
+
+ count = ARRAY_SIZE(mt7621_clks_base) +
+ ARRAY_SIZE(mt7621_fixed_clks) + ARRAY_SIZE(mt7621_gates);
+ clk_data = kzalloc(struct_size(clk_data, hws, count), GFP_KERNEL);
+ if (!clk_data)
+ goto free_clk_priv;
+
+ ret = mt7621_register_early_clocks(node, clk_data, priv);
+ if (ret) {
+ pr_err("Couldn't register top clocks\n");
+ goto free_clk_data;
+ }
+
+ clk_data->num = count;
+
+ ret = of_clk_add_hw_provider(node, of_clk_hw_onecell_get, clk_data);
+ if (ret) {
+ pr_err("Couldn't add clk hw provider\n");
+ goto unreg_clk_top;
+ }
+
+ return;
+
+unreg_clk_top:
+ for (i = 0; i < ARRAY_SIZE(mt7621_clks_base); i++) {
+ struct mt7621_clk *sclk = &mt7621_clks_base[i];
+
+ clk_hw_unregister(&sclk->hw);
+ }
+
+free_clk_data:
+ kfree(clk_data);
+
+free_clk_priv:
+ kfree(priv);
+}
+CLK_OF_DECLARE_DRIVER(mt7621_clk, "mediatek,mt7621-sysc", mt7621_clk_init);
+
+static int mt7621_clk_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct clk_hw_onecell_data *clk_data;
+ struct device *dev = &pdev->dev;
+ struct mt7621_clk_priv *priv;
+ int ret, i, count;
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->sysc = syscon_node_to_regmap(np);
+ if (IS_ERR(priv->sysc)) {
+ ret = PTR_ERR(priv->sysc);
+ dev_err(dev, "Could not get sysc syscon regmap\n");
+ return ret;
+ }
+
+ priv->memc = syscon_regmap_lookup_by_phandle(np, "ralink,memctl");
+ if (IS_ERR(priv->memc)) {
+ ret = PTR_ERR(priv->memc);
+ dev_err(dev, "Could not get memc syscon regmap\n");
+ return ret;
+ }
+
+ count = ARRAY_SIZE(mt7621_clks_base) +
+ ARRAY_SIZE(mt7621_fixed_clks) + ARRAY_SIZE(mt7621_gates);
+ clk_data = devm_kzalloc(dev, struct_size(clk_data, hws, count),
+ GFP_KERNEL);
+ if (!clk_data)
+ return -ENOMEM;
+
+ for (i = 0; i < ARRAY_SIZE(mt7621_clks_base); i++)
+ clk_data->hws[i] = mt7621_clk_early[i];
+
+ ret = mt7621_register_fixed_clocks(dev, clk_data);
+ if (ret) {
+ dev_err(dev, "Couldn't register fixed clocks\n");
+ return ret;
+ }
+
+ ret = mt7621_register_gates(dev, clk_data, priv);
+ if (ret) {
+ dev_err(dev, "Couldn't register fixed clock gates\n");
+ goto unreg_clk_fixed;
+ }
+
+ clk_data->num = count;
+
+ ret = devm_of_clk_add_hw_provider(dev, of_clk_hw_onecell_get, clk_data);
+ if (ret) {
+ dev_err(dev, "Couldn't add clk hw provider\n");
+ goto unreg_clk_gates;
+ }
+
+ return 0;
+
+unreg_clk_gates:
+ for (i = 0; i < ARRAY_SIZE(mt7621_gates); i++) {
+ struct mt7621_gate *sclk = &mt7621_gates[i];
+
+ clk_hw_unregister(&sclk->hw);
+ }
+
+unreg_clk_fixed:
+ for (i = 0; i < ARRAY_SIZE(mt7621_fixed_clks); i++) {
+ struct mt7621_fixed_clk *sclk = &mt7621_fixed_clks[i];
+
+ clk_hw_unregister_fixed_rate(sclk->hw);
+ }
+
+ return ret;
+}
+
+static const struct of_device_id mt7621_clk_of_match[] = {
+ { .compatible = "mediatek,mt7621-sysc" },
+ {}
+};
+
+static struct platform_driver mt7621_clk_driver = {
+ .probe = mt7621_clk_probe,
+ .driver = {
+ .name = "mt7621-clk",
+ .of_match_table = mt7621_clk_of_match,
+ },
+};
+builtin_platform_driver(mt7621_clk_driver);
diff --git a/drivers/clk/renesas/clk-div6.c b/drivers/clk/renesas/clk-div6.c
index 5ca183e70166..8fb68e703a6b 100644
--- a/drivers/clk/renesas/clk-div6.c
+++ b/drivers/clk/renesas/clk-div6.c
@@ -216,7 +216,7 @@ struct clk * __init cpg_div6_register(const char *name,
struct raw_notifier_head *notifiers)
{
unsigned int valid_parents;
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct div6_clock *clock;
struct clk *clk;
unsigned int i;
@@ -267,7 +267,6 @@ struct clk * __init cpg_div6_register(const char *name,
/* Register the clock. */
init.name = name;
init.ops = &cpg_div6_clock_ops;
- init.flags = 0;
init.parent_names = parent_names;
init.num_parents = valid_parents;
diff --git a/drivers/clk/renesas/clk-mstp.c b/drivers/clk/renesas/clk-mstp.c
index 003e9ce45757..90804ac06fa5 100644
--- a/drivers/clk/renesas/clk-mstp.c
+++ b/drivers/clk/renesas/clk-mstp.c
@@ -150,7 +150,7 @@ static struct clk * __init cpg_mstp_clock_register(const char *name,
const char *parent_name, unsigned int index,
struct mstp_clock_group *group)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct mstp_clock *clock;
struct clk *clk;
diff --git a/drivers/clk/renesas/r8a7795-cpg-mssr.c b/drivers/clk/renesas/r8a7795-cpg-mssr.c
index 068018ae3c6e..c32d2c678046 100644
--- a/drivers/clk/renesas/r8a7795-cpg-mssr.c
+++ b/drivers/clk/renesas/r8a7795-cpg-mssr.c
@@ -128,6 +128,11 @@ static struct mssr_mod_clk r8a7795_mod_clks[] __initdata = {
DEF_MOD("fdp1-2", 117, R8A7795_CLK_S2D1), /* ES1.x */
DEF_MOD("fdp1-1", 118, R8A7795_CLK_S0D1),
DEF_MOD("fdp1-0", 119, R8A7795_CLK_S0D1),
+ DEF_MOD("tmu4", 121, R8A7795_CLK_S0D6),
+ DEF_MOD("tmu3", 122, R8A7795_CLK_S3D2),
+ DEF_MOD("tmu2", 123, R8A7795_CLK_S3D2),
+ DEF_MOD("tmu1", 124, R8A7795_CLK_S3D2),
+ DEF_MOD("tmu0", 125, R8A7795_CLK_CP),
DEF_MOD("scif5", 202, R8A7795_CLK_S3D4),
DEF_MOD("scif4", 203, R8A7795_CLK_S3D4),
DEF_MOD("scif3", 204, R8A7795_CLK_S3D4),
@@ -362,6 +367,7 @@ static const unsigned int r8a7795es1_mod_nullify[] __initconst = {
static const struct mssr_mod_reparent r8a7795es1_mod_reparent[] __initconst = {
{ MOD_CLK_ID(118), R8A7795_CLK_S2D1 }, /* FDP1-1 */
{ MOD_CLK_ID(119), R8A7795_CLK_S2D1 }, /* FDP1-0 */
+ { MOD_CLK_ID(121), R8A7795_CLK_S3D2 }, /* TMU4 */
{ MOD_CLK_ID(217), R8A7795_CLK_S3D1 }, /* SYS-DMAC2 */
{ MOD_CLK_ID(218), R8A7795_CLK_S3D1 }, /* SYS-DMAC1 */
{ MOD_CLK_ID(219), R8A7795_CLK_S3D1 }, /* SYS-DMAC0 */
diff --git a/drivers/clk/renesas/r8a77965-cpg-mssr.c b/drivers/clk/renesas/r8a77965-cpg-mssr.c
index 46a157732759..bc1be8bcbbe4 100644
--- a/drivers/clk/renesas/r8a77965-cpg-mssr.c
+++ b/drivers/clk/renesas/r8a77965-cpg-mssr.c
@@ -250,6 +250,7 @@ static const struct mssr_mod_clk r8a77965_mod_clks[] __initconst = {
DEF_MOD("ssi2", 1013, MOD_CLK_ID(1005)),
DEF_MOD("ssi1", 1014, MOD_CLK_ID(1005)),
DEF_MOD("ssi0", 1015, MOD_CLK_ID(1005)),
+ DEF_MOD("dab", 1016, R8A77965_CLK_S0D6),
DEF_MOD("scu-all", 1017, R8A77965_CLK_S3D4),
DEF_MOD("scu-dvc1", 1018, MOD_CLK_ID(1017)),
DEF_MOD("scu-dvc0", 1019, MOD_CLK_ID(1017)),
diff --git a/drivers/clk/renesas/r8a77990-cpg-mssr.c b/drivers/clk/renesas/r8a77990-cpg-mssr.c
index 2d172f80b34c..a582f2ec3294 100644
--- a/drivers/clk/renesas/r8a77990-cpg-mssr.c
+++ b/drivers/clk/renesas/r8a77990-cpg-mssr.c
@@ -232,6 +232,7 @@ static const struct mssr_mod_clk r8a77990_mod_clks[] __initconst = {
DEF_MOD("ssi2", 1013, MOD_CLK_ID(1005)),
DEF_MOD("ssi1", 1014, MOD_CLK_ID(1005)),
DEF_MOD("ssi0", 1015, MOD_CLK_ID(1005)),
+ DEF_MOD("dab", 1016, R8A77990_CLK_S3D1),
DEF_MOD("scu-all", 1017, R8A77990_CLK_S3D4),
DEF_MOD("scu-dvc1", 1018, MOD_CLK_ID(1017)),
DEF_MOD("scu-dvc0", 1019, MOD_CLK_ID(1017)),
diff --git a/drivers/clk/renesas/r8a779a0-cpg-mssr.c b/drivers/clk/renesas/r8a779a0-cpg-mssr.c
index f23fe9d5e5e1..867c565cb58f 100644
--- a/drivers/clk/renesas/r8a779a0-cpg-mssr.c
+++ b/drivers/clk/renesas/r8a779a0-cpg-mssr.c
@@ -144,6 +144,7 @@ static const struct cpg_core_clk r8a779a0_core_clks[] __initconst = {
DEF_FIXED("vcbus", R8A779A0_CLK_VCBUS, CLK_PLL5_DIV4, 1, 1),
DEF_FIXED("cbfusa", R8A779A0_CLK_CBFUSA, CLK_EXTAL, 2, 1),
DEF_FIXED("cp", R8A779A0_CLK_CP, CLK_EXTAL, 2, 1),
+ DEF_FIXED("cl16mck", R8A779A0_CLK_CL16MCK, CLK_PLL1_DIV2, 64, 1),
DEF_SD("sd0", R8A779A0_CLK_SD0, CLK_SDSRC, 0x870),
@@ -192,6 +193,11 @@ static const struct mssr_mod_clk r8a779a0_mod_clks[] __initconst = {
DEF_MOD("sdhi0", 706, R8A779A0_CLK_SD0),
DEF_MOD("sydm1", 709, R8A779A0_CLK_S1D2),
DEF_MOD("sydm2", 710, R8A779A0_CLK_S1D2),
+ DEF_MOD("tmu0", 713, R8A779A0_CLK_CL16MCK),
+ DEF_MOD("tmu1", 714, R8A779A0_CLK_S1D4),
+ DEF_MOD("tmu2", 715, R8A779A0_CLK_S1D4),
+ DEF_MOD("tmu3", 716, R8A779A0_CLK_S1D4),
+ DEF_MOD("tmu4", 717, R8A779A0_CLK_S1D4),
DEF_MOD("vin00", 730, R8A779A0_CLK_S1D1),
DEF_MOD("vin01", 731, R8A779A0_CLK_S1D1),
DEF_MOD("vin02", 800, R8A779A0_CLK_S1D1),
@@ -227,10 +233,15 @@ static const struct mssr_mod_clk r8a779a0_mod_clks[] __initconst = {
DEF_MOD("vspd0", 830, R8A779A0_CLK_S3D1),
DEF_MOD("vspd1", 831, R8A779A0_CLK_S3D1),
DEF_MOD("rwdt", 907, R8A779A0_CLK_R),
+ DEF_MOD("cmt0", 910, R8A779A0_CLK_R),
+ DEF_MOD("cmt1", 911, R8A779A0_CLK_R),
+ DEF_MOD("cmt2", 912, R8A779A0_CLK_R),
+ DEF_MOD("cmt3", 913, R8A779A0_CLK_R),
DEF_MOD("pfc0", 915, R8A779A0_CLK_CP),
DEF_MOD("pfc1", 916, R8A779A0_CLK_CP),
DEF_MOD("pfc2", 917, R8A779A0_CLK_CP),
DEF_MOD("pfc3", 918, R8A779A0_CLK_CP),
+ DEF_MOD("tsc", 919, R8A779A0_CLK_CL16MCK),
DEF_MOD("vspx0", 1028, R8A779A0_CLK_S1D1),
DEF_MOD("vspx1", 1029, R8A779A0_CLK_S1D1),
DEF_MOD("vspx2", 1030, R8A779A0_CLK_S1D1),
diff --git a/drivers/clk/renesas/r9a06g032-clocks.c b/drivers/clk/renesas/r9a06g032-clocks.c
index 892e91b92f2c..71b11443f6fc 100644
--- a/drivers/clk/renesas/r9a06g032-clocks.c
+++ b/drivers/clk/renesas/r9a06g032-clocks.c
@@ -279,7 +279,7 @@ static const struct r9a06g032_clkdesc r9a06g032_clocks[] = {
/*
* These are not hardware clocks, but are needed to handle the special
* case where we have a 'selector bit' that doesn't just change the
- * parent for a clock, but also the gate it's suposed to use.
+ * parent for a clock, but also the gate it's supposed to use.
*/
{
.index = R9A06G032_UART_GROUP_012,
@@ -311,7 +311,7 @@ static const struct r9a06g032_clkdesc r9a06g032_clocks[] = {
struct r9a06g032_priv {
struct clk_onecell_data data;
- spinlock_t lock; /* protects concurent access to gates */
+ spinlock_t lock; /* protects concurrent access to gates */
void __iomem *reg;
};
@@ -504,7 +504,7 @@ r9a06g032_register_gate(struct r9a06g032_priv *clocks,
{
struct clk *clk;
struct r9a06g032_clk_gate *g;
- struct clk_init_data init;
+ struct clk_init_data init = {};
g = kzalloc(sizeof(*g), GFP_KERNEL);
if (!g)
@@ -674,7 +674,7 @@ r9a06g032_register_div(struct r9a06g032_priv *clocks,
{
struct r9a06g032_clk_div *div;
struct clk *clk;
- struct clk_init_data init;
+ struct clk_init_data init = {};
unsigned int i;
div = kzalloc(sizeof(*div), GFP_KERNEL);
@@ -758,7 +758,7 @@ r9a06g032_register_bitsel(struct r9a06g032_priv *clocks,
{
struct clk *clk;
struct r9a06g032_clk_bitsel *g;
- struct clk_init_data init;
+ struct clk_init_data init = {};
const char *names[2];
/* allocate the gate */
@@ -849,7 +849,7 @@ r9a06g032_register_dualgate(struct r9a06g032_priv *clocks,
{
struct r9a06g032_clk_dualgate *g;
struct clk *clk;
- struct clk_init_data init;
+ struct clk_init_data init = {};
/* allocate the gate */
g = kzalloc(sizeof(*g), GFP_KERNEL);
diff --git a/drivers/clk/renesas/rcar-cpg-lib.c b/drivers/clk/renesas/rcar-cpg-lib.c
index 7e7e5d1341d5..5678768ee1f2 100644
--- a/drivers/clk/renesas/rcar-cpg-lib.c
+++ b/drivers/clk/renesas/rcar-cpg-lib.c
@@ -226,7 +226,7 @@ struct clk * __init cpg_sd_clk_register(const char *name,
void __iomem *base, unsigned int offset, const char *parent_name,
struct raw_notifier_head *notifiers, bool skip_first)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct sd_clock *clock;
struct clk *clk;
u32 val;
diff --git a/drivers/clk/renesas/rcar-gen2-cpg.c b/drivers/clk/renesas/rcar-gen2-cpg.c
index d4fa3dc3e2a2..edae874fa2b6 100644
--- a/drivers/clk/renesas/rcar-gen2-cpg.c
+++ b/drivers/clk/renesas/rcar-gen2-cpg.c
@@ -137,7 +137,7 @@ static struct clk * __init cpg_z_clk_register(const char *name,
const char *parent_name,
void __iomem *base)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct cpg_z_clk *zclk;
struct clk *clk;
@@ -147,7 +147,6 @@ static struct clk * __init cpg_z_clk_register(const char *name,
init.name = name;
init.ops = &cpg_z_clk_ops;
- init.flags = 0;
init.parent_names = &parent_name;
init.num_parents = 1;
diff --git a/drivers/clk/renesas/rcar-gen3-cpg.c b/drivers/clk/renesas/rcar-gen3-cpg.c
index 17826599e9dd..caa0f9414e45 100644
--- a/drivers/clk/renesas/rcar-gen3-cpg.c
+++ b/drivers/clk/renesas/rcar-gen3-cpg.c
@@ -143,7 +143,7 @@ static struct clk * __init cpg_z_clk_register(const char *name,
unsigned int div,
unsigned int offset)
{
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct cpg_z_clk *zclk;
struct clk *clk;
diff --git a/drivers/clk/renesas/rcar-usb2-clock-sel.c b/drivers/clk/renesas/rcar-usb2-clock-sel.c
index 3abafd78f7c8..34a85dc95beb 100644
--- a/drivers/clk/renesas/rcar-usb2-clock-sel.c
+++ b/drivers/clk/renesas/rcar-usb2-clock-sel.c
@@ -144,7 +144,7 @@ static int rcar_usb2_clock_sel_probe(struct platform_device *pdev)
struct device_node *np = dev->of_node;
struct usb2_clock_sel_priv *priv;
struct clk *clk;
- struct clk_init_data init;
+ struct clk_init_data init = {};
int ret;
priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
@@ -188,9 +188,6 @@ static int rcar_usb2_clock_sel_probe(struct platform_device *pdev)
init.name = "rcar_usb2_clock_sel";
init.ops = &usb2_clock_sel_clock_ops;
- init.flags = 0;
- init.parent_names = NULL;
- init.num_parents = 0;
priv->hw.init = &init;
clk = clk_register(NULL, &priv->hw);
diff --git a/drivers/clk/renesas/renesas-cpg-mssr.c b/drivers/clk/renesas/renesas-cpg-mssr.c
index bffbc3d2faf5..fc531d35b269 100644
--- a/drivers/clk/renesas/renesas-cpg-mssr.c
+++ b/drivers/clk/renesas/renesas-cpg-mssr.c
@@ -408,7 +408,7 @@ static void __init cpg_mssr_register_mod_clk(const struct mssr_mod_clk *mod,
struct mstp_clock *clock = NULL;
struct device *dev = priv->dev;
unsigned int id = mod->id;
- struct clk_init_data init;
+ struct clk_init_data init = {};
struct clk *parent, *clk;
const char *parent_name;
unsigned int i;
diff --git a/drivers/clk/rockchip/Kconfig b/drivers/clk/rockchip/Kconfig
index effd05032e85..2dfd6a383393 100644
--- a/drivers/clk/rockchip/Kconfig
+++ b/drivers/clk/rockchip/Kconfig
@@ -11,78 +11,85 @@ config COMMON_CLK_ROCKCHIP
if COMMON_CLK_ROCKCHIP
config CLK_PX30
bool "Rockchip PX30 clock controller support"
- depends on (ARM64 || COMPILE_TEST)
+ depends on ARM64 || COMPILE_TEST
default y
help
Build the driver for PX30 Clock Driver.
config CLK_RV110X
bool "Rockchip RV110x clock controller support"
- depends on (ARM || COMPILE_TEST)
+ depends on ARM || COMPILE_TEST
default y
help
Build the driver for RV110x Clock Driver.
config CLK_RK3036
bool "Rockchip RK3036 clock controller support"
- depends on (ARM || COMPILE_TEST)
+ depends on ARM || COMPILE_TEST
default y
help
Build the driver for RK3036 Clock Driver.
config CLK_RK312X
bool "Rockchip RK312x clock controller support"
- depends on (ARM || COMPILE_TEST)
+ depends on ARM || COMPILE_TEST
default y
help
Build the driver for RK312x Clock Driver.
config CLK_RK3188
bool "Rockchip RK3188 clock controller support"
- depends on (ARM || COMPILE_TEST)
+ depends on ARM || COMPILE_TEST
default y
help
Build the driver for RK3188 Clock Driver.
config CLK_RK322X
bool "Rockchip RK322x clock controller support"
- depends on (ARM || COMPILE_TEST)
+ depends on ARM || COMPILE_TEST
default y
help
Build the driver for RK322x Clock Driver.
config CLK_RK3288
bool "Rockchip RK3288 clock controller support"
- depends on (ARM || COMPILE_TEST)
+ depends on ARM || COMPILE_TEST
default y
help
Build the driver for RK3288 Clock Driver.
config CLK_RK3308
bool "Rockchip RK3308 clock controller support"
- depends on (ARM64 || COMPILE_TEST)
+ depends on ARM64 || COMPILE_TEST
default y
help
Build the driver for RK3308 Clock Driver.
config CLK_RK3328
bool "Rockchip RK3328 clock controller support"
- depends on (ARM64 || COMPILE_TEST)
+ depends on ARM64 || COMPILE_TEST
default y
help
Build the driver for RK3328 Clock Driver.
config CLK_RK3368
bool "Rockchip RK3368 clock controller support"
- depends on (ARM64 || COMPILE_TEST)
+ depends on ARM64 || COMPILE_TEST
default y
help
Build the driver for RK3368 Clock Driver.
config CLK_RK3399
tristate "Rockchip RK3399 clock controller support"
- depends on (ARM64 || COMPILE_TEST)
+ depends on ARM64 || COMPILE_TEST
default y
help
Build the driver for RK3399 Clock Driver.
+
+config CLK_RK3568
+ tristate "Rockchip RK3568 clock controller support"
+ depends on ARM64 || COMPILE_TEST
+ default y
+ help
+ Build the driver for RK3568 Clock Driver.
endif
diff --git a/drivers/clk/rockchip/Makefile b/drivers/clk/rockchip/Makefile
index a99e4d9bbae1..2b78f1247372 100644
--- a/drivers/clk/rockchip/Makefile
+++ b/drivers/clk/rockchip/Makefile
@@ -26,3 +26,4 @@ obj-$(CONFIG_CLK_RK3308) += clk-rk3308.o
obj-$(CONFIG_CLK_RK3328) += clk-rk3328.o
obj-$(CONFIG_CLK_RK3368) += clk-rk3368.o
obj-$(CONFIG_CLK_RK3399) += clk-rk3399.o
+obj-$(CONFIG_CLK_RK3568) += clk-rk3568.o
diff --git a/drivers/clk/rockchip/clk-cpu.c b/drivers/clk/rockchip/clk-cpu.c
index fa9027fb1920..47288197c9d7 100644
--- a/drivers/clk/rockchip/clk-cpu.c
+++ b/drivers/clk/rockchip/clk-cpu.c
@@ -84,10 +84,10 @@ static unsigned long rockchip_cpuclk_recalc_rate(struct clk_hw *hw,
{
struct rockchip_cpuclk *cpuclk = to_rockchip_cpuclk_hw(hw);
const struct rockchip_cpuclk_reg_data *reg_data = cpuclk->reg_data;
- u32 clksel0 = readl_relaxed(cpuclk->reg_base + reg_data->core_reg);
+ u32 clksel0 = readl_relaxed(cpuclk->reg_base + reg_data->core_reg[0]);
- clksel0 >>= reg_data->div_core_shift;
- clksel0 &= reg_data->div_core_mask;
+ clksel0 >>= reg_data->div_core_shift[0];
+ clksel0 &= reg_data->div_core_mask[0];
return parent_rate / (clksel0 + 1);
}
@@ -120,6 +120,7 @@ static int rockchip_cpuclk_pre_rate_change(struct rockchip_cpuclk *cpuclk,
const struct rockchip_cpuclk_rate_table *rate;
unsigned long alt_prate, alt_div;
unsigned long flags;
+ int i = 0;
/* check validity of the new rate */
rate = rockchip_get_cpuclk_settings(cpuclk, ndata->new_rate);
@@ -142,10 +143,10 @@ static int rockchip_cpuclk_pre_rate_change(struct rockchip_cpuclk *cpuclk,
if (alt_prate > ndata->old_rate) {
/* calculate dividers */
alt_div = DIV_ROUND_UP(alt_prate, ndata->old_rate) - 1;
- if (alt_div > reg_data->div_core_mask) {
+ if (alt_div > reg_data->div_core_mask[0]) {
pr_warn("%s: limiting alt-divider %lu to %d\n",
- __func__, alt_div, reg_data->div_core_mask);
- alt_div = reg_data->div_core_mask;
+ __func__, alt_div, reg_data->div_core_mask[0]);
+ alt_div = reg_data->div_core_mask[0];
}
/*
@@ -158,19 +159,17 @@ static int rockchip_cpuclk_pre_rate_change(struct rockchip_cpuclk *cpuclk,
pr_debug("%s: setting div %lu as alt-rate %lu > old-rate %lu\n",
__func__, alt_div, alt_prate, ndata->old_rate);
- writel(HIWORD_UPDATE(alt_div, reg_data->div_core_mask,
- reg_data->div_core_shift) |
- HIWORD_UPDATE(reg_data->mux_core_alt,
- reg_data->mux_core_mask,
- reg_data->mux_core_shift),
- cpuclk->reg_base + reg_data->core_reg);
- } else {
- /* select alternate parent */
- writel(HIWORD_UPDATE(reg_data->mux_core_alt,
- reg_data->mux_core_mask,
- reg_data->mux_core_shift),
- cpuclk->reg_base + reg_data->core_reg);
+ for (i = 0; i < reg_data->num_cores; i++) {
+ writel(HIWORD_UPDATE(alt_div, reg_data->div_core_mask[i],
+ reg_data->div_core_shift[i]),
+ cpuclk->reg_base + reg_data->core_reg[i]);
+ }
}
+ /* select alternate parent */
+ writel(HIWORD_UPDATE(reg_data->mux_core_alt,
+ reg_data->mux_core_mask,
+ reg_data->mux_core_shift),
+ cpuclk->reg_base + reg_data->core_reg[0]);
spin_unlock_irqrestore(cpuclk->lock, flags);
return 0;
@@ -182,6 +181,7 @@ static int rockchip_cpuclk_post_rate_change(struct rockchip_cpuclk *cpuclk,
const struct rockchip_cpuclk_reg_data *reg_data = cpuclk->reg_data;
const struct rockchip_cpuclk_rate_table *rate;
unsigned long flags;
+ int i = 0;
rate = rockchip_get_cpuclk_settings(cpuclk, ndata->new_rate);
if (!rate) {
@@ -202,12 +202,17 @@ static int rockchip_cpuclk_post_rate_change(struct rockchip_cpuclk *cpuclk,
* primary parent by the extra dividers that were needed for the alt.
*/
- writel(HIWORD_UPDATE(0, reg_data->div_core_mask,
- reg_data->div_core_shift) |
- HIWORD_UPDATE(reg_data->mux_core_main,
- reg_data->mux_core_mask,
- reg_data->mux_core_shift),
- cpuclk->reg_base + reg_data->core_reg);
+ writel(HIWORD_UPDATE(reg_data->mux_core_main,
+ reg_data->mux_core_mask,
+ reg_data->mux_core_shift),
+ cpuclk->reg_base + reg_data->core_reg[0]);
+
+ /* remove dividers */
+ for (i = 0; i < reg_data->num_cores; i++) {
+ writel(HIWORD_UPDATE(0, reg_data->div_core_mask[i],
+ reg_data->div_core_shift[i]),
+ cpuclk->reg_base + reg_data->core_reg[i]);
+ }
if (ndata->old_rate > ndata->new_rate)
rockchip_cpuclk_set_dividers(cpuclk, rate);
diff --git a/drivers/clk/rockchip/clk-px30.c b/drivers/clk/rockchip/clk-px30.c
index 6fb9c98b7d24..b58619eb412b 100644
--- a/drivers/clk/rockchip/clk-px30.c
+++ b/drivers/clk/rockchip/clk-px30.c
@@ -124,9 +124,10 @@ static struct rockchip_cpuclk_rate_table px30_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data px30_cpuclk_data = {
- .core_reg = PX30_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0xf,
+ .core_reg[0] = PX30_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0xf,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 7,
diff --git a/drivers/clk/rockchip/clk-rk3036.c b/drivers/clk/rockchip/clk-rk3036.c
index 6a46f85ad837..91d56ad45817 100644
--- a/drivers/clk/rockchip/clk-rk3036.c
+++ b/drivers/clk/rockchip/clk-rk3036.c
@@ -102,9 +102,10 @@ static struct rockchip_cpuclk_rate_table rk3036_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3036_cpuclk_data = {
- .core_reg = RK2928_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK2928_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 7,
diff --git a/drivers/clk/rockchip/clk-rk3128.c b/drivers/clk/rockchip/clk-rk3128.c
index 4b1122e98e16..aa53797dbfc1 100644
--- a/drivers/clk/rockchip/clk-rk3128.c
+++ b/drivers/clk/rockchip/clk-rk3128.c
@@ -117,9 +117,10 @@ static struct rockchip_cpuclk_rate_table rk3128_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3128_cpuclk_data = {
- .core_reg = RK2928_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK2928_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 7,
diff --git a/drivers/clk/rockchip/clk-rk3188.c b/drivers/clk/rockchip/clk-rk3188.c
index 0b76ad34de00..9c8af4d1dae0 100644
--- a/drivers/clk/rockchip/clk-rk3188.c
+++ b/drivers/clk/rockchip/clk-rk3188.c
@@ -145,9 +145,10 @@ static struct rockchip_cpuclk_rate_table rk3066_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3066_cpuclk_data = {
- .core_reg = RK2928_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK2928_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 8,
@@ -184,9 +185,10 @@ static struct rockchip_cpuclk_rate_table rk3188_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3188_cpuclk_data = {
- .core_reg = RK2928_CLKSEL_CON(0),
- .div_core_shift = 9,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK2928_CLKSEL_CON(0),
+ .div_core_shift[0] = 9,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 8,
diff --git a/drivers/clk/rockchip/clk-rk3228.c b/drivers/clk/rockchip/clk-rk3228.c
index 47d6482dda9d..a24a35553e13 100644
--- a/drivers/clk/rockchip/clk-rk3228.c
+++ b/drivers/clk/rockchip/clk-rk3228.c
@@ -119,9 +119,10 @@ static struct rockchip_cpuclk_rate_table rk3228_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3228_cpuclk_data = {
- .core_reg = RK2928_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK2928_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 6,
diff --git a/drivers/clk/rockchip/clk-rk3288.c b/drivers/clk/rockchip/clk-rk3288.c
index 93c794695c46..baa5aebd3277 100644
--- a/drivers/clk/rockchip/clk-rk3288.c
+++ b/drivers/clk/rockchip/clk-rk3288.c
@@ -179,9 +179,10 @@ static struct rockchip_cpuclk_rate_table rk3288_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3288_cpuclk_data = {
- .core_reg = RK3288_CLKSEL_CON(0),
- .div_core_shift = 8,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK3288_CLKSEL_CON(0),
+ .div_core_shift[0] = 8,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 15,
diff --git a/drivers/clk/rockchip/clk-rk3308.c b/drivers/clk/rockchip/clk-rk3308.c
index 5bf15f2a44b7..2c3bd0c749f2 100644
--- a/drivers/clk/rockchip/clk-rk3308.c
+++ b/drivers/clk/rockchip/clk-rk3308.c
@@ -109,9 +109,10 @@ static struct rockchip_cpuclk_rate_table rk3308_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3308_cpuclk_data = {
- .core_reg = RK3308_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0xf,
+ .core_reg[0] = RK3308_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0xf,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 6,
diff --git a/drivers/clk/rockchip/clk-rk3328.c b/drivers/clk/rockchip/clk-rk3328.c
index 2429b7c2a8b3..267ab54937d3 100644
--- a/drivers/clk/rockchip/clk-rk3328.c
+++ b/drivers/clk/rockchip/clk-rk3328.c
@@ -130,9 +130,10 @@ static struct rockchip_cpuclk_rate_table rk3328_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rk3328_cpuclk_data = {
- .core_reg = RK3328_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK3328_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 3,
.mux_core_shift = 6,
diff --git a/drivers/clk/rockchip/clk-rk3368.c b/drivers/clk/rockchip/clk-rk3368.c
index 9a0dab9448db..2c50cc2cc6db 100644
--- a/drivers/clk/rockchip/clk-rk3368.c
+++ b/drivers/clk/rockchip/clk-rk3368.c
@@ -154,9 +154,10 @@ static struct clk_div_table div_ddrphy_t[] = {
#define IFLAGS ROCKCHIP_INVERTER_HIWORD_MASK
static const struct rockchip_cpuclk_reg_data rk3368_cpuclkb_data = {
- .core_reg = RK3368_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK3368_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 7,
@@ -164,11 +165,12 @@ static const struct rockchip_cpuclk_reg_data rk3368_cpuclkb_data = {
};
static const struct rockchip_cpuclk_reg_data rk3368_cpuclkl_data = {
- .core_reg = RK3368_CLKSEL_CON(2),
- .div_core_shift = 0,
+ .core_reg[0] = RK3368_CLKSEL_CON(2),
+ .div_core_shift[0] = 0,
.mux_core_alt = 1,
+ .num_cores = 1,
.mux_core_main = 0,
- .div_core_mask = 0x1f,
+ .div_core_mask[0] = 0x1f,
.mux_core_shift = 7,
.mux_core_mask = 0x1,
};
diff --git a/drivers/clk/rockchip/clk-rk3399.c b/drivers/clk/rockchip/clk-rk3399.c
index 7df2f1e00347..62a4f2543960 100644
--- a/drivers/clk/rockchip/clk-rk3399.c
+++ b/drivers/clk/rockchip/clk-rk3399.c
@@ -291,9 +291,10 @@ static struct rockchip_clk_branch rk3399_pmuclk_wifi_fracmux __initdata =
RK3399_PMU_CLKSEL_CON(1), 14, 1, MFLAGS);
static const struct rockchip_cpuclk_reg_data rk3399_cpuclkl_data = {
- .core_reg = RK3399_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK3399_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 3,
.mux_core_main = 0,
.mux_core_shift = 6,
@@ -301,9 +302,10 @@ static const struct rockchip_cpuclk_reg_data rk3399_cpuclkl_data = {
};
static const struct rockchip_cpuclk_reg_data rk3399_cpuclkb_data = {
- .core_reg = RK3399_CLKSEL_CON(2),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RK3399_CLKSEL_CON(2),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 3,
.mux_core_main = 1,
.mux_core_shift = 6,
@@ -1655,4 +1657,3 @@ builtin_platform_driver_probe(clk_rk3399_driver, clk_rk3399_probe);
MODULE_DESCRIPTION("Rockchip RK3399 Clock Driver");
MODULE_LICENSE("GPL");
-MODULE_ALIAS("platform:clk-rk3399");
diff --git a/drivers/clk/rockchip/clk-rk3568.c b/drivers/clk/rockchip/clk-rk3568.c
new file mode 100644
index 000000000000..946ea2f45bf3
--- /dev/null
+++ b/drivers/clk/rockchip/clk-rk3568.c
@@ -0,0 +1,1725 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Rockchip Electronics Co. Ltd.
+ * Author: Elaine Zhang <zhangqing@rock-chips.com>
+ */
+
+#include <linux/clk-provider.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include <linux/of_address.h>
+#include <linux/syscore_ops.h>
+#include <dt-bindings/clock/rk3568-cru.h>
+#include "clk.h"
+
+#define RK3568_GRF_SOC_STATUS0 0x580
+
+enum rk3568_pmu_plls {
+ ppll, hpll,
+};
+
+enum rk3568_plls {
+ apll, dpll, gpll, cpll, npll, vpll,
+};
+
+static struct rockchip_pll_rate_table rk3568_pll_rates[] = {
+ /* _mhz, _refdiv, _fbdiv, _postdiv1, _postdiv2, _dsmpd, _frac */
+ RK3036_PLL_RATE(2208000000, 1, 92, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2184000000, 1, 91, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2160000000, 1, 90, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2088000000, 1, 87, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2064000000, 1, 86, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2040000000, 1, 85, 1, 1, 1, 0),
+ RK3036_PLL_RATE(2016000000, 1, 84, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1992000000, 1, 83, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1920000000, 1, 80, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1896000000, 1, 79, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1800000000, 1, 75, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1704000000, 1, 71, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1608000000, 1, 67, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1600000000, 3, 200, 1, 1, 1, 0),
+ RK3036_PLL_RATE(1584000000, 1, 132, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1560000000, 1, 130, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1536000000, 1, 128, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1512000000, 1, 126, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1488000000, 1, 124, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1464000000, 1, 122, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1440000000, 1, 120, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1416000000, 1, 118, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1400000000, 3, 350, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1392000000, 1, 116, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1368000000, 1, 114, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1344000000, 1, 112, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1320000000, 1, 110, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1296000000, 1, 108, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1272000000, 1, 106, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1248000000, 1, 104, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1200000000, 1, 100, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1188000000, 1, 99, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1104000000, 1, 92, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1100000000, 3, 275, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1008000000, 1, 84, 2, 1, 1, 0),
+ RK3036_PLL_RATE(1000000000, 3, 250, 2, 1, 1, 0),
+ RK3036_PLL_RATE(912000000, 1, 76, 2, 1, 1, 0),
+ RK3036_PLL_RATE(816000000, 1, 68, 2, 1, 1, 0),
+ RK3036_PLL_RATE(800000000, 3, 200, 2, 1, 1, 0),
+ RK3036_PLL_RATE(700000000, 3, 350, 4, 1, 1, 0),
+ RK3036_PLL_RATE(696000000, 1, 116, 4, 1, 1, 0),
+ RK3036_PLL_RATE(600000000, 1, 100, 4, 1, 1, 0),
+ RK3036_PLL_RATE(594000000, 1, 99, 4, 1, 1, 0),
+ RK3036_PLL_RATE(500000000, 1, 125, 6, 1, 1, 0),
+ RK3036_PLL_RATE(408000000, 1, 68, 2, 2, 1, 0),
+ RK3036_PLL_RATE(312000000, 1, 78, 6, 1, 1, 0),
+ RK3036_PLL_RATE(216000000, 1, 72, 4, 2, 1, 0),
+ RK3036_PLL_RATE(200000000, 1, 100, 3, 4, 1, 0),
+ RK3036_PLL_RATE(148500000, 1, 99, 4, 4, 1, 0),
+ RK3036_PLL_RATE(100000000, 1, 150, 6, 6, 1, 0),
+ RK3036_PLL_RATE(96000000, 1, 96, 6, 4, 1, 0),
+ RK3036_PLL_RATE(74250000, 2, 99, 4, 4, 1, 0),
+ { /* sentinel */ },
+};
+
+#define RK3568_DIV_ATCLK_CORE_MASK 0x1f
+#define RK3568_DIV_ATCLK_CORE_SHIFT 0
+#define RK3568_DIV_GICCLK_CORE_MASK 0x1f
+#define RK3568_DIV_GICCLK_CORE_SHIFT 8
+#define RK3568_DIV_PCLK_CORE_MASK 0x1f
+#define RK3568_DIV_PCLK_CORE_SHIFT 0
+#define RK3568_DIV_PERIPHCLK_CORE_MASK 0x1f
+#define RK3568_DIV_PERIPHCLK_CORE_SHIFT 8
+#define RK3568_DIV_ACLK_CORE_MASK 0x1f
+#define RK3568_DIV_ACLK_CORE_SHIFT 8
+
+#define RK3568_DIV_SCLK_CORE_MASK 0xf
+#define RK3568_DIV_SCLK_CORE_SHIFT 0
+#define RK3568_MUX_SCLK_CORE_MASK 0x3
+#define RK3568_MUX_SCLK_CORE_SHIFT 8
+#define RK3568_MUX_SCLK_CORE_NPLL_MASK 0x1
+#define RK3568_MUX_SCLK_CORE_NPLL_SHIFT 15
+#define RK3568_MUX_CLK_CORE_APLL_MASK 0x1
+#define RK3568_MUX_CLK_CORE_APLL_SHIFT 7
+#define RK3568_MUX_CLK_PVTPLL_MASK 0x1
+#define RK3568_MUX_CLK_PVTPLL_SHIFT 15
+
+#define RK3568_CLKSEL1(_sclk_core) \
+{ \
+ .reg = RK3568_CLKSEL_CON(2), \
+ .val = HIWORD_UPDATE(_sclk_core, RK3568_MUX_SCLK_CORE_NPLL_MASK, \
+ RK3568_MUX_SCLK_CORE_NPLL_SHIFT) | \
+ HIWORD_UPDATE(_sclk_core, RK3568_MUX_SCLK_CORE_MASK, \
+ RK3568_MUX_SCLK_CORE_SHIFT) | \
+ HIWORD_UPDATE(1, RK3568_DIV_SCLK_CORE_MASK, \
+ RK3568_DIV_SCLK_CORE_SHIFT), \
+}
+
+#define RK3568_CLKSEL2(_aclk_core) \
+{ \
+ .reg = RK3568_CLKSEL_CON(5), \
+ .val = HIWORD_UPDATE(_aclk_core, RK3568_DIV_ACLK_CORE_MASK, \
+ RK3568_DIV_ACLK_CORE_SHIFT), \
+}
+
+#define RK3568_CLKSEL3(_atclk_core, _gic_core) \
+{ \
+ .reg = RK3568_CLKSEL_CON(3), \
+ .val = HIWORD_UPDATE(_atclk_core, RK3568_DIV_ATCLK_CORE_MASK, \
+ RK3568_DIV_ATCLK_CORE_SHIFT) | \
+ HIWORD_UPDATE(_gic_core, RK3568_DIV_GICCLK_CORE_MASK, \
+ RK3568_DIV_GICCLK_CORE_SHIFT), \
+}
+
+#define RK3568_CLKSEL4(_pclk_core, _periph_core) \
+{ \
+ .reg = RK3568_CLKSEL_CON(4), \
+ .val = HIWORD_UPDATE(_pclk_core, RK3568_DIV_PCLK_CORE_MASK, \
+ RK3568_DIV_PCLK_CORE_SHIFT) | \
+ HIWORD_UPDATE(_periph_core, RK3568_DIV_PERIPHCLK_CORE_MASK, \
+ RK3568_DIV_PERIPHCLK_CORE_SHIFT), \
+}
+
+#define RK3568_CPUCLK_RATE(_prate, _sclk, _acore, _atcore, _gicclk, _pclk, _periph) \
+{ \
+ .prate = _prate##U, \
+ .divs = { \
+ RK3568_CLKSEL1(_sclk), \
+ RK3568_CLKSEL2(_acore), \
+ RK3568_CLKSEL3(_atcore, _gicclk), \
+ RK3568_CLKSEL4(_pclk, _periph), \
+ }, \
+}
+
+static struct rockchip_cpuclk_rate_table rk3568_cpuclk_rates[] __initdata = {
+ RK3568_CPUCLK_RATE(1800000000, 0, 1, 7, 7, 7, 7),
+ RK3568_CPUCLK_RATE(1704000000, 0, 1, 7, 7, 7, 7),
+ RK3568_CPUCLK_RATE(1608000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1584000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1560000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1536000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1512000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1488000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1464000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1440000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1416000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1392000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1368000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1344000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1320000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1296000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1272000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1248000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1224000000, 0, 1, 5, 5, 5, 5),
+ RK3568_CPUCLK_RATE(1200000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(1104000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(1008000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(912000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(816000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(696000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(600000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(408000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(312000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(216000000, 0, 1, 3, 3, 3, 3),
+ RK3568_CPUCLK_RATE(96000000, 0, 1, 3, 3, 3, 3),
+};
+
+static const struct rockchip_cpuclk_reg_data rk3568_cpuclk_data = {
+ .core_reg[0] = RK3568_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .core_reg[1] = RK3568_CLKSEL_CON(0),
+ .div_core_shift[1] = 8,
+ .div_core_mask[1] = 0x1f,
+ .core_reg[2] = RK3568_CLKSEL_CON(1),
+ .div_core_shift[2] = 0,
+ .div_core_mask[2] = 0x1f,
+ .core_reg[3] = RK3568_CLKSEL_CON(1),
+ .div_core_shift[3] = 8,
+ .div_core_mask[3] = 0x1f,
+ .num_cores = 4,
+ .mux_core_alt = 1,
+ .mux_core_main = 0,
+ .mux_core_shift = 6,
+ .mux_core_mask = 0x1,
+};
+
+PNAME(mux_pll_p) = { "xin24m" };
+PNAME(mux_usb480m_p) = { "xin24m", "usb480m_phy", "clk_rtc_32k" };
+PNAME(mux_armclk_p) = { "apll", "gpll" };
+PNAME(clk_i2s0_8ch_tx_p) = { "clk_i2s0_8ch_tx_src", "clk_i2s0_8ch_tx_frac", "i2s0_mclkin", "xin_osc0_half" };
+PNAME(clk_i2s0_8ch_rx_p) = { "clk_i2s0_8ch_rx_src", "clk_i2s0_8ch_rx_frac", "i2s0_mclkin", "xin_osc0_half" };
+PNAME(clk_i2s1_8ch_tx_p) = { "clk_i2s1_8ch_tx_src", "clk_i2s1_8ch_tx_frac", "i2s1_mclkin", "xin_osc0_half" };
+PNAME(clk_i2s1_8ch_rx_p) = { "clk_i2s1_8ch_rx_src", "clk_i2s1_8ch_rx_frac", "i2s1_mclkin", "xin_osc0_half" };
+PNAME(clk_i2s2_2ch_p) = { "clk_i2s2_2ch_src", "clk_i2s2_2ch_frac", "i2s2_mclkin", "xin_osc0_half "};
+PNAME(clk_i2s3_2ch_tx_p) = { "clk_i2s3_2ch_tx_src", "clk_i2s3_2ch_tx_frac", "i2s3_mclkin", "xin_osc0_half" };
+PNAME(clk_i2s3_2ch_rx_p) = { "clk_i2s3_2ch_rx_src", "clk_i2s3_2ch_rx_frac", "i2s3_mclkin", "xin_osc0_half" };
+PNAME(mclk_spdif_8ch_p) = { "mclk_spdif_8ch_src", "mclk_spdif_8ch_frac" };
+PNAME(sclk_audpwm_p) = { "sclk_audpwm_src", "sclk_audpwm_frac" };
+PNAME(sclk_uart1_p) = { "clk_uart1_src", "clk_uart1_frac", "xin24m" };
+PNAME(sclk_uart2_p) = { "clk_uart2_src", "clk_uart2_frac", "xin24m" };
+PNAME(sclk_uart3_p) = { "clk_uart3_src", "clk_uart3_frac", "xin24m" };
+PNAME(sclk_uart4_p) = { "clk_uart4_src", "clk_uart4_frac", "xin24m" };
+PNAME(sclk_uart5_p) = { "clk_uart5_src", "clk_uart5_frac", "xin24m" };
+PNAME(sclk_uart6_p) = { "clk_uart6_src", "clk_uart6_frac", "xin24m" };
+PNAME(sclk_uart7_p) = { "clk_uart7_src", "clk_uart7_frac", "xin24m" };
+PNAME(sclk_uart8_p) = { "clk_uart8_src", "clk_uart8_frac", "xin24m" };
+PNAME(sclk_uart9_p) = { "clk_uart9_src", "clk_uart9_frac", "xin24m" };
+PNAME(sclk_uart0_p) = { "sclk_uart0_div", "sclk_uart0_frac", "xin24m" };
+PNAME(clk_rtc32k_pmu_p) = { "clk_32k_pvtm", "xin32k", "clk_rtc32k_frac" };
+PNAME(mpll_gpll_cpll_npll_p) = { "mpll", "gpll", "cpll", "npll" };
+PNAME(gpll_cpll_npll_p) = { "gpll", "cpll", "npll" };
+PNAME(npll_gpll_p) = { "npll", "gpll" };
+PNAME(cpll_gpll_p) = { "cpll", "gpll" };
+PNAME(gpll_cpll_p) = { "gpll", "cpll" };
+PNAME(gpll_cpll_npll_vpll_p) = { "gpll", "cpll", "npll", "vpll" };
+PNAME(apll_gpll_npll_p) = { "apll", "gpll", "npll" };
+PNAME(sclk_core_pre_p) = { "sclk_core_src", "npll" };
+PNAME(gpll150_gpll100_gpll75_xin24m_p) = { "gpll_150m", "gpll_100m", "gpll_75m", "xin24m" };
+PNAME(clk_gpu_pre_mux_p) = { "clk_gpu_src", "gpu_pvtpll_out" };
+PNAME(clk_npu_pre_ndft_p) = { "clk_npu_src", "dummy"};
+PNAME(clk_npu_p) = { "clk_npu_pre_ndft", "npu_pvtpll_out" };
+PNAME(dpll_gpll_cpll_p) = { "dpll", "gpll", "cpll" };
+PNAME(clk_ddr1x_p) = { "clk_ddrphy1x_src", "dpll" };
+PNAME(gpll200_gpll150_gpll100_xin24m_p) = { "gpll_200m", "gpll_150m", "gpll_100m", "xin24m" };
+PNAME(gpll100_gpll75_gpll50_p) = { "gpll_100m", "gpll_75m", "cpll_50m" };
+PNAME(i2s0_mclkout_tx_p) = { "clk_i2s0_8ch_tx", "xin_osc0_half" };
+PNAME(i2s0_mclkout_rx_p) = { "clk_i2s0_8ch_rx", "xin_osc0_half" };
+PNAME(i2s1_mclkout_tx_p) = { "clk_i2s1_8ch_tx", "xin_osc0_half" };
+PNAME(i2s1_mclkout_rx_p) = { "clk_i2s1_8ch_rx", "xin_osc0_half" };
+PNAME(i2s2_mclkout_p) = { "clk_i2s2_2ch", "xin_osc0_half" };
+PNAME(i2s3_mclkout_tx_p) = { "clk_i2s3_2ch_tx", "xin_osc0_half" };
+PNAME(i2s3_mclkout_rx_p) = { "clk_i2s3_2ch_rx", "xin_osc0_half" };
+PNAME(mclk_pdm_p) = { "gpll_300m", "cpll_250m", "gpll_200m", "gpll_100m" };
+PNAME(clk_i2c_p) = { "gpll_200m", "gpll_100m", "xin24m", "cpll_100m" };
+PNAME(gpll200_gpll150_gpll100_p) = { "gpll_200m", "gpll_150m", "gpll_100m" };
+PNAME(gpll300_gpll200_gpll100_p) = { "gpll_300m", "gpll_200m", "gpll_100m" };
+PNAME(clk_nandc_p) = { "gpll_200m", "gpll_150m", "cpll_100m", "xin24m" };
+PNAME(sclk_sfc_p) = { "xin24m", "cpll_50m", "gpll_75m", "gpll_100m", "cpll_125m", "gpll_150m" };
+PNAME(gpll200_gpll150_cpll125_p) = { "gpll_200m", "gpll_150m", "cpll_125m" };
+PNAME(cclk_emmc_p) = { "xin24m", "gpll_200m", "gpll_150m", "cpll_100m", "cpll_50m", "clk_osc0_div_375k" };
+PNAME(aclk_pipe_p) = { "gpll_400m", "gpll_300m", "gpll_200m", "xin24m" };
+PNAME(gpll200_cpll125_p) = { "gpll_200m", "cpll_125m" };
+PNAME(gpll300_gpll200_gpll100_xin24m_p) = { "gpll_300m", "gpll_200m", "gpll_100m", "xin24m" };
+PNAME(clk_sdmmc_p) = { "xin24m", "gpll_400m", "gpll_300m", "cpll_100m", "cpll_50m", "clk_osc0_div_750k" };
+PNAME(cpll125_cpll50_cpll25_xin24m_p) = { "cpll_125m", "cpll_50m", "cpll_25m", "xin24m" };
+PNAME(clk_gmac_ptp_p) = { "cpll_62p5", "gpll_100m", "cpll_50m", "xin24m" };
+PNAME(cpll333_gpll300_gpll200_p) = { "cpll_333m", "gpll_300m", "gpll_200m" };
+PNAME(cpll_gpll_hpll_p) = { "cpll", "gpll", "hpll" };
+PNAME(gpll_usb480m_xin24m_p) = { "gpll", "usb480m", "xin24m", "xin24m" };
+PNAME(gpll300_cpll250_gpll100_xin24m_p) = { "gpll_300m", "cpll_250m", "gpll_100m", "xin24m" };
+PNAME(cpll_gpll_hpll_vpll_p) = { "cpll", "gpll", "hpll", "vpll" };
+PNAME(hpll_vpll_gpll_cpll_p) = { "hpll", "vpll", "gpll", "cpll" };
+PNAME(gpll400_cpll333_gpll200_p) = { "gpll_400m", "cpll_333m", "gpll_200m" };
+PNAME(gpll100_gpll75_cpll50_xin24m_p) = { "gpll_100m", "gpll_75m", "cpll_50m", "xin24m" };
+PNAME(xin24m_gpll100_cpll100_p) = { "xin24m", "gpll_100m", "cpll_100m" };
+PNAME(gpll_cpll_usb480m_p) = { "gpll", "cpll", "usb480m" };
+PNAME(gpll100_xin24m_cpll100_p) = { "gpll_100m", "xin24m", "cpll_100m" };
+PNAME(gpll200_xin24m_cpll100_p) = { "gpll_200m", "xin24m", "cpll_100m" };
+PNAME(xin24m_32k_p) = { "xin24m", "clk_rtc_32k" };
+PNAME(cpll500_gpll400_gpll300_xin24m_p) = { "cpll_500m", "gpll_400m", "gpll_300m", "xin24m" };
+PNAME(gpll400_gpll300_gpll200_xin24m_p) = { "gpll_400m", "gpll_300m", "gpll_200m", "xin24m" };
+PNAME(xin24m_cpll100_p) = { "xin24m", "cpll_100m" };
+PNAME(ppll_usb480m_cpll_gpll_p) = { "ppll", "usb480m", "cpll", "gpll"};
+PNAME(clk_usbphy0_ref_p) = { "clk_ref24m", "xin_osc0_usbphy0_g" };
+PNAME(clk_usbphy1_ref_p) = { "clk_ref24m", "xin_osc0_usbphy1_g" };
+PNAME(clk_mipidsiphy0_ref_p) = { "clk_ref24m", "xin_osc0_mipidsiphy0_g" };
+PNAME(clk_mipidsiphy1_ref_p) = { "clk_ref24m", "xin_osc0_mipidsiphy1_g" };
+PNAME(clk_wifi_p) = { "clk_wifi_osc0", "clk_wifi_div" };
+PNAME(clk_pciephy0_ref_p) = { "clk_pciephy0_osc0", "clk_pciephy0_div" };
+PNAME(clk_pciephy1_ref_p) = { "clk_pciephy1_osc0", "clk_pciephy1_div" };
+PNAME(clk_pciephy2_ref_p) = { "clk_pciephy2_osc0", "clk_pciephy2_div" };
+PNAME(mux_gmac0_p) = { "clk_mac0_2top", "gmac0_clkin" };
+PNAME(mux_gmac0_rgmii_speed_p) = { "clk_gmac0", "clk_gmac0", "clk_gmac0_tx_div50", "clk_gmac0_tx_div5" };
+PNAME(mux_gmac0_rmii_speed_p) = { "clk_gmac0_rx_div20", "clk_gmac0_rx_div2" };
+PNAME(mux_gmac0_rx_tx_p) = { "clk_gmac0_rgmii_speed", "clk_gmac0_rmii_speed", "clk_gmac0_xpcs_mii" };
+PNAME(mux_gmac1_p) = { "clk_mac1_2top", "gmac1_clkin" };
+PNAME(mux_gmac1_rgmii_speed_p) = { "clk_gmac1", "clk_gmac1", "clk_gmac1_tx_div50", "clk_gmac1_tx_div5" };
+PNAME(mux_gmac1_rmii_speed_p) = { "clk_gmac1_rx_div20", "clk_gmac1_rx_div2" };
+PNAME(mux_gmac1_rx_tx_p) = { "clk_gmac1_rgmii_speed", "clk_gmac1_rmii_speed", "clk_gmac1_xpcs_mii" };
+PNAME(clk_hdmi_ref_p) = { "hpll", "hpll_ph0" };
+PNAME(clk_pdpmu_p) = { "ppll", "gpll" };
+PNAME(clk_mac_2top_p) = { "cpll_125m", "cpll_50m", "cpll_25m", "ppll" };
+PNAME(clk_pwm0_p) = { "xin24m", "clk_pdpmu" };
+PNAME(aclk_rkvdec_pre_p) = { "gpll", "cpll" };
+PNAME(clk_rkvdec_core_p) = { "gpll", "cpll", "dummy_npll", "dummy_vpll" };
+
+static struct rockchip_pll_clock rk3568_pmu_pll_clks[] __initdata = {
+ [ppll] = PLL(pll_rk3328, PLL_PPLL, "ppll", mux_pll_p,
+ 0, RK3568_PMU_PLL_CON(0),
+ RK3568_PMU_MODE_CON0, 0, 4, 0, rk3568_pll_rates),
+ [hpll] = PLL(pll_rk3328, PLL_HPLL, "hpll", mux_pll_p,
+ 0, RK3568_PMU_PLL_CON(16),
+ RK3568_PMU_MODE_CON0, 2, 7, 0, rk3568_pll_rates),
+};
+
+static struct rockchip_pll_clock rk3568_pll_clks[] __initdata = {
+ [apll] = PLL(pll_rk3328, PLL_APLL, "apll", mux_pll_p,
+ 0, RK3568_PLL_CON(0),
+ RK3568_MODE_CON0, 0, 0, 0, rk3568_pll_rates),
+ [dpll] = PLL(pll_rk3328, PLL_DPLL, "dpll", mux_pll_p,
+ 0, RK3568_PLL_CON(8),
+ RK3568_MODE_CON0, 2, 1, 0, NULL),
+ [cpll] = PLL(pll_rk3328, PLL_CPLL, "cpll", mux_pll_p,
+ 0, RK3568_PLL_CON(24),
+ RK3568_MODE_CON0, 4, 2, 0, rk3568_pll_rates),
+ [gpll] = PLL(pll_rk3328, PLL_GPLL, "gpll", mux_pll_p,
+ 0, RK3568_PLL_CON(16),
+ RK3568_MODE_CON0, 6, 3, 0, rk3568_pll_rates),
+ [npll] = PLL(pll_rk3328, PLL_NPLL, "npll", mux_pll_p,
+ 0, RK3568_PLL_CON(32),
+ RK3568_MODE_CON0, 10, 5, 0, rk3568_pll_rates),
+ [vpll] = PLL(pll_rk3328, PLL_VPLL, "vpll", mux_pll_p,
+ 0, RK3568_PLL_CON(40),
+ RK3568_MODE_CON0, 12, 6, 0, rk3568_pll_rates),
+};
+
+#define MFLAGS CLK_MUX_HIWORD_MASK
+#define DFLAGS CLK_DIVIDER_HIWORD_MASK
+#define GFLAGS (CLK_GATE_HIWORD_MASK | CLK_GATE_SET_TO_DISABLE)
+
+static struct rockchip_clk_branch rk3568_i2s0_8ch_tx_fracmux __initdata =
+ MUX(CLK_I2S0_8CH_TX, "clk_i2s0_8ch_tx", clk_i2s0_8ch_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(11), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_i2s0_8ch_rx_fracmux __initdata =
+ MUX(CLK_I2S0_8CH_RX, "clk_i2s0_8ch_rx", clk_i2s0_8ch_rx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(13), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_i2s1_8ch_tx_fracmux __initdata =
+ MUX(CLK_I2S1_8CH_TX, "clk_i2s1_8ch_tx", clk_i2s1_8ch_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(15), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_i2s1_8ch_rx_fracmux __initdata =
+ MUX(CLK_I2S1_8CH_RX, "clk_i2s1_8ch_rx", clk_i2s1_8ch_rx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(17), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_i2s2_2ch_fracmux __initdata =
+ MUX(CLK_I2S2_2CH, "clk_i2s2_2ch", clk_i2s2_2ch_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(19), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_i2s3_2ch_tx_fracmux __initdata =
+ MUX(CLK_I2S3_2CH_TX, "clk_i2s3_2ch_tx", clk_i2s3_2ch_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(21), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_i2s3_2ch_rx_fracmux __initdata =
+ MUX(CLK_I2S3_2CH_RX, "clk_i2s3_2ch_rx", clk_i2s3_2ch_rx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(83), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_spdif_8ch_fracmux __initdata =
+ MUX(MCLK_SPDIF_8CH, "mclk_spdif_8ch", mclk_spdif_8ch_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(23), 15, 1, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_audpwm_fracmux __initdata =
+ MUX(SCLK_AUDPWM, "sclk_audpwm", sclk_audpwm_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(25), 15, 1, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart1_fracmux __initdata =
+ MUX(0, "sclk_uart1_mux", sclk_uart1_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(52), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart2_fracmux __initdata =
+ MUX(0, "sclk_uart2_mux", sclk_uart2_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(54), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart3_fracmux __initdata =
+ MUX(0, "sclk_uart3_mux", sclk_uart3_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(56), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart4_fracmux __initdata =
+ MUX(0, "sclk_uart4_mux", sclk_uart4_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(58), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart5_fracmux __initdata =
+ MUX(0, "sclk_uart5_mux", sclk_uart5_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(60), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart6_fracmux __initdata =
+ MUX(0, "sclk_uart6_mux", sclk_uart6_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(62), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart7_fracmux __initdata =
+ MUX(0, "sclk_uart7_mux", sclk_uart7_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(64), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart8_fracmux __initdata =
+ MUX(0, "sclk_uart8_mux", sclk_uart8_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(66), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart9_fracmux __initdata =
+ MUX(0, "sclk_uart9_mux", sclk_uart9_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(68), 12, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_uart0_fracmux __initdata =
+ MUX(0, "sclk_uart0_mux", sclk_uart0_p, CLK_SET_RATE_PARENT,
+ RK3568_PMU_CLKSEL_CON(4), 10, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_rtc32k_pmu_fracmux __initdata =
+ MUX(CLK_RTC_32K, "clk_rtc_32k", clk_rtc32k_pmu_p, CLK_SET_RATE_PARENT | CLK_SET_RATE_NO_REPARENT,
+ RK3568_PMU_CLKSEL_CON(0), 6, 2, MFLAGS);
+
+static struct rockchip_clk_branch rk3568_clk_branches[] __initdata = {
+ /*
+ * Clock-Architecture Diagram 1
+ */
+ /* SRC_CLK */
+ COMPOSITE_NOMUX(0, "gpll_400m", "gpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(75), 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 0, GFLAGS),
+ COMPOSITE_NOMUX(0, "gpll_300m", "gpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(75), 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 1, GFLAGS),
+ COMPOSITE_NOMUX(0, "gpll_200m", "gpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(76), 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 2, GFLAGS),
+ COMPOSITE_NOMUX(0, "gpll_150m", "gpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(76), 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 3, GFLAGS),
+ COMPOSITE_NOMUX(0, "gpll_100m", "gpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(77), 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 4, GFLAGS),
+ COMPOSITE_NOMUX(0, "gpll_75m", "gpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(77), 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 5, GFLAGS),
+ COMPOSITE_NOMUX(0, "gpll_20m", "gpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(78), 0, 6, DFLAGS,
+ RK3568_CLKGATE_CON(35), 6, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_500M, "cpll_500m", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(78), 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 7, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_333M, "cpll_333m", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(79), 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 8, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_250M, "cpll_250m", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(79), 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 9, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_125M, "cpll_125m", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(80), 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 10, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_62P5M, "cpll_62p5", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(80), 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 11, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_50M, "cpll_50m", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(81), 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 12, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_25M, "cpll_25m", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(81), 8, 6, DFLAGS,
+ RK3568_CLKGATE_CON(35), 13, GFLAGS),
+ COMPOSITE_NOMUX(CPLL_100M, "cpll_100m", "cpll", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(82), 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(35), 14, GFLAGS),
+ COMPOSITE_NOMUX(0, "clk_osc0_div_750k", "xin24m", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(82), 8, 6, DFLAGS,
+ RK3568_CLKGATE_CON(35), 15, GFLAGS),
+ FACTOR(0, "clk_osc0_div_375k", "clk_osc0_div_750k", 0, 1, 2),
+ FACTOR(0, "xin_osc0_half", "xin24m", 0, 1, 2),
+ MUX(USB480M, "usb480m", mux_usb480m_p, CLK_SET_RATE_PARENT,
+ RK3568_MODE_CON0, 14, 2, MFLAGS),
+
+ /* PD_CORE */
+ COMPOSITE(0, "sclk_core_src", apll_gpll_npll_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(2), 8, 2, MFLAGS, 0, 4, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(0), 5, GFLAGS),
+ COMPOSITE_NODIV(0, "sclk_core", sclk_core_pre_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(2), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(0), 7, GFLAGS),
+
+ COMPOSITE_NOMUX(0, "atclk_core", "armclk", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(3), 0, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(0), 8, GFLAGS),
+ COMPOSITE_NOMUX(0, "gicclk_core", "armclk", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(3), 8, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(0), 9, GFLAGS),
+ COMPOSITE_NOMUX(0, "pclk_core_pre", "armclk", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(4), 0, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(0), 10, GFLAGS),
+ COMPOSITE_NOMUX(0, "periphclk_core_pre", "armclk", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(4), 8, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(0), 11, GFLAGS),
+ COMPOSITE_NOMUX(0, "tsclk_core", "periphclk_core_pre", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(5), 0, 4, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(0), 14, GFLAGS),
+ COMPOSITE_NOMUX(0, "cntclk_core", "periphclk_core_pre", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(5), 4, 4, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(0), 15, GFLAGS),
+ COMPOSITE_NOMUX(0, "aclk_core", "sclk_core", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(5), 8, 5, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(1), 0, GFLAGS),
+
+ COMPOSITE_NODIV(ACLK_CORE_NIU2BUS, "aclk_core_niu2bus", gpll150_gpll100_gpll75_xin24m_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(5), 14, 2, MFLAGS,
+ RK3568_CLKGATE_CON(1), 2, GFLAGS),
+
+ GATE(CLK_CORE_PVTM, "clk_core_pvtm", "xin24m", 0,
+ RK3568_CLKGATE_CON(1), 10, GFLAGS),
+ GATE(CLK_CORE_PVTM_CORE, "clk_core_pvtm_core", "armclk", 0,
+ RK3568_CLKGATE_CON(1), 11, GFLAGS),
+ GATE(CLK_CORE_PVTPLL, "clk_core_pvtpll", "armclk", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(1), 12, GFLAGS),
+ GATE(PCLK_CORE_PVTM, "pclk_core_pvtm", "pclk_core_pre", 0,
+ RK3568_CLKGATE_CON(1), 9, GFLAGS),
+
+ /* PD_GPU */
+ COMPOSITE(CLK_GPU_SRC, "clk_gpu_src", mpll_gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(6), 6, 2, MFLAGS | CLK_MUX_READ_ONLY, 0, 4, DFLAGS | CLK_DIVIDER_READ_ONLY,
+ RK3568_CLKGATE_CON(2), 0, GFLAGS),
+ MUX(CLK_GPU_PRE_MUX, "clk_gpu_pre_mux", clk_gpu_pre_mux_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(6), 11, 1, MFLAGS | CLK_MUX_READ_ONLY),
+ DIV(ACLK_GPU_PRE, "aclk_gpu_pre", "clk_gpu_pre_mux", 0,
+ RK3568_CLKSEL_CON(6), 8, 2, DFLAGS),
+ DIV(PCLK_GPU_PRE, "pclk_gpu_pre", "clk_gpu_pre_mux", 0,
+ RK3568_CLKSEL_CON(6), 12, 4, DFLAGS),
+ GATE(CLK_GPU, "clk_gpu", "clk_gpu_pre_mux", 0,
+ RK3568_CLKGATE_CON(2), 3, GFLAGS),
+
+ GATE(PCLK_GPU_PVTM, "pclk_gpu_pvtm", "pclk_gpu_pre", 0,
+ RK3568_CLKGATE_CON(2), 6, GFLAGS),
+ GATE(CLK_GPU_PVTM, "clk_gpu_pvtm", "xin24m", 0,
+ RK3568_CLKGATE_CON(2), 7, GFLAGS),
+ GATE(CLK_GPU_PVTM_CORE, "clk_gpu_pvtm_core", "clk_gpu_src", 0,
+ RK3568_CLKGATE_CON(2), 8, GFLAGS),
+ GATE(CLK_GPU_PVTPLL, "clk_gpu_pvtpll", "clk_gpu_src", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(2), 9, GFLAGS),
+
+ /* PD_NPU */
+ COMPOSITE(CLK_NPU_SRC, "clk_npu_src", npll_gpll_p, 0,
+ RK3568_CLKSEL_CON(7), 6, 1, MFLAGS, 0, 4, DFLAGS,
+ RK3568_CLKGATE_CON(3), 0, GFLAGS),
+ MUX(CLK_NPU_PRE_NDFT, "clk_npu_pre_ndft", clk_npu_pre_ndft_p, CLK_SET_RATE_PARENT | CLK_OPS_PARENT_ENABLE,
+ RK3568_CLKSEL_CON(7), 8, 1, MFLAGS),
+ MUX(CLK_NPU, "clk_npu", clk_npu_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(7), 15, 1, MFLAGS),
+ COMPOSITE_NOMUX(HCLK_NPU_PRE, "hclk_npu_pre", "clk_npu", 0,
+ RK3568_CLKSEL_CON(8), 0, 4, DFLAGS,
+ RK3568_CLKGATE_CON(3), 2, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_NPU_PRE, "pclk_npu_pre", "clk_npu", 0,
+ RK3568_CLKSEL_CON(8), 4, 4, DFLAGS,
+ RK3568_CLKGATE_CON(3), 3, GFLAGS),
+ GATE(ACLK_NPU_PRE, "aclk_npu_pre", "clk_npu", 0,
+ RK3568_CLKGATE_CON(3), 4, GFLAGS),
+ GATE(ACLK_NPU, "aclk_npu", "aclk_npu_pre", 0,
+ RK3568_CLKGATE_CON(3), 7, GFLAGS),
+ GATE(HCLK_NPU, "hclk_npu", "hclk_npu_pre", 0,
+ RK3568_CLKGATE_CON(3), 8, GFLAGS),
+
+ GATE(PCLK_NPU_PVTM, "pclk_npu_pvtm", "pclk_npu_pre", 0,
+ RK3568_CLKGATE_CON(3), 9, GFLAGS),
+ GATE(CLK_NPU_PVTM, "clk_npu_pvtm", "xin24m", 0,
+ RK3568_CLKGATE_CON(3), 10, GFLAGS),
+ GATE(CLK_NPU_PVTM_CORE, "clk_npu_pvtm_core", "clk_npu_pre_ndft", 0,
+ RK3568_CLKGATE_CON(3), 11, GFLAGS),
+ GATE(CLK_NPU_PVTPLL, "clk_npu_pvtpll", "clk_npu_pre_ndft", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(3), 12, GFLAGS),
+
+ /* PD_DDR */
+ COMPOSITE(CLK_DDRPHY1X_SRC, "clk_ddrphy1x_src", dpll_gpll_cpll_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(9), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(4), 0, GFLAGS),
+ MUXGRF(CLK_DDR1X, "clk_ddr1x", clk_ddr1x_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(9), 15, 1, MFLAGS),
+
+ COMPOSITE_NOMUX(CLK_MSCH, "clk_msch", "clk_ddr1x", CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(10), 0, 2, DFLAGS,
+ RK3568_CLKGATE_CON(4), 2, GFLAGS),
+ GATE(CLK24_DDRMON, "clk24_ddrmon", "xin24m", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(4), 15, GFLAGS),
+
+ /* PD_GIC_AUDIO */
+ COMPOSITE_NODIV(ACLK_GIC_AUDIO, "aclk_gic_audio", gpll200_gpll150_gpll100_xin24m_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(10), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(5), 0, GFLAGS),
+ COMPOSITE_NODIV(HCLK_GIC_AUDIO, "hclk_gic_audio", gpll150_gpll100_gpll75_xin24m_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(10), 10, 2, MFLAGS,
+ RK3568_CLKGATE_CON(5), 1, GFLAGS),
+ GATE(HCLK_SDMMC_BUFFER, "hclk_sdmmc_buffer", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(5), 8, GFLAGS),
+ COMPOSITE_NODIV(DCLK_SDMMC_BUFFER, "dclk_sdmmc_buffer", gpll100_gpll75_gpll50_p, 0,
+ RK3568_CLKSEL_CON(10), 12, 2, MFLAGS,
+ RK3568_CLKGATE_CON(5), 9, GFLAGS),
+ GATE(ACLK_GIC600, "aclk_gic600", "aclk_gic_audio", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(5), 4, GFLAGS),
+ GATE(ACLK_SPINLOCK, "aclk_spinlock", "aclk_gic_audio", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(5), 7, GFLAGS),
+ GATE(HCLK_I2S0_8CH, "hclk_i2s0_8ch", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(5), 10, GFLAGS),
+ GATE(HCLK_I2S1_8CH, "hclk_i2s1_8ch", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(5), 11, GFLAGS),
+ GATE(HCLK_I2S2_2CH, "hclk_i2s2_2ch", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(5), 12, GFLAGS),
+ GATE(HCLK_I2S3_2CH, "hclk_i2s3_2ch", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(5), 13, GFLAGS),
+
+ COMPOSITE(CLK_I2S0_8CH_TX_SRC, "clk_i2s0_8ch_tx_src", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(11), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(6), 0, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_I2S0_8CH_TX_FRAC, "clk_i2s0_8ch_tx_frac", "clk_i2s0_8ch_tx_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(12), 0,
+ RK3568_CLKGATE_CON(6), 1, GFLAGS,
+ &rk3568_i2s0_8ch_tx_fracmux),
+ GATE(MCLK_I2S0_8CH_TX, "mclk_i2s0_8ch_tx", "clk_i2s0_8ch_tx", 0,
+ RK3568_CLKGATE_CON(6), 2, GFLAGS),
+ COMPOSITE_NODIV(I2S0_MCLKOUT_TX, "i2s0_mclkout_tx", i2s0_mclkout_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(11), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(6), 3, GFLAGS),
+
+ COMPOSITE(CLK_I2S0_8CH_RX_SRC, "clk_i2s0_8ch_rx_src", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(13), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(6), 4, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_I2S0_8CH_RX_FRAC, "clk_i2s0_8ch_rx_frac", "clk_i2s0_8ch_rx_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(14), 0,
+ RK3568_CLKGATE_CON(6), 5, GFLAGS,
+ &rk3568_i2s0_8ch_rx_fracmux),
+ GATE(MCLK_I2S0_8CH_RX, "mclk_i2s0_8ch_rx", "clk_i2s0_8ch_rx", 0,
+ RK3568_CLKGATE_CON(6), 6, GFLAGS),
+ COMPOSITE_NODIV(I2S0_MCLKOUT_RX, "i2s0_mclkout_rx", i2s0_mclkout_rx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(13), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(6), 7, GFLAGS),
+
+ COMPOSITE(CLK_I2S1_8CH_TX_SRC, "clk_i2s1_8ch_tx_src", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(15), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(6), 8, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_I2S1_8CH_TX_FRAC, "clk_i2s1_8ch_tx_frac", "clk_i2s1_8ch_tx_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(16), 0,
+ RK3568_CLKGATE_CON(6), 9, GFLAGS,
+ &rk3568_i2s1_8ch_tx_fracmux),
+ GATE(MCLK_I2S1_8CH_TX, "mclk_i2s1_8ch_tx", "clk_i2s1_8ch_tx", 0,
+ RK3568_CLKGATE_CON(6), 10, GFLAGS),
+ COMPOSITE_NODIV(I2S1_MCLKOUT_TX, "i2s1_mclkout_tx", i2s1_mclkout_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(15), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(6), 11, GFLAGS),
+
+ COMPOSITE(CLK_I2S1_8CH_RX_SRC, "clk_i2s1_8ch_rx_src", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(17), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(6), 12, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_I2S1_8CH_RX_FRAC, "clk_i2s1_8ch_rx_frac", "clk_i2s1_8ch_rx_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(18), 0,
+ RK3568_CLKGATE_CON(6), 13, GFLAGS,
+ &rk3568_i2s1_8ch_rx_fracmux),
+ GATE(MCLK_I2S1_8CH_RX, "mclk_i2s1_8ch_rx", "clk_i2s1_8ch_rx", 0,
+ RK3568_CLKGATE_CON(6), 14, GFLAGS),
+ COMPOSITE_NODIV(I2S1_MCLKOUT_RX, "i2s1_mclkout_rx", i2s1_mclkout_rx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(17), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(6), 15, GFLAGS),
+
+ COMPOSITE(CLK_I2S2_2CH_SRC, "clk_i2s2_2ch_src", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(19), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(7), 0, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_I2S2_2CH_FRAC, "clk_i2s2_2ch_frac", "clk_i2s2_2ch_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(20), 0,
+ RK3568_CLKGATE_CON(7), 1, GFLAGS,
+ &rk3568_i2s2_2ch_fracmux),
+ GATE(MCLK_I2S2_2CH, "mclk_i2s2_2ch", "clk_i2s2_2ch", 0,
+ RK3568_CLKGATE_CON(7), 2, GFLAGS),
+ COMPOSITE_NODIV(I2S2_MCLKOUT, "i2s2_mclkout", i2s2_mclkout_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(19), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(7), 3, GFLAGS),
+
+ COMPOSITE(CLK_I2S3_2CH_TX_SRC, "clk_i2s3_2ch_tx_src", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(21), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(7), 4, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_I2S3_2CH_TX_FRAC, "clk_i2s3_2ch_tx_frac", "clk_i2s3_2ch_tx_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(22), 0,
+ RK3568_CLKGATE_CON(7), 5, GFLAGS,
+ &rk3568_i2s3_2ch_tx_fracmux),
+ GATE(MCLK_I2S3_2CH_TX, "mclk_i2s3_2ch_tx", "clk_i2s3_2ch_tx", 0,
+ RK3568_CLKGATE_CON(7), 6, GFLAGS),
+ COMPOSITE_NODIV(I2S3_MCLKOUT_TX, "i2s3_mclkout_tx", i2s3_mclkout_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(21), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(7), 7, GFLAGS),
+
+ COMPOSITE(CLK_I2S3_2CH_RX_SRC, "clk_i2s3_2ch_rx_src", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(83), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(7), 8, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_I2S3_2CH_RX_FRAC, "clk_i2s3_2ch_rx_frac", "clk_i2s3_2ch_rx_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(84), 0,
+ RK3568_CLKGATE_CON(7), 9, GFLAGS,
+ &rk3568_i2s3_2ch_rx_fracmux),
+ GATE(MCLK_I2S3_2CH_RX, "mclk_i2s3_2ch_rx", "clk_i2s3_2ch_rx", 0,
+ RK3568_CLKGATE_CON(7), 10, GFLAGS),
+ COMPOSITE_NODIV(I2S3_MCLKOUT_RX, "i2s3_mclkout_rx", i2s3_mclkout_rx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(83), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(7), 11, GFLAGS),
+
+ GATE(HCLK_PDM, "hclk_pdm", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(5), 14, GFLAGS),
+ COMPOSITE_NODIV(MCLK_PDM, "mclk_pdm", mclk_pdm_p, 0,
+ RK3568_CLKSEL_CON(23), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(5), 15, GFLAGS),
+ GATE(HCLK_VAD, "hclk_vad", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(7), 12, GFLAGS),
+ GATE(HCLK_SPDIF_8CH, "hclk_spdif_8ch", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(7), 13, GFLAGS),
+
+ COMPOSITE(MCLK_SPDIF_8CH_SRC, "mclk_spdif_8ch_src", cpll_gpll_p, 0,
+ RK3568_CLKSEL_CON(23), 14, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(7), 14, GFLAGS),
+ COMPOSITE_FRACMUX(MCLK_SPDIF_8CH_FRAC, "mclk_spdif_8ch_frac", "mclk_spdif_8ch_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(24), 0,
+ RK3568_CLKGATE_CON(7), 15, GFLAGS,
+ &rk3568_spdif_8ch_fracmux),
+
+ GATE(HCLK_AUDPWM, "hclk_audpwm", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(8), 0, GFLAGS),
+ COMPOSITE(SCLK_AUDPWM_SRC, "sclk_audpwm_src", gpll_cpll_p, 0,
+ RK3568_CLKSEL_CON(25), 14, 1, MFLAGS, 0, 6, DFLAGS,
+ RK3568_CLKGATE_CON(8), 1, GFLAGS),
+ COMPOSITE_FRACMUX(SCLK_AUDPWM_FRAC, "sclk_audpwm_frac", "sclk_audpwm_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(26), 0,
+ RK3568_CLKGATE_CON(8), 2, GFLAGS,
+ &rk3568_audpwm_fracmux),
+
+ GATE(HCLK_ACDCDIG, "hclk_acdcdig", "hclk_gic_audio", 0,
+ RK3568_CLKGATE_CON(8), 3, GFLAGS),
+ COMPOSITE_NODIV(CLK_ACDCDIG_I2C, "clk_acdcdig_i2c", clk_i2c_p, 0,
+ RK3568_CLKSEL_CON(23), 10, 2, MFLAGS,
+ RK3568_CLKGATE_CON(8), 4, GFLAGS),
+ GATE(CLK_ACDCDIG_DAC, "clk_acdcdig_dac", "mclk_i2s3_2ch_tx", 0,
+ RK3568_CLKGATE_CON(8), 5, GFLAGS),
+ GATE(CLK_ACDCDIG_ADC, "clk_acdcdig_adc", "mclk_i2s3_2ch_rx", 0,
+ RK3568_CLKGATE_CON(8), 6, GFLAGS),
+
+ /* PD_SECURE_FLASH */
+ COMPOSITE_NODIV(ACLK_SECURE_FLASH, "aclk_secure_flash", gpll200_gpll150_gpll100_xin24m_p, 0,
+ RK3568_CLKSEL_CON(27), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(8), 7, GFLAGS),
+ COMPOSITE_NODIV(HCLK_SECURE_FLASH, "hclk_secure_flash", gpll150_gpll100_gpll75_xin24m_p, 0,
+ RK3568_CLKSEL_CON(27), 2, 2, MFLAGS,
+ RK3568_CLKGATE_CON(8), 8, GFLAGS),
+ GATE(ACLK_CRYPTO_NS, "aclk_crypto_ns", "aclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(8), 11, GFLAGS),
+ GATE(HCLK_CRYPTO_NS, "hclk_crypto_ns", "hclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(8), 12, GFLAGS),
+ COMPOSITE_NODIV(CLK_CRYPTO_NS_CORE, "clk_crypto_ns_core", gpll200_gpll150_gpll100_p, 0,
+ RK3568_CLKSEL_CON(27), 4, 2, MFLAGS,
+ RK3568_CLKGATE_CON(8), 13, GFLAGS),
+ COMPOSITE_NODIV(CLK_CRYPTO_NS_PKA, "clk_crypto_ns_pka", gpll300_gpll200_gpll100_p, 0,
+ RK3568_CLKSEL_CON(27), 6, 2, MFLAGS,
+ RK3568_CLKGATE_CON(8), 14, GFLAGS),
+ GATE(CLK_CRYPTO_NS_RNG, "clk_crypto_ns_rng", "hclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(8), 15, GFLAGS),
+ GATE(HCLK_TRNG_NS, "hclk_trng_ns", "hclk_secure_flash", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(9), 10, GFLAGS),
+ GATE(CLK_TRNG_NS, "clk_trng_ns", "hclk_secure_flash", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(9), 11, GFLAGS),
+ GATE(PCLK_OTPC_NS, "pclk_otpc_ns", "hclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(26), 9, GFLAGS),
+ GATE(CLK_OTPC_NS_SBPI, "clk_otpc_ns_sbpi", "xin24m", 0,
+ RK3568_CLKGATE_CON(26), 10, GFLAGS),
+ GATE(CLK_OTPC_NS_USR, "clk_otpc_ns_usr", "xin_osc0_half", 0,
+ RK3568_CLKGATE_CON(26), 11, GFLAGS),
+ GATE(HCLK_NANDC, "hclk_nandc", "hclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(9), 0, GFLAGS),
+ COMPOSITE_NODIV(NCLK_NANDC, "nclk_nandc", clk_nandc_p, 0,
+ RK3568_CLKSEL_CON(28), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(9), 1, GFLAGS),
+ GATE(HCLK_SFC, "hclk_sfc", "hclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(9), 2, GFLAGS),
+ GATE(HCLK_SFC_XIP, "hclk_sfc_xip", "hclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(9), 3, GFLAGS),
+ COMPOSITE_NODIV(SCLK_SFC, "sclk_sfc", sclk_sfc_p, 0,
+ RK3568_CLKSEL_CON(28), 4, 3, MFLAGS,
+ RK3568_CLKGATE_CON(9), 4, GFLAGS),
+ GATE(ACLK_EMMC, "aclk_emmc", "aclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(9), 5, GFLAGS),
+ GATE(HCLK_EMMC, "hclk_emmc", "hclk_secure_flash", 0,
+ RK3568_CLKGATE_CON(9), 6, GFLAGS),
+ COMPOSITE_NODIV(BCLK_EMMC, "bclk_emmc", gpll200_gpll150_cpll125_p, 0,
+ RK3568_CLKSEL_CON(28), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(9), 7, GFLAGS),
+ COMPOSITE_NODIV(CCLK_EMMC, "cclk_emmc", cclk_emmc_p, 0,
+ RK3568_CLKSEL_CON(28), 12, 3, MFLAGS,
+ RK3568_CLKGATE_CON(9), 8, GFLAGS),
+ GATE(TCLK_EMMC, "tclk_emmc", "xin24m", 0,
+ RK3568_CLKGATE_CON(9), 9, GFLAGS),
+ MMC(SCLK_EMMC_DRV, "emmc_drv", "cclk_emmc", RK3568_EMMC_CON0, 1),
+ MMC(SCLK_EMMC_SAMPLE, "emmc_sample", "cclk_emmc", RK3568_EMMC_CON1, 1),
+
+ /* PD_PIPE */
+ COMPOSITE_NODIV(ACLK_PIPE, "aclk_pipe", aclk_pipe_p, 0,
+ RK3568_CLKSEL_CON(29), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(10), 0, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_PIPE, "pclk_pipe", "aclk_pipe", 0,
+ RK3568_CLKSEL_CON(29), 4, 4, DFLAGS,
+ RK3568_CLKGATE_CON(10), 1, GFLAGS),
+ GATE(ACLK_PCIE20_MST, "aclk_pcie20_mst", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 0, GFLAGS),
+ GATE(ACLK_PCIE20_SLV, "aclk_pcie20_slv", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 1, GFLAGS),
+ GATE(ACLK_PCIE20_DBI, "aclk_pcie20_dbi", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 2, GFLAGS),
+ GATE(PCLK_PCIE20, "pclk_pcie20", "pclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 3, GFLAGS),
+ GATE(CLK_PCIE20_AUX_NDFT, "clk_pcie20_aux_ndft", "xin24m", 0,
+ RK3568_CLKGATE_CON(12), 4, GFLAGS),
+ GATE(ACLK_PCIE30X1_MST, "aclk_pcie30x1_mst", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 8, GFLAGS),
+ GATE(ACLK_PCIE30X1_SLV, "aclk_pcie30x1_slv", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 9, GFLAGS),
+ GATE(ACLK_PCIE30X1_DBI, "aclk_pcie30x1_dbi", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 10, GFLAGS),
+ GATE(PCLK_PCIE30X1, "pclk_pcie30x1", "pclk_pipe", 0,
+ RK3568_CLKGATE_CON(12), 11, GFLAGS),
+ GATE(CLK_PCIE30X1_AUX_NDFT, "clk_pcie30x1_aux_ndft", "xin24m", 0,
+ RK3568_CLKGATE_CON(12), 12, GFLAGS),
+ GATE(ACLK_PCIE30X2_MST, "aclk_pcie30x2_mst", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(13), 0, GFLAGS),
+ GATE(ACLK_PCIE30X2_SLV, "aclk_pcie30x2_slv", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(13), 1, GFLAGS),
+ GATE(ACLK_PCIE30X2_DBI, "aclk_pcie30x2_dbi", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(13), 2, GFLAGS),
+ GATE(PCLK_PCIE30X2, "pclk_pcie30x2", "pclk_pipe", 0,
+ RK3568_CLKGATE_CON(13), 3, GFLAGS),
+ GATE(CLK_PCIE30X2_AUX_NDFT, "clk_pcie30x2_aux_ndft", "xin24m", 0,
+ RK3568_CLKGATE_CON(13), 4, GFLAGS),
+ GATE(ACLK_SATA0, "aclk_sata0", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(11), 0, GFLAGS),
+ GATE(CLK_SATA0_PMALIVE, "clk_sata0_pmalive", "gpll_20m", 0,
+ RK3568_CLKGATE_CON(11), 1, GFLAGS),
+ GATE(CLK_SATA0_RXOOB, "clk_sata0_rxoob", "cpll_50m", 0,
+ RK3568_CLKGATE_CON(11), 2, GFLAGS),
+ GATE(ACLK_SATA1, "aclk_sata1", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(11), 4, GFLAGS),
+ GATE(CLK_SATA1_PMALIVE, "clk_sata1_pmalive", "gpll_20m", 0,
+ RK3568_CLKGATE_CON(11), 5, GFLAGS),
+ GATE(CLK_SATA1_RXOOB, "clk_sata1_rxoob", "cpll_50m", 0,
+ RK3568_CLKGATE_CON(11), 6, GFLAGS),
+ GATE(ACLK_SATA2, "aclk_sata2", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(11), 8, GFLAGS),
+ GATE(CLK_SATA2_PMALIVE, "clk_sata2_pmalive", "gpll_20m", 0,
+ RK3568_CLKGATE_CON(11), 9, GFLAGS),
+ GATE(CLK_SATA2_RXOOB, "clk_sata2_rxoob", "cpll_50m", 0,
+ RK3568_CLKGATE_CON(11), 10, GFLAGS),
+ GATE(ACLK_USB3OTG0, "aclk_usb3otg0", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(10), 8, GFLAGS),
+ GATE(CLK_USB3OTG0_REF, "clk_usb3otg0_ref", "xin24m", 0,
+ RK3568_CLKGATE_CON(10), 9, GFLAGS),
+ COMPOSITE_NODIV(CLK_USB3OTG0_SUSPEND, "clk_usb3otg0_suspend", xin24m_32k_p, 0,
+ RK3568_CLKSEL_CON(29), 8, 1, MFLAGS,
+ RK3568_CLKGATE_CON(10), 10, GFLAGS),
+ GATE(ACLK_USB3OTG1, "aclk_usb3otg1", "aclk_pipe", 0,
+ RK3568_CLKGATE_CON(10), 12, GFLAGS),
+ GATE(CLK_USB3OTG1_REF, "clk_usb3otg1_ref", "xin24m", 0,
+ RK3568_CLKGATE_CON(10), 13, GFLAGS),
+ COMPOSITE_NODIV(CLK_USB3OTG1_SUSPEND, "clk_usb3otg1_suspend", xin24m_32k_p, 0,
+ RK3568_CLKSEL_CON(29), 9, 1, MFLAGS,
+ RK3568_CLKGATE_CON(10), 14, GFLAGS),
+ COMPOSITE_NODIV(CLK_XPCS_EEE, "clk_xpcs_eee", gpll200_cpll125_p, 0,
+ RK3568_CLKSEL_CON(29), 13, 1, MFLAGS,
+ RK3568_CLKGATE_CON(10), 4, GFLAGS),
+ GATE(PCLK_XPCS, "pclk_xpcs", "pclk_pipe", 0,
+ RK3568_CLKGATE_CON(13), 6, GFLAGS),
+
+ /* PD_PHP */
+ COMPOSITE_NODIV(ACLK_PHP, "aclk_php", gpll300_gpll200_gpll100_xin24m_p, 0,
+ RK3568_CLKSEL_CON(30), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(14), 8, GFLAGS),
+ COMPOSITE_NODIV(HCLK_PHP, "hclk_php", gpll150_gpll100_gpll75_xin24m_p, 0,
+ RK3568_CLKSEL_CON(30), 2, 2, MFLAGS,
+ RK3568_CLKGATE_CON(14), 9, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_PHP, "pclk_php", "aclk_php", 0,
+ RK3568_CLKSEL_CON(30), 4, 4, DFLAGS,
+ RK3568_CLKGATE_CON(14), 10, GFLAGS),
+ GATE(HCLK_SDMMC0, "hclk_sdmmc0", "hclk_php", 0,
+ RK3568_CLKGATE_CON(15), 0, GFLAGS),
+ COMPOSITE_NODIV(CLK_SDMMC0, "clk_sdmmc0", clk_sdmmc_p, 0,
+ RK3568_CLKSEL_CON(30), 8, 3, MFLAGS,
+ RK3568_CLKGATE_CON(15), 1, GFLAGS),
+ MMC(SCLK_SDMMC0_DRV, "sdmmc0_drv", "clk_sdmmc0", RK3568_SDMMC0_CON0, 1),
+ MMC(SCLK_SDMMC0_SAMPLE, "sdmmc0_sample", "clk_sdmmc0", RK3568_SDMMC0_CON1, 1),
+
+ GATE(HCLK_SDMMC1, "hclk_sdmmc1", "hclk_php", 0,
+ RK3568_CLKGATE_CON(15), 2, GFLAGS),
+ COMPOSITE_NODIV(CLK_SDMMC1, "clk_sdmmc1", clk_sdmmc_p, 0,
+ RK3568_CLKSEL_CON(30), 12, 3, MFLAGS,
+ RK3568_CLKGATE_CON(15), 3, GFLAGS),
+ MMC(SCLK_SDMMC1_DRV, "sdmmc1_drv", "clk_sdmmc1", RK3568_SDMMC1_CON0, 1),
+ MMC(SCLK_SDMMC1_SAMPLE, "sdmmc1_sample", "clk_sdmmc1", RK3568_SDMMC1_CON1, 1),
+
+ GATE(ACLK_GMAC0, "aclk_gmac0", "aclk_php", 0,
+ RK3568_CLKGATE_CON(15), 5, GFLAGS),
+ GATE(PCLK_GMAC0, "pclk_gmac0", "pclk_php", 0,
+ RK3568_CLKGATE_CON(15), 6, GFLAGS),
+ COMPOSITE_NODIV(CLK_MAC0_2TOP, "clk_mac0_2top", clk_mac_2top_p, 0,
+ RK3568_CLKSEL_CON(31), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(15), 7, GFLAGS),
+ COMPOSITE_NODIV(CLK_MAC0_OUT, "clk_mac0_out", cpll125_cpll50_cpll25_xin24m_p, 0,
+ RK3568_CLKSEL_CON(31), 14, 2, MFLAGS,
+ RK3568_CLKGATE_CON(15), 8, GFLAGS),
+ GATE(CLK_MAC0_REFOUT, "clk_mac0_refout", "clk_mac0_2top", 0,
+ RK3568_CLKGATE_CON(15), 12, GFLAGS),
+ COMPOSITE_NODIV(CLK_GMAC0_PTP_REF, "clk_gmac0_ptp_ref", clk_gmac_ptp_p, 0,
+ RK3568_CLKSEL_CON(31), 12, 2, MFLAGS,
+ RK3568_CLKGATE_CON(15), 4, GFLAGS),
+ MUX(SCLK_GMAC0, "clk_gmac0", mux_gmac0_p, CLK_SET_RATE_PARENT | CLK_SET_RATE_NO_REPARENT,
+ RK3568_CLKSEL_CON(31), 2, 1, MFLAGS),
+ FACTOR(0, "clk_gmac0_tx_div5", "clk_gmac0", 0, 1, 5),
+ FACTOR(0, "clk_gmac0_tx_div50", "clk_gmac0", 0, 1, 50),
+ FACTOR(0, "clk_gmac0_rx_div2", "clk_gmac0", 0, 1, 2),
+ FACTOR(0, "clk_gmac0_rx_div20", "clk_gmac0", 0, 1, 20),
+ MUX(SCLK_GMAC0_RGMII_SPEED, "clk_gmac0_rgmii_speed", mux_gmac0_rgmii_speed_p, 0,
+ RK3568_CLKSEL_CON(31), 4, 2, MFLAGS),
+ MUX(SCLK_GMAC0_RMII_SPEED, "clk_gmac0_rmii_speed", mux_gmac0_rmii_speed_p, 0,
+ RK3568_CLKSEL_CON(31), 3, 1, MFLAGS),
+ MUX(SCLK_GMAC0_RX_TX, "clk_gmac0_rx_tx", mux_gmac0_rx_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(31), 0, 2, MFLAGS),
+
+ /* PD_USB */
+ COMPOSITE_NODIV(ACLK_USB, "aclk_usb", gpll300_gpll200_gpll100_xin24m_p, 0,
+ RK3568_CLKSEL_CON(32), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(16), 0, GFLAGS),
+ COMPOSITE_NODIV(HCLK_USB, "hclk_usb", gpll150_gpll100_gpll75_xin24m_p, 0,
+ RK3568_CLKSEL_CON(32), 2, 2, MFLAGS,
+ RK3568_CLKGATE_CON(16), 1, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_USB, "pclk_usb", "aclk_usb", 0,
+ RK3568_CLKSEL_CON(32), 4, 4, DFLAGS,
+ RK3568_CLKGATE_CON(16), 2, GFLAGS),
+ GATE(HCLK_USB2HOST0, "hclk_usb2host0", "hclk_usb", 0,
+ RK3568_CLKGATE_CON(16), 12, GFLAGS),
+ GATE(HCLK_USB2HOST0_ARB, "hclk_usb2host0_arb", "hclk_usb", 0,
+ RK3568_CLKGATE_CON(16), 13, GFLAGS),
+ GATE(HCLK_USB2HOST1, "hclk_usb2host1", "hclk_usb", 0,
+ RK3568_CLKGATE_CON(16), 14, GFLAGS),
+ GATE(HCLK_USB2HOST1_ARB, "hclk_usb2host1_arb", "hclk_usb", 0,
+ RK3568_CLKGATE_CON(16), 15, GFLAGS),
+ GATE(HCLK_SDMMC2, "hclk_sdmmc2", "hclk_usb", 0,
+ RK3568_CLKGATE_CON(17), 0, GFLAGS),
+ COMPOSITE_NODIV(CLK_SDMMC2, "clk_sdmmc2", clk_sdmmc_p, 0,
+ RK3568_CLKSEL_CON(32), 8, 3, MFLAGS,
+ RK3568_CLKGATE_CON(17), 1, GFLAGS),
+ MMC(SCLK_SDMMC2_DRV, "sdmmc2_drv", "clk_sdmmc2", RK3568_SDMMC2_CON0, 1),
+ MMC(SCLK_SDMMC2_SAMPLE, "sdmmc2_sample", "clk_sdmmc2", RK3568_SDMMC2_CON1, 1),
+
+ GATE(ACLK_GMAC1, "aclk_gmac1", "aclk_usb", 0,
+ RK3568_CLKGATE_CON(17), 3, GFLAGS),
+ GATE(PCLK_GMAC1, "pclk_gmac1", "pclk_usb", 0,
+ RK3568_CLKGATE_CON(17), 4, GFLAGS),
+ COMPOSITE_NODIV(CLK_MAC1_2TOP, "clk_mac1_2top", clk_mac_2top_p, 0,
+ RK3568_CLKSEL_CON(33), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(17), 5, GFLAGS),
+ COMPOSITE_NODIV(CLK_MAC1_OUT, "clk_mac1_out", cpll125_cpll50_cpll25_xin24m_p, 0,
+ RK3568_CLKSEL_CON(33), 14, 2, MFLAGS,
+ RK3568_CLKGATE_CON(17), 6, GFLAGS),
+ GATE(CLK_MAC1_REFOUT, "clk_mac1_refout", "clk_mac1_2top", 0,
+ RK3568_CLKGATE_CON(17), 10, GFLAGS),
+ COMPOSITE_NODIV(CLK_GMAC1_PTP_REF, "clk_gmac1_ptp_ref", clk_gmac_ptp_p, 0,
+ RK3568_CLKSEL_CON(33), 12, 2, MFLAGS,
+ RK3568_CLKGATE_CON(17), 2, GFLAGS),
+ MUX(SCLK_GMAC1, "clk_gmac1", mux_gmac1_p, CLK_SET_RATE_PARENT | CLK_SET_RATE_NO_REPARENT,
+ RK3568_CLKSEL_CON(33), 2, 1, MFLAGS),
+ FACTOR(0, "clk_gmac1_tx_div5", "clk_gmac1", 0, 1, 5),
+ FACTOR(0, "clk_gmac1_tx_div50", "clk_gmac1", 0, 1, 50),
+ FACTOR(0, "clk_gmac1_rx_div2", "clk_gmac1", 0, 1, 2),
+ FACTOR(0, "clk_gmac1_rx_div20", "clk_gmac1", 0, 1, 20),
+ MUX(SCLK_GMAC1_RGMII_SPEED, "clk_gmac1_rgmii_speed", mux_gmac1_rgmii_speed_p, 0,
+ RK3568_CLKSEL_CON(33), 4, 2, MFLAGS),
+ MUX(SCLK_GMAC1_RMII_SPEED, "clk_gmac1_rmii_speed", mux_gmac1_rmii_speed_p, 0,
+ RK3568_CLKSEL_CON(33), 3, 1, MFLAGS),
+ MUX(SCLK_GMAC1_RX_TX, "clk_gmac1_rx_tx", mux_gmac1_rx_tx_p, CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(33), 0, 2, MFLAGS),
+
+ /* PD_PERI */
+ COMPOSITE_NODIV(ACLK_PERIMID, "aclk_perimid", gpll300_gpll200_gpll100_xin24m_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(10), 4, 2, MFLAGS,
+ RK3568_CLKGATE_CON(14), 0, GFLAGS),
+ COMPOSITE_NODIV(HCLK_PERIMID, "hclk_perimid", gpll150_gpll100_gpll75_xin24m_p, CLK_IGNORE_UNUSED,
+ RK3568_CLKSEL_CON(10), 6, 2, MFLAGS,
+ RK3568_CLKGATE_CON(14), 1, GFLAGS),
+
+ /* PD_VI */
+ COMPOSITE_NODIV(ACLK_VI, "aclk_vi", gpll400_gpll300_gpll200_xin24m_p, 0,
+ RK3568_CLKSEL_CON(34), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(18), 0, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_VI, "hclk_vi", "aclk_vi", 0,
+ RK3568_CLKSEL_CON(34), 4, 4, DFLAGS,
+ RK3568_CLKGATE_CON(18), 1, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_VI, "pclk_vi", "aclk_vi", 0,
+ RK3568_CLKSEL_CON(34), 8, 4, DFLAGS,
+ RK3568_CLKGATE_CON(18), 2, GFLAGS),
+ GATE(ACLK_VICAP, "aclk_vicap", "aclk_vi", 0,
+ RK3568_CLKGATE_CON(18), 9, GFLAGS),
+ GATE(HCLK_VICAP, "hclk_vicap", "hclk_vi", 0,
+ RK3568_CLKGATE_CON(18), 10, GFLAGS),
+ COMPOSITE_NODIV(DCLK_VICAP, "dclk_vicap", cpll333_gpll300_gpll200_p, 0,
+ RK3568_CLKSEL_CON(34), 14, 2, MFLAGS,
+ RK3568_CLKGATE_CON(18), 11, GFLAGS),
+ GATE(ICLK_VICAP_G, "iclk_vicap_g", "iclk_vicap", 0,
+ RK3568_CLKGATE_CON(18), 13, GFLAGS),
+ GATE(ACLK_ISP, "aclk_isp", "aclk_vi", 0,
+ RK3568_CLKGATE_CON(19), 0, GFLAGS),
+ GATE(HCLK_ISP, "hclk_isp", "hclk_vi", 0,
+ RK3568_CLKGATE_CON(19), 1, GFLAGS),
+ COMPOSITE(CLK_ISP, "clk_isp", cpll_gpll_hpll_p, 0,
+ RK3568_CLKSEL_CON(35), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(19), 2, GFLAGS),
+ GATE(PCLK_CSI2HOST1, "pclk_csi2host1", "pclk_vi", 0,
+ RK3568_CLKGATE_CON(19), 4, GFLAGS),
+ COMPOSITE(CLK_CIF_OUT, "clk_cif_out", gpll_usb480m_xin24m_p, 0,
+ RK3568_CLKSEL_CON(35), 14, 2, MFLAGS, 8, 6, DFLAGS,
+ RK3568_CLKGATE_CON(19), 8, GFLAGS),
+ COMPOSITE(CLK_CAM0_OUT, "clk_cam0_out", gpll_usb480m_xin24m_p, 0,
+ RK3568_CLKSEL_CON(36), 6, 2, MFLAGS, 0, 6, DFLAGS,
+ RK3568_CLKGATE_CON(19), 9, GFLAGS),
+ COMPOSITE(CLK_CAM1_OUT, "clk_cam1_out", gpll_usb480m_xin24m_p, 0,
+ RK3568_CLKSEL_CON(36), 14, 2, MFLAGS, 8, 6, DFLAGS,
+ RK3568_CLKGATE_CON(19), 10, GFLAGS),
+
+ /* PD_VO */
+ COMPOSITE_NODIV(ACLK_VO, "aclk_vo", gpll300_cpll250_gpll100_xin24m_p, 0,
+ RK3568_CLKSEL_CON(37), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(20), 0, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_VO, "hclk_vo", "aclk_vo", 0,
+ RK3568_CLKSEL_CON(37), 8, 4, DFLAGS,
+ RK3568_CLKGATE_CON(20), 1, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_VO, "pclk_vo", "aclk_vo", 0,
+ RK3568_CLKSEL_CON(37), 12, 4, DFLAGS,
+ RK3568_CLKGATE_CON(20), 2, GFLAGS),
+ COMPOSITE(ACLK_VOP_PRE, "aclk_vop_pre", cpll_gpll_hpll_vpll_p, 0,
+ RK3568_CLKSEL_CON(38), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(20), 6, GFLAGS),
+ GATE(ACLK_VOP, "aclk_vop", "aclk_vop_pre", 0,
+ RK3568_CLKGATE_CON(20), 8, GFLAGS),
+ GATE(HCLK_VOP, "hclk_vop", "hclk_vo", 0,
+ RK3568_CLKGATE_CON(20), 9, GFLAGS),
+ COMPOSITE(DCLK_VOP0, "dclk_vop0", hpll_vpll_gpll_cpll_p, CLK_SET_RATE_PARENT | CLK_SET_RATE_NO_REPARENT,
+ RK3568_CLKSEL_CON(39), 10, 2, MFLAGS, 0, 8, DFLAGS,
+ RK3568_CLKGATE_CON(20), 10, GFLAGS),
+ COMPOSITE(DCLK_VOP1, "dclk_vop1", hpll_vpll_gpll_cpll_p, CLK_SET_RATE_PARENT | CLK_SET_RATE_NO_REPARENT,
+ RK3568_CLKSEL_CON(40), 10, 2, MFLAGS, 0, 8, DFLAGS,
+ RK3568_CLKGATE_CON(20), 11, GFLAGS),
+ COMPOSITE(DCLK_VOP2, "dclk_vop2", hpll_vpll_gpll_cpll_p, 0,
+ RK3568_CLKSEL_CON(41), 10, 2, MFLAGS, 0, 8, DFLAGS,
+ RK3568_CLKGATE_CON(20), 12, GFLAGS),
+ GATE(CLK_VOP_PWM, "clk_vop_pwm", "xin24m", 0,
+ RK3568_CLKGATE_CON(20), 13, GFLAGS),
+ GATE(ACLK_HDCP, "aclk_hdcp", "aclk_vo", 0,
+ RK3568_CLKGATE_CON(21), 0, GFLAGS),
+ GATE(HCLK_HDCP, "hclk_hdcp", "hclk_vo", 0,
+ RK3568_CLKGATE_CON(21), 1, GFLAGS),
+ GATE(PCLK_HDCP, "pclk_hdcp", "pclk_vo", 0,
+ RK3568_CLKGATE_CON(21), 2, GFLAGS),
+ GATE(PCLK_HDMI_HOST, "pclk_hdmi_host", "pclk_vo", 0,
+ RK3568_CLKGATE_CON(21), 3, GFLAGS),
+ GATE(CLK_HDMI_SFR, "clk_hdmi_sfr", "xin24m", 0,
+ RK3568_CLKGATE_CON(21), 4, GFLAGS),
+ GATE(CLK_HDMI_CEC, "clk_hdmi_cec", "clk_rtc_32k", 0,
+ RK3568_CLKGATE_CON(21), 5, GFLAGS),
+ GATE(PCLK_DSITX_0, "pclk_dsitx_0", "pclk_vo", 0,
+ RK3568_CLKGATE_CON(21), 6, GFLAGS),
+ GATE(PCLK_DSITX_1, "pclk_dsitx_1", "pclk_vo", 0,
+ RK3568_CLKGATE_CON(21), 7, GFLAGS),
+ GATE(PCLK_EDP_CTRL, "pclk_edp_ctrl", "pclk_vo", 0,
+ RK3568_CLKGATE_CON(21), 8, GFLAGS),
+ COMPOSITE_NODIV(CLK_EDP_200M, "clk_edp_200m", gpll200_gpll150_cpll125_p, 0,
+ RK3568_CLKSEL_CON(38), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(21), 9, GFLAGS),
+
+ /* PD_VPU */
+ COMPOSITE(ACLK_VPU_PRE, "aclk_vpu_pre", gpll_cpll_p, 0,
+ RK3568_CLKSEL_CON(42), 7, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(22), 0, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_VPU_PRE, "hclk_vpu_pre", "aclk_vpu_pre", 0,
+ RK3568_CLKSEL_CON(42), 8, 4, DFLAGS,
+ RK3568_CLKGATE_CON(22), 1, GFLAGS),
+ GATE(ACLK_VPU, "aclk_vpu", "aclk_vpu_pre", 0,
+ RK3568_CLKGATE_CON(22), 4, GFLAGS),
+ GATE(HCLK_VPU, "hclk_vpu", "hclk_vpu_pre", 0,
+ RK3568_CLKGATE_CON(22), 5, GFLAGS),
+
+ /* PD_RGA */
+ COMPOSITE_NODIV(ACLK_RGA_PRE, "aclk_rga_pre", gpll300_cpll250_gpll100_xin24m_p, 0,
+ RK3568_CLKSEL_CON(43), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(23), 0, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_RGA_PRE, "hclk_rga_pre", "aclk_rga_pre", 0,
+ RK3568_CLKSEL_CON(43), 8, 4, DFLAGS,
+ RK3568_CLKGATE_CON(23), 1, GFLAGS),
+ COMPOSITE_NOMUX(PCLK_RGA_PRE, "pclk_rga_pre", "aclk_rga_pre", 0,
+ RK3568_CLKSEL_CON(43), 12, 4, DFLAGS,
+ RK3568_CLKGATE_CON(22), 12, GFLAGS),
+ GATE(ACLK_RGA, "aclk_rga", "aclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 4, GFLAGS),
+ GATE(HCLK_RGA, "hclk_rga", "hclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 5, GFLAGS),
+ COMPOSITE_NODIV(CLK_RGA_CORE, "clk_rga_core", gpll300_gpll200_gpll100_p, 0,
+ RK3568_CLKSEL_CON(43), 2, 2, MFLAGS,
+ RK3568_CLKGATE_CON(23), 6, GFLAGS),
+ GATE(ACLK_IEP, "aclk_iep", "aclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 7, GFLAGS),
+ GATE(HCLK_IEP, "hclk_iep", "hclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 8, GFLAGS),
+ COMPOSITE_NODIV(CLK_IEP_CORE, "clk_iep_core", gpll300_gpll200_gpll100_p, 0,
+ RK3568_CLKSEL_CON(43), 4, 2, MFLAGS,
+ RK3568_CLKGATE_CON(23), 9, GFLAGS),
+ GATE(HCLK_EBC, "hclk_ebc", "hclk_rga_pre", 0, RK3568_CLKGATE_CON(23), 10, GFLAGS),
+ COMPOSITE_NODIV(DCLK_EBC, "dclk_ebc", gpll400_cpll333_gpll200_p, 0,
+ RK3568_CLKSEL_CON(43), 6, 2, MFLAGS,
+ RK3568_CLKGATE_CON(23), 11, GFLAGS),
+ GATE(ACLK_JDEC, "aclk_jdec", "aclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 12, GFLAGS),
+ GATE(HCLK_JDEC, "hclk_jdec", "hclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 13, GFLAGS),
+ GATE(ACLK_JENC, "aclk_jenc", "aclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 14, GFLAGS),
+ GATE(HCLK_JENC, "hclk_jenc", "hclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(23), 15, GFLAGS),
+ GATE(PCLK_EINK, "pclk_eink", "pclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(22), 14, GFLAGS),
+ GATE(HCLK_EINK, "hclk_eink", "hclk_rga_pre", 0,
+ RK3568_CLKGATE_CON(22), 15, GFLAGS),
+
+ /* PD_RKVENC */
+ COMPOSITE(ACLK_RKVENC_PRE, "aclk_rkvenc_pre", gpll_cpll_npll_p, 0,
+ RK3568_CLKSEL_CON(44), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(24), 0, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_RKVENC_PRE, "hclk_rkvenc_pre", "aclk_rkvenc_pre", 0,
+ RK3568_CLKSEL_CON(44), 8, 4, DFLAGS,
+ RK3568_CLKGATE_CON(24), 1, GFLAGS),
+ GATE(ACLK_RKVENC, "aclk_rkvenc", "aclk_rkvenc_pre", 0,
+ RK3568_CLKGATE_CON(24), 6, GFLAGS),
+ GATE(HCLK_RKVENC, "hclk_rkvenc", "hclk_rkvenc_pre", 0,
+ RK3568_CLKGATE_CON(24), 7, GFLAGS),
+ COMPOSITE(CLK_RKVENC_CORE, "clk_rkvenc_core", gpll_cpll_npll_vpll_p, 0,
+ RK3568_CLKSEL_CON(45), 14, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(24), 8, GFLAGS),
+ COMPOSITE(ACLK_RKVDEC_PRE, "aclk_rkvdec_pre", aclk_rkvdec_pre_p, CLK_SET_RATE_NO_REPARENT,
+ RK3568_CLKSEL_CON(47), 7, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(25), 0, GFLAGS),
+ COMPOSITE_NOMUX(HCLK_RKVDEC_PRE, "hclk_rkvdec_pre", "aclk_rkvdec_pre", 0,
+ RK3568_CLKSEL_CON(47), 8, 4, DFLAGS,
+ RK3568_CLKGATE_CON(25), 1, GFLAGS),
+ GATE(ACLK_RKVDEC, "aclk_rkvdec", "aclk_rkvdec_pre", 0,
+ RK3568_CLKGATE_CON(25), 4, GFLAGS),
+ GATE(HCLK_RKVDEC, "hclk_rkvdec", "hclk_rkvdec_pre", 0,
+ RK3568_CLKGATE_CON(25), 5, GFLAGS),
+ COMPOSITE(CLK_RKVDEC_CA, "clk_rkvdec_ca", gpll_cpll_npll_vpll_p, 0,
+ RK3568_CLKSEL_CON(48), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(25), 6, GFLAGS),
+ COMPOSITE(CLK_RKVDEC_CORE, "clk_rkvdec_core", clk_rkvdec_core_p, CLK_SET_RATE_NO_REPARENT,
+ RK3568_CLKSEL_CON(49), 14, 2, MFLAGS, 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(25), 7, GFLAGS),
+ COMPOSITE(CLK_RKVDEC_HEVC_CA, "clk_rkvdec_hevc_ca", gpll_cpll_npll_vpll_p, 0,
+ RK3568_CLKSEL_CON(49), 6, 2, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(25), 8, GFLAGS),
+
+ /* PD_BUS */
+ COMPOSITE_NODIV(ACLK_BUS, "aclk_bus", gpll200_gpll150_gpll100_xin24m_p, 0,
+ RK3568_CLKSEL_CON(50), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(26), 0, GFLAGS),
+ COMPOSITE_NODIV(PCLK_BUS, "pclk_bus", gpll100_gpll75_cpll50_xin24m_p, 0,
+ RK3568_CLKSEL_CON(50), 4, 2, MFLAGS,
+ RK3568_CLKGATE_CON(26), 1, GFLAGS),
+ GATE(PCLK_TSADC, "pclk_tsadc", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(26), 4, GFLAGS),
+ COMPOSITE(CLK_TSADC_TSEN, "clk_tsadc_tsen", xin24m_gpll100_cpll100_p, 0,
+ RK3568_CLKSEL_CON(51), 4, 2, MFLAGS, 0, 3, DFLAGS,
+ RK3568_CLKGATE_CON(26), 5, GFLAGS),
+ COMPOSITE_NOMUX(CLK_TSADC, "clk_tsadc", "clk_tsadc_tsen", 0,
+ RK3568_CLKSEL_CON(51), 8, 7, DFLAGS,
+ RK3568_CLKGATE_CON(26), 6, GFLAGS),
+ GATE(PCLK_SARADC, "pclk_saradc", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(26), 7, GFLAGS),
+ GATE(CLK_SARADC, "clk_saradc", "xin24m", 0,
+ RK3568_CLKGATE_CON(26), 8, GFLAGS),
+ GATE(PCLK_SCR, "pclk_scr", "pclk_bus", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(26), 12, GFLAGS),
+ GATE(PCLK_WDT_NS, "pclk_wdt_ns", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(26), 13, GFLAGS),
+ GATE(TCLK_WDT_NS, "tclk_wdt_ns", "xin24m", 0,
+ RK3568_CLKGATE_CON(26), 14, GFLAGS),
+ GATE(ACLK_MCU, "aclk_mcu", "aclk_bus", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(32), 13, GFLAGS),
+ GATE(PCLK_INTMUX, "pclk_intmux", "pclk_bus", CLK_IGNORE_UNUSED,
+ RK3568_CLKGATE_CON(32), 14, GFLAGS),
+ GATE(PCLK_MAILBOX, "pclk_mailbox", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(32), 15, GFLAGS),
+
+ GATE(PCLK_UART1, "pclk_uart1", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(27), 12, GFLAGS),
+ COMPOSITE(CLK_UART1_SRC, "clk_uart1_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(52), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(27), 13, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART1_FRAC, "clk_uart1_frac", "clk_uart1_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(53), 0,
+ RK3568_CLKGATE_CON(27), 14, GFLAGS,
+ &rk3568_uart1_fracmux),
+ GATE(SCLK_UART1, "sclk_uart1", "sclk_uart1_mux", 0,
+ RK3568_CLKGATE_CON(27), 15, GFLAGS),
+
+ GATE(PCLK_UART2, "pclk_uart2", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(28), 0, GFLAGS),
+ COMPOSITE(CLK_UART2_SRC, "clk_uart2_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(54), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(28), 1, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART2_FRAC, "clk_uart2_frac", "clk_uart2_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(55), 0,
+ RK3568_CLKGATE_CON(28), 2, GFLAGS,
+ &rk3568_uart2_fracmux),
+ GATE(SCLK_UART2, "sclk_uart2", "sclk_uart2_mux", 0,
+ RK3568_CLKGATE_CON(28), 3, GFLAGS),
+
+ GATE(PCLK_UART3, "pclk_uart3", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(28), 4, GFLAGS),
+ COMPOSITE(CLK_UART3_SRC, "clk_uart3_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(56), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(28), 5, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART3_FRAC, "clk_uart3_frac", "clk_uart3_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(57), 0,
+ RK3568_CLKGATE_CON(28), 6, GFLAGS,
+ &rk3568_uart3_fracmux),
+ GATE(SCLK_UART3, "sclk_uart3", "sclk_uart3_mux", 0,
+ RK3568_CLKGATE_CON(28), 7, GFLAGS),
+
+ GATE(PCLK_UART4, "pclk_uart4", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(28), 8, GFLAGS),
+ COMPOSITE(CLK_UART4_SRC, "clk_uart4_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(58), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(28), 9, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART4_FRAC, "clk_uart4_frac", "clk_uart4_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(59), 0,
+ RK3568_CLKGATE_CON(28), 10, GFLAGS,
+ &rk3568_uart4_fracmux),
+ GATE(SCLK_UART4, "sclk_uart4", "sclk_uart4_mux", 0,
+ RK3568_CLKGATE_CON(28), 11, GFLAGS),
+
+ GATE(PCLK_UART5, "pclk_uart5", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(28), 12, GFLAGS),
+ COMPOSITE(CLK_UART5_SRC, "clk_uart5_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(60), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(28), 13, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART5_FRAC, "clk_uart5_frac", "clk_uart5_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(61), 0,
+ RK3568_CLKGATE_CON(28), 14, GFLAGS,
+ &rk3568_uart5_fracmux),
+ GATE(SCLK_UART5, "sclk_uart5", "sclk_uart5_mux", 0,
+ RK3568_CLKGATE_CON(28), 15, GFLAGS),
+
+ GATE(PCLK_UART6, "pclk_uart6", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(29), 0, GFLAGS),
+ COMPOSITE(CLK_UART6_SRC, "clk_uart6_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(62), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(29), 1, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART6_FRAC, "clk_uart6_frac", "clk_uart6_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(63), 0,
+ RK3568_CLKGATE_CON(29), 2, GFLAGS,
+ &rk3568_uart6_fracmux),
+ GATE(SCLK_UART6, "sclk_uart6", "sclk_uart6_mux", 0,
+ RK3568_CLKGATE_CON(29), 3, GFLAGS),
+
+ GATE(PCLK_UART7, "pclk_uart7", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(29), 4, GFLAGS),
+ COMPOSITE(CLK_UART7_SRC, "clk_uart7_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(64), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(29), 5, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART7_FRAC, "clk_uart7_frac", "clk_uart7_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(65), 0,
+ RK3568_CLKGATE_CON(29), 6, GFLAGS,
+ &rk3568_uart7_fracmux),
+ GATE(SCLK_UART7, "sclk_uart7", "sclk_uart7_mux", 0,
+ RK3568_CLKGATE_CON(29), 7, GFLAGS),
+
+ GATE(PCLK_UART8, "pclk_uart8", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(29), 8, GFLAGS),
+ COMPOSITE(CLK_UART8_SRC, "clk_uart8_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(66), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(29), 9, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART8_FRAC, "clk_uart8_frac", "clk_uart8_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(67), 0,
+ RK3568_CLKGATE_CON(29), 10, GFLAGS,
+ &rk3568_uart8_fracmux),
+ GATE(SCLK_UART8, "sclk_uart8", "sclk_uart8_mux", 0,
+ RK3568_CLKGATE_CON(29), 11, GFLAGS),
+
+ GATE(PCLK_UART9, "pclk_uart9", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(29), 12, GFLAGS),
+ COMPOSITE(CLK_UART9_SRC, "clk_uart9_src", gpll_cpll_usb480m_p, 0,
+ RK3568_CLKSEL_CON(68), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_CLKGATE_CON(29), 13, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART9_FRAC, "clk_uart9_frac", "clk_uart9_src", CLK_SET_RATE_PARENT,
+ RK3568_CLKSEL_CON(69), 0,
+ RK3568_CLKGATE_CON(29), 14, GFLAGS,
+ &rk3568_uart9_fracmux),
+ GATE(SCLK_UART9, "sclk_uart9", "sclk_uart9_mux", 0,
+ RK3568_CLKGATE_CON(29), 15, GFLAGS),
+
+ GATE(PCLK_CAN0, "pclk_can0", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(27), 5, GFLAGS),
+ COMPOSITE(CLK_CAN0, "clk_can0", gpll_cpll_p, 0,
+ RK3568_CLKSEL_CON(70), 7, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(27), 6, GFLAGS),
+ GATE(PCLK_CAN1, "pclk_can1", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(27), 7, GFLAGS),
+ COMPOSITE(CLK_CAN1, "clk_can1", gpll_cpll_p, 0,
+ RK3568_CLKSEL_CON(70), 15, 1, MFLAGS, 8, 5, DFLAGS,
+ RK3568_CLKGATE_CON(27), 8, GFLAGS),
+ GATE(PCLK_CAN2, "pclk_can2", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(27), 9, GFLAGS),
+ COMPOSITE(CLK_CAN2, "clk_can2", gpll_cpll_p, 0,
+ RK3568_CLKSEL_CON(71), 7, 1, MFLAGS, 0, 5, DFLAGS,
+ RK3568_CLKGATE_CON(27), 10, GFLAGS),
+ COMPOSITE_NODIV(CLK_I2C, "clk_i2c", clk_i2c_p, 0,
+ RK3568_CLKSEL_CON(71), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(32), 10, GFLAGS),
+ GATE(PCLK_I2C1, "pclk_i2c1", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 0, GFLAGS),
+ GATE(CLK_I2C1, "clk_i2c1", "clk_i2c", 0,
+ RK3568_CLKGATE_CON(30), 1, GFLAGS),
+ GATE(PCLK_I2C2, "pclk_i2c2", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 2, GFLAGS),
+ GATE(CLK_I2C2, "clk_i2c2", "clk_i2c", 0,
+ RK3568_CLKGATE_CON(30), 3, GFLAGS),
+ GATE(PCLK_I2C3, "pclk_i2c3", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 4, GFLAGS),
+ GATE(CLK_I2C3, "clk_i2c3", "clk_i2c", 0,
+ RK3568_CLKGATE_CON(30), 5, GFLAGS),
+ GATE(PCLK_I2C4, "pclk_i2c4", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 6, GFLAGS),
+ GATE(CLK_I2C4, "clk_i2c4", "clk_i2c", 0,
+ RK3568_CLKGATE_CON(30), 7, GFLAGS),
+ GATE(PCLK_I2C5, "pclk_i2c5", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 8, GFLAGS),
+ GATE(CLK_I2C5, "clk_i2c5", "clk_i2c", 0,
+ RK3568_CLKGATE_CON(30), 9, GFLAGS),
+ GATE(PCLK_SPI0, "pclk_spi0", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 10, GFLAGS),
+ COMPOSITE_NODIV(CLK_SPI0, "clk_spi0", gpll200_xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(72), 0, 1, MFLAGS,
+ RK3568_CLKGATE_CON(30), 11, GFLAGS),
+ GATE(PCLK_SPI1, "pclk_spi1", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 12, GFLAGS),
+ COMPOSITE_NODIV(CLK_SPI1, "clk_spi1", gpll200_xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(72), 2, 1, MFLAGS,
+ RK3568_CLKGATE_CON(30), 13, GFLAGS),
+ GATE(PCLK_SPI2, "pclk_spi2", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(30), 14, GFLAGS),
+ COMPOSITE_NODIV(CLK_SPI2, "clk_spi2", gpll200_xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(72), 4, 1, MFLAGS,
+ RK3568_CLKGATE_CON(30), 15, GFLAGS),
+ GATE(PCLK_SPI3, "pclk_spi3", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(31), 0, GFLAGS),
+ COMPOSITE_NODIV(CLK_SPI3, "clk_spi3", gpll200_xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(72), 6, 1, MFLAGS, RK3568_CLKGATE_CON(31), 1, GFLAGS),
+ GATE(PCLK_PWM1, "pclk_pwm1", "pclk_bus", 0, RK3568_CLKGATE_CON(31), 10, GFLAGS),
+ COMPOSITE_NODIV(CLK_PWM1, "clk_pwm1", gpll100_xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(72), 8, 1, MFLAGS,
+ RK3568_CLKGATE_CON(31), 11, GFLAGS),
+ GATE(CLK_PWM1_CAPTURE, "clk_pwm1_capture", "xin24m", 0,
+ RK3568_CLKGATE_CON(31), 12, GFLAGS),
+ GATE(PCLK_PWM2, "pclk_pwm2", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(31), 13, GFLAGS),
+ COMPOSITE_NODIV(CLK_PWM2, "clk_pwm2", gpll100_xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(72), 10, 1, MFLAGS,
+ RK3568_CLKGATE_CON(31), 14, GFLAGS),
+ GATE(CLK_PWM2_CAPTURE, "clk_pwm2_capture", "xin24m", 0,
+ RK3568_CLKGATE_CON(31), 15, GFLAGS),
+ GATE(PCLK_PWM3, "pclk_pwm3", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(32), 0, GFLAGS),
+ COMPOSITE_NODIV(CLK_PWM3, "clk_pwm3", gpll100_xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(72), 12, 1, MFLAGS,
+ RK3568_CLKGATE_CON(32), 1, GFLAGS),
+ GATE(CLK_PWM3_CAPTURE, "clk_pwm3_capture", "xin24m", 0,
+ RK3568_CLKGATE_CON(32), 2, GFLAGS),
+ COMPOSITE_NODIV(DBCLK_GPIO, "dbclk_gpio", xin24m_32k_p, 0,
+ RK3568_CLKSEL_CON(72), 14, 1, MFLAGS,
+ RK3568_CLKGATE_CON(32), 11, GFLAGS),
+ GATE(PCLK_GPIO1, "pclk_gpio1", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(31), 2, GFLAGS),
+ GATE(DBCLK_GPIO1, "dbclk_gpio1", "dbclk_gpio", 0,
+ RK3568_CLKGATE_CON(31), 3, GFLAGS),
+ GATE(PCLK_GPIO2, "pclk_gpio2", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(31), 4, GFLAGS),
+ GATE(DBCLK_GPIO2, "dbclk_gpio2", "dbclk_gpio", 0,
+ RK3568_CLKGATE_CON(31), 5, GFLAGS),
+ GATE(PCLK_GPIO3, "pclk_gpio3", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(31), 6, GFLAGS),
+ GATE(DBCLK_GPIO3, "dbclk_gpio3", "dbclk_gpio", 0,
+ RK3568_CLKGATE_CON(31), 7, GFLAGS),
+ GATE(PCLK_GPIO4, "pclk_gpio4", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(31), 8, GFLAGS),
+ GATE(DBCLK_GPIO4, "dbclk_gpio4", "dbclk_gpio", 0,
+ RK3568_CLKGATE_CON(31), 9, GFLAGS),
+ GATE(PCLK_TIMER, "pclk_timer", "pclk_bus", 0,
+ RK3568_CLKGATE_CON(32), 3, GFLAGS),
+ GATE(CLK_TIMER0, "clk_timer0", "xin24m", 0,
+ RK3568_CLKGATE_CON(32), 4, GFLAGS),
+ GATE(CLK_TIMER1, "clk_timer1", "xin24m", 0,
+ RK3568_CLKGATE_CON(32), 5, GFLAGS),
+ GATE(CLK_TIMER2, "clk_timer2", "xin24m", 0,
+ RK3568_CLKGATE_CON(32), 6, GFLAGS),
+ GATE(CLK_TIMER3, "clk_timer3", "xin24m", 0,
+ RK3568_CLKGATE_CON(32), 7, GFLAGS),
+ GATE(CLK_TIMER4, "clk_timer4", "xin24m", 0,
+ RK3568_CLKGATE_CON(32), 8, GFLAGS),
+ GATE(CLK_TIMER5, "clk_timer5", "xin24m", 0,
+ RK3568_CLKGATE_CON(32), 9, GFLAGS),
+
+ /* PD_TOP */
+ COMPOSITE_NODIV(ACLK_TOP_HIGH, "aclk_top_high", cpll500_gpll400_gpll300_xin24m_p, 0,
+ RK3568_CLKSEL_CON(73), 0, 2, MFLAGS,
+ RK3568_CLKGATE_CON(33), 0, GFLAGS),
+ COMPOSITE_NODIV(ACLK_TOP_LOW, "aclk_top_low", gpll400_gpll300_gpll200_xin24m_p, 0,
+ RK3568_CLKSEL_CON(73), 4, 2, MFLAGS,
+ RK3568_CLKGATE_CON(33), 1, GFLAGS),
+ COMPOSITE_NODIV(HCLK_TOP, "hclk_top", gpll150_gpll100_gpll75_xin24m_p, 0,
+ RK3568_CLKSEL_CON(73), 8, 2, MFLAGS,
+ RK3568_CLKGATE_CON(33), 2, GFLAGS),
+ COMPOSITE_NODIV(PCLK_TOP, "pclk_top", gpll100_gpll75_cpll50_xin24m_p, 0,
+ RK3568_CLKSEL_CON(73), 12, 2, MFLAGS,
+ RK3568_CLKGATE_CON(33), 3, GFLAGS),
+ GATE(PCLK_PCIE30PHY, "pclk_pcie30phy", "pclk_top", 0,
+ RK3568_CLKGATE_CON(33), 8, GFLAGS),
+ COMPOSITE_NODIV(CLK_OPTC_ARB, "clk_optc_arb", xin24m_cpll100_p, 0,
+ RK3568_CLKSEL_CON(73), 15, 1, MFLAGS,
+ RK3568_CLKGATE_CON(33), 9, GFLAGS),
+ GATE(PCLK_MIPICSIPHY, "pclk_mipicsiphy", "pclk_top", 0,
+ RK3568_CLKGATE_CON(33), 13, GFLAGS),
+ GATE(PCLK_MIPIDSIPHY0, "pclk_mipidsiphy0", "pclk_top", 0,
+ RK3568_CLKGATE_CON(33), 14, GFLAGS),
+ GATE(PCLK_MIPIDSIPHY1, "pclk_mipidsiphy1", "pclk_top", 0,
+ RK3568_CLKGATE_CON(33), 15, GFLAGS),
+ GATE(PCLK_PIPEPHY0, "pclk_pipephy0", "pclk_top", 0,
+ RK3568_CLKGATE_CON(34), 4, GFLAGS),
+ GATE(PCLK_PIPEPHY1, "pclk_pipephy1", "pclk_top", 0,
+ RK3568_CLKGATE_CON(34), 5, GFLAGS),
+ GATE(PCLK_PIPEPHY2, "pclk_pipephy2", "pclk_top", 0,
+ RK3568_CLKGATE_CON(34), 6, GFLAGS),
+ GATE(PCLK_CPU_BOOST, "pclk_cpu_boost", "pclk_top", 0,
+ RK3568_CLKGATE_CON(34), 11, GFLAGS),
+ GATE(CLK_CPU_BOOST, "clk_cpu_boost", "xin24m", 0,
+ RK3568_CLKGATE_CON(34), 12, GFLAGS),
+ GATE(PCLK_OTPPHY, "pclk_otpphy", "pclk_top", 0,
+ RK3568_CLKGATE_CON(34), 13, GFLAGS),
+ GATE(PCLK_EDPPHY_GRF, "pclk_edpphy_grf", "pclk_top", 0,
+ RK3568_CLKGATE_CON(34), 14, GFLAGS),
+};
+
+static struct rockchip_clk_branch rk3568_clk_pmu_branches[] __initdata = {
+ /* PD_PMU */
+ FACTOR(0, "ppll_ph0", "ppll", 0, 1, 2),
+ FACTOR(0, "ppll_ph180", "ppll", 0, 1, 2),
+ FACTOR(0, "hpll_ph0", "hpll", 0, 1, 2),
+
+ MUX(CLK_PDPMU, "clk_pdpmu", clk_pdpmu_p, 0,
+ RK3568_PMU_CLKSEL_CON(2), 15, 1, MFLAGS),
+ COMPOSITE_NOMUX(PCLK_PDPMU, "pclk_pdpmu", "clk_pdpmu", 0,
+ RK3568_PMU_CLKSEL_CON(2), 0, 5, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(0), 2, GFLAGS),
+ GATE(PCLK_PMU, "pclk_pmu", "pclk_pdpmu", 0,
+ RK3568_PMU_CLKGATE_CON(0), 6, GFLAGS),
+ GATE(CLK_PMU, "clk_pmu", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(0), 7, GFLAGS),
+ GATE(PCLK_I2C0, "pclk_i2c0", "pclk_pdpmu", 0,
+ RK3568_PMU_CLKGATE_CON(1), 0, GFLAGS),
+ COMPOSITE_NOMUX(CLK_I2C0, "clk_i2c0", "clk_pdpmu", 0,
+ RK3568_PMU_CLKSEL_CON(3), 0, 7, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(1), 1, GFLAGS),
+ GATE(PCLK_UART0, "pclk_uart0", "pclk_pdpmu", 0,
+ RK3568_PMU_CLKGATE_CON(1), 2, GFLAGS),
+
+ COMPOSITE_FRACMUX(CLK_RTC32K_FRAC, "clk_rtc32k_frac", "xin24m", CLK_IGNORE_UNUSED,
+ RK3568_PMU_CLKSEL_CON(1), 0,
+ RK3568_PMU_CLKGATE_CON(0), 1, GFLAGS,
+ &rk3568_rtc32k_pmu_fracmux),
+
+ COMPOSITE_NOMUX(XIN_OSC0_DIV, "xin_osc0_div", "xin24m", CLK_IGNORE_UNUSED,
+ RK3568_PMU_CLKSEL_CON(0), 0, 5, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(0), 0, GFLAGS),
+
+ COMPOSITE(CLK_UART0_DIV, "sclk_uart0_div", ppll_usb480m_cpll_gpll_p, 0,
+ RK3568_PMU_CLKSEL_CON(4), 8, 2, MFLAGS, 0, 7, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(1), 3, GFLAGS),
+ COMPOSITE_FRACMUX(CLK_UART0_FRAC, "sclk_uart0_frac", "sclk_uart0_div", CLK_SET_RATE_PARENT,
+ RK3568_PMU_CLKSEL_CON(5), 0,
+ RK3568_PMU_CLKGATE_CON(1), 4, GFLAGS,
+ &rk3568_uart0_fracmux),
+ GATE(SCLK_UART0, "sclk_uart0", "sclk_uart0_mux", 0,
+ RK3568_PMU_CLKGATE_CON(1), 5, GFLAGS),
+
+ GATE(PCLK_GPIO0, "pclk_gpio0", "pclk_pdpmu", 0,
+ RK3568_PMU_CLKGATE_CON(1), 9, GFLAGS),
+ COMPOSITE_NODIV(DBCLK_GPIO0, "dbclk_gpio0", xin24m_32k_p, 0,
+ RK3568_PMU_CLKSEL_CON(6), 15, 1, MFLAGS,
+ RK3568_PMU_CLKGATE_CON(1), 10, GFLAGS),
+ GATE(PCLK_PWM0, "pclk_pwm0", "pclk_pdpmu", 0,
+ RK3568_PMU_CLKGATE_CON(1), 6, GFLAGS),
+ COMPOSITE(CLK_PWM0, "clk_pwm0", clk_pwm0_p, 0,
+ RK3568_PMU_CLKSEL_CON(6), 7, 1, MFLAGS, 0, 7, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(1), 7, GFLAGS),
+ GATE(CLK_CAPTURE_PWM0_NDFT, "clk_capture_pwm0_ndft", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(1), 8, GFLAGS),
+ GATE(PCLK_PMUPVTM, "pclk_pmupvtm", "pclk_pdpmu", 0,
+ RK3568_PMU_CLKGATE_CON(1), 11, GFLAGS),
+ GATE(CLK_PMUPVTM, "clk_pmupvtm", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(1), 12, GFLAGS),
+ GATE(CLK_CORE_PMUPVTM, "clk_core_pmupvtm", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(1), 13, GFLAGS),
+ COMPOSITE_NOMUX(CLK_REF24M, "clk_ref24m", "clk_pdpmu", 0,
+ RK3568_PMU_CLKSEL_CON(7), 0, 6, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(2), 0, GFLAGS),
+ GATE(XIN_OSC0_USBPHY0_G, "xin_osc0_usbphy0_g", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 1, GFLAGS),
+ MUX(CLK_USBPHY0_REF, "clk_usbphy0_ref", clk_usbphy0_ref_p, 0,
+ RK3568_PMU_CLKSEL_CON(8), 0, 1, MFLAGS),
+ GATE(XIN_OSC0_USBPHY1_G, "xin_osc0_usbphy1_g", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 2, GFLAGS),
+ MUX(CLK_USBPHY1_REF, "clk_usbphy1_ref", clk_usbphy1_ref_p, 0,
+ RK3568_PMU_CLKSEL_CON(8), 1, 1, MFLAGS),
+ GATE(XIN_OSC0_MIPIDSIPHY0_G, "xin_osc0_mipidsiphy0_g", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 3, GFLAGS),
+ MUX(CLK_MIPIDSIPHY0_REF, "clk_mipidsiphy0_ref", clk_mipidsiphy0_ref_p, 0,
+ RK3568_PMU_CLKSEL_CON(8), 2, 1, MFLAGS),
+ GATE(XIN_OSC0_MIPIDSIPHY1_G, "xin_osc0_mipidsiphy1_g", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 4, GFLAGS),
+ MUX(CLK_MIPIDSIPHY1_REF, "clk_mipidsiphy1_ref", clk_mipidsiphy1_ref_p, 0,
+ RK3568_PMU_CLKSEL_CON(8), 3, 1, MFLAGS),
+ COMPOSITE_NOMUX(CLK_WIFI_DIV, "clk_wifi_div", "clk_pdpmu", 0,
+ RK3568_PMU_CLKSEL_CON(8), 8, 6, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(2), 5, GFLAGS),
+ GATE(CLK_WIFI_OSC0, "clk_wifi_osc0", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 6, GFLAGS),
+ MUX(CLK_WIFI, "clk_wifi", clk_wifi_p, CLK_SET_RATE_PARENT,
+ RK3568_PMU_CLKSEL_CON(8), 15, 1, MFLAGS),
+ COMPOSITE_NOMUX(CLK_PCIEPHY0_DIV, "clk_pciephy0_div", "ppll_ph0", 0,
+ RK3568_PMU_CLKSEL_CON(9), 0, 3, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(2), 7, GFLAGS),
+ GATE(CLK_PCIEPHY0_OSC0, "clk_pciephy0_osc0", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 8, GFLAGS),
+ MUX(CLK_PCIEPHY0_REF, "clk_pciephy0_ref", clk_pciephy0_ref_p, CLK_SET_RATE_PARENT,
+ RK3568_PMU_CLKSEL_CON(9), 3, 1, MFLAGS),
+ COMPOSITE_NOMUX(CLK_PCIEPHY1_DIV, "clk_pciephy1_div", "ppll_ph0", 0,
+ RK3568_PMU_CLKSEL_CON(9), 4, 3, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(2), 9, GFLAGS),
+ GATE(CLK_PCIEPHY1_OSC0, "clk_pciephy1_osc0", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 10, GFLAGS),
+ MUX(CLK_PCIEPHY1_REF, "clk_pciephy1_ref", clk_pciephy1_ref_p, CLK_SET_RATE_PARENT,
+ RK3568_PMU_CLKSEL_CON(9), 7, 1, MFLAGS),
+ COMPOSITE_NOMUX(CLK_PCIEPHY2_DIV, "clk_pciephy2_div", "ppll_ph0", 0,
+ RK3568_PMU_CLKSEL_CON(9), 8, 3, DFLAGS,
+ RK3568_PMU_CLKGATE_CON(2), 11, GFLAGS),
+ GATE(CLK_PCIEPHY2_OSC0, "clk_pciephy2_osc0", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 12, GFLAGS),
+ MUX(CLK_PCIEPHY2_REF, "clk_pciephy2_ref", clk_pciephy2_ref_p, CLK_SET_RATE_PARENT,
+ RK3568_PMU_CLKSEL_CON(9), 11, 1, MFLAGS),
+ GATE(CLK_PCIE30PHY_REF_M, "clk_pcie30phy_ref_m", "ppll_ph0", 0,
+ RK3568_PMU_CLKGATE_CON(2), 13, GFLAGS),
+ GATE(CLK_PCIE30PHY_REF_N, "clk_pcie30phy_ref_n", "ppll_ph180", 0,
+ RK3568_PMU_CLKGATE_CON(2), 14, GFLAGS),
+ GATE(XIN_OSC0_EDPPHY_G, "xin_osc0_edpphy_g", "xin24m", 0,
+ RK3568_PMU_CLKGATE_CON(2), 15, GFLAGS),
+ MUX(CLK_HDMI_REF, "clk_hdmi_ref", clk_hdmi_ref_p, 0,
+ RK3568_PMU_CLKSEL_CON(8), 7, 1, MFLAGS),
+};
+
+static const char *const rk3568_cru_critical_clocks[] __initconst = {
+ "armclk",
+ "pclk_core_pre",
+ "aclk_bus",
+ "pclk_bus",
+ "aclk_top_high",
+ "aclk_top_low",
+ "hclk_top",
+ "pclk_top",
+ "aclk_perimid",
+ "hclk_perimid",
+ "aclk_secure_flash",
+ "hclk_secure_flash",
+ "aclk_core_niu2bus",
+ "npll",
+ "clk_optc_arb",
+ "hclk_php",
+ "pclk_php",
+ "hclk_usb",
+};
+
+static const char *const rk3568_pmucru_critical_clocks[] __initconst = {
+ "pclk_pdpmu",
+ "pclk_pmu",
+ "clk_pmu",
+};
+
+static void __init rk3568_pmu_clk_init(struct device_node *np)
+{
+ struct rockchip_clk_provider *ctx;
+ void __iomem *reg_base;
+
+ reg_base = of_iomap(np, 0);
+ if (!reg_base) {
+ pr_err("%s: could not map cru pmu region\n", __func__);
+ return;
+ }
+
+ ctx = rockchip_clk_init(np, reg_base, CLKPMU_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip pmu clk init failed\n", __func__);
+ return;
+ }
+
+ rockchip_clk_register_plls(ctx, rk3568_pmu_pll_clks,
+ ARRAY_SIZE(rk3568_pmu_pll_clks),
+ RK3568_GRF_SOC_STATUS0);
+
+ rockchip_clk_register_branches(ctx, rk3568_clk_pmu_branches,
+ ARRAY_SIZE(rk3568_clk_pmu_branches));
+
+ rockchip_register_softrst(np, 1, reg_base + RK3568_PMU_SOFTRST_CON(0),
+ ROCKCHIP_SOFTRST_HIWORD_MASK);
+
+ rockchip_clk_protect_critical(rk3568_pmucru_critical_clocks,
+ ARRAY_SIZE(rk3568_pmucru_critical_clocks));
+
+ rockchip_clk_of_add_provider(np, ctx);
+}
+
+CLK_OF_DECLARE(rk3568_cru_pmu, "rockchip,rk3568-pmucru", rk3568_pmu_clk_init);
+
+static void __init rk3568_clk_init(struct device_node *np)
+{
+ struct rockchip_clk_provider *ctx;
+ void __iomem *reg_base;
+
+ reg_base = of_iomap(np, 0);
+ if (!reg_base) {
+ pr_err("%s: could not map cru region\n", __func__);
+ return;
+ }
+
+ ctx = rockchip_clk_init(np, reg_base, CLK_NR_CLKS);
+ if (IS_ERR(ctx)) {
+ pr_err("%s: rockchip clk init failed\n", __func__);
+ iounmap(reg_base);
+ return;
+ }
+
+ rockchip_clk_register_plls(ctx, rk3568_pll_clks,
+ ARRAY_SIZE(rk3568_pll_clks),
+ RK3568_GRF_SOC_STATUS0);
+
+ rockchip_clk_register_armclk(ctx, ARMCLK, "armclk",
+ mux_armclk_p, ARRAY_SIZE(mux_armclk_p),
+ &rk3568_cpuclk_data, rk3568_cpuclk_rates,
+ ARRAY_SIZE(rk3568_cpuclk_rates));
+
+ rockchip_clk_register_branches(ctx, rk3568_clk_branches,
+ ARRAY_SIZE(rk3568_clk_branches));
+
+ rockchip_register_softrst(np, 30, reg_base + RK3568_SOFTRST_CON(0),
+ ROCKCHIP_SOFTRST_HIWORD_MASK);
+
+ rockchip_register_restart_notifier(ctx, RK3568_GLB_SRST_FST, NULL);
+
+ rockchip_clk_protect_critical(rk3568_cru_critical_clocks,
+ ARRAY_SIZE(rk3568_cru_critical_clocks));
+
+ rockchip_clk_of_add_provider(np, ctx);
+}
+
+CLK_OF_DECLARE(rk3568_cru, "rockchip,rk3568-cru", rk3568_clk_init);
+
+struct clk_rk3568_inits {
+ void (*inits)(struct device_node *np);
+};
+
+static const struct clk_rk3568_inits clk_rk3568_pmucru_init = {
+ .inits = rk3568_pmu_clk_init,
+};
+
+static const struct clk_rk3568_inits clk_3568_cru_init = {
+ .inits = rk3568_clk_init,
+};
+
+static const struct of_device_id clk_rk3568_match_table[] = {
+ {
+ .compatible = "rockchip,rk3568-cru",
+ .data = &clk_3568_cru_init,
+ }, {
+ .compatible = "rockchip,rk3568-pmucru",
+ .data = &clk_rk3568_pmucru_init,
+ },
+ { }
+};
+MODULE_DEVICE_TABLE(of, clk_rk3568_match_table);
+
+static int __init clk_rk3568_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ const struct of_device_id *match;
+ const struct clk_rk3568_inits *init_data;
+
+ match = of_match_device(clk_rk3568_match_table, &pdev->dev);
+ if (!match || !match->data)
+ return -EINVAL;
+
+ init_data = match->data;
+ if (init_data->inits)
+ init_data->inits(np);
+
+ return 0;
+}
+
+static struct platform_driver clk_rk3568_driver = {
+ .driver = {
+ .name = "clk-rk3568",
+ .of_match_table = clk_rk3568_match_table,
+ .suppress_bind_attrs = true,
+ },
+};
+builtin_platform_driver_probe(clk_rk3568_driver, clk_rk3568_probe);
+
+MODULE_DESCRIPTION("Rockchip RK3568 Clock Driver");
+MODULE_LICENSE("GPL");
diff --git a/drivers/clk/rockchip/clk-rv1108.c b/drivers/clk/rockchip/clk-rv1108.c
index 5947d3192866..5f49af3c970a 100644
--- a/drivers/clk/rockchip/clk-rv1108.c
+++ b/drivers/clk/rockchip/clk-rv1108.c
@@ -106,9 +106,10 @@ static struct rockchip_cpuclk_rate_table rv1108_cpuclk_rates[] __initdata = {
};
static const struct rockchip_cpuclk_reg_data rv1108_cpuclk_data = {
- .core_reg = RV1108_CLKSEL_CON(0),
- .div_core_shift = 0,
- .div_core_mask = 0x1f,
+ .core_reg[0] = RV1108_CLKSEL_CON(0),
+ .div_core_shift[0] = 0,
+ .div_core_mask[0] = 0x1f,
+ .num_cores = 1,
.mux_core_alt = 1,
.mux_core_main = 0,
.mux_core_shift = 8,
diff --git a/drivers/clk/rockchip/clk.h b/drivers/clk/rockchip/clk.h
index 2271a84124b0..571cee7bbfdc 100644
--- a/drivers/clk/rockchip/clk.h
+++ b/drivers/clk/rockchip/clk.h
@@ -188,6 +188,34 @@ struct clk;
#define RK3399_PMU_CLKGATE_CON(x) ((x) * 0x4 + 0x100)
#define RK3399_PMU_SOFTRST_CON(x) ((x) * 0x4 + 0x110)
+#define RK3568_PLL_CON(x) RK2928_PLL_CON(x)
+#define RK3568_MODE_CON0 0xc0
+#define RK3568_MISC_CON0 0xc4
+#define RK3568_MISC_CON1 0xc8
+#define RK3568_MISC_CON2 0xcc
+#define RK3568_GLB_CNT_TH 0xd0
+#define RK3568_GLB_SRST_FST 0xd4
+#define RK3568_GLB_SRST_SND 0xd8
+#define RK3568_GLB_RST_CON 0xdc
+#define RK3568_GLB_RST_ST 0xe0
+#define RK3568_CLKSEL_CON(x) ((x) * 0x4 + 0x100)
+#define RK3568_CLKGATE_CON(x) ((x) * 0x4 + 0x300)
+#define RK3568_SOFTRST_CON(x) ((x) * 0x4 + 0x400)
+#define RK3568_SDMMC0_CON0 0x580
+#define RK3568_SDMMC0_CON1 0x584
+#define RK3568_SDMMC1_CON0 0x588
+#define RK3568_SDMMC1_CON1 0x58c
+#define RK3568_SDMMC2_CON0 0x590
+#define RK3568_SDMMC2_CON1 0x594
+#define RK3568_EMMC_CON0 0x598
+#define RK3568_EMMC_CON1 0x59c
+
+#define RK3568_PMU_PLL_CON(x) RK2928_PLL_CON(x)
+#define RK3568_PMU_MODE_CON0 0x80
+#define RK3568_PMU_CLKSEL_CON(x) ((x) * 0x4 + 0x100)
+#define RK3568_PMU_CLKGATE_CON(x) ((x) * 0x4 + 0x180)
+#define RK3568_PMU_SOFTRST_CON(x) ((x) * 0x4 + 0x200)
+
enum rockchip_pll_type {
pll_rk3036,
pll_rk3066,
@@ -322,7 +350,8 @@ struct rockchip_cpuclk_clksel {
u32 val;
};
-#define ROCKCHIP_CPUCLK_NUM_DIVIDERS 2
+#define ROCKCHIP_CPUCLK_NUM_DIVIDERS 5
+#define ROCKCHIP_CPUCLK_MAX_CORES 4
struct rockchip_cpuclk_rate_table {
unsigned long prate;
struct rockchip_cpuclk_clksel divs[ROCKCHIP_CPUCLK_NUM_DIVIDERS];
@@ -330,22 +359,23 @@ struct rockchip_cpuclk_rate_table {
/**
* struct rockchip_cpuclk_reg_data - register offsets and masks of the cpuclock
- * @core_reg: register offset of the core settings register
- * @div_core_shift: core divider offset used to divide the pll value
- * @div_core_mask: core divider mask
- * @mux_core_alt: mux value to select alternate parent
+ * @core_reg[]: register offset of the cores setting register
+ * @div_core_shift[]: cores divider offset used to divide the pll value
+ * @div_core_mask[]: cores divider mask
+ * @num_cores: number of cpu cores
* @mux_core_main: mux value to select main parent of core
* @mux_core_shift: offset of the core multiplexer
* @mux_core_mask: core multiplexer mask
*/
struct rockchip_cpuclk_reg_data {
- int core_reg;
- u8 div_core_shift;
- u32 div_core_mask;
- u8 mux_core_alt;
- u8 mux_core_main;
- u8 mux_core_shift;
- u32 mux_core_mask;
+ int core_reg[ROCKCHIP_CPUCLK_MAX_CORES];
+ u8 div_core_shift[ROCKCHIP_CPUCLK_MAX_CORES];
+ u32 div_core_mask[ROCKCHIP_CPUCLK_MAX_CORES];
+ int num_cores;
+ u8 mux_core_alt;
+ u8 mux_core_main;
+ u8 mux_core_shift;
+ u32 mux_core_mask;
};
struct clk *rockchip_clk_register_cpuclk(const char *name,
diff --git a/drivers/clk/samsung/clk-exynos4412-isp.c b/drivers/clk/samsung/clk-exynos4412-isp.c
index 4b9e73608c21..b69e381b8c0c 100644
--- a/drivers/clk/samsung/clk-exynos4412-isp.c
+++ b/drivers/clk/samsung/clk-exynos4412-isp.c
@@ -115,10 +115,8 @@ static int __init exynos4x12_isp_clk_probe(struct platform_device *pdev)
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
reg_base = devm_ioremap_resource(dev, res);
- if (IS_ERR(reg_base)) {
- dev_err(dev, "failed to map registers\n");
+ if (IS_ERR(reg_base))
return PTR_ERR(reg_base);
- }
exynos4x12_save_isp = samsung_clk_alloc_reg_dump(exynos4x12_clk_isp_save,
ARRAY_SIZE(exynos4x12_clk_isp_save));
diff --git a/drivers/clk/samsung/clk-exynos7.c b/drivers/clk/samsung/clk-exynos7.c
index 87ee1bad9a9a..4a5d2a914bd6 100644
--- a/drivers/clk/samsung/clk-exynos7.c
+++ b/drivers/clk/samsung/clk-exynos7.c
@@ -537,8 +537,13 @@ static const struct samsung_gate_clock top1_gate_clks[] __initconst = {
GATE(CLK_ACLK_FSYS0_200, "aclk_fsys0_200", "dout_aclk_fsys0_200",
ENABLE_ACLK_TOP13, 28, CLK_SET_RATE_PARENT |
CLK_IS_CRITICAL, 0),
+ /*
+ * This clock is required for the CMU_FSYS1 registers access, keep it
+ * enabled permanently until proper runtime PM support is added.
+ */
GATE(CLK_ACLK_FSYS1_200, "aclk_fsys1_200", "dout_aclk_fsys1_200",
- ENABLE_ACLK_TOP13, 24, CLK_SET_RATE_PARENT, 0),
+ ENABLE_ACLK_TOP13, 24, CLK_SET_RATE_PARENT |
+ CLK_IS_CRITICAL, 0),
GATE(CLK_SCLK_PHY_FSYS1_26M, "sclk_phy_fsys1_26m",
"dout_sclk_phy_fsys1_26m", ENABLE_SCLK_TOP1_FSYS11,
diff --git a/drivers/clk/samsung/clk-s5pv210-audss.c b/drivers/clk/samsung/clk-s5pv210-audss.c
index 14985ebd043b..a7827a120695 100644
--- a/drivers/clk/samsung/clk-s5pv210-audss.c
+++ b/drivers/clk/samsung/clk-s5pv210-audss.c
@@ -72,10 +72,8 @@ static int s5pv210_audss_clk_probe(struct platform_device *pdev)
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
reg_base = devm_ioremap_resource(&pdev->dev, res);
- if (IS_ERR(reg_base)) {
- dev_err(&pdev->dev, "failed to map audss registers\n");
+ if (IS_ERR(reg_base))
return PTR_ERR(reg_base);
- }
clk_data = devm_kzalloc(&pdev->dev,
struct_size(clk_data, hws, AUDSS_MAX_CLKS),
diff --git a/drivers/clk/sifive/Kconfig b/drivers/clk/sifive/Kconfig
index 1c14eb20c066..9132c3c4aa86 100644
--- a/drivers/clk/sifive/Kconfig
+++ b/drivers/clk/sifive/Kconfig
@@ -10,6 +10,8 @@ if CLK_SIFIVE
config CLK_SIFIVE_PRCI
bool "PRCI driver for SiFive SoCs"
+ select RESET_CONTROLLER
+ select RESET_SIMPLE
select CLK_ANALOGBITS_WRPLL_CLN28HPC
help
Supports the Power Reset Clock interface (PRCI) IP block found in
diff --git a/drivers/clk/sifive/fu740-prci.c b/drivers/clk/sifive/fu740-prci.c
index 764d1097aa51..53f6e00a03b9 100644
--- a/drivers/clk/sifive/fu740-prci.c
+++ b/drivers/clk/sifive/fu740-prci.c
@@ -72,6 +72,12 @@ static const struct clk_ops sifive_fu740_prci_hfpclkplldiv_clk_ops = {
.recalc_rate = sifive_prci_hfpclkplldiv_recalc_rate,
};
+static const struct clk_ops sifive_fu740_prci_pcie_aux_clk_ops = {
+ .enable = sifive_prci_pcie_aux_clock_enable,
+ .disable = sifive_prci_pcie_aux_clock_disable,
+ .is_enabled = sifive_prci_pcie_aux_clock_is_enabled,
+};
+
/* List of clock controls provided by the PRCI */
struct __prci_clock __prci_init_clocks_fu740[] = {
[PRCI_CLK_COREPLL] = {
@@ -120,4 +126,9 @@ struct __prci_clock __prci_init_clocks_fu740[] = {
.parent_name = "hfpclkpll",
.ops = &sifive_fu740_prci_hfpclkplldiv_clk_ops,
},
+ [PRCI_CLK_PCIE_AUX] = {
+ .name = "pcie_aux",
+ .parent_name = "hfclk",
+ .ops = &sifive_fu740_prci_pcie_aux_clk_ops,
+ },
};
diff --git a/drivers/clk/sifive/fu740-prci.h b/drivers/clk/sifive/fu740-prci.h
index 13ef971f7764..511a0bf7ba2b 100644
--- a/drivers/clk/sifive/fu740-prci.h
+++ b/drivers/clk/sifive/fu740-prci.h
@@ -9,7 +9,7 @@
#include "sifive-prci.h"
-#define NUM_CLOCK_FU740 8
+#define NUM_CLOCK_FU740 9
extern struct __prci_clock __prci_init_clocks_fu740[NUM_CLOCK_FU740];
diff --git a/drivers/clk/sifive/sifive-prci.c b/drivers/clk/sifive/sifive-prci.c
index 1490b01ce629..0d79ba31a793 100644
--- a/drivers/clk/sifive/sifive-prci.c
+++ b/drivers/clk/sifive/sifive-prci.c
@@ -453,6 +453,47 @@ void sifive_prci_hfpclkpllsel_use_hfpclkpll(struct __prci_data *pd)
r = __prci_readl(pd, PRCI_HFPCLKPLLSEL_OFFSET); /* barrier */
}
+/* PCIE AUX clock APIs for enable, disable. */
+int sifive_prci_pcie_aux_clock_is_enabled(struct clk_hw *hw)
+{
+ struct __prci_clock *pc = clk_hw_to_prci_clock(hw);
+ struct __prci_data *pd = pc->pd;
+ u32 r;
+
+ r = __prci_readl(pd, PRCI_PCIE_AUX_OFFSET);
+
+ if (r & PRCI_PCIE_AUX_EN_MASK)
+ return 1;
+ else
+ return 0;
+}
+
+int sifive_prci_pcie_aux_clock_enable(struct clk_hw *hw)
+{
+ struct __prci_clock *pc = clk_hw_to_prci_clock(hw);
+ struct __prci_data *pd = pc->pd;
+ u32 r __maybe_unused;
+
+ if (sifive_prci_pcie_aux_clock_is_enabled(hw))
+ return 0;
+
+ __prci_writel(1, PRCI_PCIE_AUX_OFFSET, pd);
+ r = __prci_readl(pd, PRCI_PCIE_AUX_OFFSET); /* barrier */
+
+ return 0;
+}
+
+void sifive_prci_pcie_aux_clock_disable(struct clk_hw *hw)
+{
+ struct __prci_clock *pc = clk_hw_to_prci_clock(hw);
+ struct __prci_data *pd = pc->pd;
+ u32 r __maybe_unused;
+
+ __prci_writel(0, PRCI_PCIE_AUX_OFFSET, pd);
+ r = __prci_readl(pd, PRCI_PCIE_AUX_OFFSET); /* barrier */
+
+}
+
/**
* __prci_register_clocks() - register clock controls in the PRCI
* @dev: Linux struct device
@@ -547,6 +588,19 @@ static int sifive_prci_probe(struct platform_device *pdev)
if (IS_ERR(pd->va))
return PTR_ERR(pd->va);
+ pd->reset.rcdev.owner = THIS_MODULE;
+ pd->reset.rcdev.nr_resets = PRCI_RST_NR;
+ pd->reset.rcdev.ops = &reset_simple_ops;
+ pd->reset.rcdev.of_node = pdev->dev.of_node;
+ pd->reset.active_low = true;
+ pd->reset.membase = pd->va + PRCI_DEVICESRESETREG_OFFSET;
+ spin_lock_init(&pd->reset.lock);
+
+ r = devm_reset_controller_register(&pdev->dev, &pd->reset.rcdev);
+ if (r) {
+ dev_err(dev, "could not register reset controller: %d\n", r);
+ return r;
+ }
r = __prci_register_clocks(dev, pd, desc);
if (r) {
dev_err(dev, "could not register clocks: %d\n", r);
diff --git a/drivers/clk/sifive/sifive-prci.h b/drivers/clk/sifive/sifive-prci.h
index dbdbd1722688..91658a88af4e 100644
--- a/drivers/clk/sifive/sifive-prci.h
+++ b/drivers/clk/sifive/sifive-prci.h
@@ -11,6 +11,7 @@
#include <linux/clk/analogbits-wrpll-cln28hpc.h>
#include <linux/clk-provider.h>
+#include <linux/reset/reset-simple.h>
#include <linux/platform_device.h>
/*
@@ -67,6 +68,11 @@
#define PRCI_DDRPLLCFG1_CKE_SHIFT 31
#define PRCI_DDRPLLCFG1_CKE_MASK (0x1 << PRCI_DDRPLLCFG1_CKE_SHIFT)
+/* PCIEAUX */
+#define PRCI_PCIE_AUX_OFFSET 0x14
+#define PRCI_PCIE_AUX_EN_SHIFT 0
+#define PRCI_PCIE_AUX_EN_MASK (0x1 << PRCI_PCIE_AUX_EN_SHIFT)
+
/* GEMGXLPLLCFG0 */
#define PRCI_GEMGXLPLLCFG0_OFFSET 0x1c
#define PRCI_GEMGXLPLLCFG0_DIVR_SHIFT 0
@@ -116,6 +122,8 @@
#define PRCI_DEVICESRESETREG_CHIPLINK_RST_N_MASK \
(0x1 << PRCI_DEVICESRESETREG_CHIPLINK_RST_N_SHIFT)
+#define PRCI_RST_NR 7
+
/* CLKMUXSTATUSREG */
#define PRCI_CLKMUXSTATUSREG_OFFSET 0x2c
#define PRCI_CLKMUXSTATUSREG_TLCLKSEL_STATUS_SHIFT 1
@@ -216,6 +224,7 @@
*/
struct __prci_data {
void __iomem *va;
+ struct reset_simple_data reset;
struct clk_hw_onecell_data hw_clks;
};
@@ -296,4 +305,8 @@ unsigned long sifive_prci_tlclksel_recalc_rate(struct clk_hw *hw,
unsigned long sifive_prci_hfpclkplldiv_recalc_rate(struct clk_hw *hw,
unsigned long parent_rate);
+int sifive_prci_pcie_aux_clock_is_enabled(struct clk_hw *hw);
+int sifive_prci_pcie_aux_clock_enable(struct clk_hw *hw);
+void sifive_prci_pcie_aux_clock_disable(struct clk_hw *hw);
+
#endif /* __SIFIVE_CLK_SIFIVE_PRCI_H */
diff --git a/drivers/clk/socfpga/Kconfig b/drivers/clk/socfpga/Kconfig
new file mode 100644
index 000000000000..0cf16b894efb
--- /dev/null
+++ b/drivers/clk/socfpga/Kconfig
@@ -0,0 +1,19 @@
+# SPDX-License-Identifier: GPL-2.0
+config CLK_INTEL_SOCFPGA
+ bool "Intel SoCFPGA family clock support" if COMPILE_TEST && !ARCH_INTEL_SOCFPGA
+ default ARCH_INTEL_SOCFPGA
+ help
+ Support for the clock controllers present on Intel SoCFPGA and eASIC
+ devices like Aria, Cyclone, Stratix 10, Agilex and N5X eASIC.
+
+if CLK_INTEL_SOCFPGA
+
+config CLK_INTEL_SOCFPGA32
+ bool "Intel Aria / Cyclone clock controller support" if COMPILE_TEST && (!ARM || !ARCH_INTEL_SOCFPGA)
+ default ARM && ARCH_INTEL_SOCFPGA
+
+config CLK_INTEL_SOCFPGA64
+ bool "Intel Stratix / Agilex / N5X clock controller support" if COMPILE_TEST && (!ARM64 || !ARCH_INTEL_SOCFPGA)
+ default ARM64 && ARCH_INTEL_SOCFPGA
+
+endif # CLK_INTEL_SOCFPGA
diff --git a/drivers/clk/socfpga/Makefile b/drivers/clk/socfpga/Makefile
index bf736f8d201a..e8dfce339c91 100644
--- a/drivers/clk/socfpga/Makefile
+++ b/drivers/clk/socfpga/Makefile
@@ -1,7 +1,6 @@
# SPDX-License-Identifier: GPL-2.0
-obj-$(CONFIG_ARCH_SOCFPGA) += clk.o clk-gate.o clk-pll.o clk-periph.o
-obj-$(CONFIG_ARCH_SOCFPGA) += clk-pll-a10.o clk-periph-a10.o clk-gate-a10.o
-obj-$(CONFIG_ARCH_STRATIX10) += clk-s10.o
-obj-$(CONFIG_ARCH_STRATIX10) += clk-pll-s10.o clk-periph-s10.o clk-gate-s10.o
-obj-$(CONFIG_ARCH_AGILEX) += clk-agilex.o
-obj-$(CONFIG_ARCH_AGILEX) += clk-pll-s10.o clk-periph-s10.o clk-gate-s10.o
+obj-$(CONFIG_CLK_INTEL_SOCFPGA32) += clk.o clk-gate.o clk-pll.o clk-periph.o \
+ clk-pll-a10.o clk-periph-a10.o clk-gate-a10.o
+obj-$(CONFIG_CLK_INTEL_SOCFPGA64) += clk-s10.o \
+ clk-pll-s10.o clk-periph-s10.o clk-gate-s10.o \
+ clk-agilex.o
diff --git a/drivers/clk/socfpga/clk-agilex.c b/drivers/clk/socfpga/clk-agilex.c
index 7689bdd0a914..92a6d740a799 100644
--- a/drivers/clk/socfpga/clk-agilex.c
+++ b/drivers/clk/socfpga/clk-agilex.c
@@ -303,18 +303,18 @@ static const struct stratix10_gate_clock agilex_gate_clks[] = {
static int n5x_clk_register_c_perip(const struct n5x_perip_c_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = n5x_register_periph(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = n5x_register_periph(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
}
@@ -322,18 +322,18 @@ static int n5x_clk_register_c_perip(const struct n5x_perip_c_clock *clks,
static int agilex_clk_register_c_perip(const struct stratix10_perip_c_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = s10_register_periph(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = s10_register_periph(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
}
@@ -341,37 +341,38 @@ static int agilex_clk_register_c_perip(const struct stratix10_perip_c_clock *clk
static int agilex_clk_register_cnt_perip(const struct stratix10_perip_cnt_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = s10_register_cnt_periph(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = s10_register_cnt_periph(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
}
-static int agilex_clk_register_gate(const struct stratix10_gate_clock *clks, int nums, struct stratix10_clock_data *data)
+static int agilex_clk_register_gate(const struct stratix10_gate_clock *clks,
+ int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = s10_register_gate(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = s10_register_gate(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
@@ -380,18 +381,18 @@ static int agilex_clk_register_gate(const struct stratix10_gate_clock *clks,
static int agilex_clk_register_pll(const struct stratix10_pll_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = agilex_register_pll(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = agilex_register_pll(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
@@ -400,64 +401,49 @@ static int agilex_clk_register_pll(const struct stratix10_pll_clock *clks,
static int n5x_clk_register_pll(const struct stratix10_pll_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = n5x_register_pll(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = n5x_register_pll(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
}
-static struct stratix10_clock_data *__socfpga_agilex_clk_init(struct platform_device *pdev,
- int nr_clks)
+static int agilex_clkmgr_init(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct device *dev = &pdev->dev;
struct stratix10_clock_data *clk_data;
- struct clk **clk_table;
struct resource *res;
void __iomem *base;
- int ret;
+ int i, num_clks;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
base = devm_ioremap_resource(dev, res);
if (IS_ERR(base))
- return ERR_CAST(base);
-
- clk_data = devm_kzalloc(dev, sizeof(*clk_data), GFP_KERNEL);
- if (!clk_data)
- return ERR_PTR(-ENOMEM);
+ return PTR_ERR(base);
- clk_data->base = base;
- clk_table = devm_kcalloc(dev, nr_clks, sizeof(*clk_table), GFP_KERNEL);
- if (!clk_table)
- return ERR_PTR(-ENOMEM);
-
- clk_data->clk_data.clks = clk_table;
- clk_data->clk_data.clk_num = nr_clks;
- ret = of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data->clk_data);
- if (ret)
- return ERR_PTR(ret);
+ num_clks = AGILEX_NUM_CLKS;
- return clk_data;
-}
+ clk_data = devm_kzalloc(dev, struct_size(clk_data, clk_data.hws,
+ num_clks), GFP_KERNEL);
+ if (!clk_data)
+ return -ENOMEM;
-static int agilex_clkmgr_init(struct platform_device *pdev)
-{
- struct stratix10_clock_data *clk_data;
+ for (i = 0; i < num_clks; i++)
+ clk_data->clk_data.hws[i] = ERR_PTR(-ENOENT);
- clk_data = __socfpga_agilex_clk_init(pdev, AGILEX_NUM_CLKS);
- if (IS_ERR(clk_data))
- return PTR_ERR(clk_data);
+ clk_data->base = base;
+ clk_data->clk_data.num = num_clks;
agilex_clk_register_pll(agilex_pll_clks, ARRAY_SIZE(agilex_pll_clks), clk_data);
@@ -470,16 +456,36 @@ static int agilex_clkmgr_init(struct platform_device *pdev)
agilex_clk_register_gate(agilex_gate_clks, ARRAY_SIZE(agilex_gate_clks),
clk_data);
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, &clk_data->clk_data);
return 0;
}
static int n5x_clkmgr_init(struct platform_device *pdev)
{
+ struct device_node *np = pdev->dev.of_node;
+ struct device *dev = &pdev->dev;
struct stratix10_clock_data *clk_data;
+ struct resource *res;
+ void __iomem *base;
+ int i, num_clks;
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ base = devm_ioremap_resource(dev, res);
+ if (IS_ERR(base))
+ return PTR_ERR(base);
+
+ num_clks = AGILEX_NUM_CLKS;
+
+ clk_data = devm_kzalloc(dev, struct_size(clk_data, clk_data.hws,
+ num_clks), GFP_KERNEL);
+ if (!clk_data)
+ return -ENOMEM;
+
+ for (i = 0; i < num_clks; i++)
+ clk_data->clk_data.hws[i] = ERR_PTR(-ENOENT);
- clk_data = __socfpga_agilex_clk_init(pdev, AGILEX_NUM_CLKS);
- if (IS_ERR(clk_data))
- return PTR_ERR(clk_data);
+ clk_data->base = base;
+ clk_data->clk_data.num = num_clks;
n5x_clk_register_pll(agilex_pll_clks, ARRAY_SIZE(agilex_pll_clks), clk_data);
@@ -492,6 +498,7 @@ static int n5x_clkmgr_init(struct platform_device *pdev)
agilex_clk_register_gate(agilex_gate_clks, ARRAY_SIZE(agilex_gate_clks),
clk_data);
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, &clk_data->clk_data);
return 0;
}
diff --git a/drivers/clk/socfpga/clk-gate-a10.c b/drivers/clk/socfpga/clk-gate-a10.c
index cd5df9103614..738c53391e39 100644
--- a/drivers/clk/socfpga/clk-gate-a10.c
+++ b/drivers/clk/socfpga/clk-gate-a10.c
@@ -98,7 +98,7 @@ static void __init __socfpga_gate_init(struct device_node *node,
u32 div_reg[3];
u32 clk_phase[2];
u32 fixed_div;
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_gate_clk *socfpga_clk;
const char *clk_name = node->name;
const char *parent_name[SOCFPGA_MAX_PARENTS];
@@ -146,6 +146,7 @@ static void __init __socfpga_gate_init(struct device_node *node,
if (IS_ERR(socfpga_clk->sys_mgr_base_addr)) {
pr_err("%s: failed to find altr,sys-mgr regmap!\n",
__func__);
+ kfree(socfpga_clk);
return;
}
}
@@ -159,13 +160,13 @@ static void __init __socfpga_gate_init(struct device_node *node,
init.num_parents = of_clk_parent_fill(node, parent_name, SOCFPGA_MAX_PARENTS);
init.parent_names = parent_name;
socfpga_clk->hw.hw.init = &init;
+ hw_clk = &socfpga_clk->hw.hw;
- clk = clk_register(NULL, &socfpga_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ if (clk_hw_register(NULL, hw_clk)) {
kfree(socfpga_clk);
return;
}
- rc = of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ rc = of_clk_add_provider(node, of_clk_src_simple_get, hw_clk);
if (WARN_ON(rc))
return;
}
diff --git a/drivers/clk/socfpga/clk-gate-s10.c b/drivers/clk/socfpga/clk-gate-s10.c
index 083b2ec21fdd..b84f2627551e 100644
--- a/drivers/clk/socfpga/clk-gate-s10.c
+++ b/drivers/clk/socfpga/clk-gate-s10.c
@@ -31,7 +31,7 @@ static unsigned long socfpga_dbg_clk_recalc_rate(struct clk_hw *hwclk,
unsigned long parent_rate)
{
struct socfpga_gate_clk *socfpgaclk = to_socfpga_gate_clk(hwclk);
- u32 div = 1, val;
+ u32 div, val;
val = readl(socfpgaclk->div_reg) >> socfpgaclk->shift;
val &= GENMASK(socfpgaclk->width - 1, 0);
@@ -65,12 +65,13 @@ static const struct clk_ops dbgclk_ops = {
.get_parent = socfpga_gate_get_parent,
};
-struct clk *s10_register_gate(const struct stratix10_gate_clock *clks, void __iomem *regbase)
+struct clk_hw *s10_register_gate(const struct stratix10_gate_clock *clks, void __iomem *regbase)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_gate_clk *socfpga_clk;
struct clk_init_data init;
const char *parent_name = clks->parent_name;
+ int ret;
socfpga_clk = kzalloc(sizeof(*socfpga_clk), GFP_KERNEL);
if (!socfpga_clk)
@@ -112,10 +113,12 @@ struct clk *s10_register_gate(const struct stratix10_gate_clock *clks, void __io
init.parent_data = clks->parent_data;
socfpga_clk->hw.hw.init = &init;
- clk = clk_register(NULL, &socfpga_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ hw_clk = &socfpga_clk->hw.hw;
+
+ ret = clk_hw_register(NULL, &socfpga_clk->hw.hw);
+ if (ret) {
kfree(socfpga_clk);
- return NULL;
+ return ERR_PTR(ret);
}
- return clk;
+ return hw_clk;
}
diff --git a/drivers/clk/socfpga/clk-gate.c b/drivers/clk/socfpga/clk-gate.c
index 43ecd507bf83..1ec9678d8cd3 100644
--- a/drivers/clk/socfpga/clk-gate.c
+++ b/drivers/clk/socfpga/clk-gate.c
@@ -99,7 +99,7 @@ static unsigned long socfpga_clk_recalc_rate(struct clk_hw *hwclk,
val = readl(socfpgaclk->div_reg) >> socfpgaclk->shift;
val &= GENMASK(socfpgaclk->width - 1, 0);
/* Check for GPIO_DB_CLK by its offset */
- if ((int) socfpgaclk->div_reg & SOCFPGA_GPIO_DB_CLK_OFFSET)
+ if ((uintptr_t) socfpgaclk->div_reg & SOCFPGA_GPIO_DB_CLK_OFFSET)
div = val + 1;
else
div = (1 << val);
@@ -174,13 +174,14 @@ void __init socfpga_gate_init(struct device_node *node)
u32 div_reg[3];
u32 clk_phase[2];
u32 fixed_div;
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_gate_clk *socfpga_clk;
const char *clk_name = node->name;
const char *parent_name[SOCFPGA_MAX_PARENTS];
struct clk_init_data init;
struct clk_ops *ops;
int rc;
+ int err;
socfpga_clk = kzalloc(sizeof(*socfpga_clk), GFP_KERNEL);
if (WARN_ON(!socfpga_clk))
@@ -238,12 +239,14 @@ void __init socfpga_gate_init(struct device_node *node)
init.parent_names = parent_name;
socfpga_clk->hw.hw.init = &init;
- clk = clk_register(NULL, &socfpga_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ hw_clk = &socfpga_clk->hw.hw;
+
+ err = clk_hw_register(NULL, hw_clk);
+ if (err) {
kfree(socfpga_clk);
return;
}
- rc = of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ rc = of_clk_add_provider(node, of_clk_src_simple_get, hw_clk);
if (WARN_ON(rc))
return;
}
diff --git a/drivers/clk/socfpga/clk-periph-a10.c b/drivers/clk/socfpga/clk-periph-a10.c
index 3e0c55727b89..b9cdde4b8441 100644
--- a/drivers/clk/socfpga/clk-periph-a10.c
+++ b/drivers/clk/socfpga/clk-periph-a10.c
@@ -61,7 +61,7 @@ static __init void __socfpga_periph_init(struct device_node *node,
const struct clk_ops *ops)
{
u32 reg;
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_periph_clk *periph_clk;
const char *clk_name = node->name;
const char *parent_name[SOCFPGA_MAX_PARENTS];
@@ -104,12 +104,13 @@ static __init void __socfpga_periph_init(struct device_node *node,
periph_clk->hw.hw.init = &init;
- clk = clk_register(NULL, &periph_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ hw_clk = &periph_clk->hw.hw;
+
+ if (clk_hw_register(NULL, hw_clk)) {
kfree(periph_clk);
return;
}
- rc = of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ rc = of_clk_add_provider(node, of_clk_src_simple_get, hw_clk);
if (rc < 0) {
pr_err("Could not register clock provider for node:%s\n",
clk_name);
@@ -119,7 +120,7 @@ static __init void __socfpga_periph_init(struct device_node *node,
return;
err_clk:
- clk_unregister(clk);
+ clk_hw_unregister(hw_clk);
}
void __init socfpga_a10_periph_init(struct device_node *node)
diff --git a/drivers/clk/socfpga/clk-periph-s10.c b/drivers/clk/socfpga/clk-periph-s10.c
index 0ff2b9d24035..e5a5fef76df7 100644
--- a/drivers/clk/socfpga/clk-periph-s10.c
+++ b/drivers/clk/socfpga/clk-periph-s10.c
@@ -93,14 +93,15 @@ static const struct clk_ops peri_cnt_clk_ops = {
.get_parent = clk_periclk_get_parent,
};
-struct clk *s10_register_periph(const struct stratix10_perip_c_clock *clks,
+struct clk_hw *s10_register_periph(const struct stratix10_perip_c_clock *clks,
void __iomem *reg)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_periph_clk *periph_clk;
struct clk_init_data init;
const char *name = clks->name;
const char *parent_name = clks->parent_name;
+ int ret;
periph_clk = kzalloc(sizeof(*periph_clk), GFP_KERNEL);
if (WARN_ON(!periph_clk))
@@ -118,23 +119,25 @@ struct clk *s10_register_periph(const struct stratix10_perip_c_clock *clks,
init.parent_data = clks->parent_data;
periph_clk->hw.hw.init = &init;
+ hw_clk = &periph_clk->hw.hw;
- clk = clk_register(NULL, &periph_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ ret = clk_hw_register(NULL, hw_clk);
+ if (ret) {
kfree(periph_clk);
- return NULL;
+ return ERR_PTR(ret);
}
- return clk;
+ return hw_clk;
}
-struct clk *n5x_register_periph(const struct n5x_perip_c_clock *clks,
+struct clk_hw *n5x_register_periph(const struct n5x_perip_c_clock *clks,
void __iomem *regbase)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_periph_clk *periph_clk;
struct clk_init_data init;
const char *name = clks->name;
const char *parent_name = clks->parent_name;
+ int ret;
periph_clk = kzalloc(sizeof(*periph_clk), GFP_KERNEL);
if (WARN_ON(!periph_clk))
@@ -151,23 +154,25 @@ struct clk *n5x_register_periph(const struct n5x_perip_c_clock *clks,
init.parent_names = parent_name ? &parent_name : NULL;
periph_clk->hw.hw.init = &init;
+ hw_clk = &periph_clk->hw.hw;
- clk = clk_register(NULL, &periph_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ ret = clk_hw_register(NULL, hw_clk);
+ if (ret) {
kfree(periph_clk);
- return NULL;
+ return ERR_PTR(ret);
}
- return clk;
+ return hw_clk;
}
-struct clk *s10_register_cnt_periph(const struct stratix10_perip_cnt_clock *clks,
+struct clk_hw *s10_register_cnt_periph(const struct stratix10_perip_cnt_clock *clks,
void __iomem *regbase)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_periph_clk *periph_clk;
struct clk_init_data init;
const char *name = clks->name;
const char *parent_name = clks->parent_name;
+ int ret;
periph_clk = kzalloc(sizeof(*periph_clk), GFP_KERNEL);
if (WARN_ON(!periph_clk))
@@ -195,11 +200,12 @@ struct clk *s10_register_cnt_periph(const struct stratix10_perip_cnt_clock *clks
init.parent_data = clks->parent_data;
periph_clk->hw.hw.init = &init;
+ hw_clk = &periph_clk->hw.hw;
- clk = clk_register(NULL, &periph_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ ret = clk_hw_register(NULL, hw_clk);
+ if (ret) {
kfree(periph_clk);
- return NULL;
+ return ERR_PTR(ret);
}
- return clk;
+ return hw_clk;
}
diff --git a/drivers/clk/socfpga/clk-periph.c b/drivers/clk/socfpga/clk-periph.c
index 5e0c4b45f77f..43707e2d7248 100644
--- a/drivers/clk/socfpga/clk-periph.c
+++ b/drivers/clk/socfpga/clk-periph.c
@@ -51,7 +51,7 @@ static __init void __socfpga_periph_init(struct device_node *node,
const struct clk_ops *ops)
{
u32 reg;
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_periph_clk *periph_clk;
const char *clk_name = node->name;
const char *parent_name[SOCFPGA_MAX_PARENTS];
@@ -94,13 +94,13 @@ static __init void __socfpga_periph_init(struct device_node *node,
init.parent_names = parent_name;
periph_clk->hw.hw.init = &init;
+ hw_clk = &periph_clk->hw.hw;
- clk = clk_register(NULL, &periph_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ if (clk_hw_register(NULL, hw_clk)) {
kfree(periph_clk);
return;
}
- rc = of_clk_add_provider(node, of_clk_src_simple_get, clk);
+ rc = of_clk_add_provider(node, of_clk_src_simple_get, hw_clk);
}
void __init socfpga_periph_init(struct device_node *node)
diff --git a/drivers/clk/socfpga/clk-pll-a10.c b/drivers/clk/socfpga/clk-pll-a10.c
index 3338f054fe98..bee0f7da5b6e 100644
--- a/drivers/clk/socfpga/clk-pll-a10.c
+++ b/drivers/clk/socfpga/clk-pll-a10.c
@@ -63,11 +63,11 @@ static const struct clk_ops clk_pll_ops = {
.get_parent = clk_pll_get_parent,
};
-static struct clk * __init __socfpga_pll_init(struct device_node *node,
+static struct clk_hw * __init __socfpga_pll_init(struct device_node *node,
const struct clk_ops *ops)
{
u32 reg;
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_pll *pll_clk;
const char *clk_name = node->name;
const char *parent_name[SOCFGPA_MAX_PARENTS];
@@ -101,14 +101,14 @@ static struct clk * __init __socfpga_pll_init(struct device_node *node,
pll_clk->hw.hw.init = &init;
pll_clk->hw.bit_idx = SOCFPGA_PLL_EXT_ENA;
+ hw_clk = &pll_clk->hw.hw;
- clk = clk_register(NULL, &pll_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ if (clk_hw_register(NULL, hw_clk)) {
kfree(pll_clk);
return NULL;
}
- of_clk_add_provider(node, of_clk_src_simple_get, clk);
- return clk;
+ of_clk_add_provider(node, of_clk_src_simple_get, hw_clk);
+ return hw_clk;
}
void __init socfpga_a10_pll_init(struct device_node *node)
diff --git a/drivers/clk/socfpga/clk-pll-s10.c b/drivers/clk/socfpga/clk-pll-s10.c
index f6f66e08e1f4..70076a80149d 100644
--- a/drivers/clk/socfpga/clk-pll-s10.c
+++ b/drivers/clk/socfpga/clk-pll-s10.c
@@ -107,7 +107,7 @@ static unsigned long clk_boot_clk_recalc_rate(struct clk_hw *hwclk,
unsigned long parent_rate)
{
struct socfpga_pll *socfpgaclk = to_socfpga_clk(hwclk);
- u32 div = 1;
+ u32 div;
div = ((readl(socfpgaclk->hw.reg) &
SWCTRLBTCLKSEL_MASK) >>
@@ -187,13 +187,14 @@ static const struct clk_ops clk_boot_ops = {
.prepare = clk_pll_prepare,
};
-struct clk *s10_register_pll(const struct stratix10_pll_clock *clks,
+struct clk_hw *s10_register_pll(const struct stratix10_pll_clock *clks,
void __iomem *reg)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_pll *pll_clk;
struct clk_init_data init;
const char *name = clks->name;
+ int ret;
pll_clk = kzalloc(sizeof(*pll_clk), GFP_KERNEL);
if (WARN_ON(!pll_clk))
@@ -216,21 +217,24 @@ struct clk *s10_register_pll(const struct stratix10_pll_clock *clks,
pll_clk->hw.bit_idx = SOCFPGA_PLL_POWER;
- clk = clk_register(NULL, &pll_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ hw_clk = &pll_clk->hw.hw;
+
+ ret = clk_hw_register(NULL, hw_clk);
+ if (ret) {
kfree(pll_clk);
- return NULL;
+ return ERR_PTR(ret);
}
- return clk;
+ return hw_clk;
}
-struct clk *agilex_register_pll(const struct stratix10_pll_clock *clks,
+struct clk_hw *agilex_register_pll(const struct stratix10_pll_clock *clks,
void __iomem *reg)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_pll *pll_clk;
struct clk_init_data init;
const char *name = clks->name;
+ int ret;
pll_clk = kzalloc(sizeof(*pll_clk), GFP_KERNEL);
if (WARN_ON(!pll_clk))
@@ -252,22 +256,24 @@ struct clk *agilex_register_pll(const struct stratix10_pll_clock *clks,
pll_clk->hw.hw.init = &init;
pll_clk->hw.bit_idx = SOCFPGA_PLL_POWER;
+ hw_clk = &pll_clk->hw.hw;
- clk = clk_register(NULL, &pll_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ ret = clk_hw_register(NULL, hw_clk);
+ if (ret) {
kfree(pll_clk);
- return NULL;
+ return ERR_PTR(ret);
}
- return clk;
+ return hw_clk;
}
-struct clk *n5x_register_pll(const struct stratix10_pll_clock *clks,
+struct clk_hw *n5x_register_pll(const struct stratix10_pll_clock *clks,
void __iomem *reg)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_pll *pll_clk;
struct clk_init_data init;
const char *name = clks->name;
+ int ret;
pll_clk = kzalloc(sizeof(*pll_clk), GFP_KERNEL);
if (WARN_ON(!pll_clk))
@@ -289,11 +295,12 @@ struct clk *n5x_register_pll(const struct stratix10_pll_clock *clks,
pll_clk->hw.hw.init = &init;
pll_clk->hw.bit_idx = SOCFPGA_PLL_POWER;
+ hw_clk = &pll_clk->hw.hw;
- clk = clk_register(NULL, &pll_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ ret = clk_hw_register(NULL, hw_clk);
+ if (ret) {
kfree(pll_clk);
- return NULL;
+ return ERR_PTR(ret);
}
- return clk;
+ return hw_clk;
}
diff --git a/drivers/clk/socfpga/clk-pll.c b/drivers/clk/socfpga/clk-pll.c
index 3cf99df7d005..dcb573d44034 100644
--- a/drivers/clk/socfpga/clk-pll.c
+++ b/drivers/clk/socfpga/clk-pll.c
@@ -70,16 +70,18 @@ static const struct clk_ops clk_pll_ops = {
.get_parent = clk_pll_get_parent,
};
-static __init struct clk *__socfpga_pll_init(struct device_node *node,
+static __init struct clk_hw *__socfpga_pll_init(struct device_node *node,
const struct clk_ops *ops)
{
u32 reg;
- struct clk *clk;
+ struct clk_hw *hw_clk;
struct socfpga_pll *pll_clk;
const char *clk_name = node->name;
const char *parent_name[SOCFPGA_MAX_PARENTS];
struct clk_init_data init;
struct device_node *clkmgr_np;
+ int rc;
+ int err;
of_property_read_u32(node, "reg", &reg);
@@ -105,13 +107,15 @@ static __init struct clk *__socfpga_pll_init(struct device_node *node,
pll_clk->hw.bit_idx = SOCFPGA_PLL_EXT_ENA;
- clk = clk_register(NULL, &pll_clk->hw.hw);
- if (WARN_ON(IS_ERR(clk))) {
+ hw_clk = &pll_clk->hw.hw;
+
+ err = clk_hw_register(NULL, hw_clk);
+ if (err) {
kfree(pll_clk);
- return NULL;
+ return ERR_PTR(err);
}
- of_clk_add_provider(node, of_clk_src_simple_get, clk);
- return clk;
+ rc = of_clk_add_provider(node, of_clk_src_simple_get, hw_clk);
+ return hw_clk;
}
void __init socfpga_pll_init(struct device_node *node)
diff --git a/drivers/clk/socfpga/clk-s10.c b/drivers/clk/socfpga/clk-s10.c
index 661a8e9bfb9b..f0bd77138ecb 100644
--- a/drivers/clk/socfpga/clk-s10.c
+++ b/drivers/clk/socfpga/clk-s10.c
@@ -274,18 +274,18 @@ static const struct stratix10_gate_clock s10_gate_clks[] = {
static int s10_clk_register_c_perip(const struct stratix10_perip_c_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = s10_register_periph(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = s10_register_periph(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
}
@@ -293,18 +293,18 @@ static int s10_clk_register_c_perip(const struct stratix10_perip_c_clock *clks,
static int s10_clk_register_cnt_perip(const struct stratix10_perip_cnt_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = s10_register_cnt_periph(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = s10_register_cnt_periph(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
@@ -313,18 +313,18 @@ static int s10_clk_register_cnt_perip(const struct stratix10_perip_cnt_clock *cl
static int s10_clk_register_gate(const struct stratix10_gate_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = s10_register_gate(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = s10_register_gate(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
@@ -333,62 +333,50 @@ static int s10_clk_register_gate(const struct stratix10_gate_clock *clks,
static int s10_clk_register_pll(const struct stratix10_pll_clock *clks,
int nums, struct stratix10_clock_data *data)
{
- struct clk *clk;
+ struct clk_hw *hw_clk;
void __iomem *base = data->base;
int i;
for (i = 0; i < nums; i++) {
- clk = s10_register_pll(&clks[i], base);
- if (IS_ERR(clk)) {
+ hw_clk = s10_register_pll(&clks[i], base);
+ if (IS_ERR(hw_clk)) {
pr_err("%s: failed to register clock %s\n",
__func__, clks[i].name);
continue;
}
- data->clk_data.clks[clks[i].id] = clk;
+ data->clk_data.hws[clks[i].id] = hw_clk;
}
return 0;
}
-static struct stratix10_clock_data *__socfpga_s10_clk_init(struct platform_device *pdev,
- int nr_clks)
+static int s10_clkmgr_init(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct device *dev = &pdev->dev;
struct stratix10_clock_data *clk_data;
- struct clk **clk_table;
struct resource *res;
void __iomem *base;
+ int i, num_clks;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
base = devm_ioremap_resource(dev, res);
if (IS_ERR(base)) {
pr_err("%s: failed to map clock registers\n", __func__);
- return ERR_CAST(base);
+ return PTR_ERR(base);
}
- clk_data = devm_kzalloc(dev, sizeof(*clk_data), GFP_KERNEL);
+ num_clks = STRATIX10_NUM_CLKS;
+ clk_data = devm_kzalloc(dev, struct_size(clk_data, clk_data.hws,
+ num_clks), GFP_KERNEL);
if (!clk_data)
- return ERR_PTR(-ENOMEM);
+ return -ENOMEM;
- clk_data->base = base;
- clk_table = devm_kcalloc(dev, nr_clks, sizeof(*clk_table), GFP_KERNEL);
- if (!clk_table)
- return ERR_PTR(-ENOMEM);
-
- clk_data->clk_data.clks = clk_table;
- clk_data->clk_data.clk_num = nr_clks;
- of_clk_add_provider(np, of_clk_src_onecell_get, &clk_data->clk_data);
- return clk_data;
-}
+ for (i = 0; i < num_clks; i++)
+ clk_data->clk_data.hws[i] = ERR_PTR(-ENOENT);
-static int s10_clkmgr_init(struct platform_device *pdev)
-{
- struct stratix10_clock_data *clk_data;
-
- clk_data = __socfpga_s10_clk_init(pdev, STRATIX10_NUM_CLKS);
- if (IS_ERR(clk_data))
- return PTR_ERR(clk_data);
+ clk_data->base = base;
+ clk_data->clk_data.num = num_clks;
s10_clk_register_pll(s10_pll_clks, ARRAY_SIZE(s10_pll_clks), clk_data);
@@ -401,6 +389,8 @@ static int s10_clkmgr_init(struct platform_device *pdev)
s10_clk_register_gate(s10_gate_clks, ARRAY_SIZE(s10_gate_clks),
clk_data);
+
+ of_clk_add_hw_provider(np, of_clk_hw_onecell_get, &clk_data->clk_data);
return 0;
}
diff --git a/drivers/clk/socfpga/stratix10-clk.h b/drivers/clk/socfpga/stratix10-clk.h
index 420deed677ce..61eaf3a41fbb 100644
--- a/drivers/clk/socfpga/stratix10-clk.h
+++ b/drivers/clk/socfpga/stratix10-clk.h
@@ -7,7 +7,7 @@
#define __STRATIX10_CLK_H
struct stratix10_clock_data {
- struct clk_onecell_data clk_data;
+ struct clk_hw_onecell_data clk_data;
void __iomem *base;
};
@@ -71,18 +71,18 @@ struct stratix10_gate_clock {
u8 fixed_div;
};
-struct clk *s10_register_pll(const struct stratix10_pll_clock *,
- void __iomem *);
-struct clk *agilex_register_pll(const struct stratix10_pll_clock *,
- void __iomem *);
-struct clk *n5x_register_pll(const struct stratix10_pll_clock *clks,
+struct clk_hw *s10_register_pll(const struct stratix10_pll_clock *clks,
void __iomem *reg);
-struct clk *s10_register_periph(const struct stratix10_perip_c_clock *,
+struct clk_hw *agilex_register_pll(const struct stratix10_pll_clock *clks,
void __iomem *reg);
-struct clk *n5x_register_periph(const struct n5x_perip_c_clock *clks,
+struct clk_hw *n5x_register_pll(const struct stratix10_pll_clock *clks,
+ void __iomem *reg);
+struct clk_hw *s10_register_periph(const struct stratix10_perip_c_clock *clks,
+ void __iomem *reg);
+struct clk_hw *n5x_register_periph(const struct n5x_perip_c_clock *clks,
void __iomem *reg);
-struct clk *s10_register_cnt_periph(const struct stratix10_perip_cnt_clock *,
- void __iomem *);
-struct clk *s10_register_gate(const struct stratix10_gate_clock *,
- void __iomem *);
+struct clk_hw *s10_register_cnt_periph(const struct stratix10_perip_cnt_clock *clks,
+ void __iomem *reg);
+struct clk_hw *s10_register_gate(const struct stratix10_gate_clock *clks,
+ void __iomem *reg);
#endif /* __STRATIX10_CLK_H */
diff --git a/drivers/clk/sunxi-ng/ccu-sun8i-v3s.c b/drivers/clk/sunxi-ng/ccu-sun8i-v3s.c
index 0e36ca3bf3d5..a774942cb153 100644
--- a/drivers/clk/sunxi-ng/ccu-sun8i-v3s.c
+++ b/drivers/clk/sunxi-ng/ccu-sun8i-v3s.c
@@ -40,18 +40,29 @@ static SUNXI_CCU_NKMP_WITH_GATE_LOCK(pll_cpu_clk, "pll-cpu",
* the base (2x, 4x and 8x), and one variable divider (the one true
* pll audio).
*
- * We don't have any need for the variable divider for now, so we just
- * hardcode it to match with the clock names
+ * With sigma-delta modulation for fractional-N on the audio PLL,
+ * we have to use specific dividers. This means the variable divider
+ * can no longer be used, as the audio codec requests the exact clock
+ * rates we support through this mechanism. So we now hard code the
+ * variable divider to 1. This means the clock rates will no longer
+ * match the clock names.
*/
#define SUN8I_V3S_PLL_AUDIO_REG 0x008
-static SUNXI_CCU_NM_WITH_GATE_LOCK(pll_audio_base_clk, "pll-audio-base",
- "osc24M", 0x008,
- 8, 7, /* N */
- 0, 5, /* M */
- BIT(31), /* gate */
- BIT(28), /* lock */
- 0);
+static struct ccu_sdm_setting pll_audio_sdm_table[] = {
+ { .rate = 22579200, .pattern = 0xc0010d84, .m = 8, .n = 7 },
+ { .rate = 24576000, .pattern = 0xc000ac02, .m = 14, .n = 14 },
+};
+
+static SUNXI_CCU_NM_WITH_SDM_GATE_LOCK(pll_audio_base_clk, "pll-audio-base",
+ "osc24M", 0x008,
+ 8, 7, /* N */
+ 0, 5, /* M */
+ pll_audio_sdm_table, BIT(24),
+ 0x284, BIT(31),
+ BIT(31), /* gate */
+ BIT(28), /* lock */
+ CLK_SET_RATE_UNGATE);
static SUNXI_CCU_NM_WITH_FRAC_GATE_LOCK(pll_video_clk, "pll-video",
"osc24M", 0x0010,
@@ -524,10 +535,10 @@ static struct ccu_common *sun8i_v3_ccu_clks[] = {
&mipi_csi_clk.common,
};
-/* We hardcode the divider to 4 for now */
+/* We hardcode the divider to 1 for SDM support */
static CLK_FIXED_FACTOR_HWS(pll_audio_clk, "pll-audio",
clk_parent_pll_audio,
- 4, 1, CLK_SET_RATE_PARENT);
+ 1, 1, CLK_SET_RATE_PARENT);
static CLK_FIXED_FACTOR_HWS(pll_audio_2x_clk, "pll-audio-2x",
clk_parent_pll_audio,
2, 1, CLK_SET_RATE_PARENT);
diff --git a/drivers/clk/sunxi/clk-sun6i-ar100.c b/drivers/clk/sunxi/clk-sun6i-ar100.c
index e1b7d0929cf7..54babc2b4b9e 100644
--- a/drivers/clk/sunxi/clk-sun6i-ar100.c
+++ b/drivers/clk/sunxi/clk-sun6i-ar100.c
@@ -16,7 +16,7 @@
#include "clk-factors.h"
-/**
+/*
* sun6i_get_ar100_factors - Calculates factors p, m for AR100
*
* AR100 rate is calculated as follows
diff --git a/drivers/clk/sunxi/clk-sun9i-core.c b/drivers/clk/sunxi/clk-sun9i-core.c
index 305ad78d5fb4..d93c7a53c6c0 100644
--- a/drivers/clk/sunxi/clk-sun9i-core.c
+++ b/drivers/clk/sunxi/clk-sun9i-core.c
@@ -14,7 +14,7 @@
#include "clk-factors.h"
-/**
+/*
* sun9i_a80_get_pll4_factors() - calculates n, p, m factors for PLL4
* PLL4 rate is calculated as follows
* rate = (parent_rate * n >> p) / (m + 1);
@@ -90,7 +90,7 @@ static void __init sun9i_a80_pll4_setup(struct device_node *node)
CLK_OF_DECLARE(sun9i_a80_pll4, "allwinner,sun9i-a80-pll4-clk", sun9i_a80_pll4_setup);
-/**
+/*
* sun9i_a80_get_gt_factors() - calculates m factor for GT
* GT rate is calculated as follows
* rate = parent_rate / (m + 1);
@@ -145,7 +145,7 @@ static void __init sun9i_a80_gt_setup(struct device_node *node)
CLK_OF_DECLARE(sun9i_a80_gt, "allwinner,sun9i-a80-gt-clk", sun9i_a80_gt_setup);
-/**
+/*
* sun9i_a80_get_ahb_factors() - calculates p factor for AHB0/1/2
* AHB rate is calculated as follows
* rate = parent_rate >> p;
@@ -225,7 +225,7 @@ static void __init sun9i_a80_apb0_setup(struct device_node *node)
CLK_OF_DECLARE(sun9i_a80_apb0, "allwinner,sun9i-a80-apb0-clk", sun9i_a80_apb0_setup);
-/**
+/*
* sun9i_a80_get_apb1_factors() - calculates m, p factors for APB1
* APB1 rate is calculated as follows
* rate = (parent_rate >> p) / (m + 1);
diff --git a/drivers/clk/sunxi/clk-usb.c b/drivers/clk/sunxi/clk-usb.c
index d78a78495bed..5460218f3467 100644
--- a/drivers/clk/sunxi/clk-usb.c
+++ b/drivers/clk/sunxi/clk-usb.c
@@ -15,7 +15,7 @@
#include <linux/spinlock.h>
-/**
+/*
* sunxi_usb_reset... - reset bits in usb clk registers handling
*/
diff --git a/drivers/clk/tegra/clk-pll.c b/drivers/clk/tegra/clk-pll.c
index c5cc0a2dac6f..0193cebe8c5a 100644
--- a/drivers/clk/tegra/clk-pll.c
+++ b/drivers/clk/tegra/clk-pll.c
@@ -2515,18 +2515,6 @@ static int clk_plle_tegra210_enable(struct clk_hw *hw)
pll_writel(val, PLLE_SS_CTRL, pll);
udelay(1);
- val = pll_readl_misc(pll);
- val &= ~PLLE_MISC_IDDQ_SW_CTRL;
- pll_writel_misc(val, pll);
-
- val = pll_readl(pll->params->aux_reg, pll);
- val |= (PLLE_AUX_USE_LOCKDET | PLLE_AUX_SS_SEQ_INCLUDE);
- val &= ~(PLLE_AUX_ENABLE_SWCTL | PLLE_AUX_SS_SWCTL);
- pll_writel(val, pll->params->aux_reg, pll);
- udelay(1);
- val |= PLLE_AUX_SEQ_ENABLE;
- pll_writel(val, pll->params->aux_reg, pll);
-
out:
if (pll->lock)
spin_unlock_irqrestore(pll->lock, flags);
diff --git a/drivers/clk/tegra/clk-tegra210.c b/drivers/clk/tegra/clk-tegra210.c
index 68cbb98af567..b9099012dc7b 100644
--- a/drivers/clk/tegra/clk-tegra210.c
+++ b/drivers/clk/tegra/clk-tegra210.c
@@ -1,6 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
- * Copyright (c) 2012-2014 NVIDIA CORPORATION. All rights reserved.
+ * Copyright (c) 2012-2020 NVIDIA CORPORATION. All rights reserved.
*/
#include <linux/io.h>
@@ -403,6 +403,14 @@ static unsigned long tegra210_input_freq[] = {
#define PLLRE_BASE_DEFAULT_MASK 0x1c000000
#define PLLRE_MISC0_WRITE_MASK 0x67ffffff
+/* PLLE */
+#define PLLE_MISC_IDDQ_SW_CTRL (1 << 14)
+#define PLLE_AUX_USE_LOCKDET (1 << 3)
+#define PLLE_AUX_SS_SEQ_INCLUDE (1 << 31)
+#define PLLE_AUX_ENABLE_SWCTL (1 << 4)
+#define PLLE_AUX_SS_SWCTL (1 << 6)
+#define PLLE_AUX_SEQ_ENABLE (1 << 24)
+
/* PLLX */
#define PLLX_USE_DYN_RAMP 1
#define PLLX_BASE_LOCK (1 << 27)
@@ -489,6 +497,49 @@ static unsigned long tegra210_input_freq[] = {
#define PLLU_MISC0_WRITE_MASK 0xbfffffff
#define PLLU_MISC1_WRITE_MASK 0x00000007
+bool tegra210_plle_hw_sequence_is_enabled(void)
+{
+ u32 value;
+
+ value = readl_relaxed(clk_base + PLLE_AUX);
+ if (value & PLLE_AUX_SEQ_ENABLE)
+ return true;
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(tegra210_plle_hw_sequence_is_enabled);
+
+int tegra210_plle_hw_sequence_start(void)
+{
+ u32 value;
+
+ if (tegra210_plle_hw_sequence_is_enabled())
+ return 0;
+
+ /* skip if PLLE is not enabled yet */
+ value = readl_relaxed(clk_base + PLLE_MISC0);
+ if (!(value & PLLE_MISC_LOCK))
+ return -EIO;
+
+ value &= ~PLLE_MISC_IDDQ_SW_CTRL;
+ writel_relaxed(value, clk_base + PLLE_MISC0);
+
+ value = readl_relaxed(clk_base + PLLE_AUX);
+ value |= (PLLE_AUX_USE_LOCKDET | PLLE_AUX_SS_SEQ_INCLUDE);
+ value &= ~(PLLE_AUX_ENABLE_SWCTL | PLLE_AUX_SS_SWCTL);
+ writel_relaxed(value, clk_base + PLLE_AUX);
+
+ fence_udelay(1, clk_base);
+
+ value |= PLLE_AUX_SEQ_ENABLE;
+ writel_relaxed(value, clk_base + PLLE_AUX);
+
+ fence_udelay(1, clk_base);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(tegra210_plle_hw_sequence_start);
+
void tegra210_xusb_pll_hw_control_enable(void)
{
u32 val;
diff --git a/drivers/clk/ti/clk-54xx.c b/drivers/clk/ti/clk-54xx.c
index f0542391ca4b..90e0a9ea6351 100644
--- a/drivers/clk/ti/clk-54xx.c
+++ b/drivers/clk/ti/clk-54xx.c
@@ -156,6 +156,8 @@ static const struct omap_clkctrl_reg_data omap5_l3main1_clkctrl_regs[] __initcon
static const struct omap_clkctrl_reg_data omap5_l3main2_clkctrl_regs[] __initconst = {
{ OMAP5_L3_MAIN_2_CLKCTRL, NULL, 0, "l3_iclk_div" },
+ { OMAP5_L3_MAIN_2_GPMC_CLKCTRL, NULL, CLKF_HW_SUP, "l3_iclk_div" },
+ { OMAP5_L3_MAIN_2_OCMC_RAM_CLKCTRL, NULL, CLKF_HW_SUP, "l3_iclk_div" },
{ 0 },
};
diff --git a/drivers/clk/uniphier/clk-uniphier-mux.c b/drivers/clk/uniphier/clk-uniphier-mux.c
index 462c84321b2d..1998e9d4cfc0 100644
--- a/drivers/clk/uniphier/clk-uniphier-mux.c
+++ b/drivers/clk/uniphier/clk-uniphier-mux.c
@@ -31,10 +31,10 @@ static int uniphier_clk_mux_set_parent(struct clk_hw *hw, u8 index)
static u8 uniphier_clk_mux_get_parent(struct clk_hw *hw)
{
struct uniphier_clk_mux *mux = to_uniphier_clk_mux(hw);
- int num_parents = clk_hw_get_num_parents(hw);
+ unsigned int num_parents = clk_hw_get_num_parents(hw);
int ret;
unsigned int val;
- u8 i;
+ unsigned int i;
ret = regmap_read(mux->regmap, mux->reg, &val);
if (ret)
diff --git a/drivers/clk/zynqmp/Kconfig b/drivers/clk/zynqmp/Kconfig
index 17086059be8b..00df6be25c5d 100644
--- a/drivers/clk/zynqmp/Kconfig
+++ b/drivers/clk/zynqmp/Kconfig
@@ -2,8 +2,8 @@
config COMMON_CLK_ZYNQMP
bool "Support for Xilinx ZynqMP Ultrascale+ clock controllers"
- depends on ARCH_ZYNQMP || COMPILE_TEST
- depends on ZYNQMP_FIRMWARE
+ depends on ZYNQMP_FIRMWARE || COMPILE_TEST
+ default ZYNQMP_FIRMWARE
help
Support for the Zynqmp Ultrascale clock controller.
It has a dependency on the PMU firmware.
diff --git a/drivers/clk/zynqmp/pll.c b/drivers/clk/zynqmp/pll.c
index 92f449ed38e5..abe6afbf3407 100644
--- a/drivers/clk/zynqmp/pll.c
+++ b/drivers/clk/zynqmp/pll.c
@@ -14,10 +14,12 @@
* struct zynqmp_pll - PLL clock
* @hw: Handle between common and hardware-specific interfaces
* @clk_id: PLL clock ID
+ * @set_pll_mode: Whether an IOCTL_SET_PLL_FRAC_MODE request be sent to ATF
*/
struct zynqmp_pll {
struct clk_hw hw;
u32 clk_id;
+ bool set_pll_mode;
};
#define to_zynqmp_pll(_hw) container_of(_hw, struct zynqmp_pll, hw)
@@ -81,6 +83,8 @@ static inline void zynqmp_pll_set_mode(struct clk_hw *hw, bool on)
if (ret)
pr_warn_once("%s() PLL set frac mode failed for %s, ret = %d\n",
__func__, clk_name, ret);
+ else
+ clk->set_pll_mode = true;
}
/**
@@ -100,9 +104,7 @@ static long zynqmp_pll_round_rate(struct clk_hw *hw, unsigned long rate,
/* Enable the fractional mode if needed */
rate_div = (rate * FRAC_DIV) / *prate;
f = rate_div % FRAC_DIV;
- zynqmp_pll_set_mode(hw, !!f);
-
- if (zynqmp_pll_get_mode(hw) == PLL_MODE_FRAC) {
+ if (f) {
if (rate > PS_PLL_VCO_MAX) {
fbdiv = rate / PS_PLL_VCO_MAX;
rate = rate / (fbdiv + 1);
@@ -173,10 +175,12 @@ static int zynqmp_pll_set_rate(struct clk_hw *hw, unsigned long rate,
long rate_div, frac, m, f;
int ret;
- if (zynqmp_pll_get_mode(hw) == PLL_MODE_FRAC) {
- rate_div = (rate * FRAC_DIV) / parent_rate;
+ rate_div = (rate * FRAC_DIV) / parent_rate;
+ f = rate_div % FRAC_DIV;
+ zynqmp_pll_set_mode(hw, !!f);
+
+ if (f) {
m = rate_div / FRAC_DIV;
- f = rate_div % FRAC_DIV;
m = clamp_t(u32, m, (PLL_FBDIV_MIN), (PLL_FBDIV_MAX));
rate = parent_rate * m;
frac = (parent_rate * f) / FRAC_DIV;
@@ -240,9 +244,15 @@ static int zynqmp_pll_enable(struct clk_hw *hw)
u32 clk_id = clk->clk_id;
int ret;
- if (zynqmp_pll_is_enabled(hw))
+ /*
+ * Don't skip enabling clock if there is an IOCTL_SET_PLL_FRAC_MODE request
+ * that has been sent to ATF.
+ */
+ if (zynqmp_pll_is_enabled(hw) && (!clk->set_pll_mode))
return 0;
+ clk->set_pll_mode = false;
+
ret = zynqmp_pm_clock_enable(clk_id);
if (ret)
pr_warn_once("%s() clock enable failed for %s, ret = %d\n",
diff --git a/drivers/clocksource/arm_arch_timer.c b/drivers/clocksource/arm_arch_timer.c
index d0177824c518..fe1a82627d57 100644
--- a/drivers/clocksource/arm_arch_timer.c
+++ b/drivers/clocksource/arm_arch_timer.c
@@ -16,6 +16,7 @@
#include <linux/cpu_pm.h>
#include <linux/clockchips.h>
#include <linux/clocksource.h>
+#include <linux/clocksource_ids.h>
#include <linux/interrupt.h>
#include <linux/of_irq.h>
#include <linux/of_address.h>
@@ -24,6 +25,8 @@
#include <linux/sched/clock.h>
#include <linux/sched_clock.h>
#include <linux/acpi.h>
+#include <linux/arm-smccc.h>
+#include <linux/ptp_kvm.h>
#include <asm/arch_timer.h>
#include <asm/virt.h>
@@ -51,7 +54,7 @@
static unsigned arch_timers_present __initdata;
-static void __iomem *arch_counter_base;
+static void __iomem *arch_counter_base __ro_after_init;
struct arch_timer {
void __iomem *base;
@@ -60,15 +63,24 @@ struct arch_timer {
#define to_arch_timer(e) container_of(e, struct arch_timer, evt)
-static u32 arch_timer_rate;
-static int arch_timer_ppi[ARCH_TIMER_MAX_TIMER_PPI];
+static u32 arch_timer_rate __ro_after_init;
+u32 arch_timer_rate1 __ro_after_init;
+static int arch_timer_ppi[ARCH_TIMER_MAX_TIMER_PPI] __ro_after_init;
+
+static const char *arch_timer_ppi_names[ARCH_TIMER_MAX_TIMER_PPI] = {
+ [ARCH_TIMER_PHYS_SECURE_PPI] = "sec-phys",
+ [ARCH_TIMER_PHYS_NONSECURE_PPI] = "phys",
+ [ARCH_TIMER_VIRT_PPI] = "virt",
+ [ARCH_TIMER_HYP_PPI] = "hyp-phys",
+ [ARCH_TIMER_HYP_VIRT_PPI] = "hyp-virt",
+};
static struct clock_event_device __percpu *arch_timer_evt;
-static enum arch_timer_ppi_nr arch_timer_uses_ppi = ARCH_TIMER_VIRT_PPI;
-static bool arch_timer_c3stop;
-static bool arch_timer_mem_use_virtual;
-static bool arch_counter_suspend_stop;
+static enum arch_timer_ppi_nr arch_timer_uses_ppi __ro_after_init = ARCH_TIMER_VIRT_PPI;
+static bool arch_timer_c3stop __ro_after_init;
+static bool arch_timer_mem_use_virtual __ro_after_init;
+static bool arch_counter_suspend_stop __ro_after_init;
#ifdef CONFIG_GENERIC_GETTIMEOFDAY
static enum vdso_clock_mode vdso_default = VDSO_CLOCKMODE_ARCHTIMER;
#else
@@ -76,7 +88,7 @@ static enum vdso_clock_mode vdso_default = VDSO_CLOCKMODE_NONE;
#endif /* CONFIG_GENERIC_GETTIMEOFDAY */
static cpumask_t evtstrm_available = CPU_MASK_NONE;
-static bool evtstrm_enable = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM);
+static bool evtstrm_enable __ro_after_init = IS_ENABLED(CONFIG_ARM_ARCH_TIMER_EVTSTREAM);
static int __init early_evtstrm_cfg(char *buf)
{
@@ -176,7 +188,7 @@ static notrace u64 arch_counter_get_cntvct(void)
* to exist on arm64. arm doesn't use this before DT is probed so even
* if we don't have the cp15 accessors we won't have a problem.
*/
-u64 (*arch_timer_read_counter)(void) = arch_counter_get_cntvct;
+u64 (*arch_timer_read_counter)(void) __ro_after_init = arch_counter_get_cntvct;
EXPORT_SYMBOL_GPL(arch_timer_read_counter);
static u64 arch_counter_read(struct clocksource *cs)
@@ -191,6 +203,7 @@ static u64 arch_counter_read_cc(const struct cyclecounter *cc)
static struct clocksource clocksource_counter = {
.name = "arch_sys_counter",
+ .id = CSID_ARM_ARCH_COUNTER,
.rating = 400,
.read = arch_counter_read,
.mask = CLOCKSOURCE_MASK(56),
@@ -925,7 +938,7 @@ static int validate_timer_rate(void)
* rate was probed first, and don't verify that others match. If the first node
* probed has a clock-frequency property, this overrides the HW register.
*/
-static void arch_timer_of_configure_rate(u32 rate, struct device_node *np)
+static void __init arch_timer_of_configure_rate(u32 rate, struct device_node *np)
{
/* Who has more than one independent system counter? */
if (arch_timer_rate)
@@ -939,7 +952,7 @@ static void arch_timer_of_configure_rate(u32 rate, struct device_node *np)
pr_warn("frequency not available\n");
}
-static void arch_timer_banner(unsigned type)
+static void __init arch_timer_banner(unsigned type)
{
pr_info("%s%s%s timer(s) running at %lu.%02luMHz (%s%s%s).\n",
type & ARCH_TIMER_TYPE_CP15 ? "cp15" : "",
@@ -1280,8 +1293,9 @@ static void __init arch_timer_populate_kvm_info(void)
static int __init arch_timer_of_init(struct device_node *np)
{
- int i, ret;
+ int i, irq, ret;
u32 rate;
+ bool has_names;
if (arch_timers_present & ARCH_TIMER_TYPE_CP15) {
pr_warn("multiple nodes in dt, skipping\n");
@@ -1289,8 +1303,17 @@ static int __init arch_timer_of_init(struct device_node *np)
}
arch_timers_present |= ARCH_TIMER_TYPE_CP15;
- for (i = ARCH_TIMER_PHYS_SECURE_PPI; i < ARCH_TIMER_MAX_TIMER_PPI; i++)
- arch_timer_ppi[i] = irq_of_parse_and_map(np, i);
+
+ has_names = of_property_read_bool(np, "interrupt-names");
+
+ for (i = ARCH_TIMER_PHYS_SECURE_PPI; i < ARCH_TIMER_MAX_TIMER_PPI; i++) {
+ if (has_names)
+ irq = of_irq_get_byname(np, arch_timer_ppi_names[i]);
+ else
+ irq = of_irq_get(np, i);
+ if (irq > 0)
+ arch_timer_ppi[i] = irq;
+ }
arch_timer_populate_kvm_info();
@@ -1657,3 +1680,35 @@ static int __init arch_timer_acpi_init(struct acpi_table_header *table)
}
TIMER_ACPI_DECLARE(arch_timer, ACPI_SIG_GTDT, arch_timer_acpi_init);
#endif
+
+int kvm_arch_ptp_get_crosststamp(u64 *cycle, struct timespec64 *ts,
+ struct clocksource **cs)
+{
+ struct arm_smccc_res hvc_res;
+ u32 ptp_counter;
+ ktime_t ktime;
+
+ if (!IS_ENABLED(CONFIG_HAVE_ARM_SMCCC_DISCOVERY))
+ return -EOPNOTSUPP;
+
+ if (arch_timer_uses_ppi == ARCH_TIMER_VIRT_PPI)
+ ptp_counter = KVM_PTP_VIRT_COUNTER;
+ else
+ ptp_counter = KVM_PTP_PHYS_COUNTER;
+
+ arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_PTP_FUNC_ID,
+ ptp_counter, &hvc_res);
+
+ if ((int)(hvc_res.a0) < 0)
+ return -EOPNOTSUPP;
+
+ ktime = (u64)hvc_res.a0 << 32 | hvc_res.a1;
+ *ts = ktime_to_timespec64(ktime);
+ if (cycle)
+ *cycle = (u64)hvc_res.a2 << 32 | hvc_res.a3;
+ if (cs)
+ *cs = &clocksource_counter;
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(kvm_arch_ptp_get_crosststamp);
diff --git a/drivers/clocksource/clksrc-dbx500-prcmu.c b/drivers/clocksource/clksrc-dbx500-prcmu.c
index 996900d017c6..2fc93e46cea3 100644
--- a/drivers/clocksource/clksrc-dbx500-prcmu.c
+++ b/drivers/clocksource/clksrc-dbx500-prcmu.c
@@ -18,7 +18,7 @@
#define RATE_32K 32768
-#define TIMER_MODE_CONTINOUS 0x1
+#define TIMER_MODE_CONTINUOUS 0x1
#define TIMER_DOWNCOUNT_VAL 0xffffffff
#define PRCMU_TIMER_REF 0
@@ -55,13 +55,13 @@ static int __init clksrc_dbx500_prcmu_init(struct device_node *node)
/*
* The A9 sub system expects the timer to be configured as
- * a continous looping timer.
+ * a continuous looping timer.
* The PRCMU should configure it but if it for some reason
* don't we do it here.
*/
if (readl(clksrc_dbx500_timer_base + PRCMU_TIMER_MODE) !=
- TIMER_MODE_CONTINOUS) {
- writel(TIMER_MODE_CONTINOUS,
+ TIMER_MODE_CONTINUOUS) {
+ writel(TIMER_MODE_CONTINUOUS,
clksrc_dbx500_timer_base + PRCMU_TIMER_MODE);
writel(TIMER_DOWNCOUNT_VAL,
clksrc_dbx500_timer_base + PRCMU_TIMER_REF);
diff --git a/drivers/clocksource/dw_apb_timer_of.c b/drivers/clocksource/dw_apb_timer_of.c
index 42e7e43b8fcd..3819ef5b7098 100644
--- a/drivers/clocksource/dw_apb_timer_of.c
+++ b/drivers/clocksource/dw_apb_timer_of.c
@@ -38,7 +38,7 @@ static int __init timer_get_base_and_rate(struct device_node *np,
}
/*
- * Not all implementations use a periphal clock, so don't panic
+ * Not all implementations use a peripheral clock, so don't panic
* if it's not present
*/
pclk = of_clk_get_by_name(np, "pclk");
@@ -52,18 +52,34 @@ static int __init timer_get_base_and_rate(struct device_node *np,
return 0;
timer_clk = of_clk_get_by_name(np, "timer");
- if (IS_ERR(timer_clk))
- return PTR_ERR(timer_clk);
+ if (IS_ERR(timer_clk)) {
+ ret = PTR_ERR(timer_clk);
+ goto out_pclk_disable;
+ }
ret = clk_prepare_enable(timer_clk);
if (ret)
- return ret;
+ goto out_timer_clk_put;
*rate = clk_get_rate(timer_clk);
- if (!(*rate))
- return -EINVAL;
+ if (!(*rate)) {
+ ret = -EINVAL;
+ goto out_timer_clk_disable;
+ }
return 0;
+
+out_timer_clk_disable:
+ clk_disable_unprepare(timer_clk);
+out_timer_clk_put:
+ clk_put(timer_clk);
+out_pclk_disable:
+ if (!IS_ERR(pclk)) {
+ clk_disable_unprepare(pclk);
+ clk_put(pclk);
+ }
+ iounmap(*base);
+ return ret;
}
static int __init add_clockevent(struct device_node *event_timer)
diff --git a/drivers/clocksource/hyperv_timer.c b/drivers/clocksource/hyperv_timer.c
index 269a691bd2c4..d6ece7bbce89 100644
--- a/drivers/clocksource/hyperv_timer.c
+++ b/drivers/clocksource/hyperv_timer.c
@@ -18,6 +18,9 @@
#include <linux/sched_clock.h>
#include <linux/mm.h>
#include <linux/cpuhotplug.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/acpi.h>
#include <clocksource/hyperv_timer.h>
#include <asm/hyperv-tlfs.h>
#include <asm/mshyperv.h>
@@ -43,14 +46,13 @@ static u64 hv_sched_clock_offset __ro_after_init;
*/
static bool direct_mode_enabled;
-static int stimer0_irq;
-static int stimer0_vector;
+static int stimer0_irq = -1;
static int stimer0_message_sint;
+static DEFINE_PER_CPU(long, stimer0_evt);
/*
- * ISR for when stimer0 is operating in Direct Mode. Direct Mode
- * does not use VMbus or any VMbus messages, so process here and not
- * in the VMbus driver code.
+ * Common code for stimer0 interrupts coming via Direct Mode or
+ * as a VMbus message.
*/
void hv_stimer0_isr(void)
{
@@ -61,6 +63,16 @@ void hv_stimer0_isr(void)
}
EXPORT_SYMBOL_GPL(hv_stimer0_isr);
+/*
+ * stimer0 interrupt handler for architectures that support
+ * per-cpu interrupts, which also implies Direct Mode.
+ */
+static irqreturn_t hv_stimer0_percpu_isr(int irq, void *dev_id)
+{
+ hv_stimer0_isr();
+ return IRQ_HANDLED;
+}
+
static int hv_ce_set_next_event(unsigned long delta,
struct clock_event_device *evt)
{
@@ -68,16 +80,16 @@ static int hv_ce_set_next_event(unsigned long delta,
current_tick = hv_read_reference_counter();
current_tick += delta;
- hv_init_timer(0, current_tick);
+ hv_set_register(HV_REGISTER_STIMER0_COUNT, current_tick);
return 0;
}
static int hv_ce_shutdown(struct clock_event_device *evt)
{
- hv_init_timer(0, 0);
- hv_init_timer_config(0, 0);
- if (direct_mode_enabled)
- hv_disable_stimer0_percpu_irq(stimer0_irq);
+ hv_set_register(HV_REGISTER_STIMER0_COUNT, 0);
+ hv_set_register(HV_REGISTER_STIMER0_CONFIG, 0);
+ if (direct_mode_enabled && stimer0_irq >= 0)
+ disable_percpu_irq(stimer0_irq);
return 0;
}
@@ -95,8 +107,9 @@ static int hv_ce_set_oneshot(struct clock_event_device *evt)
* on the specified hardware vector/IRQ.
*/
timer_cfg.direct_mode = 1;
- timer_cfg.apic_vector = stimer0_vector;
- hv_enable_stimer0_percpu_irq(stimer0_irq);
+ timer_cfg.apic_vector = HYPERV_STIMER0_VECTOR;
+ if (stimer0_irq >= 0)
+ enable_percpu_irq(stimer0_irq, IRQ_TYPE_NONE);
} else {
/*
* When it expires, the timer will generate a VMbus message,
@@ -105,7 +118,7 @@ static int hv_ce_set_oneshot(struct clock_event_device *evt)
timer_cfg.direct_mode = 0;
timer_cfg.sintx = stimer0_message_sint;
}
- hv_init_timer_config(0, timer_cfg.as_uint64);
+ hv_set_register(HV_REGISTER_STIMER0_CONFIG, timer_cfg.as_uint64);
return 0;
}
@@ -169,10 +182,58 @@ int hv_stimer_cleanup(unsigned int cpu)
}
EXPORT_SYMBOL_GPL(hv_stimer_cleanup);
+/*
+ * These placeholders are overridden by arch specific code on
+ * architectures that need special setup of the stimer0 IRQ because
+ * they don't support per-cpu IRQs (such as x86/x64).
+ */
+void __weak hv_setup_stimer0_handler(void (*handler)(void))
+{
+};
+
+void __weak hv_remove_stimer0_handler(void)
+{
+};
+
+/* Called only on architectures with per-cpu IRQs (i.e., not x86/x64) */
+static int hv_setup_stimer0_irq(void)
+{
+ int ret;
+
+ ret = acpi_register_gsi(NULL, HYPERV_STIMER0_VECTOR,
+ ACPI_EDGE_SENSITIVE, ACPI_ACTIVE_HIGH);
+ if (ret < 0) {
+ pr_err("Can't register Hyper-V stimer0 GSI. Error %d", ret);
+ return ret;
+ }
+ stimer0_irq = ret;
+
+ ret = request_percpu_irq(stimer0_irq, hv_stimer0_percpu_isr,
+ "Hyper-V stimer0", &stimer0_evt);
+ if (ret) {
+ pr_err("Can't request Hyper-V stimer0 IRQ %d. Error %d",
+ stimer0_irq, ret);
+ acpi_unregister_gsi(stimer0_irq);
+ stimer0_irq = -1;
+ }
+ return ret;
+}
+
+static void hv_remove_stimer0_irq(void)
+{
+ if (stimer0_irq == -1) {
+ hv_remove_stimer0_handler();
+ } else {
+ free_percpu_irq(stimer0_irq, &stimer0_evt);
+ acpi_unregister_gsi(stimer0_irq);
+ stimer0_irq = -1;
+ }
+}
+
/* hv_stimer_alloc - Global initialization of the clockevent and stimer0 */
-int hv_stimer_alloc(void)
+int hv_stimer_alloc(bool have_percpu_irqs)
{
- int ret = 0;
+ int ret;
/*
* Synthetic timers are always available except on old versions of
@@ -188,29 +249,37 @@ int hv_stimer_alloc(void)
direct_mode_enabled = ms_hyperv.misc_features &
HV_STIMER_DIRECT_MODE_AVAILABLE;
- if (direct_mode_enabled) {
- ret = hv_setup_stimer0_irq(&stimer0_irq, &stimer0_vector,
- hv_stimer0_isr);
+
+ /*
+ * If Direct Mode isn't enabled, the remainder of the initialization
+ * is done later by hv_stimer_legacy_init()
+ */
+ if (!direct_mode_enabled)
+ return 0;
+
+ if (have_percpu_irqs) {
+ ret = hv_setup_stimer0_irq();
if (ret)
- goto free_percpu;
+ goto free_clock_event;
+ } else {
+ hv_setup_stimer0_handler(hv_stimer0_isr);
+ }
- /*
- * Since we are in Direct Mode, stimer initialization
- * can be done now with a CPUHP value in the same range
- * as other clockevent devices.
- */
- ret = cpuhp_setup_state(CPUHP_AP_HYPERV_TIMER_STARTING,
- "clockevents/hyperv/stimer:starting",
- hv_stimer_init, hv_stimer_cleanup);
- if (ret < 0)
- goto free_stimer0_irq;
+ /*
+ * Since we are in Direct Mode, stimer initialization
+ * can be done now with a CPUHP value in the same range
+ * as other clockevent devices.
+ */
+ ret = cpuhp_setup_state(CPUHP_AP_HYPERV_TIMER_STARTING,
+ "clockevents/hyperv/stimer:starting",
+ hv_stimer_init, hv_stimer_cleanup);
+ if (ret < 0) {
+ hv_remove_stimer0_irq();
+ goto free_clock_event;
}
return ret;
-free_stimer0_irq:
- hv_remove_stimer0_irq(stimer0_irq);
- stimer0_irq = 0;
-free_percpu:
+free_clock_event:
free_percpu(hv_clock_event);
hv_clock_event = NULL;
return ret;
@@ -254,23 +323,6 @@ void hv_stimer_legacy_cleanup(unsigned int cpu)
}
EXPORT_SYMBOL_GPL(hv_stimer_legacy_cleanup);
-
-/* hv_stimer_free - Free global resources allocated by hv_stimer_alloc() */
-void hv_stimer_free(void)
-{
- if (!hv_clock_event)
- return;
-
- if (direct_mode_enabled) {
- cpuhp_remove_state(CPUHP_AP_HYPERV_TIMER_STARTING);
- hv_remove_stimer0_irq(stimer0_irq);
- stimer0_irq = 0;
- }
- free_percpu(hv_clock_event);
- hv_clock_event = NULL;
-}
-EXPORT_SYMBOL_GPL(hv_stimer_free);
-
/*
* Do a global cleanup of clockevents for the cases of kexec and
* vmbus exit
@@ -287,12 +339,17 @@ void hv_stimer_global_cleanup(void)
hv_stimer_legacy_cleanup(cpu);
}
- /*
- * If Direct Mode is enabled, the cpuhp teardown callback
- * (hv_stimer_cleanup) will be run on all CPUs to stop the
- * stimers.
- */
- hv_stimer_free();
+ if (!hv_clock_event)
+ return;
+
+ if (direct_mode_enabled) {
+ cpuhp_remove_state(CPUHP_AP_HYPERV_TIMER_STARTING);
+ hv_remove_stimer0_irq();
+ stimer0_irq = -1;
+ }
+ free_percpu(hv_clock_event);
+ hv_clock_event = NULL;
+
}
EXPORT_SYMBOL_GPL(hv_stimer_global_cleanup);
@@ -302,14 +359,6 @@ EXPORT_SYMBOL_GPL(hv_stimer_global_cleanup);
* the other that uses the TSC reference page feature as defined in the
* TLFS. The MSR version is for compatibility with old versions of
* Hyper-V and 32-bit x86. The TSC reference page version is preferred.
- *
- * The Hyper-V clocksource ratings of 250 are chosen to be below the
- * TSC clocksource rating of 300. In configurations where Hyper-V offers
- * an InvariantTSC, the TSC is not marked "unstable", so the TSC clocksource
- * is available and preferred. With the higher rating, it will be the
- * default. On older hardware and Hyper-V versions, the TSC is marked
- * "unstable", so no TSC clocksource is created and the selected Hyper-V
- * clocksource will be the default.
*/
u64 (*hv_read_reference_counter)(void);
@@ -331,7 +380,7 @@ static u64 notrace read_hv_clock_tsc(void)
u64 current_tick = hv_read_tsc_page(hv_get_tsc_page());
if (current_tick == U64_MAX)
- hv_get_time_ref_count(current_tick);
+ current_tick = hv_get_register(HV_REGISTER_TIME_REF_COUNT);
return current_tick;
}
@@ -352,9 +401,9 @@ static void suspend_hv_clock_tsc(struct clocksource *arg)
u64 tsc_msr;
/* Disable the TSC page */
- hv_get_reference_tsc(tsc_msr);
+ tsc_msr = hv_get_register(HV_REGISTER_REFERENCE_TSC);
tsc_msr &= ~BIT_ULL(0);
- hv_set_reference_tsc(tsc_msr);
+ hv_set_register(HV_REGISTER_REFERENCE_TSC, tsc_msr);
}
@@ -364,39 +413,44 @@ static void resume_hv_clock_tsc(struct clocksource *arg)
u64 tsc_msr;
/* Re-enable the TSC page */
- hv_get_reference_tsc(tsc_msr);
+ tsc_msr = hv_get_register(HV_REGISTER_REFERENCE_TSC);
tsc_msr &= GENMASK_ULL(11, 0);
tsc_msr |= BIT_ULL(0) | (u64)phys_addr;
- hv_set_reference_tsc(tsc_msr);
+ hv_set_register(HV_REGISTER_REFERENCE_TSC, tsc_msr);
}
+#ifdef HAVE_VDSO_CLOCKMODE_HVCLOCK
static int hv_cs_enable(struct clocksource *cs)
{
- hv_enable_vdso_clocksource();
+ vclocks_set_used(VDSO_CLOCKMODE_HVCLOCK);
return 0;
}
+#endif
static struct clocksource hyperv_cs_tsc = {
.name = "hyperv_clocksource_tsc_page",
- .rating = 250,
+ .rating = 500,
.read = read_hv_clock_tsc_cs,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
.suspend= suspend_hv_clock_tsc,
.resume = resume_hv_clock_tsc,
+#ifdef HAVE_VDSO_CLOCKMODE_HVCLOCK
.enable = hv_cs_enable,
+ .vdso_clock_mode = VDSO_CLOCKMODE_HVCLOCK,
+#else
+ .vdso_clock_mode = VDSO_CLOCKMODE_NONE,
+#endif
};
static u64 notrace read_hv_clock_msr(void)
{
- u64 current_tick;
/*
* Read the partition counter to get the current tick count. This count
* is set to 0 when the partition is created and is incremented in
* 100 nanosecond units.
*/
- hv_get_time_ref_count(current_tick);
- return current_tick;
+ return hv_get_register(HV_REGISTER_TIME_REF_COUNT);
}
static u64 notrace read_hv_clock_msr_cs(struct clocksource *arg)
@@ -412,12 +466,36 @@ static u64 notrace read_hv_sched_clock_msr(void)
static struct clocksource hyperv_cs_msr = {
.name = "hyperv_clocksource_msr",
- .rating = 250,
+ .rating = 500,
.read = read_hv_clock_msr_cs,
.mask = CLOCKSOURCE_MASK(64),
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
+/*
+ * Reference to pv_ops must be inline so objtool
+ * detection of noinstr violations can work correctly.
+ */
+#ifdef CONFIG_GENERIC_SCHED_CLOCK
+static __always_inline void hv_setup_sched_clock(void *sched_clock)
+{
+ /*
+ * We're on an architecture with generic sched clock (not x86/x64).
+ * The Hyper-V sched clock read function returns nanoseconds, not
+ * the normal 100ns units of the Hyper-V synthetic clock.
+ */
+ sched_clock_register(sched_clock, 64, NSEC_PER_SEC);
+}
+#elif defined CONFIG_PARAVIRT
+static __always_inline void hv_setup_sched_clock(void *sched_clock)
+{
+ /* We're on x86/x64 *and* using PV ops */
+ paravirt_set_sched_clock(sched_clock);
+}
+#else /* !CONFIG_GENERIC_SCHED_CLOCK && !CONFIG_PARAVIRT */
+static __always_inline void hv_setup_sched_clock(void *sched_clock) {}
+#endif /* CONFIG_GENERIC_SCHED_CLOCK */
+
static bool __init hv_init_tsc_clocksource(void)
{
u64 tsc_msr;
@@ -429,6 +507,22 @@ static bool __init hv_init_tsc_clocksource(void)
if (hv_root_partition)
return false;
+ /*
+ * If Hyper-V offers TSC_INVARIANT, then the virtualized TSC correctly
+ * handles frequency and offset changes due to live migration,
+ * pause/resume, and other VM management operations. So lower the
+ * Hyper-V Reference TSC rating, causing the generic TSC to be used.
+ * TSC_INVARIANT is not offered on ARM64, so the Hyper-V Reference
+ * TSC will be preferred over the virtualized ARM64 arch counter.
+ * While the Hyper-V MSR clocksource won't be used since the
+ * Reference TSC clocksource is present, change its rating as
+ * well for consistency.
+ */
+ if (ms_hyperv.features & HV_ACCESS_TSC_INVARIANT) {
+ hyperv_cs_tsc.rating = 250;
+ hyperv_cs_msr.rating = 250;
+ }
+
hv_read_reference_counter = read_hv_clock_tsc;
phys_addr = virt_to_phys(hv_get_tsc_page());
@@ -439,12 +533,11 @@ static bool __init hv_init_tsc_clocksource(void)
* (which already has at least the low 12 bits set to zero since
* it is page aligned). Also set the "enable" bit, which is bit 0.
*/
- hv_get_reference_tsc(tsc_msr);
+ tsc_msr = hv_get_register(HV_REGISTER_REFERENCE_TSC);
tsc_msr &= GENMASK_ULL(11, 0);
tsc_msr = tsc_msr | 0x1 | (u64)phys_addr;
- hv_set_reference_tsc(tsc_msr);
+ hv_set_register(HV_REGISTER_REFERENCE_TSC, tsc_msr);
- hv_set_clocksource_vdso(hyperv_cs_tsc);
clocksource_register_hz(&hyperv_cs_tsc, NSEC_PER_SEC/100);
hv_sched_clock_offset = hv_read_reference_counter();
@@ -457,7 +550,7 @@ void __init hv_init_clocksource(void)
{
/*
* Try to set up the TSC page clocksource. If it succeeds, we're
- * done. Otherwise, set up the MSR clocksoruce. At least one of
+ * done. Otherwise, set up the MSR clocksource. At least one of
* these will always be available except on very old versions of
* Hyper-V on x86. In that case we won't have a Hyper-V
* clocksource, but Linux will still run with a clocksource based
diff --git a/drivers/clocksource/ingenic-ost.c b/drivers/clocksource/ingenic-ost.c
index 029efc2731b4..06d25754e606 100644
--- a/drivers/clocksource/ingenic-ost.c
+++ b/drivers/clocksource/ingenic-ost.c
@@ -88,9 +88,9 @@ static int __init ingenic_ost_probe(struct platform_device *pdev)
return PTR_ERR(ost->regs);
map = device_node_to_regmap(dev->parent->of_node);
- if (!map) {
+ if (IS_ERR(map)) {
dev_err(dev, "regmap not found");
- return -EINVAL;
+ return PTR_ERR(map);
}
ost->clk = devm_clk_get(dev, "ost");
@@ -167,13 +167,14 @@ static const struct ingenic_ost_soc_info jz4725b_ost_soc_info = {
.is64bit = false,
};
-static const struct ingenic_ost_soc_info jz4770_ost_soc_info = {
+static const struct ingenic_ost_soc_info jz4760b_ost_soc_info = {
.is64bit = true,
};
static const struct of_device_id ingenic_ost_of_match[] = {
{ .compatible = "ingenic,jz4725b-ost", .data = &jz4725b_ost_soc_info, },
- { .compatible = "ingenic,jz4770-ost", .data = &jz4770_ost_soc_info, },
+ { .compatible = "ingenic,jz4760b-ost", .data = &jz4760b_ost_soc_info, },
+ { .compatible = "ingenic,jz4770-ost", .data = &jz4760b_ost_soc_info, },
{ }
};
diff --git a/drivers/clocksource/ingenic-timer.c b/drivers/clocksource/ingenic-timer.c
index 905fd6b163a8..24ed0f1f089b 100644
--- a/drivers/clocksource/ingenic-timer.c
+++ b/drivers/clocksource/ingenic-timer.c
@@ -264,6 +264,7 @@ static const struct ingenic_soc_info jz4725b_soc_info = {
static const struct of_device_id ingenic_tcu_of_match[] = {
{ .compatible = "ingenic,jz4740-tcu", .data = &jz4740_soc_info, },
{ .compatible = "ingenic,jz4725b-tcu", .data = &jz4725b_soc_info, },
+ { .compatible = "ingenic,jz4760-tcu", .data = &jz4740_soc_info, },
{ .compatible = "ingenic,jz4770-tcu", .data = &jz4740_soc_info, },
{ .compatible = "ingenic,x1000-tcu", .data = &jz4740_soc_info, },
{ /* sentinel */ }
@@ -358,6 +359,7 @@ err_free_ingenic_tcu:
TIMER_OF_DECLARE(jz4740_tcu_intc, "ingenic,jz4740-tcu", ingenic_tcu_init);
TIMER_OF_DECLARE(jz4725b_tcu_intc, "ingenic,jz4725b-tcu", ingenic_tcu_init);
+TIMER_OF_DECLARE(jz4760_tcu_intc, "ingenic,jz4760-tcu", ingenic_tcu_init);
TIMER_OF_DECLARE(jz4770_tcu_intc, "ingenic,jz4770-tcu", ingenic_tcu_init);
TIMER_OF_DECLARE(x1000_tcu_intc, "ingenic,x1000-tcu", ingenic_tcu_init);
diff --git a/drivers/clocksource/sh_cmt.c b/drivers/clocksource/sh_cmt.c
index c98f8851fd68..d7ed99f0001f 100644
--- a/drivers/clocksource/sh_cmt.c
+++ b/drivers/clocksource/sh_cmt.c
@@ -339,8 +339,9 @@ static int sh_cmt_enable(struct sh_cmt_channel *ch)
sh_cmt_write_cmcsr(ch, SH_CMT16_CMCSR_CMIE |
SH_CMT16_CMCSR_CKS512);
} else {
- sh_cmt_write_cmcsr(ch, SH_CMT32_CMCSR_CMM |
- SH_CMT32_CMCSR_CMTOUT_IE |
+ u32 cmtout = ch->cmt->info->model <= SH_CMT_48BIT ?
+ SH_CMT32_CMCSR_CMTOUT_IE : 0;
+ sh_cmt_write_cmcsr(ch, cmtout | SH_CMT32_CMCSR_CMM |
SH_CMT32_CMCSR_CMR_IRQ |
SH_CMT32_CMCSR_CKS_RCLK8);
}
diff --git a/drivers/clocksource/timer-atmel-tcb.c b/drivers/clocksource/timer-atmel-tcb.c
index 787dbebbb432..27af17c99590 100644
--- a/drivers/clocksource/timer-atmel-tcb.c
+++ b/drivers/clocksource/timer-atmel-tcb.c
@@ -455,9 +455,9 @@ static int __init tcb_clksrc_init(struct device_node *node)
tcaddr = tc.regs;
if (bits == 32) {
- /* use apropriate function to read 32 bit counter */
+ /* use appropriate function to read 32 bit counter */
clksrc.read = tc_get_cycles32;
- /* setup ony channel 0 */
+ /* setup only channel 0 */
tcb_setup_single_chan(&tc, best_divisor_idx);
tc_sched_clock = tc_sched_clock_read32;
tc_delay_timer.read_current_timer = tc_delay_timer_read32;
diff --git a/drivers/clocksource/timer-fsl-ftm.c b/drivers/clocksource/timer-fsl-ftm.c
index 12a2ed7cfaff..93f336ec875a 100644
--- a/drivers/clocksource/timer-fsl-ftm.c
+++ b/drivers/clocksource/timer-fsl-ftm.c
@@ -116,7 +116,7 @@ static int ftm_set_next_event(unsigned long delta,
* to the MOD register latches the value into a buffer. The MOD
* register is updated with the value of its write buffer with
* the following scenario:
- * a, the counter source clock is diabled.
+ * a, the counter source clock is disabled.
*/
ftm_counter_disable(priv->clkevt_base);
diff --git a/drivers/clocksource/timer-microchip-pit64b.c b/drivers/clocksource/timer-microchip-pit64b.c
index ab623b25a47b..cfa4ec7ef396 100644
--- a/drivers/clocksource/timer-microchip-pit64b.c
+++ b/drivers/clocksource/timer-microchip-pit64b.c
@@ -237,7 +237,7 @@ static void __init mchp_pit64b_pres_compute(u32 *pres, u32 clk_rate,
break;
}
- /* Use the bigest prescaler if we didn't match one. */
+ /* Use the biggest prescaler if we didn't match one. */
if (*pres == MCHP_PIT64B_PRES_MAX)
*pres = MCHP_PIT64B_PRES_MAX - 1;
}
diff --git a/drivers/clocksource/timer-npcm7xx.c b/drivers/clocksource/timer-npcm7xx.c
index 9780ffd8010e..a00520cbb660 100644
--- a/drivers/clocksource/timer-npcm7xx.c
+++ b/drivers/clocksource/timer-npcm7xx.c
@@ -208,5 +208,6 @@ static int __init npcm7xx_timer_init(struct device_node *np)
return 0;
}
+TIMER_OF_DECLARE(wpcm450, "nuvoton,wpcm450-timer", npcm7xx_timer_init);
TIMER_OF_DECLARE(npcm7xx, "nuvoton,npcm750-timer", npcm7xx_timer_init);
diff --git a/drivers/clocksource/timer-of.c b/drivers/clocksource/timer-of.c
index 572da477c6d3..529cc6a51cdb 100644
--- a/drivers/clocksource/timer-of.c
+++ b/drivers/clocksource/timer-of.c
@@ -211,10 +211,10 @@ out_fail:
}
/**
- * timer_of_cleanup - release timer_of ressources
+ * timer_of_cleanup - release timer_of resources
* @to: timer_of structure
*
- * Release the ressources that has been used in timer_of_init().
+ * Release the resources that has been used in timer_of_init().
* This function should be called in init error cases
*/
void __init timer_of_cleanup(struct timer_of *to)
diff --git a/drivers/clocksource/timer-pistachio.c b/drivers/clocksource/timer-pistachio.c
index a2dd85d0c1d7..6f37181a8c63 100644
--- a/drivers/clocksource/timer-pistachio.c
+++ b/drivers/clocksource/timer-pistachio.c
@@ -71,7 +71,7 @@ static u64 notrace
pistachio_clocksource_read_cycles(struct clocksource *cs)
{
struct pistachio_clocksource *pcs = to_pistachio_clocksource(cs);
- u32 counter, overflw;
+ u32 counter, overflow;
unsigned long flags;
/*
@@ -80,7 +80,7 @@ pistachio_clocksource_read_cycles(struct clocksource *cs)
*/
raw_spin_lock_irqsave(&pcs->lock, flags);
- overflw = gpt_readl(pcs->base, TIMER_CURRENT_OVERFLOW_VALUE, 0);
+ overflow = gpt_readl(pcs->base, TIMER_CURRENT_OVERFLOW_VALUE, 0);
counter = gpt_readl(pcs->base, TIMER_CURRENT_VALUE, 0);
raw_spin_unlock_irqrestore(&pcs->lock, flags);
diff --git a/drivers/clocksource/timer-ti-dm-systimer.c b/drivers/clocksource/timer-ti-dm-systimer.c
index 33b3e8aa2cc5..b6f97960d8ee 100644
--- a/drivers/clocksource/timer-ti-dm-systimer.c
+++ b/drivers/clocksource/timer-ti-dm-systimer.c
@@ -2,6 +2,7 @@
#include <linux/clk.h>
#include <linux/clocksource.h>
#include <linux/clockchips.h>
+#include <linux/cpuhotplug.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/iopoll.h>
@@ -449,13 +450,13 @@ static int dmtimer_set_next_event(unsigned long cycles,
struct dmtimer_systimer *t = &clkevt->t;
void __iomem *pend = t->base + t->pend;
- writel_relaxed(0xffffffff - cycles, t->base + t->counter);
while (readl_relaxed(pend) & WP_TCRR)
cpu_relax();
+ writel_relaxed(0xffffffff - cycles, t->base + t->counter);
- writel_relaxed(OMAP_TIMER_CTRL_ST, t->base + t->ctrl);
while (readl_relaxed(pend) & WP_TCLR)
cpu_relax();
+ writel_relaxed(OMAP_TIMER_CTRL_ST, t->base + t->ctrl);
return 0;
}
@@ -490,18 +491,18 @@ static int dmtimer_set_periodic(struct clock_event_device *evt)
dmtimer_clockevent_shutdown(evt);
/* Looks like we need to first set the load value separately */
- writel_relaxed(clkevt->period, t->base + t->load);
while (readl_relaxed(pend) & WP_TLDR)
cpu_relax();
+ writel_relaxed(clkevt->period, t->base + t->load);
- writel_relaxed(clkevt->period, t->base + t->counter);
while (readl_relaxed(pend) & WP_TCRR)
cpu_relax();
+ writel_relaxed(clkevt->period, t->base + t->counter);
- writel_relaxed(OMAP_TIMER_CTRL_AR | OMAP_TIMER_CTRL_ST,
- t->base + t->ctrl);
while (readl_relaxed(pend) & WP_TCLR)
cpu_relax();
+ writel_relaxed(OMAP_TIMER_CTRL_AR | OMAP_TIMER_CTRL_ST,
+ t->base + t->ctrl);
return 0;
}
@@ -530,17 +531,17 @@ static void omap_clockevent_unidle(struct clock_event_device *evt)
writel_relaxed(OMAP_TIMER_INT_OVERFLOW, t->base + t->wakeup);
}
-static int __init dmtimer_clockevent_init(struct device_node *np)
+static int __init dmtimer_clkevt_init_common(struct dmtimer_clockevent *clkevt,
+ struct device_node *np,
+ unsigned int features,
+ const struct cpumask *cpumask,
+ const char *name,
+ int rating)
{
- struct dmtimer_clockevent *clkevt;
struct clock_event_device *dev;
struct dmtimer_systimer *t;
int error;
- clkevt = kzalloc(sizeof(*clkevt), GFP_KERNEL);
- if (!clkevt)
- return -ENOMEM;
-
t = &clkevt->t;
dev = &clkevt->dev;
@@ -548,24 +549,23 @@ static int __init dmtimer_clockevent_init(struct device_node *np)
* We mostly use cpuidle_coupled with ARM local timers for runtime,
* so there's probably no use for CLOCK_EVT_FEAT_DYNIRQ here.
*/
- dev->features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT;
- dev->rating = 300;
+ dev->features = features;
+ dev->rating = rating;
dev->set_next_event = dmtimer_set_next_event;
dev->set_state_shutdown = dmtimer_clockevent_shutdown;
dev->set_state_periodic = dmtimer_set_periodic;
dev->set_state_oneshot = dmtimer_clockevent_shutdown;
+ dev->set_state_oneshot_stopped = dmtimer_clockevent_shutdown;
dev->tick_resume = dmtimer_clockevent_shutdown;
- dev->cpumask = cpu_possible_mask;
+ dev->cpumask = cpumask;
dev->irq = irq_of_parse_and_map(np, 0);
- if (!dev->irq) {
- error = -ENXIO;
- goto err_out_free;
- }
+ if (!dev->irq)
+ return -ENXIO;
error = dmtimer_systimer_setup(np, &clkevt->t);
if (error)
- goto err_out_free;
+ return error;
clkevt->period = 0xffffffff - DIV_ROUND_CLOSEST(t->rate, HZ);
@@ -577,38 +577,132 @@ static int __init dmtimer_clockevent_init(struct device_node *np)
writel_relaxed(OMAP_TIMER_CTRL_POSTED, t->base + t->ifctrl);
error = request_irq(dev->irq, dmtimer_clockevent_interrupt,
- IRQF_TIMER, "clockevent", clkevt);
+ IRQF_TIMER, name, clkevt);
if (error)
goto err_out_unmap;
writel_relaxed(OMAP_TIMER_INT_OVERFLOW, t->base + t->irq_ena);
writel_relaxed(OMAP_TIMER_INT_OVERFLOW, t->base + t->wakeup);
- pr_info("TI gptimer clockevent: %s%lu Hz at %pOF\n",
- of_find_property(np, "ti,timer-alwon", NULL) ?
+ pr_info("TI gptimer %s: %s%lu Hz at %pOF\n",
+ name, of_find_property(np, "ti,timer-alwon", NULL) ?
"always-on " : "", t->rate, np->parent);
- clockevents_config_and_register(dev, t->rate,
- 3, /* Timer internal resynch latency */
+ return 0;
+
+err_out_unmap:
+ iounmap(t->base);
+
+ return error;
+}
+
+static int __init dmtimer_clockevent_init(struct device_node *np)
+{
+ struct dmtimer_clockevent *clkevt;
+ int error;
+
+ clkevt = kzalloc(sizeof(*clkevt), GFP_KERNEL);
+ if (!clkevt)
+ return -ENOMEM;
+
+ error = dmtimer_clkevt_init_common(clkevt, np,
+ CLOCK_EVT_FEAT_PERIODIC |
+ CLOCK_EVT_FEAT_ONESHOT,
+ cpu_possible_mask, "clockevent",
+ 300);
+ if (error)
+ goto err_out_free;
+
+ clockevents_config_and_register(&clkevt->dev, clkevt->t.rate,
+ 3, /* Timer internal resync latency */
0xffffffff);
if (of_machine_is_compatible("ti,am33xx") ||
of_machine_is_compatible("ti,am43")) {
- dev->suspend = omap_clockevent_idle;
- dev->resume = omap_clockevent_unidle;
+ clkevt->dev.suspend = omap_clockevent_idle;
+ clkevt->dev.resume = omap_clockevent_unidle;
}
return 0;
-err_out_unmap:
- iounmap(t->base);
-
err_out_free:
kfree(clkevt);
return error;
}
+/* Dmtimer as percpu timer. See dra7 ARM architected timer wrap erratum i940 */
+static DEFINE_PER_CPU(struct dmtimer_clockevent, dmtimer_percpu_timer);
+
+static int __init dmtimer_percpu_timer_init(struct device_node *np, int cpu)
+{
+ struct dmtimer_clockevent *clkevt;
+ int error;
+
+ if (!cpu_possible(cpu))
+ return -EINVAL;
+
+ if (!of_property_read_bool(np->parent, "ti,no-reset-on-init") ||
+ !of_property_read_bool(np->parent, "ti,no-idle"))
+ pr_warn("Incomplete dtb for percpu dmtimer %pOF\n", np->parent);
+
+ clkevt = per_cpu_ptr(&dmtimer_percpu_timer, cpu);
+
+ error = dmtimer_clkevt_init_common(clkevt, np, CLOCK_EVT_FEAT_ONESHOT,
+ cpumask_of(cpu), "percpu-dmtimer",
+ 500);
+ if (error)
+ return error;
+
+ return 0;
+}
+
+/* See TRM for timer internal resynch latency */
+static int omap_dmtimer_starting_cpu(unsigned int cpu)
+{
+ struct dmtimer_clockevent *clkevt = per_cpu_ptr(&dmtimer_percpu_timer, cpu);
+ struct clock_event_device *dev = &clkevt->dev;
+ struct dmtimer_systimer *t = &clkevt->t;
+
+ clockevents_config_and_register(dev, t->rate, 3, ULONG_MAX);
+ irq_force_affinity(dev->irq, cpumask_of(cpu));
+
+ return 0;
+}
+
+static int __init dmtimer_percpu_timer_startup(void)
+{
+ struct dmtimer_clockevent *clkevt = per_cpu_ptr(&dmtimer_percpu_timer, 0);
+ struct dmtimer_systimer *t = &clkevt->t;
+
+ if (t->sysc) {
+ cpuhp_setup_state(CPUHP_AP_TI_GP_TIMER_STARTING,
+ "clockevents/omap/gptimer:starting",
+ omap_dmtimer_starting_cpu, NULL);
+ }
+
+ return 0;
+}
+subsys_initcall(dmtimer_percpu_timer_startup);
+
+static int __init dmtimer_percpu_quirk_init(struct device_node *np, u32 pa)
+{
+ struct device_node *arm_timer;
+
+ arm_timer = of_find_compatible_node(NULL, NULL, "arm,armv7-timer");
+ if (of_device_is_available(arm_timer)) {
+ pr_warn_once("ARM architected timer wrap issue i940 detected\n");
+ return 0;
+ }
+
+ if (pa == 0x48034000) /* dra7 dmtimer3 */
+ return dmtimer_percpu_timer_init(np, 0);
+ else if (pa == 0x48036000) /* dra7 dmtimer4 */
+ return dmtimer_percpu_timer_init(np, 1);
+
+ return 0;
+}
+
/* Clocksource */
static struct dmtimer_clocksource *
to_dmtimer_clocksource(struct clocksource *cs)
@@ -742,6 +836,9 @@ static int __init dmtimer_systimer_init(struct device_node *np)
if (clockevent == pa)
return dmtimer_clockevent_init(np);
+ if (of_machine_is_compatible("ti,dra7"))
+ return dmtimer_percpu_quirk_init(np, pa);
+
return 0;
}
diff --git a/drivers/clocksource/timer-vf-pit.c b/drivers/clocksource/timer-vf-pit.c
index 1a86a4e7e344..911c92146eca 100644
--- a/drivers/clocksource/timer-vf-pit.c
+++ b/drivers/clocksource/timer-vf-pit.c
@@ -136,7 +136,7 @@ static int __init pit_clockevent_init(unsigned long rate, int irq)
/*
* The value for the LDVAL register trigger is calculated as:
* LDVAL trigger = (period / clock period) - 1
- * The pit is a 32-bit down count timer, when the conter value
+ * The pit is a 32-bit down count timer, when the counter value
* reaches 0, it will generate an interrupt, thus the minimal
* LDVAL trigger value is 1. And then the min_delta is
* minimal LDVAL trigger value + 1, and the max_delta is full 32-bit.
diff --git a/drivers/staging/comedi/Kconfig b/drivers/comedi/Kconfig
index 049b659fa6ad..3cb61fa2c5c3 100644
--- a/drivers/staging/comedi/Kconfig
+++ b/drivers/comedi/Kconfig
@@ -1319,4 +1319,37 @@ config COMEDI_NI_TIO
config COMEDI_NI_ROUTING
tristate
+config COMEDI_TESTS
+ tristate "Comedi unit tests"
+ help
+ Enable comedi unit-test modules to be built.
+
+ Note that the answer to this question won't directly affect the
+ kernel: saying N will just cause the configurator to skip all
+ the questions about comedi unit-test modules.
+
+if COMEDI_TESTS
+
+config COMEDI_TESTS_EXAMPLE
+ tristate "Comedi example unit-test module"
+ help
+ Enable support for an example unit-test module. This is just a
+ silly example to be used as a basis for writing other unit-test
+ modules.
+
+ To compile this as a module, choose M here: the module will be called
+ comedi_example_test.
+
+config COMEDI_TESTS_NI_ROUTES
+ tristate "NI routing unit-test module"
+ select COMEDI_NI_ROUTING
+ help
+ Enable support for a unit-test module to test the signal routing
+ code used by comedi drivers for various National Instruments cards.
+
+ To compile this as a module, choose M here: the module will be called
+ ni_routes_test.
+
+endif # COMEDI_TESTS
+
endif # COMEDI
diff --git a/drivers/staging/comedi/Makefile b/drivers/comedi/Makefile
index 072ed83a5a6a..072ed83a5a6a 100644
--- a/drivers/staging/comedi/Makefile
+++ b/drivers/comedi/Makefile
diff --git a/drivers/staging/comedi/TODO b/drivers/comedi/TODO
index f733c017f181..f733c017f181 100644
--- a/drivers/staging/comedi/TODO
+++ b/drivers/comedi/TODO
diff --git a/drivers/staging/comedi/comedi.h b/drivers/comedi/comedi.h
index b5d00a006dbb..b5d00a006dbb 100644
--- a/drivers/staging/comedi/comedi.h
+++ b/drivers/comedi/comedi.h
diff --git a/drivers/staging/comedi/comedi_buf.c b/drivers/comedi/comedi_buf.c
index 3ef3ddabf139..06bfc859ab31 100644
--- a/drivers/staging/comedi/comedi_buf.c
+++ b/drivers/comedi/comedi_buf.c
@@ -371,35 +371,35 @@ static unsigned int comedi_buf_munge(struct comedi_subdevice *s,
if (!s->munge || (async->cmd.flags & CMDF_RAWDATA)) {
async->munge_count += num_bytes;
- count = num_bytes;
- } else {
- /* don't munge partial samples */
- num_bytes -= num_bytes % num_sample_bytes;
- while (count < num_bytes) {
- int block_size = num_bytes - count;
- unsigned int buf_end;
+ return num_bytes;
+ }
- buf_end = async->prealloc_bufsz - async->munge_ptr;
- if (block_size > buf_end)
- block_size = buf_end;
+ /* don't munge partial samples */
+ num_bytes -= num_bytes % num_sample_bytes;
+ while (count < num_bytes) {
+ int block_size = num_bytes - count;
+ unsigned int buf_end;
- s->munge(s->device, s,
- async->prealloc_buf + async->munge_ptr,
- block_size, async->munge_chan);
+ buf_end = async->prealloc_bufsz - async->munge_ptr;
+ if (block_size > buf_end)
+ block_size = buf_end;
- /*
- * ensure data is munged in buffer before the
- * async buffer munge_count is incremented
- */
- smp_wmb();
-
- async->munge_chan += block_size / num_sample_bytes;
- async->munge_chan %= async->cmd.chanlist_len;
- async->munge_count += block_size;
- async->munge_ptr += block_size;
- async->munge_ptr %= async->prealloc_bufsz;
- count += block_size;
- }
+ s->munge(s->device, s,
+ async->prealloc_buf + async->munge_ptr,
+ block_size, async->munge_chan);
+
+ /*
+ * ensure data is munged in buffer before the
+ * async buffer munge_count is incremented
+ */
+ smp_wmb();
+
+ async->munge_chan += block_size / num_sample_bytes;
+ async->munge_chan %= async->cmd.chanlist_len;
+ async->munge_count += block_size;
+ async->munge_ptr += block_size;
+ async->munge_ptr %= async->prealloc_bufsz;
+ count += block_size;
}
return count;
diff --git a/drivers/staging/comedi/comedi_fops.c b/drivers/comedi/comedi_fops.c
index df77b6bf5c64..df77b6bf5c64 100644
--- a/drivers/staging/comedi/comedi_fops.c
+++ b/drivers/comedi/comedi_fops.c
diff --git a/drivers/staging/comedi/comedi_internal.h b/drivers/comedi/comedi_internal.h
index 9b3631a654c8..9b3631a654c8 100644
--- a/drivers/staging/comedi/comedi_internal.h
+++ b/drivers/comedi/comedi_internal.h
diff --git a/drivers/staging/comedi/comedi_pci.c b/drivers/comedi/comedi_pci.c
index 54739af7eb71..54739af7eb71 100644
--- a/drivers/staging/comedi/comedi_pci.c
+++ b/drivers/comedi/comedi_pci.c
diff --git a/drivers/staging/comedi/comedi_pci.h b/drivers/comedi/comedi_pci.h
index 4e069440cbdc..4e069440cbdc 100644
--- a/drivers/staging/comedi/comedi_pci.h
+++ b/drivers/comedi/comedi_pci.h
diff --git a/drivers/staging/comedi/comedi_pcmcia.c b/drivers/comedi/comedi_pcmcia.c
index bb273bb202e6..bb273bb202e6 100644
--- a/drivers/staging/comedi/comedi_pcmcia.c
+++ b/drivers/comedi/comedi_pcmcia.c
diff --git a/drivers/staging/comedi/comedi_pcmcia.h b/drivers/comedi/comedi_pcmcia.h
index f2f6e779645b..f2f6e779645b 100644
--- a/drivers/staging/comedi/comedi_pcmcia.h
+++ b/drivers/comedi/comedi_pcmcia.h
diff --git a/drivers/staging/comedi/comedi_usb.c b/drivers/comedi/comedi_usb.c
index eea8ebf32ed0..eea8ebf32ed0 100644
--- a/drivers/staging/comedi/comedi_usb.c
+++ b/drivers/comedi/comedi_usb.c
diff --git a/drivers/staging/comedi/comedi_usb.h b/drivers/comedi/comedi_usb.h
index 601e29d3891c..601e29d3891c 100644
--- a/drivers/staging/comedi/comedi_usb.h
+++ b/drivers/comedi/comedi_usb.h
diff --git a/drivers/staging/comedi/comedidev.h b/drivers/comedi/comedidev.h
index 0e1b95ef9a4d..0e1b95ef9a4d 100644
--- a/drivers/staging/comedi/comedidev.h
+++ b/drivers/comedi/comedidev.h
diff --git a/drivers/staging/comedi/comedilib.h b/drivers/comedi/comedilib.h
index 0223c9cd9215..0223c9cd9215 100644
--- a/drivers/staging/comedi/comedilib.h
+++ b/drivers/comedi/comedilib.h
diff --git a/drivers/staging/comedi/drivers.c b/drivers/comedi/drivers.c
index 750a6ff3c03c..750a6ff3c03c 100644
--- a/drivers/staging/comedi/drivers.c
+++ b/drivers/comedi/drivers.c
diff --git a/drivers/staging/comedi/drivers/8255.c b/drivers/comedi/drivers/8255.c
index e23335c75867..e23335c75867 100644
--- a/drivers/staging/comedi/drivers/8255.c
+++ b/drivers/comedi/drivers/8255.c
diff --git a/drivers/staging/comedi/drivers/8255.h b/drivers/comedi/drivers/8255.h
index ceae3ca52e60..ceae3ca52e60 100644
--- a/drivers/staging/comedi/drivers/8255.h
+++ b/drivers/comedi/drivers/8255.h
diff --git a/drivers/staging/comedi/drivers/8255_pci.c b/drivers/comedi/drivers/8255_pci.c
index 5a810f0e532a..5a810f0e532a 100644
--- a/drivers/staging/comedi/drivers/8255_pci.c
+++ b/drivers/comedi/drivers/8255_pci.c
diff --git a/drivers/staging/comedi/drivers/Makefile b/drivers/comedi/drivers/Makefile
index b24ac00cab73..b24ac00cab73 100644
--- a/drivers/staging/comedi/drivers/Makefile
+++ b/drivers/comedi/drivers/Makefile
diff --git a/drivers/staging/comedi/drivers/addi_apci_1032.c b/drivers/comedi/drivers/addi_apci_1032.c
index 35b75f0c9200..81a246fbcc01 100644
--- a/drivers/staging/comedi/drivers/addi_apci_1032.c
+++ b/drivers/comedi/drivers/addi_apci_1032.c
@@ -260,6 +260,7 @@ static irqreturn_t apci1032_interrupt(int irq, void *d)
struct apci1032_private *devpriv = dev->private;
struct comedi_subdevice *s = dev->read_subdev;
unsigned int ctrl;
+ unsigned short val;
/* check interrupt is from this device */
if ((inl(devpriv->amcc_iobase + AMCC_OP_REG_INTCSR) &
@@ -275,7 +276,8 @@ static irqreturn_t apci1032_interrupt(int irq, void *d)
outl(ctrl & ~APCI1032_CTRL_INT_ENA, dev->iobase + APCI1032_CTRL_REG);
s->state = inl(dev->iobase + APCI1032_STATUS_REG) & 0xffff;
- comedi_buf_write_samples(s, &s->state, 1);
+ val = s->state;
+ comedi_buf_write_samples(s, &val, 1);
comedi_handle_events(dev, s);
/* enable the interrupt */
diff --git a/drivers/staging/comedi/drivers/addi_apci_1500.c b/drivers/comedi/drivers/addi_apci_1500.c
index 11efb21555e3..b04c15dcfb57 100644
--- a/drivers/staging/comedi/drivers/addi_apci_1500.c
+++ b/drivers/comedi/drivers/addi_apci_1500.c
@@ -208,7 +208,7 @@ static irqreturn_t apci1500_interrupt(int irq, void *d)
struct comedi_device *dev = d;
struct apci1500_private *devpriv = dev->private;
struct comedi_subdevice *s = dev->read_subdev;
- unsigned int status = 0;
+ unsigned short status = 0;
unsigned int val;
val = inl(devpriv->amcc + AMCC_OP_REG_INTCSR);
@@ -238,14 +238,14 @@ static irqreturn_t apci1500_interrupt(int irq, void *d)
*
* Mask Meaning
* ---------- ------------------------------------------
- * 0x00000001 Event 1 has occurred
- * 0x00000010 Event 2 has occurred
- * 0x00000100 Counter/timer 1 has run down (not implemented)
- * 0x00001000 Counter/timer 2 has run down (not implemented)
- * 0x00010000 Counter 3 has run down (not implemented)
- * 0x00100000 Watchdog has run down (not implemented)
- * 0x01000000 Voltage error
- * 0x10000000 Short-circuit error
+ * 0b00000001 Event 1 has occurred
+ * 0b00000010 Event 2 has occurred
+ * 0b00000100 Counter/timer 1 has run down (not implemented)
+ * 0b00001000 Counter/timer 2 has run down (not implemented)
+ * 0b00010000 Counter 3 has run down (not implemented)
+ * 0b00100000 Watchdog has run down (not implemented)
+ * 0b01000000 Voltage error
+ * 0b10000000 Short-circuit error
*/
comedi_buf_write_samples(s, &status, 1);
comedi_handle_events(dev, s);
diff --git a/drivers/staging/comedi/drivers/addi_apci_1516.c b/drivers/comedi/drivers/addi_apci_1516.c
index 274ec9fb030c..274ec9fb030c 100644
--- a/drivers/staging/comedi/drivers/addi_apci_1516.c
+++ b/drivers/comedi/drivers/addi_apci_1516.c
diff --git a/drivers/staging/comedi/drivers/addi_apci_1564.c b/drivers/comedi/drivers/addi_apci_1564.c
index 06fc7ed96200..06fc7ed96200 100644
--- a/drivers/staging/comedi/drivers/addi_apci_1564.c
+++ b/drivers/comedi/drivers/addi_apci_1564.c
diff --git a/drivers/staging/comedi/drivers/addi_apci_16xx.c b/drivers/comedi/drivers/addi_apci_16xx.c
index 9bbef3b15f3f..c306aa41df97 100644
--- a/drivers/staging/comedi/drivers/addi_apci_16xx.c
+++ b/drivers/comedi/drivers/addi_apci_16xx.c
@@ -110,7 +110,7 @@ static int apci16xx_auto_attach(struct comedi_device *dev,
dev->iobase = pci_resource_start(pcidev, 0);
/*
- * Work out the nubmer of subdevices needed to support all the
+ * Work out the number of subdevices needed to support all the
* digital i/o channels on the board. Each subdevice supports
* up to 32 channels.
*/
diff --git a/drivers/staging/comedi/drivers/addi_apci_2032.c b/drivers/comedi/drivers/addi_apci_2032.c
index e9a2b37a4ae0..e9a2b37a4ae0 100644
--- a/drivers/staging/comedi/drivers/addi_apci_2032.c
+++ b/drivers/comedi/drivers/addi_apci_2032.c
diff --git a/drivers/staging/comedi/drivers/addi_apci_2200.c b/drivers/comedi/drivers/addi_apci_2200.c
index 4c5aee784bd9..4c5aee784bd9 100644
--- a/drivers/staging/comedi/drivers/addi_apci_2200.c
+++ b/drivers/comedi/drivers/addi_apci_2200.c
diff --git a/drivers/staging/comedi/drivers/addi_apci_3120.c b/drivers/comedi/drivers/addi_apci_3120.c
index 1ed3b33d1a30..1ed3b33d1a30 100644
--- a/drivers/staging/comedi/drivers/addi_apci_3120.c
+++ b/drivers/comedi/drivers/addi_apci_3120.c
diff --git a/drivers/staging/comedi/drivers/addi_apci_3501.c b/drivers/comedi/drivers/addi_apci_3501.c
index f0c9642f3f1a..f0c9642f3f1a 100644
--- a/drivers/staging/comedi/drivers/addi_apci_3501.c
+++ b/drivers/comedi/drivers/addi_apci_3501.c
diff --git a/drivers/staging/comedi/drivers/addi_apci_3xxx.c b/drivers/comedi/drivers/addi_apci_3xxx.c
index a90d59377e18..a90d59377e18 100644
--- a/drivers/staging/comedi/drivers/addi_apci_3xxx.c
+++ b/drivers/comedi/drivers/addi_apci_3xxx.c
diff --git a/drivers/staging/comedi/drivers/addi_tcw.h b/drivers/comedi/drivers/addi_tcw.h
index 2b44d3a04484..2b44d3a04484 100644
--- a/drivers/staging/comedi/drivers/addi_tcw.h
+++ b/drivers/comedi/drivers/addi_tcw.h
diff --git a/drivers/staging/comedi/drivers/addi_watchdog.c b/drivers/comedi/drivers/addi_watchdog.c
index 69b323fb869f..69b323fb869f 100644
--- a/drivers/staging/comedi/drivers/addi_watchdog.c
+++ b/drivers/comedi/drivers/addi_watchdog.c
diff --git a/drivers/staging/comedi/drivers/addi_watchdog.h b/drivers/comedi/drivers/addi_watchdog.h
index 7523084a0742..7523084a0742 100644
--- a/drivers/staging/comedi/drivers/addi_watchdog.h
+++ b/drivers/comedi/drivers/addi_watchdog.h
diff --git a/drivers/staging/comedi/drivers/adl_pci6208.c b/drivers/comedi/drivers/adl_pci6208.c
index 9ae4cc523dd4..9ae4cc523dd4 100644
--- a/drivers/staging/comedi/drivers/adl_pci6208.c
+++ b/drivers/comedi/drivers/adl_pci6208.c
diff --git a/drivers/staging/comedi/drivers/adl_pci7x3x.c b/drivers/comedi/drivers/adl_pci7x3x.c
index 8fc45638ff59..8fc45638ff59 100644
--- a/drivers/staging/comedi/drivers/adl_pci7x3x.c
+++ b/drivers/comedi/drivers/adl_pci7x3x.c
diff --git a/drivers/staging/comedi/drivers/adl_pci8164.c b/drivers/comedi/drivers/adl_pci8164.c
index d5e1bda81557..d5e1bda81557 100644
--- a/drivers/staging/comedi/drivers/adl_pci8164.c
+++ b/drivers/comedi/drivers/adl_pci8164.c
diff --git a/drivers/staging/comedi/drivers/adl_pci9111.c b/drivers/comedi/drivers/adl_pci9111.c
index a062c5ab20e9..a062c5ab20e9 100644
--- a/drivers/staging/comedi/drivers/adl_pci9111.c
+++ b/drivers/comedi/drivers/adl_pci9111.c
diff --git a/drivers/staging/comedi/drivers/adl_pci9118.c b/drivers/comedi/drivers/adl_pci9118.c
index cda3a4267dca..cda3a4267dca 100644
--- a/drivers/staging/comedi/drivers/adl_pci9118.c
+++ b/drivers/comedi/drivers/adl_pci9118.c
diff --git a/drivers/staging/comedi/drivers/adq12b.c b/drivers/comedi/drivers/adq12b.c
index d719f76709ef..d719f76709ef 100644
--- a/drivers/staging/comedi/drivers/adq12b.c
+++ b/drivers/comedi/drivers/adq12b.c
diff --git a/drivers/staging/comedi/drivers/adv_pci1710.c b/drivers/comedi/drivers/adv_pci1710.c
index 692893c7e5c3..090607760be6 100644
--- a/drivers/staging/comedi/drivers/adv_pci1710.c
+++ b/drivers/comedi/drivers/adv_pci1710.c
@@ -300,11 +300,11 @@ static int pci1710_ai_eoc(struct comedi_device *dev,
static int pci1710_ai_read_sample(struct comedi_device *dev,
struct comedi_subdevice *s,
unsigned int cur_chan,
- unsigned int *val)
+ unsigned short *val)
{
const struct boardtype *board = dev->board_ptr;
struct pci1710_private *devpriv = dev->private;
- unsigned int sample;
+ unsigned short sample;
unsigned int chan;
sample = inw(dev->iobase + PCI171X_AD_DATA_REG);
@@ -345,7 +345,7 @@ static int pci1710_ai_insn_read(struct comedi_device *dev,
pci1710_ai_setup_chanlist(dev, s, &insn->chanspec, 1, 1);
for (i = 0; i < insn->n; i++) {
- unsigned int val;
+ unsigned short val;
/* start conversion */
outw(0, dev->iobase + PCI171X_SOFTTRG_REG);
@@ -395,7 +395,7 @@ static void pci1710_handle_every_sample(struct comedi_device *dev,
{
struct comedi_cmd *cmd = &s->async->cmd;
unsigned int status;
- unsigned int val;
+ unsigned short val;
int ret;
status = inw(dev->iobase + PCI171X_STATUS_REG);
@@ -455,7 +455,7 @@ static void pci1710_handle_fifo(struct comedi_device *dev,
}
for (i = 0; i < devpriv->max_samples; i++) {
- unsigned int val;
+ unsigned short val;
int ret;
ret = pci1710_ai_read_sample(dev, s, s->async->cur_chan, &val);
diff --git a/drivers/staging/comedi/drivers/adv_pci1720.c b/drivers/comedi/drivers/adv_pci1720.c
index 2fcd7e8e7d85..2fcd7e8e7d85 100644
--- a/drivers/staging/comedi/drivers/adv_pci1720.c
+++ b/drivers/comedi/drivers/adv_pci1720.c
diff --git a/drivers/staging/comedi/drivers/adv_pci1723.c b/drivers/comedi/drivers/adv_pci1723.c
index 23660a9fdb9c..23660a9fdb9c 100644
--- a/drivers/staging/comedi/drivers/adv_pci1723.c
+++ b/drivers/comedi/drivers/adv_pci1723.c
diff --git a/drivers/staging/comedi/drivers/adv_pci1724.c b/drivers/comedi/drivers/adv_pci1724.c
index e8ab573c839f..e8ab573c839f 100644
--- a/drivers/staging/comedi/drivers/adv_pci1724.c
+++ b/drivers/comedi/drivers/adv_pci1724.c
diff --git a/drivers/staging/comedi/drivers/adv_pci1760.c b/drivers/comedi/drivers/adv_pci1760.c
index 6de8ab97d346..6de8ab97d346 100644
--- a/drivers/staging/comedi/drivers/adv_pci1760.c
+++ b/drivers/comedi/drivers/adv_pci1760.c
diff --git a/drivers/staging/comedi/drivers/adv_pci_dio.c b/drivers/comedi/drivers/adv_pci_dio.c
index 8e222b6ff2b4..54c7419c8ca6 100644
--- a/drivers/staging/comedi/drivers/adv_pci_dio.c
+++ b/drivers/comedi/drivers/adv_pci_dio.c
@@ -604,7 +604,6 @@ static int pci_dio_auto_attach(struct comedi_device *dev,
? pci_dio_insn_bits_di_w
: pci_dio_insn_bits_di_b;
s->private = (void *)d->addr;
-
}
}
diff --git a/drivers/staging/comedi/drivers/aio_aio12_8.c b/drivers/comedi/drivers/aio_aio12_8.c
index 4829115921a3..4829115921a3 100644
--- a/drivers/staging/comedi/drivers/aio_aio12_8.c
+++ b/drivers/comedi/drivers/aio_aio12_8.c
diff --git a/drivers/staging/comedi/drivers/aio_iiro_16.c b/drivers/comedi/drivers/aio_iiro_16.c
index fe3876235075..fe3876235075 100644
--- a/drivers/staging/comedi/drivers/aio_iiro_16.c
+++ b/drivers/comedi/drivers/aio_iiro_16.c
diff --git a/drivers/staging/comedi/drivers/amcc_s5933.h b/drivers/comedi/drivers/amcc_s5933.h
index f738b91b2052..f738b91b2052 100644
--- a/drivers/staging/comedi/drivers/amcc_s5933.h
+++ b/drivers/comedi/drivers/amcc_s5933.h
diff --git a/drivers/staging/comedi/drivers/amplc_dio200.c b/drivers/comedi/drivers/amplc_dio200.c
index fa19c9e7c56b..fa19c9e7c56b 100644
--- a/drivers/staging/comedi/drivers/amplc_dio200.c
+++ b/drivers/comedi/drivers/amplc_dio200.c
diff --git a/drivers/staging/comedi/drivers/amplc_dio200.h b/drivers/comedi/drivers/amplc_dio200.h
index 745baaf940ee..745baaf940ee 100644
--- a/drivers/staging/comedi/drivers/amplc_dio200.h
+++ b/drivers/comedi/drivers/amplc_dio200.h
diff --git a/drivers/staging/comedi/drivers/amplc_dio200_common.c b/drivers/comedi/drivers/amplc_dio200_common.c
index a3454130d5f8..a3454130d5f8 100644
--- a/drivers/staging/comedi/drivers/amplc_dio200_common.c
+++ b/drivers/comedi/drivers/amplc_dio200_common.c
diff --git a/drivers/staging/comedi/drivers/amplc_dio200_pci.c b/drivers/comedi/drivers/amplc_dio200_pci.c
index 1bd7a42c8464..1bd7a42c8464 100644
--- a/drivers/staging/comedi/drivers/amplc_dio200_pci.c
+++ b/drivers/comedi/drivers/amplc_dio200_pci.c
diff --git a/drivers/staging/comedi/drivers/amplc_pc236.c b/drivers/comedi/drivers/amplc_pc236.c
index c377af1d5246..c377af1d5246 100644
--- a/drivers/staging/comedi/drivers/amplc_pc236.c
+++ b/drivers/comedi/drivers/amplc_pc236.c
diff --git a/drivers/staging/comedi/drivers/amplc_pc236.h b/drivers/comedi/drivers/amplc_pc236.h
index 7e72729f7492..7e72729f7492 100644
--- a/drivers/staging/comedi/drivers/amplc_pc236.h
+++ b/drivers/comedi/drivers/amplc_pc236.h
diff --git a/drivers/staging/comedi/drivers/amplc_pc236_common.c b/drivers/comedi/drivers/amplc_pc236_common.c
index 043752663188..981d281e87a1 100644
--- a/drivers/staging/comedi/drivers/amplc_pc236_common.c
+++ b/drivers/comedi/drivers/amplc_pc236_common.c
@@ -126,7 +126,9 @@ static irqreturn_t pc236_interrupt(int irq, void *d)
handled = pc236_intr_check(dev);
if (dev->attached && handled) {
- comedi_buf_write_samples(s, &s->state, 1);
+ unsigned short val = 0;
+
+ comedi_buf_write_samples(s, &val, 1);
comedi_handle_events(dev, s);
}
return IRQ_RETVAL(handled);
diff --git a/drivers/staging/comedi/drivers/amplc_pc263.c b/drivers/comedi/drivers/amplc_pc263.c
index 68da6098ee84..68da6098ee84 100644
--- a/drivers/staging/comedi/drivers/amplc_pc263.c
+++ b/drivers/comedi/drivers/amplc_pc263.c
diff --git a/drivers/staging/comedi/drivers/amplc_pci224.c b/drivers/comedi/drivers/amplc_pci224.c
index bcf6d61af863..bcf6d61af863 100644
--- a/drivers/staging/comedi/drivers/amplc_pci224.c
+++ b/drivers/comedi/drivers/amplc_pci224.c
diff --git a/drivers/staging/comedi/drivers/amplc_pci230.c b/drivers/comedi/drivers/amplc_pci230.c
index 8911dc2bd2c6..8911dc2bd2c6 100644
--- a/drivers/staging/comedi/drivers/amplc_pci230.c
+++ b/drivers/comedi/drivers/amplc_pci230.c
diff --git a/drivers/staging/comedi/drivers/amplc_pci236.c b/drivers/comedi/drivers/amplc_pci236.c
index e7f6fa4d101a..e7f6fa4d101a 100644
--- a/drivers/staging/comedi/drivers/amplc_pci236.c
+++ b/drivers/comedi/drivers/amplc_pci236.c
diff --git a/drivers/staging/comedi/drivers/amplc_pci263.c b/drivers/comedi/drivers/amplc_pci263.c
index 9217973f1141..9217973f1141 100644
--- a/drivers/staging/comedi/drivers/amplc_pci263.c
+++ b/drivers/comedi/drivers/amplc_pci263.c
diff --git a/drivers/staging/comedi/drivers/c6xdigio.c b/drivers/comedi/drivers/c6xdigio.c
index 786fd15698df..786fd15698df 100644
--- a/drivers/staging/comedi/drivers/c6xdigio.c
+++ b/drivers/comedi/drivers/c6xdigio.c
diff --git a/drivers/staging/comedi/drivers/cb_das16_cs.c b/drivers/comedi/drivers/cb_das16_cs.c
index a5d171e71c33..a5d171e71c33 100644
--- a/drivers/staging/comedi/drivers/cb_das16_cs.c
+++ b/drivers/comedi/drivers/cb_das16_cs.c
diff --git a/drivers/staging/comedi/drivers/cb_pcidas.c b/drivers/comedi/drivers/cb_pcidas.c
index d740c4782775..2f20bd56ec6c 100644
--- a/drivers/staging/comedi/drivers/cb_pcidas.c
+++ b/drivers/comedi/drivers/cb_pcidas.c
@@ -1281,7 +1281,7 @@ static int cb_pcidas_auto_attach(struct comedi_device *dev,
devpriv->amcc + AMCC_OP_REG_INTCSR);
ret = request_irq(pcidev->irq, cb_pcidas_interrupt, IRQF_SHARED,
- dev->board_name, dev);
+ "cb_pcidas", dev);
if (ret) {
dev_dbg(dev->class_dev, "unable to allocate irq %d\n",
pcidev->irq);
diff --git a/drivers/staging/comedi/drivers/cb_pcidas64.c b/drivers/comedi/drivers/cb_pcidas64.c
index fa987bb0e7cd..41a8fea7f48a 100644
--- a/drivers/staging/comedi/drivers/cb_pcidas64.c
+++ b/drivers/comedi/drivers/cb_pcidas64.c
@@ -229,11 +229,6 @@ enum daq_atrig_low_4020_contents {
EXT_START_TRIG_BNC_BIT = 0x2000,
};
-static inline u16 analog_trig_low_threshold_bits(u16 threshold)
-{
- return threshold & 0xfff;
-}
-
enum adc_control0_contents {
ADC_GATE_SRC_MASK = 0x3, /* bits that select gate */
ADC_SOFT_GATE_BITS = 0x1, /* software gate */
@@ -4035,7 +4030,7 @@ static int auto_attach(struct comedi_device *dev,
init_stc_registers(dev);
retval = request_irq(pcidev->irq, handle_interrupt, IRQF_SHARED,
- dev->board_name, dev);
+ "cb_pcidas64", dev);
if (retval) {
dev_dbg(dev->class_dev, "unable to allocate irq %u\n",
pcidev->irq);
diff --git a/drivers/staging/comedi/drivers/cb_pcidda.c b/drivers/comedi/drivers/cb_pcidda.c
index 78cf1603638c..78cf1603638c 100644
--- a/drivers/staging/comedi/drivers/cb_pcidda.c
+++ b/drivers/comedi/drivers/cb_pcidda.c
diff --git a/drivers/staging/comedi/drivers/cb_pcimdas.c b/drivers/comedi/drivers/cb_pcimdas.c
index 2292f69da4f4..2292f69da4f4 100644
--- a/drivers/staging/comedi/drivers/cb_pcimdas.c
+++ b/drivers/comedi/drivers/cb_pcimdas.c
diff --git a/drivers/staging/comedi/drivers/cb_pcimdda.c b/drivers/comedi/drivers/cb_pcimdda.c
index 21fc7b3c5f60..21fc7b3c5f60 100644
--- a/drivers/staging/comedi/drivers/cb_pcimdda.c
+++ b/drivers/comedi/drivers/cb_pcimdda.c
diff --git a/drivers/staging/comedi/drivers/comedi_8254.c b/drivers/comedi/drivers/comedi_8254.c
index d1d509e9add9..d1d509e9add9 100644
--- a/drivers/staging/comedi/drivers/comedi_8254.c
+++ b/drivers/comedi/drivers/comedi_8254.c
diff --git a/drivers/staging/comedi/drivers/comedi_8254.h b/drivers/comedi/drivers/comedi_8254.h
index d8264417e53c..d8264417e53c 100644
--- a/drivers/staging/comedi/drivers/comedi_8254.h
+++ b/drivers/comedi/drivers/comedi_8254.h
diff --git a/drivers/staging/comedi/drivers/comedi_8255.c b/drivers/comedi/drivers/comedi_8255.c
index b7ca465933ee..b7ca465933ee 100644
--- a/drivers/staging/comedi/drivers/comedi_8255.c
+++ b/drivers/comedi/drivers/comedi_8255.c
diff --git a/drivers/staging/comedi/drivers/comedi_bond.c b/drivers/comedi/drivers/comedi_bond.c
index 4392b5927a99..4392b5927a99 100644
--- a/drivers/staging/comedi/drivers/comedi_bond.c
+++ b/drivers/comedi/drivers/comedi_bond.c
diff --git a/drivers/staging/comedi/drivers/comedi_isadma.c b/drivers/comedi/drivers/comedi_isadma.c
index c729094298c2..c729094298c2 100644
--- a/drivers/staging/comedi/drivers/comedi_isadma.c
+++ b/drivers/comedi/drivers/comedi_isadma.c
diff --git a/drivers/staging/comedi/drivers/comedi_isadma.h b/drivers/comedi/drivers/comedi_isadma.h
index 9d2b12db7e6e..9d2b12db7e6e 100644
--- a/drivers/staging/comedi/drivers/comedi_isadma.h
+++ b/drivers/comedi/drivers/comedi_isadma.h
diff --git a/drivers/staging/comedi/drivers/comedi_parport.c b/drivers/comedi/drivers/comedi_parport.c
index 9361b2dcf949..5338b5eea440 100644
--- a/drivers/staging/comedi/drivers/comedi_parport.c
+++ b/drivers/comedi/drivers/comedi_parport.c
@@ -210,12 +210,13 @@ static irqreturn_t parport_interrupt(int irq, void *d)
struct comedi_device *dev = d;
struct comedi_subdevice *s = dev->read_subdev;
unsigned int ctrl;
+ unsigned short val = 0;
ctrl = inb(dev->iobase + PARPORT_CTRL_REG);
if (!(ctrl & PARPORT_CTRL_IRQ_ENA))
return IRQ_NONE;
- comedi_buf_write_samples(s, &s->state, 1);
+ comedi_buf_write_samples(s, &val, 1);
comedi_handle_events(dev, s);
return IRQ_HANDLED;
diff --git a/drivers/staging/comedi/drivers/comedi_test.c b/drivers/comedi/drivers/comedi_test.c
index cbc225eb1991..cbc225eb1991 100644
--- a/drivers/staging/comedi/drivers/comedi_test.c
+++ b/drivers/comedi/drivers/comedi_test.c
diff --git a/drivers/staging/comedi/drivers/contec_pci_dio.c b/drivers/comedi/drivers/contec_pci_dio.c
index b8fdd9c1f166..b8fdd9c1f166 100644
--- a/drivers/staging/comedi/drivers/contec_pci_dio.c
+++ b/drivers/comedi/drivers/contec_pci_dio.c
diff --git a/drivers/staging/comedi/drivers/dac02.c b/drivers/comedi/drivers/dac02.c
index 5ef8114c2c85..5ef8114c2c85 100644
--- a/drivers/staging/comedi/drivers/dac02.c
+++ b/drivers/comedi/drivers/dac02.c
diff --git a/drivers/staging/comedi/drivers/daqboard2000.c b/drivers/comedi/drivers/daqboard2000.c
index f64e747078bd..f64e747078bd 100644
--- a/drivers/staging/comedi/drivers/daqboard2000.c
+++ b/drivers/comedi/drivers/daqboard2000.c
diff --git a/drivers/staging/comedi/drivers/das08.c b/drivers/comedi/drivers/das08.c
index b50743c5b822..b50743c5b822 100644
--- a/drivers/staging/comedi/drivers/das08.c
+++ b/drivers/comedi/drivers/das08.c
diff --git a/drivers/staging/comedi/drivers/das08.h b/drivers/comedi/drivers/das08.h
index ef65a7e504ee..ef65a7e504ee 100644
--- a/drivers/staging/comedi/drivers/das08.h
+++ b/drivers/comedi/drivers/das08.h
diff --git a/drivers/staging/comedi/drivers/das08_cs.c b/drivers/comedi/drivers/das08_cs.c
index 223479f9ea3c..223479f9ea3c 100644
--- a/drivers/staging/comedi/drivers/das08_cs.c
+++ b/drivers/comedi/drivers/das08_cs.c
diff --git a/drivers/staging/comedi/drivers/das08_isa.c b/drivers/comedi/drivers/das08_isa.c
index 8c4cfa821423..8c4cfa821423 100644
--- a/drivers/staging/comedi/drivers/das08_isa.c
+++ b/drivers/comedi/drivers/das08_isa.c
diff --git a/drivers/staging/comedi/drivers/das08_pci.c b/drivers/comedi/drivers/das08_pci.c
index 1cd903336a4c..1cd903336a4c 100644
--- a/drivers/staging/comedi/drivers/das08_pci.c
+++ b/drivers/comedi/drivers/das08_pci.c
diff --git a/drivers/staging/comedi/drivers/das16.c b/drivers/comedi/drivers/das16.c
index 4ac2622b0fac..4ac2622b0fac 100644
--- a/drivers/staging/comedi/drivers/das16.c
+++ b/drivers/comedi/drivers/das16.c
diff --git a/drivers/staging/comedi/drivers/das16m1.c b/drivers/comedi/drivers/das16m1.c
index 75f3dbbe97ac..75f3dbbe97ac 100644
--- a/drivers/staging/comedi/drivers/das16m1.c
+++ b/drivers/comedi/drivers/das16m1.c
diff --git a/drivers/staging/comedi/drivers/das1800.c b/drivers/comedi/drivers/das1800.c
index f50891a6ee7d..f50891a6ee7d 100644
--- a/drivers/staging/comedi/drivers/das1800.c
+++ b/drivers/comedi/drivers/das1800.c
diff --git a/drivers/staging/comedi/drivers/das6402.c b/drivers/comedi/drivers/das6402.c
index 04e224f8b779..96f4107b8054 100644
--- a/drivers/staging/comedi/drivers/das6402.c
+++ b/drivers/comedi/drivers/das6402.c
@@ -186,7 +186,7 @@ static irqreturn_t das6402_interrupt(int irq, void *d)
if (status & DAS6402_STATUS_FFULL) {
async->events |= COMEDI_CB_OVERFLOW;
} else if (status & DAS6402_STATUS_FFNE) {
- unsigned int val;
+ unsigned short val;
val = das6402_ai_read_sample(dev, s);
comedi_buf_write_samples(s, &val, 1);
diff --git a/drivers/staging/comedi/drivers/das800.c b/drivers/comedi/drivers/das800.c
index 4ea100ff6930..bc08324f422f 100644
--- a/drivers/staging/comedi/drivers/das800.c
+++ b/drivers/comedi/drivers/das800.c
@@ -427,7 +427,7 @@ static irqreturn_t das800_interrupt(int irq, void *d)
struct comedi_cmd *cmd;
unsigned long irq_flags;
unsigned int status;
- unsigned int val;
+ unsigned short val;
bool fifo_empty;
bool fifo_overflow;
int i;
@@ -668,7 +668,7 @@ static int das800_attach(struct comedi_device *dev, struct comedi_devconfig *it)
dev->board_name = board->name;
if (irq > 1 && irq <= 7) {
- ret = request_irq(irq, das800_interrupt, 0, dev->board_name,
+ ret = request_irq(irq, das800_interrupt, 0, "das800",
dev);
if (ret == 0)
dev->irq = irq;
diff --git a/drivers/staging/comedi/drivers/dmm32at.c b/drivers/comedi/drivers/dmm32at.c
index 17e6018918bb..56682f01242f 100644
--- a/drivers/staging/comedi/drivers/dmm32at.c
+++ b/drivers/comedi/drivers/dmm32at.c
@@ -404,7 +404,7 @@ static irqreturn_t dmm32at_isr(int irq, void *d)
{
struct comedi_device *dev = d;
unsigned char intstat;
- unsigned int val;
+ unsigned short val;
int i;
if (!dev->attached) {
diff --git a/drivers/staging/comedi/drivers/dt2801.c b/drivers/comedi/drivers/dt2801.c
index 0d571d817b4e..0d571d817b4e 100644
--- a/drivers/staging/comedi/drivers/dt2801.c
+++ b/drivers/comedi/drivers/dt2801.c
diff --git a/drivers/staging/comedi/drivers/dt2811.c b/drivers/comedi/drivers/dt2811.c
index 0eb5e6ba6916..0eb5e6ba6916 100644
--- a/drivers/staging/comedi/drivers/dt2811.c
+++ b/drivers/comedi/drivers/dt2811.c
diff --git a/drivers/staging/comedi/drivers/dt2814.c b/drivers/comedi/drivers/dt2814.c
index bcf4d5444faf..ed44ce0d151b 100644
--- a/drivers/staging/comedi/drivers/dt2814.c
+++ b/drivers/comedi/drivers/dt2814.c
@@ -44,14 +44,46 @@
#define DT2814_ENB 0x10
#define DT2814_CHANMASK 0x0f
-struct dt2814_private {
- int ntrig;
- int curadchan;
-};
-
#define DT2814_TIMEOUT 10
#define DT2814_MAX_SPEED 100000 /* Arbitrary 10 khz limit */
+static int dt2814_ai_notbusy(struct comedi_device *dev,
+ struct comedi_subdevice *s,
+ struct comedi_insn *insn,
+ unsigned long context)
+{
+ unsigned int status;
+
+ status = inb(dev->iobase + DT2814_CSR);
+ if (context)
+ *(unsigned int *)context = status;
+ if (status & DT2814_BUSY)
+ return -EBUSY;
+ return 0;
+}
+
+static int dt2814_ai_clear(struct comedi_device *dev)
+{
+ unsigned int status = 0;
+ int ret;
+
+ /* Wait until not busy and get status register value. */
+ ret = comedi_timeout(dev, NULL, NULL, dt2814_ai_notbusy,
+ (unsigned long)&status);
+ if (ret)
+ return ret;
+
+ if (status & (DT2814_FINISH | DT2814_ERR)) {
+ /*
+ * There unread data, or the error flag is set.
+ * Read the data register twice to clear the condition.
+ */
+ inb(dev->iobase + DT2814_DATA);
+ inb(dev->iobase + DT2814_DATA);
+ }
+ return 0;
+}
+
static int dt2814_ai_eoc(struct comedi_device *dev,
struct comedi_subdevice *s,
struct comedi_insn *insn,
@@ -73,6 +105,7 @@ static int dt2814_ai_insn_read(struct comedi_device *dev,
int chan;
int ret;
+ dt2814_ai_clear(dev); /* clear stale data or error */
for (n = 0; n < insn->n; n++) {
chan = CR_CHAN(insn->chanspec);
@@ -169,80 +202,119 @@ static int dt2814_ai_cmdtest(struct comedi_device *dev,
static int dt2814_ai_cmd(struct comedi_device *dev, struct comedi_subdevice *s)
{
- struct dt2814_private *devpriv = dev->private;
struct comedi_cmd *cmd = &s->async->cmd;
int chan;
int trigvar;
+ dt2814_ai_clear(dev); /* clear stale data or error */
trigvar = dt2814_ns_to_timer(&cmd->scan_begin_arg, cmd->flags);
chan = CR_CHAN(cmd->chanlist[0]);
- devpriv->ntrig = cmd->stop_arg;
outb(chan | DT2814_ENB | (trigvar << 5), dev->iobase + DT2814_CSR);
return 0;
}
+static int dt2814_ai_cancel(struct comedi_device *dev,
+ struct comedi_subdevice *s)
+{
+ unsigned int status;
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->spinlock, flags);
+ status = inb(dev->iobase + DT2814_CSR);
+ if (status & DT2814_ENB) {
+ /*
+ * Clear the timed trigger enable bit.
+ *
+ * Note: turning off timed mode triggers another
+ * sample. This will be mopped up by the calls to
+ * dt2814_ai_clear().
+ */
+ outb(status & DT2814_CHANMASK, dev->iobase + DT2814_CSR);
+ }
+ spin_unlock_irqrestore(&dev->spinlock, flags);
+ return 0;
+}
+
static irqreturn_t dt2814_interrupt(int irq, void *d)
{
- int lo, hi;
struct comedi_device *dev = d;
- struct dt2814_private *devpriv = dev->private;
struct comedi_subdevice *s = dev->read_subdev;
- int data;
+ struct comedi_async *async;
+ unsigned int lo, hi;
+ unsigned short data;
+ unsigned int status;
if (!dev->attached) {
dev_err(dev->class_dev, "spurious interrupt\n");
return IRQ_HANDLED;
}
+ async = s->async;
+
+ spin_lock(&dev->spinlock);
+
+ status = inb(dev->iobase + DT2814_CSR);
+ if (!(status & DT2814_ENB)) {
+ /* Timed acquisition not enabled. Nothing to do. */
+ spin_unlock(&dev->spinlock);
+ return IRQ_HANDLED;
+ }
+
+ if (!(status & (DT2814_FINISH | DT2814_ERR))) {
+ /* Spurious interrupt? */
+ spin_unlock(&dev->spinlock);
+ return IRQ_HANDLED;
+ }
+
+ /* Read data or clear error. */
hi = inb(dev->iobase + DT2814_DATA);
lo = inb(dev->iobase + DT2814_DATA);
data = (hi << 4) | (lo >> 4);
- if (!(--devpriv->ntrig)) {
- int i;
-
- outb(0, dev->iobase + DT2814_CSR);
+ if (status & DT2814_ERR) {
+ async->events |= COMEDI_CB_ERROR;
+ } else {
+ comedi_buf_write_samples(s, &data, 1);
+ if (async->cmd.stop_src == TRIG_COUNT &&
+ async->scans_done >= async->cmd.stop_arg) {
+ async->events |= COMEDI_CB_EOA;
+ }
+ }
+ if (async->events & COMEDI_CB_CANCEL_MASK) {
/*
- * note: turning off timed mode triggers another
- * sample.
+ * Disable timed mode.
+ *
+ * Note: turning off timed mode triggers another
+ * sample. This will be mopped up by the calls to
+ * dt2814_ai_clear().
*/
+ outb(status & DT2814_CHANMASK, dev->iobase + DT2814_CSR);
+ }
- for (i = 0; i < DT2814_TIMEOUT; i++) {
- if (inb(dev->iobase + DT2814_CSR) & DT2814_FINISH)
- break;
- }
- inb(dev->iobase + DT2814_DATA);
- inb(dev->iobase + DT2814_DATA);
+ spin_unlock(&dev->spinlock);
- s->async->events |= COMEDI_CB_EOA;
- }
comedi_handle_events(dev, s);
return IRQ_HANDLED;
}
static int dt2814_attach(struct comedi_device *dev, struct comedi_devconfig *it)
{
- struct dt2814_private *devpriv;
struct comedi_subdevice *s;
int ret;
- int i;
ret = comedi_request_region(dev, it->options[0], 0x2);
if (ret)
return ret;
outb(0, dev->iobase + DT2814_CSR);
- usleep_range(100, 200);
- if (inb(dev->iobase + DT2814_CSR) & DT2814_ERR) {
+ if (dt2814_ai_clear(dev)) {
dev_err(dev->class_dev, "reset error (fatal)\n");
return -EIO;
}
- i = inb(dev->iobase + DT2814_DATA);
- i = inb(dev->iobase + DT2814_DATA);
if (it->options[1]) {
ret = request_irq(it->options[1], dt2814_interrupt, 0,
@@ -255,10 +327,6 @@ static int dt2814_attach(struct comedi_device *dev, struct comedi_devconfig *it)
if (ret)
return ret;
- devpriv = comedi_alloc_devpriv(dev, sizeof(*devpriv));
- if (!devpriv)
- return -ENOMEM;
-
s = &dev->subdevices[0];
s->type = COMEDI_SUBD_AI;
s->subdev_flags = SDF_READABLE | SDF_GROUND;
@@ -272,16 +340,30 @@ static int dt2814_attach(struct comedi_device *dev, struct comedi_devconfig *it)
s->len_chanlist = 1;
s->do_cmd = dt2814_ai_cmd;
s->do_cmdtest = dt2814_ai_cmdtest;
+ s->cancel = dt2814_ai_cancel;
}
return 0;
}
+static void dt2814_detach(struct comedi_device *dev)
+{
+ if (dev->irq) {
+ /*
+ * An extra conversion triggered on termination of an
+ * asynchronous command may still be in progress. Wait for
+ * it to finish and clear the data or error status.
+ */
+ dt2814_ai_clear(dev);
+ }
+ comedi_legacy_detach(dev);
+}
+
static struct comedi_driver dt2814_driver = {
.driver_name = "dt2814",
.module = THIS_MODULE,
.attach = dt2814_attach,
- .detach = comedi_legacy_detach,
+ .detach = dt2814_detach,
};
module_comedi_driver(dt2814_driver);
diff --git a/drivers/staging/comedi/drivers/dt2815.c b/drivers/comedi/drivers/dt2815.c
index 5906f32aa01f..5906f32aa01f 100644
--- a/drivers/staging/comedi/drivers/dt2815.c
+++ b/drivers/comedi/drivers/dt2815.c
diff --git a/drivers/staging/comedi/drivers/dt2817.c b/drivers/comedi/drivers/dt2817.c
index 7c1463e835d3..7c1463e835d3 100644
--- a/drivers/staging/comedi/drivers/dt2817.c
+++ b/drivers/comedi/drivers/dt2817.c
diff --git a/drivers/staging/comedi/drivers/dt282x.c b/drivers/comedi/drivers/dt282x.c
index 2656b4b0e3d0..2656b4b0e3d0 100644
--- a/drivers/staging/comedi/drivers/dt282x.c
+++ b/drivers/comedi/drivers/dt282x.c
diff --git a/drivers/staging/comedi/drivers/dt3000.c b/drivers/comedi/drivers/dt3000.c
index ec27aa4730d4..ec27aa4730d4 100644
--- a/drivers/staging/comedi/drivers/dt3000.c
+++ b/drivers/comedi/drivers/dt3000.c
diff --git a/drivers/staging/comedi/drivers/dt9812.c b/drivers/comedi/drivers/dt9812.c
index 634f57730c1e..634f57730c1e 100644
--- a/drivers/staging/comedi/drivers/dt9812.c
+++ b/drivers/comedi/drivers/dt9812.c
diff --git a/drivers/staging/comedi/drivers/dyna_pci10xx.c b/drivers/comedi/drivers/dyna_pci10xx.c
index c224422bb126..c224422bb126 100644
--- a/drivers/staging/comedi/drivers/dyna_pci10xx.c
+++ b/drivers/comedi/drivers/dyna_pci10xx.c
diff --git a/drivers/staging/comedi/drivers/fl512.c b/drivers/comedi/drivers/fl512.c
index b715f30659fa..b715f30659fa 100644
--- a/drivers/staging/comedi/drivers/fl512.c
+++ b/drivers/comedi/drivers/fl512.c
diff --git a/drivers/staging/comedi/drivers/gsc_hpdi.c b/drivers/comedi/drivers/gsc_hpdi.c
index e35e4a743714..e35e4a743714 100644
--- a/drivers/staging/comedi/drivers/gsc_hpdi.c
+++ b/drivers/comedi/drivers/gsc_hpdi.c
diff --git a/drivers/staging/comedi/drivers/icp_multi.c b/drivers/comedi/drivers/icp_multi.c
index 16d2b78de83c..16d2b78de83c 100644
--- a/drivers/staging/comedi/drivers/icp_multi.c
+++ b/drivers/comedi/drivers/icp_multi.c
diff --git a/drivers/staging/comedi/drivers/ii_pci20kc.c b/drivers/comedi/drivers/ii_pci20kc.c
index 399255dbe388..399255dbe388 100644
--- a/drivers/staging/comedi/drivers/ii_pci20kc.c
+++ b/drivers/comedi/drivers/ii_pci20kc.c
diff --git a/drivers/staging/comedi/drivers/jr3_pci.c b/drivers/comedi/drivers/jr3_pci.c
index 7a02c4fa3cda..7a02c4fa3cda 100644
--- a/drivers/staging/comedi/drivers/jr3_pci.c
+++ b/drivers/comedi/drivers/jr3_pci.c
diff --git a/drivers/staging/comedi/drivers/jr3_pci.h b/drivers/comedi/drivers/jr3_pci.h
index acd4e5456ceb..acd4e5456ceb 100644
--- a/drivers/staging/comedi/drivers/jr3_pci.h
+++ b/drivers/comedi/drivers/jr3_pci.h
diff --git a/drivers/staging/comedi/drivers/ke_counter.c b/drivers/comedi/drivers/ke_counter.c
index bef1b20c1c8d..bef1b20c1c8d 100644
--- a/drivers/staging/comedi/drivers/ke_counter.c
+++ b/drivers/comedi/drivers/ke_counter.c
diff --git a/drivers/staging/comedi/drivers/me4000.c b/drivers/comedi/drivers/me4000.c
index 726e40dc17b6..0d3d4cafce2e 100644
--- a/drivers/staging/comedi/drivers/me4000.c
+++ b/drivers/comedi/drivers/me4000.c
@@ -924,7 +924,7 @@ static irqreturn_t me4000_ai_isr(int irq, void *dev_id)
struct comedi_subdevice *s = dev->read_subdev;
int i;
int c = 0;
- unsigned int lval;
+ unsigned short lval;
if (!dev->attached)
return IRQ_NONE;
diff --git a/drivers/staging/comedi/drivers/me_daq.c b/drivers/comedi/drivers/me_daq.c
index ef18e387471b..ef18e387471b 100644
--- a/drivers/staging/comedi/drivers/me_daq.c
+++ b/drivers/comedi/drivers/me_daq.c
diff --git a/drivers/staging/comedi/drivers/mf6x4.c b/drivers/comedi/drivers/mf6x4.c
index 9da8dd748078..9da8dd748078 100644
--- a/drivers/staging/comedi/drivers/mf6x4.c
+++ b/drivers/comedi/drivers/mf6x4.c
diff --git a/drivers/staging/comedi/drivers/mite.c b/drivers/comedi/drivers/mite.c
index 70960e3ba878..70960e3ba878 100644
--- a/drivers/staging/comedi/drivers/mite.c
+++ b/drivers/comedi/drivers/mite.c
diff --git a/drivers/staging/comedi/drivers/mite.h b/drivers/comedi/drivers/mite.h
index c6c056069bb7..c6c056069bb7 100644
--- a/drivers/staging/comedi/drivers/mite.h
+++ b/drivers/comedi/drivers/mite.h
diff --git a/drivers/staging/comedi/drivers/mpc624.c b/drivers/comedi/drivers/mpc624.c
index 646f4c086204..646f4c086204 100644
--- a/drivers/staging/comedi/drivers/mpc624.c
+++ b/drivers/comedi/drivers/mpc624.c
diff --git a/drivers/staging/comedi/drivers/multiq3.c b/drivers/comedi/drivers/multiq3.c
index c1897aee9a9a..c1897aee9a9a 100644
--- a/drivers/staging/comedi/drivers/multiq3.c
+++ b/drivers/comedi/drivers/multiq3.c
diff --git a/drivers/staging/comedi/drivers/ni_6527.c b/drivers/comedi/drivers/ni_6527.c
index 99e744172f4d..f1a45cf7342a 100644
--- a/drivers/staging/comedi/drivers/ni_6527.c
+++ b/drivers/comedi/drivers/ni_6527.c
@@ -195,7 +195,9 @@ static irqreturn_t ni6527_interrupt(int irq, void *d)
return IRQ_NONE;
if (status & NI6527_STATUS_EDGE) {
- comedi_buf_write_samples(s, &s->state, 1);
+ unsigned short val = 0;
+
+ comedi_buf_write_samples(s, &val, 1);
comedi_handle_events(dev, s);
}
diff --git a/drivers/staging/comedi/drivers/ni_65xx.c b/drivers/comedi/drivers/ni_65xx.c
index eb3f9f7109da..7cd8497420f2 100644
--- a/drivers/staging/comedi/drivers/ni_65xx.c
+++ b/drivers/comedi/drivers/ni_65xx.c
@@ -472,6 +472,7 @@ static irqreturn_t ni_65xx_interrupt(int irq, void *d)
struct comedi_device *dev = d;
struct comedi_subdevice *s = dev->read_subdev;
unsigned int status;
+ unsigned short val = 0;
status = readb(dev->mmio + NI_65XX_STATUS_REG);
if ((status & NI_65XX_STATUS_INT) == 0)
@@ -482,7 +483,7 @@ static irqreturn_t ni_65xx_interrupt(int irq, void *d)
writeb(NI_65XX_CLR_EDGE_INT | NI_65XX_CLR_OVERFLOW_INT,
dev->mmio + NI_65XX_CLR_REG);
- comedi_buf_write_samples(s, &s->state, 1);
+ comedi_buf_write_samples(s, &val, 1);
comedi_handle_events(dev, s);
return IRQ_HANDLED;
diff --git a/drivers/staging/comedi/drivers/ni_660x.c b/drivers/comedi/drivers/ni_660x.c
index e60d0125bcb2..e60d0125bcb2 100644
--- a/drivers/staging/comedi/drivers/ni_660x.c
+++ b/drivers/comedi/drivers/ni_660x.c
diff --git a/drivers/staging/comedi/drivers/ni_670x.c b/drivers/comedi/drivers/ni_670x.c
index c197e47486be..c197e47486be 100644
--- a/drivers/staging/comedi/drivers/ni_670x.c
+++ b/drivers/comedi/drivers/ni_670x.c
diff --git a/drivers/staging/comedi/drivers/ni_at_a2150.c b/drivers/comedi/drivers/ni_at_a2150.c
index 10ad7b88713e..10ad7b88713e 100644
--- a/drivers/staging/comedi/drivers/ni_at_a2150.c
+++ b/drivers/comedi/drivers/ni_at_a2150.c
diff --git a/drivers/staging/comedi/drivers/ni_at_ao.c b/drivers/comedi/drivers/ni_at_ao.c
index 2a0fb4d460db..2a0fb4d460db 100644
--- a/drivers/staging/comedi/drivers/ni_at_ao.c
+++ b/drivers/comedi/drivers/ni_at_ao.c
diff --git a/drivers/staging/comedi/drivers/ni_atmio.c b/drivers/comedi/drivers/ni_atmio.c
index 56c78da475e7..56c78da475e7 100644
--- a/drivers/staging/comedi/drivers/ni_atmio.c
+++ b/drivers/comedi/drivers/ni_atmio.c
diff --git a/drivers/staging/comedi/drivers/ni_atmio16d.c b/drivers/comedi/drivers/ni_atmio16d.c
index dffce1aa3e69..dffce1aa3e69 100644
--- a/drivers/staging/comedi/drivers/ni_atmio16d.c
+++ b/drivers/comedi/drivers/ni_atmio16d.c
diff --git a/drivers/staging/comedi/drivers/ni_daq_700.c b/drivers/comedi/drivers/ni_daq_700.c
index d40fc89f9cef..d40fc89f9cef 100644
--- a/drivers/staging/comedi/drivers/ni_daq_700.c
+++ b/drivers/comedi/drivers/ni_daq_700.c
diff --git a/drivers/staging/comedi/drivers/ni_daq_dio24.c b/drivers/comedi/drivers/ni_daq_dio24.c
index 44fb65afc218..44fb65afc218 100644
--- a/drivers/staging/comedi/drivers/ni_daq_dio24.c
+++ b/drivers/comedi/drivers/ni_daq_dio24.c
diff --git a/drivers/staging/comedi/drivers/ni_labpc.c b/drivers/comedi/drivers/ni_labpc.c
index 1f4a07bd1d26..1f4a07bd1d26 100644
--- a/drivers/staging/comedi/drivers/ni_labpc.c
+++ b/drivers/comedi/drivers/ni_labpc.c
diff --git a/drivers/staging/comedi/drivers/ni_labpc.h b/drivers/comedi/drivers/ni_labpc.h
index 728e901f53cd..728e901f53cd 100644
--- a/drivers/staging/comedi/drivers/ni_labpc.h
+++ b/drivers/comedi/drivers/ni_labpc.h
diff --git a/drivers/staging/comedi/drivers/ni_labpc_common.c b/drivers/comedi/drivers/ni_labpc_common.c
index dd97946eacaf..dd97946eacaf 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_common.c
+++ b/drivers/comedi/drivers/ni_labpc_common.c
diff --git a/drivers/staging/comedi/drivers/ni_labpc_cs.c b/drivers/comedi/drivers/ni_labpc_cs.c
index 4f7e2fe21254..4f7e2fe21254 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_cs.c
+++ b/drivers/comedi/drivers/ni_labpc_cs.c
diff --git a/drivers/staging/comedi/drivers/ni_labpc_isadma.c b/drivers/comedi/drivers/ni_labpc_isadma.c
index a551aca6e615..a551aca6e615 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_isadma.c
+++ b/drivers/comedi/drivers/ni_labpc_isadma.c
diff --git a/drivers/staging/comedi/drivers/ni_labpc_isadma.h b/drivers/comedi/drivers/ni_labpc_isadma.h
index f06f9353cb6c..f06f9353cb6c 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_isadma.h
+++ b/drivers/comedi/drivers/ni_labpc_isadma.h
diff --git a/drivers/staging/comedi/drivers/ni_labpc_pci.c b/drivers/comedi/drivers/ni_labpc_pci.c
index ec180b0fedf7..ec180b0fedf7 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_pci.c
+++ b/drivers/comedi/drivers/ni_labpc_pci.c
diff --git a/drivers/staging/comedi/drivers/ni_labpc_regs.h b/drivers/comedi/drivers/ni_labpc_regs.h
index ace40065a25b..ace40065a25b 100644
--- a/drivers/staging/comedi/drivers/ni_labpc_regs.h
+++ b/drivers/comedi/drivers/ni_labpc_regs.h
diff --git a/drivers/staging/comedi/drivers/ni_mio_common.c b/drivers/comedi/drivers/ni_mio_common.c
index 4f80a4991f95..4f80a4991f95 100644
--- a/drivers/staging/comedi/drivers/ni_mio_common.c
+++ b/drivers/comedi/drivers/ni_mio_common.c
diff --git a/drivers/staging/comedi/drivers/ni_mio_cs.c b/drivers/comedi/drivers/ni_mio_cs.c
index 4f37b4e58f09..4f37b4e58f09 100644
--- a/drivers/staging/comedi/drivers/ni_mio_cs.c
+++ b/drivers/comedi/drivers/ni_mio_cs.c
diff --git a/drivers/staging/comedi/drivers/ni_pcidio.c b/drivers/comedi/drivers/ni_pcidio.c
index 623f8d08d13a..623f8d08d13a 100644
--- a/drivers/staging/comedi/drivers/ni_pcidio.c
+++ b/drivers/comedi/drivers/ni_pcidio.c
diff --git a/drivers/staging/comedi/drivers/ni_pcimio.c b/drivers/comedi/drivers/ni_pcimio.c
index 6c813a490ba5..6c813a490ba5 100644
--- a/drivers/staging/comedi/drivers/ni_pcimio.c
+++ b/drivers/comedi/drivers/ni_pcimio.c
diff --git a/drivers/staging/comedi/drivers/ni_routes.c b/drivers/comedi/drivers/ni_routes.c
index c426a9286f15..c426a9286f15 100644
--- a/drivers/staging/comedi/drivers/ni_routes.c
+++ b/drivers/comedi/drivers/ni_routes.c
diff --git a/drivers/staging/comedi/drivers/ni_routes.h b/drivers/comedi/drivers/ni_routes.h
index b7680fd2afe1..b7680fd2afe1 100644
--- a/drivers/staging/comedi/drivers/ni_routes.h
+++ b/drivers/comedi/drivers/ni_routes.h
diff --git a/drivers/staging/comedi/drivers/ni_routing/README b/drivers/comedi/drivers/ni_routing/README
index b65c4ebedbc4..b65c4ebedbc4 100644
--- a/drivers/staging/comedi/drivers/ni_routing/README
+++ b/drivers/comedi/drivers/ni_routing/README
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes.c b/drivers/comedi/drivers/ni_routing/ni_device_routes.c
index 7b6a74dfe48b..7b6a74dfe48b 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes.h b/drivers/comedi/drivers/ni_routing/ni_device_routes.h
index b9f1c47d19e1..b9f1c47d19e1 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes.h
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes.h
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/all.h b/drivers/comedi/drivers/ni_routing/ni_device_routes/all.h
index 78b24138acb7..78b24138acb7 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/all.h
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/all.h
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6070e.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6070e.c
index f1126a0cb285..f1126a0cb285 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6070e.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6070e.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6220.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6220.c
index 74a59222963f..74a59222963f 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6220.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6220.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6221.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6221.c
index 44dcbabf2a99..44dcbabf2a99 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6221.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6221.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6229.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6229.c
index fa5794e4e2b3..fa5794e4e2b3 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6229.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6229.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6251.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6251.c
index 645fd1cd2de4..645fd1cd2de4 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6251.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6251.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6254.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6254.c
index 056a240cd3a2..056a240cd3a2 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6254.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6254.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6259.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6259.c
index e0b5fa78c3bc..e0b5fa78c3bc 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6259.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6259.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6534.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6534.c
index a2472ed288cf..a2472ed288cf 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6534.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6534.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6602.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6602.c
index 91de9dac2d6a..91de9dac2d6a 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6602.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6602.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6713.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6713.c
index d378b36d2084..d378b36d2084 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6713.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6713.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6723.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6723.c
index e0cc57ab06e7..e0cc57ab06e7 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6723.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6723.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6733.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6733.c
index f6e1e17ab854..f6e1e17ab854 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pci-6733.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pci-6733.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6030e.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6030e.c
index 9978d632117f..9978d632117f 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6030e.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6030e.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6224.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6224.c
index 1b89e27d7aa5..1b89e27d7aa5 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6224.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6224.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6225.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6225.c
index 10dfc34bc87c..10dfc34bc87c 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6225.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6225.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6251.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6251.c
index 25db4b7363de..25db4b7363de 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6251.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6251.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6733.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6733.c
index 27da4433fc4a..27da4433fc4a 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxi-6733.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxi-6733.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6251.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6251.c
index 8354fe971d59..8354fe971d59 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6251.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6251.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6535.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6535.c
index 2ebb679e0129..2ebb679e0129 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6535.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6535.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6738.c b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6738.c
index d88504314d7f..d88504314d7f 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_device_routes/pxie-6738.c
+++ b/drivers/comedi/drivers/ni_routing/ni_device_routes/pxie-6738.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_route_values.c b/drivers/comedi/drivers/ni_routing/ni_route_values.c
index 5901762734ed..5901762734ed 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_route_values.c
+++ b/drivers/comedi/drivers/ni_routing/ni_route_values.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_route_values.h b/drivers/comedi/drivers/ni_routing/ni_route_values.h
index 80e0145fb82b..80e0145fb82b 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_route_values.h
+++ b/drivers/comedi/drivers/ni_routing/ni_route_values.h
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/all.h b/drivers/comedi/drivers/ni_routing/ni_route_values/all.h
index 7227461500b5..7227461500b5 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/all.h
+++ b/drivers/comedi/drivers/ni_routing/ni_route_values/all.h
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_660x.c b/drivers/comedi/drivers/ni_routing/ni_route_values/ni_660x.c
index f1c7e6646261..f1c7e6646261 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_660x.c
+++ b/drivers/comedi/drivers/ni_routing/ni_route_values/ni_660x.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_eseries.c b/drivers/comedi/drivers/ni_routing/ni_route_values/ni_eseries.c
index d1ab3c9ce585..d1ab3c9ce585 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_eseries.c
+++ b/drivers/comedi/drivers/ni_routing/ni_route_values/ni_eseries.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_mseries.c b/drivers/comedi/drivers/ni_routing/ni_route_values/ni_mseries.c
index c59d8afe0ae9..c59d8afe0ae9 100644
--- a/drivers/staging/comedi/drivers/ni_routing/ni_route_values/ni_mseries.c
+++ b/drivers/comedi/drivers/ni_routing/ni_route_values/ni_mseries.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/.gitignore b/drivers/comedi/drivers/ni_routing/tools/.gitignore
index e3ebffcd900e..e3ebffcd900e 100644
--- a/drivers/staging/comedi/drivers/ni_routing/tools/.gitignore
+++ b/drivers/comedi/drivers/ni_routing/tools/.gitignore
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/Makefile b/drivers/comedi/drivers/ni_routing/tools/Makefile
index 6e92a06a44cb..6e92a06a44cb 100644
--- a/drivers/staging/comedi/drivers/ni_routing/tools/Makefile
+++ b/drivers/comedi/drivers/ni_routing/tools/Makefile
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/convert_c_to_py.c b/drivers/comedi/drivers/ni_routing/tools/convert_c_to_py.c
index dedb6f2fc678..dedb6f2fc678 100644
--- a/drivers/staging/comedi/drivers/ni_routing/tools/convert_c_to_py.c
+++ b/drivers/comedi/drivers/ni_routing/tools/convert_c_to_py.c
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/convert_csv_to_c.py b/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
index 532eb6372a5a..532eb6372a5a 100755
--- a/drivers/staging/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
+++ b/drivers/comedi/drivers/ni_routing/tools/convert_csv_to_c.py
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/convert_py_to_csv.py b/drivers/comedi/drivers/ni_routing/tools/convert_py_to_csv.py
index b3e6472bac22..b3e6472bac22 100755
--- a/drivers/staging/comedi/drivers/ni_routing/tools/convert_py_to_csv.py
+++ b/drivers/comedi/drivers/ni_routing/tools/convert_py_to_csv.py
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/csv_collection.py b/drivers/comedi/drivers/ni_routing/tools/csv_collection.py
index 12617329a928..12617329a928 100644
--- a/drivers/staging/comedi/drivers/ni_routing/tools/csv_collection.py
+++ b/drivers/comedi/drivers/ni_routing/tools/csv_collection.py
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/make_blank_csv.py b/drivers/comedi/drivers/ni_routing/tools/make_blank_csv.py
index 89c90a0ba24d..89c90a0ba24d 100755
--- a/drivers/staging/comedi/drivers/ni_routing/tools/make_blank_csv.py
+++ b/drivers/comedi/drivers/ni_routing/tools/make_blank_csv.py
diff --git a/drivers/staging/comedi/drivers/ni_routing/tools/ni_names.py b/drivers/comedi/drivers/ni_routing/tools/ni_names.py
index 5f9b825968b1..5f9b825968b1 100644
--- a/drivers/staging/comedi/drivers/ni_routing/tools/ni_names.py
+++ b/drivers/comedi/drivers/ni_routing/tools/ni_names.py
diff --git a/drivers/staging/comedi/drivers/ni_stc.h b/drivers/comedi/drivers/ni_stc.h
index fbc0b753a0f5..fbc0b753a0f5 100644
--- a/drivers/staging/comedi/drivers/ni_stc.h
+++ b/drivers/comedi/drivers/ni_stc.h
diff --git a/drivers/staging/comedi/drivers/ni_tio.c b/drivers/comedi/drivers/ni_tio.c
index f6154addaa95..f6154addaa95 100644
--- a/drivers/staging/comedi/drivers/ni_tio.c
+++ b/drivers/comedi/drivers/ni_tio.c
diff --git a/drivers/staging/comedi/drivers/ni_tio.h b/drivers/comedi/drivers/ni_tio.h
index e7b05718df9b..e7b05718df9b 100644
--- a/drivers/staging/comedi/drivers/ni_tio.h
+++ b/drivers/comedi/drivers/ni_tio.h
diff --git a/drivers/staging/comedi/drivers/ni_tio_internal.h b/drivers/comedi/drivers/ni_tio_internal.h
index 20fcd60038cd..20fcd60038cd 100644
--- a/drivers/staging/comedi/drivers/ni_tio_internal.h
+++ b/drivers/comedi/drivers/ni_tio_internal.h
diff --git a/drivers/staging/comedi/drivers/ni_tiocmd.c b/drivers/comedi/drivers/ni_tiocmd.c
index ab6d9e8269f3..ab6d9e8269f3 100644
--- a/drivers/staging/comedi/drivers/ni_tiocmd.c
+++ b/drivers/comedi/drivers/ni_tiocmd.c
diff --git a/drivers/staging/comedi/drivers/ni_usb6501.c b/drivers/comedi/drivers/ni_usb6501.c
index 5b6d9d783b2f..5b6d9d783b2f 100644
--- a/drivers/staging/comedi/drivers/ni_usb6501.c
+++ b/drivers/comedi/drivers/ni_usb6501.c
diff --git a/drivers/staging/comedi/drivers/pcl711.c b/drivers/comedi/drivers/pcl711.c
index 2dbf69e30965..bd6f42fe9e3c 100644
--- a/drivers/staging/comedi/drivers/pcl711.c
+++ b/drivers/comedi/drivers/pcl711.c
@@ -184,7 +184,7 @@ static irqreturn_t pcl711_interrupt(int irq, void *d)
struct comedi_device *dev = d;
struct comedi_subdevice *s = dev->read_subdev;
struct comedi_cmd *cmd = &s->async->cmd;
- unsigned int data;
+ unsigned short data;
if (!dev->attached) {
dev_err(dev->class_dev, "spurious interrupt\n");
diff --git a/drivers/staging/comedi/drivers/pcl724.c b/drivers/comedi/drivers/pcl724.c
index 1a5799278a7a..1a5799278a7a 100644
--- a/drivers/staging/comedi/drivers/pcl724.c
+++ b/drivers/comedi/drivers/pcl724.c
diff --git a/drivers/staging/comedi/drivers/pcl726.c b/drivers/comedi/drivers/pcl726.c
index 64eb649c9813..88f25d7e76f7 100644
--- a/drivers/staging/comedi/drivers/pcl726.c
+++ b/drivers/comedi/drivers/pcl726.c
@@ -220,9 +220,11 @@ static irqreturn_t pcl726_interrupt(int irq, void *d)
struct pcl726_private *devpriv = dev->private;
if (devpriv->cmd_running) {
+ unsigned short val = 0;
+
pcl726_intr_cancel(dev, s);
- comedi_buf_write_samples(s, &s->state, 1);
+ comedi_buf_write_samples(s, &val, 1);
comedi_handle_events(dev, s);
}
diff --git a/drivers/staging/comedi/drivers/pcl730.c b/drivers/comedi/drivers/pcl730.c
index 32a29129e6e8..32a29129e6e8 100644
--- a/drivers/staging/comedi/drivers/pcl730.c
+++ b/drivers/comedi/drivers/pcl730.c
diff --git a/drivers/staging/comedi/drivers/pcl812.c b/drivers/comedi/drivers/pcl812.c
index b87ab3840eee..b87ab3840eee 100644
--- a/drivers/staging/comedi/drivers/pcl812.c
+++ b/drivers/comedi/drivers/pcl812.c
diff --git a/drivers/staging/comedi/drivers/pcl816.c b/drivers/comedi/drivers/pcl816.c
index c368a337a0ae..c368a337a0ae 100644
--- a/drivers/staging/comedi/drivers/pcl816.c
+++ b/drivers/comedi/drivers/pcl816.c
diff --git a/drivers/staging/comedi/drivers/pcl818.c b/drivers/comedi/drivers/pcl818.c
index 63e3011158f2..f4b4a686c710 100644
--- a/drivers/staging/comedi/drivers/pcl818.c
+++ b/drivers/comedi/drivers/pcl818.c
@@ -423,7 +423,7 @@ static int pcl818_ai_eoc(struct comedi_device *dev,
static bool pcl818_ai_write_sample(struct comedi_device *dev,
struct comedi_subdevice *s,
- unsigned int chan, unsigned int val)
+ unsigned int chan, unsigned short val)
{
struct pcl818_private *devpriv = dev->private;
struct comedi_cmd *cmd = &s->async->cmd;
diff --git a/drivers/staging/comedi/drivers/pcm3724.c b/drivers/comedi/drivers/pcm3724.c
index 0cb1ad060402..0cb1ad060402 100644
--- a/drivers/staging/comedi/drivers/pcm3724.c
+++ b/drivers/comedi/drivers/pcm3724.c
diff --git a/drivers/staging/comedi/drivers/pcmad.c b/drivers/comedi/drivers/pcmad.c
index eec89a0afb2f..eec89a0afb2f 100644
--- a/drivers/staging/comedi/drivers/pcmad.c
+++ b/drivers/comedi/drivers/pcmad.c
diff --git a/drivers/staging/comedi/drivers/pcmda12.c b/drivers/comedi/drivers/pcmda12.c
index 14ab1f0d1e9f..14ab1f0d1e9f 100644
--- a/drivers/staging/comedi/drivers/pcmda12.c
+++ b/drivers/comedi/drivers/pcmda12.c
diff --git a/drivers/staging/comedi/drivers/pcmmio.c b/drivers/comedi/drivers/pcmmio.c
index 24a9568d3378..24a9568d3378 100644
--- a/drivers/staging/comedi/drivers/pcmmio.c
+++ b/drivers/comedi/drivers/pcmmio.c
diff --git a/drivers/staging/comedi/drivers/pcmuio.c b/drivers/comedi/drivers/pcmuio.c
index b299d648a0eb..b299d648a0eb 100644
--- a/drivers/staging/comedi/drivers/pcmuio.c
+++ b/drivers/comedi/drivers/pcmuio.c
diff --git a/drivers/staging/comedi/drivers/plx9052.h b/drivers/comedi/drivers/plx9052.h
index e68a7afef025..e68a7afef025 100644
--- a/drivers/staging/comedi/drivers/plx9052.h
+++ b/drivers/comedi/drivers/plx9052.h
diff --git a/drivers/staging/comedi/drivers/plx9080.h b/drivers/comedi/drivers/plx9080.h
index aa0eda5a8093..aa0eda5a8093 100644
--- a/drivers/staging/comedi/drivers/plx9080.h
+++ b/drivers/comedi/drivers/plx9080.h
diff --git a/drivers/staging/comedi/drivers/quatech_daqp_cs.c b/drivers/comedi/drivers/quatech_daqp_cs.c
index fe4408ebf6b3..fe4408ebf6b3 100644
--- a/drivers/staging/comedi/drivers/quatech_daqp_cs.c
+++ b/drivers/comedi/drivers/quatech_daqp_cs.c
diff --git a/drivers/staging/comedi/drivers/rtd520.c b/drivers/comedi/drivers/rtd520.c
index 2d99a648b054..2d99a648b054 100644
--- a/drivers/staging/comedi/drivers/rtd520.c
+++ b/drivers/comedi/drivers/rtd520.c
diff --git a/drivers/staging/comedi/drivers/rti800.c b/drivers/comedi/drivers/rti800.c
index 327fd93b8b12..327fd93b8b12 100644
--- a/drivers/staging/comedi/drivers/rti800.c
+++ b/drivers/comedi/drivers/rti800.c
diff --git a/drivers/staging/comedi/drivers/rti802.c b/drivers/comedi/drivers/rti802.c
index 195e2b1ac4c1..195e2b1ac4c1 100644
--- a/drivers/staging/comedi/drivers/rti802.c
+++ b/drivers/comedi/drivers/rti802.c
diff --git a/drivers/staging/comedi/drivers/s526.c b/drivers/comedi/drivers/s526.c
index 085cf5b449e5..085cf5b449e5 100644
--- a/drivers/staging/comedi/drivers/s526.c
+++ b/drivers/comedi/drivers/s526.c
diff --git a/drivers/staging/comedi/drivers/s626.c b/drivers/comedi/drivers/s626.c
index e7aba937d896..e7aba937d896 100644
--- a/drivers/staging/comedi/drivers/s626.c
+++ b/drivers/comedi/drivers/s626.c
diff --git a/drivers/staging/comedi/drivers/s626.h b/drivers/comedi/drivers/s626.h
index 749252b1d26b..749252b1d26b 100644
--- a/drivers/staging/comedi/drivers/s626.h
+++ b/drivers/comedi/drivers/s626.h
diff --git a/drivers/staging/comedi/drivers/ssv_dnp.c b/drivers/comedi/drivers/ssv_dnp.c
index 016d315aa584..016d315aa584 100644
--- a/drivers/staging/comedi/drivers/ssv_dnp.c
+++ b/drivers/comedi/drivers/ssv_dnp.c
diff --git a/drivers/staging/comedi/drivers/tests/Makefile b/drivers/comedi/drivers/tests/Makefile
index b5d8e13d4162..5ff7cdc32a32 100644
--- a/drivers/staging/comedi/drivers/tests/Makefile
+++ b/drivers/comedi/drivers/tests/Makefile
@@ -3,5 +3,6 @@
#
ccflags-$(CONFIG_COMEDI_DEBUG) := -DDEBUG
-obj-$(CONFIG_COMEDI_TESTS) += example_test.o ni_routes_test.o
+obj-$(CONFIG_COMEDI_TESTS_EXAMPLE) += comedi_example_test.o
+obj-$(CONFIG_COMEDI_TESTS_NI_ROUTES) += ni_routes_test.o
CFLAGS_ni_routes_test.o := -DDEBUG
diff --git a/drivers/staging/comedi/drivers/tests/example_test.c b/drivers/comedi/drivers/tests/comedi_example_test.c
index fc65158b8e8e..e5aaaeab7bdd 100644
--- a/drivers/staging/comedi/drivers/tests/example_test.c
+++ b/drivers/comedi/drivers/tests/comedi_example_test.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0+
/* vim: set ts=8 sw=8 noet tw=80 nowrap: */
/*
- * comedi/drivers/tests/example_test.c
+ * comedi/drivers/tests/comedi_example_test.c
* Example set of unit tests.
*
* COMEDI - Linux Control and Measurement Device Interface
@@ -35,14 +35,14 @@ static struct comedi_device dev = {
/* *** END fake board data *** */
/* *** BEGIN fake data init *** */
-void init_fake(void)
+static void init_fake(void)
{
dev.item = 10;
}
/* *** END fake data init *** */
-void test0(void)
+static void test0(void)
{
init_fake();
unittest(dev.item != 11, "negative result\n");
@@ -52,8 +52,8 @@ void test0(void)
/* **** BEGIN simple module entry/exit functions **** */
static int __init unittest_enter(void)
{
- const unittest_fptr unit_tests[] = {
- (unittest_fptr)test0,
+ static const unittest_fptr unit_tests[] = {
+ test0,
NULL,
};
diff --git a/drivers/staging/comedi/drivers/tests/ni_routes_test.c b/drivers/comedi/drivers/tests/ni_routes_test.c
index 4061b3b5f8e9..32073850d545 100644
--- a/drivers/staging/comedi/drivers/tests/ni_routes_test.c
+++ b/drivers/comedi/drivers/tests/ni_routes_test.c
@@ -24,7 +24,7 @@
#include "../ni_routes.h"
#include "unittest.h"
-#define RVi(table, src, dest) ((table)[(dest) * NI_NUM_NAMES + (src)])
+#define RVI(table, src, dest) ((table)[(dest) * NI_NUM_NAMES + (src)])
#define O(x) ((x) + NI_NAMES_BASE)
#define B(x) ((x) - NI_NAMES_BASE)
#define V(x) ((x) | 0x80)
@@ -63,24 +63,24 @@ static const int no_val_dest = O(7), no_val_index = 4;
/* I1 and I2 should not call O(...). Mostly here to shut checkpatch.pl up */
#define I1(x1) \
- (int[]){ \
- x1, 0 \
- }
+ ((int[]){ \
+ (x1), 0 \
+ })
#define I2(x1, x2) \
- (int[]){ \
+ ((int[]){ \
(x1), (x2), 0 \
- }
+ })
#define I3(x1, x2, x3) \
- (int[]){ \
+ ((int[]){ \
(x1), (x2), (x3), 0 \
- }
+ })
/* O9 is build to call O(...) for each arg */
#define O9(x1, x2, x3, x4, x5, x6, x7, x8, x9) \
- (int[]){ \
+ ((int[]){ \
O(x1), O(x2), O(x3), O(x4), O(x5), O(x6), O(x7), O(x8), O(x9), \
0 \
- }
+ })
static struct ni_device_routes DR = {
.device = "testdev",
@@ -194,7 +194,7 @@ static bool route_set_dests_in_order(const struct ni_device_routes *devroutes)
}
/* Tests that all route_set->src are in order of the signal source. */
-bool route_set_sources_in_order(const struct ni_device_routes *devroutes)
+static bool route_set_sources_in_order(const struct ni_device_routes *devroutes)
{
int i;
@@ -211,13 +211,14 @@ bool route_set_sources_in_order(const struct ni_device_routes *devroutes)
return true;
}
-void test_ni_assign_device_routes(void)
+static void test_ni_assign_device_routes(void)
{
- const struct ni_device_routes *devroutes, *olddevroutes;
+ const struct ni_device_routes *devroutes;
const u8 *table, *oldtable;
init_pci_6070e();
- ni_assign_device_routes(ni_eseries, pci_6070e, &private.routing_tables);
+ ni_assign_device_routes(ni_eseries, pci_6070e, NULL,
+ &private.routing_tables);
devroutes = private.routing_tables.valid_routes;
table = private.routing_tables.route_values;
@@ -242,18 +243,16 @@ void test_ni_assign_device_routes(void)
unittest(route_set_sources_in_order(devroutes),
"all pci-6070e route_set->src's in order of signal source\n");
- unittest(
- RVi(table, B(PXI_Star), B(NI_AI_SampleClock)) == V(17) &&
- RVi(table, B(NI_10MHzRefClock), B(TRIGGER_LINE(0))) == 0 &&
- RVi(table, B(NI_AI_ConvertClock), B(NI_PFI(0))) == 0 &&
- RVi(table, B(NI_AI_ConvertClock), B(NI_PFI(2))) ==
- V(NI_PFI_OUTPUT_AI_CONVERT),
- "pci-6070e finds e-series route_values table\n");
+ unittest(RVI(table, B(PXI_Star), B(NI_AI_SampleClock)) == V(17) &&
+ RVI(table, B(NI_10MHzRefClock), B(TRIGGER_LINE(0))) == 0 &&
+ RVI(table, B(NI_AI_ConvertClock), B(NI_PFI(0))) == 0 &&
+ RVI(table, B(NI_AI_ConvertClock), B(NI_PFI(2))) == V(NI_PFI_OUTPUT_AI_CONVERT),
+ "pci-6070e finds e-series route_values table\n");
- olddevroutes = devroutes;
oldtable = table;
init_pci_6220();
- ni_assign_device_routes(ni_mseries, pci_6220, &private.routing_tables);
+ ni_assign_device_routes(ni_mseries, pci_6220, NULL,
+ &private.routing_tables);
devroutes = private.routing_tables.valid_routes;
table = private.routing_tables.route_values;
@@ -261,15 +260,14 @@ void test_ni_assign_device_routes(void)
"find device pci-6220\n");
unittest(oldtable != table, "pci-6220 find other route_values table\n");
- unittest(
- RVi(table, B(PXI_Star), B(NI_AI_SampleClock)) == V(20) &&
- RVi(table, B(NI_10MHzRefClock), B(TRIGGER_LINE(0))) == V(12) &&
- RVi(table, B(NI_AI_ConvertClock), B(NI_PFI(0))) == V(3) &&
- RVi(table, B(NI_AI_ConvertClock), B(NI_PFI(2))) == V(3),
- "pci-6220 finds m-series route_values table\n");
+ unittest(RVI(table, B(PXI_Star), B(NI_AI_SampleClock)) == V(20) &&
+ RVI(table, B(NI_10MHzRefClock), B(TRIGGER_LINE(0))) == V(12) &&
+ RVI(table, B(NI_AI_ConvertClock), B(NI_PFI(0))) == V(3) &&
+ RVI(table, B(NI_AI_ConvertClock), B(NI_PFI(2))) == V(3),
+ "pci-6220 finds m-series route_values table\n");
}
-void test_ni_sort_device_routes(void)
+static void test_ni_sort_device_routes(void)
{
/* We begin by sorting the device routes for use in later tests */
ni_sort_device_routes(&DR);
@@ -280,7 +278,7 @@ void test_ni_sort_device_routes(void)
"all route_set->src's of fake data in order of sig. source\n");
}
-void test_ni_find_route_set(void)
+static void test_ni_find_route_set(void)
{
unittest(!ni_find_route_set(bad_dest, &DR),
"check for nonexistent route_set\n");
@@ -296,7 +294,7 @@ void test_ni_find_route_set(void)
"find last route_set\n");
}
-void test_ni_route_set_has_source(void)
+static void test_ni_route_set_has_source(void)
{
unittest(!ni_route_set_has_source(&DR.routes[0], O(0)),
"check for bad source\n");
@@ -308,7 +306,7 @@ void test_ni_route_set_has_source(void)
"find last source\n");
}
-void test_ni_route_to_register(void)
+static void test_ni_route_to_register(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -358,7 +356,7 @@ void test_ni_route_to_register(void)
"validate indirect route through brd3 to TRIGGER_LINE(1)\n");
}
-void test_ni_lookup_route_register(void)
+static void test_ni_lookup_route_register(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -397,7 +395,7 @@ void test_ni_lookup_route_register(void)
"brd0_src1: lookup indirect route register\n");
}
-void test_route_is_valid(void)
+static void test_route_is_valid(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -412,7 +410,7 @@ void test_route_is_valid(void)
"validate last destination\n");
}
-void test_ni_is_cmd_dest(void)
+static void test_ni_is_cmd_dest(void)
{
init_pci_fake();
unittest(ni_is_cmd_dest(NI_AI_SampleClock),
@@ -429,7 +427,7 @@ void test_ni_is_cmd_dest(void)
"check that AO/SampleClockTimebase _not_ cmd destination\n");
}
-void test_channel_is_pfi(void)
+static void test_channel_is_pfi(void)
{
init_pci_fake();
unittest(channel_is_pfi(NI_PFI(0)), "check First pfi channel\n");
@@ -439,7 +437,7 @@ void test_channel_is_pfi(void)
"check first non pfi channel\n");
}
-void test_channel_is_rtsi(void)
+static void test_channel_is_rtsi(void)
{
init_pci_fake();
unittest(channel_is_rtsi(TRIGGER_LINE(0)),
@@ -452,7 +450,7 @@ void test_channel_is_rtsi(void)
"check first non rtsi channel\n");
}
-void test_ni_count_valid_routes(void)
+static void test_ni_count_valid_routes(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -460,7 +458,7 @@ void test_ni_count_valid_routes(void)
unittest(ni_count_valid_routes(T) == 57, "count all valid routes\n");
}
-void test_ni_get_valid_routes(void)
+static void test_ni_get_valid_routes(void)
{
const struct ni_route_tables *T = &private.routing_tables;
unsigned int pair_data[2];
@@ -477,7 +475,7 @@ void test_ni_get_valid_routes(void)
"destination of first valid pair from ni_get_valid_routes\n");
}
-void test_ni_find_route_source(void)
+static void test_ni_find_route_source(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -494,7 +492,7 @@ void test_ni_find_route_source(void)
"find invalid source (without checking device routes)\n");
}
-void test_route_register_is_valid(void)
+static void test_route_register_is_valid(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -509,7 +507,7 @@ void test_route_register_is_valid(void)
"find last source");
}
-void test_ni_check_trigger_arg(void)
+static void test_ni_check_trigger_arg(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -542,7 +540,7 @@ void test_ni_check_trigger_arg(void)
"check trigger arg for last src->dest\n");
}
-void test_ni_get_reg_value(void)
+static void test_ni_get_reg_value(void)
{
const struct ni_route_tables *T = &private.routing_tables;
@@ -578,23 +576,23 @@ void test_ni_get_reg_value(void)
/* **** BEGIN simple module entry/exit functions **** */
static int __init ni_routes_unittest(void)
{
- const unittest_fptr unit_tests[] = {
- (unittest_fptr)test_ni_assign_device_routes,
- (unittest_fptr)test_ni_sort_device_routes,
- (unittest_fptr)test_ni_find_route_set,
- (unittest_fptr)test_ni_route_set_has_source,
- (unittest_fptr)test_ni_route_to_register,
- (unittest_fptr)test_ni_lookup_route_register,
- (unittest_fptr)test_route_is_valid,
- (unittest_fptr)test_ni_is_cmd_dest,
- (unittest_fptr)test_channel_is_pfi,
- (unittest_fptr)test_channel_is_rtsi,
- (unittest_fptr)test_ni_count_valid_routes,
- (unittest_fptr)test_ni_get_valid_routes,
- (unittest_fptr)test_ni_find_route_source,
- (unittest_fptr)test_route_register_is_valid,
- (unittest_fptr)test_ni_check_trigger_arg,
- (unittest_fptr)test_ni_get_reg_value,
+ static const unittest_fptr unit_tests[] = {
+ test_ni_assign_device_routes,
+ test_ni_sort_device_routes,
+ test_ni_find_route_set,
+ test_ni_route_set_has_source,
+ test_ni_route_to_register,
+ test_ni_lookup_route_register,
+ test_route_is_valid,
+ test_ni_is_cmd_dest,
+ test_channel_is_pfi,
+ test_channel_is_rtsi,
+ test_ni_count_valid_routes,
+ test_ni_get_valid_routes,
+ test_ni_find_route_source,
+ test_route_register_is_valid,
+ test_ni_check_trigger_arg,
+ test_ni_get_reg_value,
NULL,
};
diff --git a/drivers/staging/comedi/drivers/tests/unittest.h b/drivers/comedi/drivers/tests/unittest.h
index b8e622ea1de1..2da3beea2479 100644
--- a/drivers/staging/comedi/drivers/tests/unittest.h
+++ b/drivers/comedi/drivers/tests/unittest.h
@@ -27,7 +27,7 @@ static struct unittest_results {
int failed;
} unittest_results;
-typedef void *(*unittest_fptr)(void);
+typedef void (*unittest_fptr)(void);
#define unittest(result, fmt, ...) ({ \
bool failed = !(result); \
diff --git a/drivers/staging/comedi/drivers/usbdux.c b/drivers/comedi/drivers/usbdux.c
index 0350f303d557..0350f303d557 100644
--- a/drivers/staging/comedi/drivers/usbdux.c
+++ b/drivers/comedi/drivers/usbdux.c
diff --git a/drivers/staging/comedi/drivers/usbduxfast.c b/drivers/comedi/drivers/usbduxfast.c
index 4af012968cb6..4af012968cb6 100644
--- a/drivers/staging/comedi/drivers/usbduxfast.c
+++ b/drivers/comedi/drivers/usbduxfast.c
diff --git a/drivers/staging/comedi/drivers/usbduxsigma.c b/drivers/comedi/drivers/usbduxsigma.c
index 54d7605e909f..54d7605e909f 100644
--- a/drivers/staging/comedi/drivers/usbduxsigma.c
+++ b/drivers/comedi/drivers/usbduxsigma.c
diff --git a/drivers/staging/comedi/drivers/vmk80xx.c b/drivers/comedi/drivers/vmk80xx.c
index 7956abcbae22..9f920819cd74 100644
--- a/drivers/staging/comedi/drivers/vmk80xx.c
+++ b/drivers/comedi/drivers/vmk80xx.c
@@ -877,5 +877,4 @@ module_comedi_usb_driver(vmk80xx_driver, vmk80xx_usb_driver);
MODULE_AUTHOR("Manuel Gebele <forensixs@gmx.de>");
MODULE_DESCRIPTION("Velleman USB Board Low-Level Driver");
-MODULE_SUPPORTED_DEVICE("K8055/K8061 aka VM110/VM140");
MODULE_LICENSE("GPL");
diff --git a/drivers/staging/comedi/drivers/z8536.h b/drivers/comedi/drivers/z8536.h
index 3ef5f9e79b89..3ef5f9e79b89 100644
--- a/drivers/staging/comedi/drivers/z8536.h
+++ b/drivers/comedi/drivers/z8536.h
diff --git a/drivers/staging/comedi/kcomedilib/Makefile b/drivers/comedi/kcomedilib/Makefile
index 8031142a105f..8031142a105f 100644
--- a/drivers/staging/comedi/kcomedilib/Makefile
+++ b/drivers/comedi/kcomedilib/Makefile
diff --git a/drivers/staging/comedi/kcomedilib/kcomedilib_main.c b/drivers/comedi/kcomedilib/kcomedilib_main.c
index df9bba1b69ed..df9bba1b69ed 100644
--- a/drivers/staging/comedi/kcomedilib/kcomedilib_main.c
+++ b/drivers/comedi/kcomedilib/kcomedilib_main.c
diff --git a/drivers/staging/comedi/proc.c b/drivers/comedi/proc.c
index 8bc8e42beb90..8bc8e42beb90 100644
--- a/drivers/staging/comedi/proc.c
+++ b/drivers/comedi/proc.c
diff --git a/drivers/staging/comedi/range.c b/drivers/comedi/range.c
index a4e6fe0fb729..a4e6fe0fb729 100644
--- a/drivers/staging/comedi/range.c
+++ b/drivers/comedi/range.c
diff --git a/drivers/counter/104-quad-8.c b/drivers/counter/104-quad-8.c
index 78766b6ec271..9691f8612be8 100644
--- a/drivers/counter/104-quad-8.c
+++ b/drivers/counter/104-quad-8.c
@@ -9,8 +9,6 @@
#include <linux/counter.h>
#include <linux/device.h>
#include <linux/errno.h>
-#include <linux/iio/iio.h>
-#include <linux/iio/types.h>
#include <linux/io.h>
#include <linux/ioport.h>
#include <linux/isa.h>
@@ -29,7 +27,7 @@ MODULE_PARM_DESC(base, "ACCES 104-QUAD-8 base addresses");
#define QUAD8_NUM_COUNTERS 8
/**
- * struct quad8_iio - IIO device private data structure
+ * struct quad8 - device private data structure
* @counter: instance of the counter_device
* @fck_prescaler: array of filter clock prescaler configurations
* @preset: array of preset values
@@ -41,9 +39,9 @@ MODULE_PARM_DESC(base, "ACCES 104-QUAD-8 base addresses");
* @synchronous_mode: array of index function synchronous mode configurations
* @index_polarity: array of index function polarity configurations
* @cable_fault_enable: differential encoder cable status enable configurations
- * @base: base port address of the IIO device
+ * @base: base port address of the device
*/
-struct quad8_iio {
+struct quad8 {
struct mutex lock;
struct counter_device counter;
unsigned int fck_prescaler[QUAD8_NUM_COUNTERS];
@@ -98,532 +96,10 @@ struct quad8_iio {
#define QUAD8_CMR_QUADRATURE_X2 0x10
#define QUAD8_CMR_QUADRATURE_X4 0x18
-
-static int quad8_read_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan, int *val, int *val2, long mask)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel;
- unsigned int flags;
- unsigned int borrow;
- unsigned int carry;
- int i;
-
- switch (mask) {
- case IIO_CHAN_INFO_RAW:
- if (chan->type == IIO_INDEX) {
- *val = !!(inb(priv->base + QUAD8_REG_INDEX_INPUT_LEVELS)
- & BIT(chan->channel));
- return IIO_VAL_INT;
- }
-
- flags = inb(base_offset + 1);
- borrow = flags & QUAD8_FLAG_BT;
- carry = !!(flags & QUAD8_FLAG_CT);
-
- /* Borrow XOR Carry effectively doubles count range */
- *val = (borrow ^ carry) << 24;
-
- mutex_lock(&priv->lock);
-
- /* Reset Byte Pointer; transfer Counter to Output Latch */
- outb(QUAD8_CTR_RLD | QUAD8_RLD_RESET_BP | QUAD8_RLD_CNTR_OUT,
- base_offset + 1);
-
- for (i = 0; i < 3; i++)
- *val |= (unsigned int)inb(base_offset) << (8 * i);
-
- mutex_unlock(&priv->lock);
-
- return IIO_VAL_INT;
- case IIO_CHAN_INFO_ENABLE:
- *val = priv->ab_enable[chan->channel];
- return IIO_VAL_INT;
- case IIO_CHAN_INFO_SCALE:
- *val = 1;
- *val2 = priv->quadrature_scale[chan->channel];
- return IIO_VAL_FRACTIONAL_LOG2;
- }
-
- return -EINVAL;
-}
-
-static int quad8_write_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan, int val, int val2, long mask)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel;
- int i;
- unsigned int ior_cfg;
-
- switch (mask) {
- case IIO_CHAN_INFO_RAW:
- if (chan->type == IIO_INDEX)
- return -EINVAL;
-
- /* Only 24-bit values are supported */
- if ((unsigned int)val > 0xFFFFFF)
- return -EINVAL;
-
- mutex_lock(&priv->lock);
-
- /* Reset Byte Pointer */
- outb(QUAD8_CTR_RLD | QUAD8_RLD_RESET_BP, base_offset + 1);
-
- /* Counter can only be set via Preset Register */
- for (i = 0; i < 3; i++)
- outb(val >> (8 * i), base_offset);
-
- /* Transfer Preset Register to Counter */
- outb(QUAD8_CTR_RLD | QUAD8_RLD_PRESET_CNTR, base_offset + 1);
-
- /* Reset Byte Pointer */
- outb(QUAD8_CTR_RLD | QUAD8_RLD_RESET_BP, base_offset + 1);
-
- /* Set Preset Register back to original value */
- val = priv->preset[chan->channel];
- for (i = 0; i < 3; i++)
- outb(val >> (8 * i), base_offset);
-
- /* Reset Borrow, Carry, Compare, and Sign flags */
- outb(QUAD8_CTR_RLD | QUAD8_RLD_RESET_FLAGS, base_offset + 1);
- /* Reset Error flag */
- outb(QUAD8_CTR_RLD | QUAD8_RLD_RESET_E, base_offset + 1);
-
- mutex_unlock(&priv->lock);
-
- return 0;
- case IIO_CHAN_INFO_ENABLE:
- /* only boolean values accepted */
- if (val < 0 || val > 1)
- return -EINVAL;
-
- mutex_lock(&priv->lock);
-
- priv->ab_enable[chan->channel] = val;
-
- ior_cfg = val | priv->preset_enable[chan->channel] << 1;
-
- /* Load I/O control configuration */
- outb(QUAD8_CTR_IOR | ior_cfg, base_offset + 1);
-
- mutex_unlock(&priv->lock);
-
- return 0;
- case IIO_CHAN_INFO_SCALE:
- mutex_lock(&priv->lock);
-
- /* Quadrature scaling only available in quadrature mode */
- if (!priv->quadrature_mode[chan->channel] &&
- (val2 || val != 1)) {
- mutex_unlock(&priv->lock);
- return -EINVAL;
- }
-
- /* Only three gain states (1, 0.5, 0.25) */
- if (val == 1 && !val2)
- priv->quadrature_scale[chan->channel] = 0;
- else if (!val)
- switch (val2) {
- case 500000:
- priv->quadrature_scale[chan->channel] = 1;
- break;
- case 250000:
- priv->quadrature_scale[chan->channel] = 2;
- break;
- default:
- mutex_unlock(&priv->lock);
- return -EINVAL;
- }
- else {
- mutex_unlock(&priv->lock);
- return -EINVAL;
- }
-
- mutex_unlock(&priv->lock);
- return 0;
- }
-
- return -EINVAL;
-}
-
-static const struct iio_info quad8_info = {
- .read_raw = quad8_read_raw,
- .write_raw = quad8_write_raw
-};
-
-static ssize_t quad8_read_preset(struct iio_dev *indio_dev, uintptr_t private,
- const struct iio_chan_spec *chan, char *buf)
-{
- const struct quad8_iio *const priv = iio_priv(indio_dev);
-
- return snprintf(buf, PAGE_SIZE, "%u\n", priv->preset[chan->channel]);
-}
-
-static ssize_t quad8_write_preset(struct iio_dev *indio_dev, uintptr_t private,
- const struct iio_chan_spec *chan, const char *buf, size_t len)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel;
- unsigned int preset;
- int ret;
- int i;
-
- ret = kstrtouint(buf, 0, &preset);
- if (ret)
- return ret;
-
- /* Only 24-bit values are supported */
- if (preset > 0xFFFFFF)
- return -EINVAL;
-
- mutex_lock(&priv->lock);
-
- priv->preset[chan->channel] = preset;
-
- /* Reset Byte Pointer */
- outb(QUAD8_CTR_RLD | QUAD8_RLD_RESET_BP, base_offset + 1);
-
- /* Set Preset Register */
- for (i = 0; i < 3; i++)
- outb(preset >> (8 * i), base_offset);
-
- mutex_unlock(&priv->lock);
-
- return len;
-}
-
-static ssize_t quad8_read_set_to_preset_on_index(struct iio_dev *indio_dev,
- uintptr_t private, const struct iio_chan_spec *chan, char *buf)
-{
- const struct quad8_iio *const priv = iio_priv(indio_dev);
-
- return snprintf(buf, PAGE_SIZE, "%u\n",
- !priv->preset_enable[chan->channel]);
-}
-
-static ssize_t quad8_write_set_to_preset_on_index(struct iio_dev *indio_dev,
- uintptr_t private, const struct iio_chan_spec *chan, const char *buf,
- size_t len)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel + 1;
- bool preset_enable;
- int ret;
- unsigned int ior_cfg;
-
- ret = kstrtobool(buf, &preset_enable);
- if (ret)
- return ret;
-
- /* Preset enable is active low in Input/Output Control register */
- preset_enable = !preset_enable;
-
- mutex_lock(&priv->lock);
-
- priv->preset_enable[chan->channel] = preset_enable;
-
- ior_cfg = priv->ab_enable[chan->channel] |
- (unsigned int)preset_enable << 1;
-
- /* Load I/O control configuration to Input / Output Control Register */
- outb(QUAD8_CTR_IOR | ior_cfg, base_offset);
-
- mutex_unlock(&priv->lock);
-
- return len;
-}
-
-static const char *const quad8_noise_error_states[] = {
- "No excessive noise is present at the count inputs",
- "Excessive noise is present at the count inputs"
-};
-
-static int quad8_get_noise_error(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel + 1;
-
- return !!(inb(base_offset) & QUAD8_FLAG_E);
-}
-
-static const struct iio_enum quad8_noise_error_enum = {
- .items = quad8_noise_error_states,
- .num_items = ARRAY_SIZE(quad8_noise_error_states),
- .get = quad8_get_noise_error
-};
-
-static const char *const quad8_count_direction_states[] = {
- "down",
- "up"
-};
-
-static int quad8_get_count_direction(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel + 1;
-
- return !!(inb(base_offset) & QUAD8_FLAG_UD);
-}
-
-static const struct iio_enum quad8_count_direction_enum = {
- .items = quad8_count_direction_states,
- .num_items = ARRAY_SIZE(quad8_count_direction_states),
- .get = quad8_get_count_direction
-};
-
-static const char *const quad8_count_modes[] = {
- "normal",
- "range limit",
- "non-recycle",
- "modulo-n"
-};
-
-static int quad8_set_count_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan, unsigned int cnt_mode)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- unsigned int mode_cfg = cnt_mode << 1;
- const int base_offset = priv->base + 2 * chan->channel + 1;
-
- mutex_lock(&priv->lock);
-
- priv->count_mode[chan->channel] = cnt_mode;
-
- /* Add quadrature mode configuration */
- if (priv->quadrature_mode[chan->channel])
- mode_cfg |= (priv->quadrature_scale[chan->channel] + 1) << 3;
-
- /* Load mode configuration to Counter Mode Register */
- outb(QUAD8_CTR_CMR | mode_cfg, base_offset);
-
- mutex_unlock(&priv->lock);
-
- return 0;
-}
-
-static int quad8_get_count_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- const struct quad8_iio *const priv = iio_priv(indio_dev);
-
- return priv->count_mode[chan->channel];
-}
-
-static const struct iio_enum quad8_count_mode_enum = {
- .items = quad8_count_modes,
- .num_items = ARRAY_SIZE(quad8_count_modes),
- .set = quad8_set_count_mode,
- .get = quad8_get_count_mode
-};
-
-static const char *const quad8_synchronous_modes[] = {
- "non-synchronous",
- "synchronous"
-};
-
-static int quad8_set_synchronous_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan, unsigned int synchronous_mode)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel + 1;
- unsigned int idr_cfg = synchronous_mode;
-
- mutex_lock(&priv->lock);
-
- idr_cfg |= priv->index_polarity[chan->channel] << 1;
-
- /* Index function must be non-synchronous in non-quadrature mode */
- if (synchronous_mode && !priv->quadrature_mode[chan->channel]) {
- mutex_unlock(&priv->lock);
- return -EINVAL;
- }
-
- priv->synchronous_mode[chan->channel] = synchronous_mode;
-
- /* Load Index Control configuration to Index Control Register */
- outb(QUAD8_CTR_IDR | idr_cfg, base_offset);
-
- mutex_unlock(&priv->lock);
-
- return 0;
-}
-
-static int quad8_get_synchronous_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- const struct quad8_iio *const priv = iio_priv(indio_dev);
-
- return priv->synchronous_mode[chan->channel];
-}
-
-static const struct iio_enum quad8_synchronous_mode_enum = {
- .items = quad8_synchronous_modes,
- .num_items = ARRAY_SIZE(quad8_synchronous_modes),
- .set = quad8_set_synchronous_mode,
- .get = quad8_get_synchronous_mode
-};
-
-static const char *const quad8_quadrature_modes[] = {
- "non-quadrature",
- "quadrature"
-};
-
-static int quad8_set_quadrature_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan, unsigned int quadrature_mode)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel + 1;
- unsigned int mode_cfg;
-
- mutex_lock(&priv->lock);
-
- mode_cfg = priv->count_mode[chan->channel] << 1;
-
- if (quadrature_mode)
- mode_cfg |= (priv->quadrature_scale[chan->channel] + 1) << 3;
- else {
- /* Quadrature scaling only available in quadrature mode */
- priv->quadrature_scale[chan->channel] = 0;
-
- /* Synchronous function not supported in non-quadrature mode */
- if (priv->synchronous_mode[chan->channel])
- quad8_set_synchronous_mode(indio_dev, chan, 0);
- }
-
- priv->quadrature_mode[chan->channel] = quadrature_mode;
-
- /* Load mode configuration to Counter Mode Register */
- outb(QUAD8_CTR_CMR | mode_cfg, base_offset);
-
- mutex_unlock(&priv->lock);
-
- return 0;
-}
-
-static int quad8_get_quadrature_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- const struct quad8_iio *const priv = iio_priv(indio_dev);
-
- return priv->quadrature_mode[chan->channel];
-}
-
-static const struct iio_enum quad8_quadrature_mode_enum = {
- .items = quad8_quadrature_modes,
- .num_items = ARRAY_SIZE(quad8_quadrature_modes),
- .set = quad8_set_quadrature_mode,
- .get = quad8_get_quadrature_mode
-};
-
-static const char *const quad8_index_polarity_modes[] = {
- "negative",
- "positive"
-};
-
-static int quad8_set_index_polarity(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan, unsigned int index_polarity)
-{
- struct quad8_iio *const priv = iio_priv(indio_dev);
- const int base_offset = priv->base + 2 * chan->channel + 1;
- unsigned int idr_cfg = index_polarity << 1;
-
- mutex_lock(&priv->lock);
-
- idr_cfg |= priv->synchronous_mode[chan->channel];
-
- priv->index_polarity[chan->channel] = index_polarity;
-
- /* Load Index Control configuration to Index Control Register */
- outb(QUAD8_CTR_IDR | idr_cfg, base_offset);
-
- mutex_unlock(&priv->lock);
-
- return 0;
-}
-
-static int quad8_get_index_polarity(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- const struct quad8_iio *const priv = iio_priv(indio_dev);
-
- return priv->index_polarity[chan->channel];
-}
-
-static const struct iio_enum quad8_index_polarity_enum = {
- .items = quad8_index_polarity_modes,
- .num_items = ARRAY_SIZE(quad8_index_polarity_modes),
- .set = quad8_set_index_polarity,
- .get = quad8_get_index_polarity
-};
-
-static const struct iio_chan_spec_ext_info quad8_count_ext_info[] = {
- {
- .name = "preset",
- .shared = IIO_SEPARATE,
- .read = quad8_read_preset,
- .write = quad8_write_preset
- },
- {
- .name = "set_to_preset_on_index",
- .shared = IIO_SEPARATE,
- .read = quad8_read_set_to_preset_on_index,
- .write = quad8_write_set_to_preset_on_index
- },
- IIO_ENUM("noise_error", IIO_SEPARATE, &quad8_noise_error_enum),
- IIO_ENUM_AVAILABLE("noise_error", &quad8_noise_error_enum),
- IIO_ENUM("count_direction", IIO_SEPARATE, &quad8_count_direction_enum),
- IIO_ENUM_AVAILABLE("count_direction", &quad8_count_direction_enum),
- IIO_ENUM("count_mode", IIO_SEPARATE, &quad8_count_mode_enum),
- IIO_ENUM_AVAILABLE("count_mode", &quad8_count_mode_enum),
- IIO_ENUM("quadrature_mode", IIO_SEPARATE, &quad8_quadrature_mode_enum),
- IIO_ENUM_AVAILABLE("quadrature_mode", &quad8_quadrature_mode_enum),
- {}
-};
-
-static const struct iio_chan_spec_ext_info quad8_index_ext_info[] = {
- IIO_ENUM("synchronous_mode", IIO_SEPARATE,
- &quad8_synchronous_mode_enum),
- IIO_ENUM_AVAILABLE("synchronous_mode", &quad8_synchronous_mode_enum),
- IIO_ENUM("index_polarity", IIO_SEPARATE, &quad8_index_polarity_enum),
- IIO_ENUM_AVAILABLE("index_polarity", &quad8_index_polarity_enum),
- {}
-};
-
-#define QUAD8_COUNT_CHAN(_chan) { \
- .type = IIO_COUNT, \
- .channel = (_chan), \
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) | \
- BIT(IIO_CHAN_INFO_ENABLE) | BIT(IIO_CHAN_INFO_SCALE), \
- .ext_info = quad8_count_ext_info, \
- .indexed = 1 \
-}
-
-#define QUAD8_INDEX_CHAN(_chan) { \
- .type = IIO_INDEX, \
- .channel = (_chan), \
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW), \
- .ext_info = quad8_index_ext_info, \
- .indexed = 1 \
-}
-
-static const struct iio_chan_spec quad8_channels[] = {
- QUAD8_COUNT_CHAN(0), QUAD8_INDEX_CHAN(0),
- QUAD8_COUNT_CHAN(1), QUAD8_INDEX_CHAN(1),
- QUAD8_COUNT_CHAN(2), QUAD8_INDEX_CHAN(2),
- QUAD8_COUNT_CHAN(3), QUAD8_INDEX_CHAN(3),
- QUAD8_COUNT_CHAN(4), QUAD8_INDEX_CHAN(4),
- QUAD8_COUNT_CHAN(5), QUAD8_INDEX_CHAN(5),
- QUAD8_COUNT_CHAN(6), QUAD8_INDEX_CHAN(6),
- QUAD8_COUNT_CHAN(7), QUAD8_INDEX_CHAN(7)
-};
-
static int quad8_signal_read(struct counter_device *counter,
struct counter_signal *signal, enum counter_signal_value *val)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
unsigned int state;
/* Only Index signal levels can be read */
@@ -641,7 +117,7 @@ static int quad8_signal_read(struct counter_device *counter,
static int quad8_count_read(struct counter_device *counter,
struct counter_count *count, unsigned long *val)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const int base_offset = priv->base + 2 * count->id;
unsigned int flags;
unsigned int borrow;
@@ -672,7 +148,7 @@ static int quad8_count_read(struct counter_device *counter,
static int quad8_count_write(struct counter_device *counter,
struct counter_count *count, unsigned long val)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const int base_offset = priv->base + 2 * count->id;
int i;
@@ -727,7 +203,7 @@ static enum counter_count_function quad8_count_functions_list[] = {
static int quad8_function_get(struct counter_device *counter,
struct counter_count *count, size_t *function)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const int id = count->id;
mutex_lock(&priv->lock);
@@ -755,7 +231,7 @@ static int quad8_function_get(struct counter_device *counter,
static int quad8_function_set(struct counter_device *counter,
struct counter_count *count, size_t function)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const int id = count->id;
unsigned int *const quadrature_mode = priv->quadrature_mode + id;
unsigned int *const scale = priv->quadrature_scale + id;
@@ -811,7 +287,7 @@ static int quad8_function_set(struct counter_device *counter,
static void quad8_direction_get(struct counter_device *counter,
struct counter_count *count, enum counter_count_direction *direction)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
unsigned int ud_flag;
const unsigned int flag_addr = priv->base + 2 * count->id + 1;
@@ -845,7 +321,7 @@ static int quad8_action_get(struct counter_device *counter,
struct counter_count *count, struct counter_synapse *synapse,
size_t *action)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
int err;
size_t function = 0;
const size_t signal_a_id = count->synapses[0].signal->id;
@@ -905,10 +381,15 @@ static const struct counter_ops quad8_ops = {
.action_get = quad8_action_get
};
+static const char *const quad8_index_polarity_modes[] = {
+ "negative",
+ "positive"
+};
+
static int quad8_index_polarity_get(struct counter_device *counter,
struct counter_signal *signal, size_t *index_polarity)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id - 16;
*index_polarity = priv->index_polarity[channel_id];
@@ -919,7 +400,7 @@ static int quad8_index_polarity_get(struct counter_device *counter,
static int quad8_index_polarity_set(struct counter_device *counter,
struct counter_signal *signal, size_t index_polarity)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id - 16;
const int base_offset = priv->base + 2 * channel_id + 1;
unsigned int idr_cfg = index_polarity << 1;
@@ -945,10 +426,15 @@ static struct counter_signal_enum_ext quad8_index_pol_enum = {
.set = quad8_index_polarity_set
};
+static const char *const quad8_synchronous_modes[] = {
+ "non-synchronous",
+ "synchronous"
+};
+
static int quad8_synchronous_mode_get(struct counter_device *counter,
struct counter_signal *signal, size_t *synchronous_mode)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id - 16;
*synchronous_mode = priv->synchronous_mode[channel_id];
@@ -959,7 +445,7 @@ static int quad8_synchronous_mode_get(struct counter_device *counter,
static int quad8_synchronous_mode_set(struct counter_device *counter,
struct counter_signal *signal, size_t synchronous_mode)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id - 16;
const int base_offset = priv->base + 2 * channel_id + 1;
unsigned int idr_cfg = synchronous_mode;
@@ -1001,7 +487,7 @@ static ssize_t quad8_count_floor_read(struct counter_device *counter,
static int quad8_count_mode_get(struct counter_device *counter,
struct counter_count *count, size_t *cnt_mode)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
/* Map 104-QUAD-8 count mode to Generic Counter count mode */
switch (priv->count_mode[count->id]) {
@@ -1025,7 +511,7 @@ static int quad8_count_mode_get(struct counter_device *counter,
static int quad8_count_mode_set(struct counter_device *counter,
struct counter_count *count, size_t cnt_mode)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
unsigned int mode_cfg;
const int base_offset = priv->base + 2 * count->id + 1;
@@ -1084,7 +570,7 @@ static ssize_t quad8_count_direction_read(struct counter_device *counter,
static ssize_t quad8_count_enable_read(struct counter_device *counter,
struct counter_count *count, void *private, char *buf)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
return sprintf(buf, "%u\n", priv->ab_enable[count->id]);
}
@@ -1092,7 +578,7 @@ static ssize_t quad8_count_enable_read(struct counter_device *counter,
static ssize_t quad8_count_enable_write(struct counter_device *counter,
struct counter_count *count, void *private, const char *buf, size_t len)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const int base_offset = priv->base + 2 * count->id;
int err;
bool ab_enable;
@@ -1116,10 +602,15 @@ static ssize_t quad8_count_enable_write(struct counter_device *counter,
return len;
}
+static const char *const quad8_noise_error_states[] = {
+ "No excessive noise is present at the count inputs",
+ "Excessive noise is present at the count inputs"
+};
+
static int quad8_error_noise_get(struct counter_device *counter,
struct counter_count *count, size_t *noise_error)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
const int base_offset = priv->base + 2 * count->id + 1;
*noise_error = !!(inb(base_offset) & QUAD8_FLAG_E);
@@ -1136,18 +627,18 @@ static struct counter_count_enum_ext quad8_error_noise_enum = {
static ssize_t quad8_count_preset_read(struct counter_device *counter,
struct counter_count *count, void *private, char *buf)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
return sprintf(buf, "%u\n", priv->preset[count->id]);
}
-static void quad8_preset_register_set(struct quad8_iio *quad8iio, int id,
- unsigned int preset)
+static void quad8_preset_register_set(struct quad8 *priv, int id,
+ unsigned int preset)
{
- const unsigned int base_offset = quad8iio->base + 2 * id;
+ const unsigned int base_offset = priv->base + 2 * id;
int i;
- quad8iio->preset[id] = preset;
+ priv->preset[id] = preset;
/* Reset Byte Pointer */
outb(QUAD8_CTR_RLD | QUAD8_RLD_RESET_BP, base_offset + 1);
@@ -1160,7 +651,7 @@ static void quad8_preset_register_set(struct quad8_iio *quad8iio, int id,
static ssize_t quad8_count_preset_write(struct counter_device *counter,
struct counter_count *count, void *private, const char *buf, size_t len)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
unsigned int preset;
int ret;
@@ -1184,7 +675,7 @@ static ssize_t quad8_count_preset_write(struct counter_device *counter,
static ssize_t quad8_count_ceiling_read(struct counter_device *counter,
struct counter_count *count, void *private, char *buf)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
mutex_lock(&priv->lock);
@@ -1205,7 +696,7 @@ static ssize_t quad8_count_ceiling_read(struct counter_device *counter,
static ssize_t quad8_count_ceiling_write(struct counter_device *counter,
struct counter_count *count, void *private, const char *buf, size_t len)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
unsigned int ceiling;
int ret;
@@ -1235,7 +726,7 @@ static ssize_t quad8_count_ceiling_write(struct counter_device *counter,
static ssize_t quad8_count_preset_enable_read(struct counter_device *counter,
struct counter_count *count, void *private, char *buf)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
return sprintf(buf, "%u\n", !priv->preset_enable[count->id]);
}
@@ -1243,7 +734,7 @@ static ssize_t quad8_count_preset_enable_read(struct counter_device *counter,
static ssize_t quad8_count_preset_enable_write(struct counter_device *counter,
struct counter_count *count, void *private, const char *buf, size_t len)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const int base_offset = priv->base + 2 * count->id + 1;
bool preset_enable;
int ret;
@@ -1274,7 +765,7 @@ static ssize_t quad8_signal_cable_fault_read(struct counter_device *counter,
struct counter_signal *signal,
void *private, char *buf)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id / 2;
bool disabled;
unsigned int status;
@@ -1304,7 +795,7 @@ static ssize_t quad8_signal_cable_fault_enable_read(
struct counter_device *counter, struct counter_signal *signal,
void *private, char *buf)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id / 2;
const unsigned int enb = !!(priv->cable_fault_enable & BIT(channel_id));
@@ -1315,7 +806,7 @@ static ssize_t quad8_signal_cable_fault_enable_write(
struct counter_device *counter, struct counter_signal *signal,
void *private, const char *buf, size_t len)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id / 2;
bool enable;
int ret;
@@ -1345,7 +836,7 @@ static ssize_t quad8_signal_cable_fault_enable_write(
static ssize_t quad8_signal_fck_prescaler_read(struct counter_device *counter,
struct counter_signal *signal, void *private, char *buf)
{
- const struct quad8_iio *const priv = counter->priv;
+ const struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id / 2;
return sprintf(buf, "%u\n", priv->fck_prescaler[channel_id]);
@@ -1355,7 +846,7 @@ static ssize_t quad8_signal_fck_prescaler_write(struct counter_device *counter,
struct counter_signal *signal, void *private, const char *buf,
size_t len)
{
- struct quad8_iio *const priv = counter->priv;
+ struct quad8 *const priv = counter->priv;
const size_t channel_id = signal->id / 2;
const int base_offset = priv->base + 2 * channel_id;
u8 prescaler;
@@ -1531,11 +1022,9 @@ static struct counter_count quad8_counts[] = {
static int quad8_probe(struct device *dev, unsigned int id)
{
- struct iio_dev *indio_dev;
- struct quad8_iio *quad8iio;
+ struct quad8 *priv;
int i, j;
unsigned int base_offset;
- int err;
if (!devm_request_region(dev, base[id], QUAD8_EXTENT, dev_name(dev))) {
dev_err(dev, "Unable to lock port addresses (0x%X-0x%X)\n",
@@ -1543,32 +1032,23 @@ static int quad8_probe(struct device *dev, unsigned int id)
return -EBUSY;
}
- /* Allocate IIO device; this also allocates driver data structure */
- indio_dev = devm_iio_device_alloc(dev, sizeof(*quad8iio));
- if (!indio_dev)
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
return -ENOMEM;
- /* Initialize IIO device */
- indio_dev->info = &quad8_info;
- indio_dev->modes = INDIO_DIRECT_MODE;
- indio_dev->num_channels = ARRAY_SIZE(quad8_channels);
- indio_dev->channels = quad8_channels;
- indio_dev->name = dev_name(dev);
-
/* Initialize Counter device and driver data */
- quad8iio = iio_priv(indio_dev);
- quad8iio->counter.name = dev_name(dev);
- quad8iio->counter.parent = dev;
- quad8iio->counter.ops = &quad8_ops;
- quad8iio->counter.counts = quad8_counts;
- quad8iio->counter.num_counts = ARRAY_SIZE(quad8_counts);
- quad8iio->counter.signals = quad8_signals;
- quad8iio->counter.num_signals = ARRAY_SIZE(quad8_signals);
- quad8iio->counter.priv = quad8iio;
- quad8iio->base = base[id];
+ priv->counter.name = dev_name(dev);
+ priv->counter.parent = dev;
+ priv->counter.ops = &quad8_ops;
+ priv->counter.counts = quad8_counts;
+ priv->counter.num_counts = ARRAY_SIZE(quad8_counts);
+ priv->counter.signals = quad8_signals;
+ priv->counter.num_signals = ARRAY_SIZE(quad8_signals);
+ priv->counter.priv = priv;
+ priv->base = base[id];
/* Initialize mutex */
- mutex_init(&quad8iio->lock);
+ mutex_init(&priv->lock);
/* Reset all counters and disable interrupt function */
outb(QUAD8_CHAN_OP_RESET_COUNTERS, base[id] + QUAD8_REG_CHAN_OP);
@@ -1602,13 +1082,8 @@ static int quad8_probe(struct device *dev, unsigned int id)
/* Enable all counters */
outb(QUAD8_CHAN_OP_ENABLE_COUNTERS, base[id] + QUAD8_REG_CHAN_OP);
- /* Register IIO device */
- err = devm_iio_device_register(dev, indio_dev);
- if (err)
- return err;
-
/* Register Counter device */
- return devm_counter_register(dev, &quad8iio->counter);
+ return devm_counter_register(dev, &priv->counter);
}
static struct isa_driver quad8_driver = {
@@ -1621,5 +1096,5 @@ static struct isa_driver quad8_driver = {
module_isa_driver(quad8_driver, num_quad8);
MODULE_AUTHOR("William Breathitt Gray <vilhelm.gray@gmail.com>");
-MODULE_DESCRIPTION("ACCES 104-QUAD-8 IIO driver");
+MODULE_DESCRIPTION("ACCES 104-QUAD-8 driver");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/counter/Kconfig b/drivers/counter/Kconfig
index 2de53ab0dd25..5328705aa09c 100644
--- a/drivers/counter/Kconfig
+++ b/drivers/counter/Kconfig
@@ -14,7 +14,7 @@ if COUNTER
config 104_QUAD_8
tristate "ACCES 104-QUAD-8 driver"
- depends on PC104 && X86 && IIO
+ depends on PC104 && X86
select ISA_BUS_API
help
Say yes here to build support for the ACCES 104-QUAD-8 quadrature
@@ -29,6 +29,16 @@ config 104_QUAD_8
The base port addresses for the devices may be configured via the base
array module parameter.
+config INTERRUPT_CNT
+ tristate "Interrupt counter driver"
+ depends on GPIOLIB
+ help
+ Select this option to enable interrupt counter driver. Any interrupt
+ source can be used by this driver as the event source.
+
+ To compile this driver as a module, choose M here: the
+ module will be called interrupt-cnt.
+
config STM32_TIMER_CNT
tristate "STM32 Timer encoder counter driver"
depends on MFD_STM32_TIMERS || COMPILE_TEST
@@ -41,7 +51,7 @@ config STM32_TIMER_CNT
config STM32_LPTIMER_CNT
tristate "STM32 LP Timer encoder counter driver"
- depends on (MFD_STM32_LPTIMER || COMPILE_TEST) && IIO
+ depends on MFD_STM32_LPTIMER || COMPILE_TEST
help
Select this option to enable STM32 Low-Power Timer quadrature encoder
and counter driver.
diff --git a/drivers/counter/Makefile b/drivers/counter/Makefile
index 0a393f71e481..cb646ed2f039 100644
--- a/drivers/counter/Makefile
+++ b/drivers/counter/Makefile
@@ -6,6 +6,7 @@
obj-$(CONFIG_COUNTER) += counter.o
obj-$(CONFIG_104_QUAD_8) += 104-quad-8.o
+obj-$(CONFIG_INTERRUPT_CNT) += interrupt-cnt.o
obj-$(CONFIG_STM32_TIMER_CNT) += stm32-timer-cnt.o
obj-$(CONFIG_STM32_LPTIMER_CNT) += stm32-lptimer-cnt.o
obj-$(CONFIG_TI_EQEP) += ti-eqep.o
diff --git a/drivers/counter/interrupt-cnt.c b/drivers/counter/interrupt-cnt.c
new file mode 100644
index 000000000000..a99ee7996977
--- /dev/null
+++ b/drivers/counter/interrupt-cnt.c
@@ -0,0 +1,244 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Copyright (c) 2021 Pengutronix, Oleksij Rempel <kernel@pengutronix.de>
+ */
+
+#include <linux/counter.h>
+#include <linux/gpio/consumer.h>
+#include <linux/interrupt.h>
+#include <linux/irq.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+
+#define INTERRUPT_CNT_NAME "interrupt-cnt"
+
+struct interrupt_cnt_priv {
+ atomic_t count;
+ struct counter_device counter;
+ struct gpio_desc *gpio;
+ int irq;
+ bool enabled;
+ struct counter_signal signals;
+ struct counter_synapse synapses;
+ struct counter_count cnts;
+};
+
+static irqreturn_t interrupt_cnt_isr(int irq, void *dev_id)
+{
+ struct interrupt_cnt_priv *priv = dev_id;
+
+ atomic_inc(&priv->count);
+
+ return IRQ_HANDLED;
+}
+
+static ssize_t interrupt_cnt_enable_read(struct counter_device *counter,
+ struct counter_count *count,
+ void *private, char *buf)
+{
+ struct interrupt_cnt_priv *priv = counter->priv;
+
+ return sysfs_emit(buf, "%d\n", priv->enabled);
+}
+
+static ssize_t interrupt_cnt_enable_write(struct counter_device *counter,
+ struct counter_count *count,
+ void *private, const char *buf,
+ size_t len)
+{
+ struct interrupt_cnt_priv *priv = counter->priv;
+ bool enable;
+ ssize_t ret;
+
+ ret = kstrtobool(buf, &enable);
+ if (ret)
+ return ret;
+
+ if (priv->enabled == enable)
+ return len;
+
+ if (enable) {
+ priv->enabled = true;
+ enable_irq(priv->irq);
+ } else {
+ disable_irq(priv->irq);
+ priv->enabled = false;
+ }
+
+ return len;
+}
+
+static const struct counter_count_ext interrupt_cnt_ext[] = {
+ {
+ .name = "enable",
+ .read = interrupt_cnt_enable_read,
+ .write = interrupt_cnt_enable_write,
+ },
+};
+
+static enum counter_synapse_action interrupt_cnt_synapse_actionss[] = {
+ COUNTER_SYNAPSE_ACTION_RISING_EDGE,
+};
+
+static int interrupt_cnt_action_get(struct counter_device *counter,
+ struct counter_count *count,
+ struct counter_synapse *synapse,
+ size_t *action)
+{
+ *action = 0;
+
+ return 0;
+}
+
+static int interrupt_cnt_read(struct counter_device *counter,
+ struct counter_count *count, unsigned long *val)
+{
+ struct interrupt_cnt_priv *priv = counter->priv;
+
+ *val = atomic_read(&priv->count);
+
+ return 0;
+}
+
+static int interrupt_cnt_write(struct counter_device *counter,
+ struct counter_count *count,
+ const unsigned long val)
+{
+ struct interrupt_cnt_priv *priv = counter->priv;
+
+ atomic_set(&priv->count, val);
+
+ return 0;
+}
+
+static enum counter_count_function interrupt_cnt_functions[] = {
+ COUNTER_COUNT_FUNCTION_INCREASE,
+};
+
+static int interrupt_cnt_function_get(struct counter_device *counter,
+ struct counter_count *count,
+ size_t *function)
+{
+ *function = 0;
+
+ return 0;
+}
+
+static int interrupt_cnt_signal_read(struct counter_device *counter,
+ struct counter_signal *signal,
+ enum counter_signal_value *val)
+{
+ struct interrupt_cnt_priv *priv = counter->priv;
+ int ret;
+
+ if (!priv->gpio)
+ return -EINVAL;
+
+ ret = gpiod_get_value(priv->gpio);
+ if (ret < 0)
+ return ret;
+
+ *val = ret ? COUNTER_SIGNAL_HIGH : COUNTER_SIGNAL_LOW;
+
+ return 0;
+}
+
+static const struct counter_ops interrupt_cnt_ops = {
+ .action_get = interrupt_cnt_action_get,
+ .count_read = interrupt_cnt_read,
+ .count_write = interrupt_cnt_write,
+ .function_get = interrupt_cnt_function_get,
+ .signal_read = interrupt_cnt_signal_read,
+};
+
+static int interrupt_cnt_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct interrupt_cnt_priv *priv;
+ int ret;
+
+ priv = devm_kzalloc(dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
+ return -ENOMEM;
+
+ priv->irq = platform_get_irq_optional(pdev, 0);
+ if (priv->irq == -ENXIO)
+ priv->irq = 0;
+ else if (priv->irq < 0)
+ return dev_err_probe(dev, priv->irq, "failed to get IRQ\n");
+
+ priv->gpio = devm_gpiod_get_optional(dev, NULL, GPIOD_IN);
+ if (IS_ERR(priv->gpio))
+ return dev_err_probe(dev, PTR_ERR(priv->gpio), "failed to get GPIO\n");
+
+ if (!priv->irq && !priv->gpio) {
+ dev_err(dev, "IRQ and GPIO are not found. At least one source should be provided\n");
+ return -ENODEV;
+ }
+
+ if (!priv->irq) {
+ int irq = gpiod_to_irq(priv->gpio);
+
+ if (irq < 0)
+ return dev_err_probe(dev, irq, "failed to get IRQ from GPIO\n");
+
+ priv->irq = irq;
+ }
+
+ priv->signals.name = devm_kasprintf(dev, GFP_KERNEL, "IRQ %d",
+ priv->irq);
+ if (!priv->signals.name)
+ return -ENOMEM;
+
+ priv->counter.signals = &priv->signals;
+ priv->counter.num_signals = 1;
+
+ priv->synapses.actions_list = interrupt_cnt_synapse_actionss;
+ priv->synapses.num_actions = ARRAY_SIZE(interrupt_cnt_synapse_actionss);
+ priv->synapses.signal = &priv->signals;
+
+ priv->cnts.name = "Channel 0 Count";
+ priv->cnts.functions_list = interrupt_cnt_functions;
+ priv->cnts.num_functions = ARRAY_SIZE(interrupt_cnt_functions);
+ priv->cnts.synapses = &priv->synapses;
+ priv->cnts.num_synapses = 1;
+ priv->cnts.ext = interrupt_cnt_ext;
+ priv->cnts.num_ext = ARRAY_SIZE(interrupt_cnt_ext);
+
+ priv->counter.priv = priv;
+ priv->counter.name = dev_name(dev);
+ priv->counter.parent = dev;
+ priv->counter.ops = &interrupt_cnt_ops;
+ priv->counter.counts = &priv->cnts;
+ priv->counter.num_counts = 1;
+
+ irq_set_status_flags(priv->irq, IRQ_NOAUTOEN);
+ ret = devm_request_irq(dev, priv->irq, interrupt_cnt_isr,
+ IRQF_TRIGGER_RISING | IRQF_NO_THREAD,
+ dev_name(dev), priv);
+ if (ret)
+ return ret;
+
+ return devm_counter_register(dev, &priv->counter);
+}
+
+static const struct of_device_id interrupt_cnt_of_match[] = {
+ { .compatible = "interrupt-counter", },
+ {}
+};
+MODULE_DEVICE_TABLE(of, interrupt_cnt_of_match);
+
+static struct platform_driver interrupt_cnt_driver = {
+ .probe = interrupt_cnt_probe,
+ .driver = {
+ .name = INTERRUPT_CNT_NAME,
+ .of_match_table = interrupt_cnt_of_match,
+ },
+};
+module_platform_driver(interrupt_cnt_driver);
+
+MODULE_ALIAS("platform:interrupt-counter");
+MODULE_AUTHOR("Oleksij Rempel <o.rempel@pengutronix.de>");
+MODULE_DESCRIPTION("Interrupt counter driver");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/counter/stm32-lptimer-cnt.c b/drivers/counter/stm32-lptimer-cnt.c
index fd6828e2d34f..937439635d53 100644
--- a/drivers/counter/stm32-lptimer-cnt.c
+++ b/drivers/counter/stm32-lptimer-cnt.c
@@ -12,8 +12,8 @@
#include <linux/bitfield.h>
#include <linux/counter.h>
-#include <linux/iio/iio.h>
#include <linux/mfd/stm32-lptimer.h>
+#include <linux/mod_devicetable.h>
#include <linux/module.h>
#include <linux/pinctrl/consumer.h>
#include <linux/platform_device.h>
@@ -107,249 +107,27 @@ static int stm32_lptim_setup(struct stm32_lptim_cnt *priv, int enable)
return regmap_update_bits(priv->regmap, STM32_LPTIM_CFGR, mask, val);
}
-static int stm32_lptim_write_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan,
- int val, int val2, long mask)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
- int ret;
-
- switch (mask) {
- case IIO_CHAN_INFO_ENABLE:
- if (val < 0 || val > 1)
- return -EINVAL;
-
- /* Check nobody uses the timer, or already disabled/enabled */
- ret = stm32_lptim_is_enabled(priv);
- if ((ret < 0) || (!ret && !val))
- return ret;
- if (val && ret)
- return -EBUSY;
-
- ret = stm32_lptim_setup(priv, val);
- if (ret)
- return ret;
- return stm32_lptim_set_enable_state(priv, val);
-
- default:
- return -EINVAL;
- }
-}
-
-static int stm32_lptim_read_raw(struct iio_dev *indio_dev,
- struct iio_chan_spec const *chan,
- int *val, int *val2, long mask)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
- u32 dat;
- int ret;
-
- switch (mask) {
- case IIO_CHAN_INFO_RAW:
- ret = regmap_read(priv->regmap, STM32_LPTIM_CNT, &dat);
- if (ret)
- return ret;
- *val = dat;
- return IIO_VAL_INT;
-
- case IIO_CHAN_INFO_ENABLE:
- ret = stm32_lptim_is_enabled(priv);
- if (ret < 0)
- return ret;
- *val = ret;
- return IIO_VAL_INT;
-
- case IIO_CHAN_INFO_SCALE:
- /* Non-quadrature mode: scale = 1 */
- *val = 1;
- *val2 = 0;
- if (priv->quadrature_mode) {
- /*
- * Quadrature encoder mode:
- * - both edges, quarter cycle, scale is 0.25
- * - either rising/falling edge scale is 0.5
- */
- if (priv->polarity > 1)
- *val2 = 2;
- else
- *val2 = 1;
- }
- return IIO_VAL_FRACTIONAL_LOG2;
-
- default:
- return -EINVAL;
- }
-}
-
-static const struct iio_info stm32_lptim_cnt_iio_info = {
- .read_raw = stm32_lptim_read_raw,
- .write_raw = stm32_lptim_write_raw,
-};
-
-static const char *const stm32_lptim_quadrature_modes[] = {
- "non-quadrature",
- "quadrature",
-};
-
-static int stm32_lptim_get_quadrature_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
-
- return priv->quadrature_mode;
-}
-
-static int stm32_lptim_set_quadrature_mode(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan,
- unsigned int type)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
-
- if (stm32_lptim_is_enabled(priv))
- return -EBUSY;
-
- priv->quadrature_mode = type;
-
- return 0;
-}
-
-static const struct iio_enum stm32_lptim_quadrature_mode_en = {
- .items = stm32_lptim_quadrature_modes,
- .num_items = ARRAY_SIZE(stm32_lptim_quadrature_modes),
- .get = stm32_lptim_get_quadrature_mode,
- .set = stm32_lptim_set_quadrature_mode,
-};
-
-static const char * const stm32_lptim_cnt_polarity[] = {
- "rising-edge", "falling-edge", "both-edges",
-};
-
-static int stm32_lptim_cnt_get_polarity(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
-
- return priv->polarity;
-}
-
-static int stm32_lptim_cnt_set_polarity(struct iio_dev *indio_dev,
- const struct iio_chan_spec *chan,
- unsigned int type)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
-
- if (stm32_lptim_is_enabled(priv))
- return -EBUSY;
-
- priv->polarity = type;
-
- return 0;
-}
-
-static const struct iio_enum stm32_lptim_cnt_polarity_en = {
- .items = stm32_lptim_cnt_polarity,
- .num_items = ARRAY_SIZE(stm32_lptim_cnt_polarity),
- .get = stm32_lptim_cnt_get_polarity,
- .set = stm32_lptim_cnt_set_polarity,
-};
-
-static ssize_t stm32_lptim_cnt_get_ceiling(struct stm32_lptim_cnt *priv,
- char *buf)
-{
- return snprintf(buf, PAGE_SIZE, "%u\n", priv->ceiling);
-}
-
-static ssize_t stm32_lptim_cnt_set_ceiling(struct stm32_lptim_cnt *priv,
- const char *buf, size_t len)
-{
- int ret;
-
- if (stm32_lptim_is_enabled(priv))
- return -EBUSY;
-
- ret = kstrtouint(buf, 0, &priv->ceiling);
- if (ret)
- return ret;
-
- if (priv->ceiling > STM32_LPTIM_MAX_ARR)
- return -EINVAL;
-
- return len;
-}
-
-static ssize_t stm32_lptim_cnt_get_preset_iio(struct iio_dev *indio_dev,
- uintptr_t private,
- const struct iio_chan_spec *chan,
- char *buf)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
-
- return stm32_lptim_cnt_get_ceiling(priv, buf);
-}
-
-static ssize_t stm32_lptim_cnt_set_preset_iio(struct iio_dev *indio_dev,
- uintptr_t private,
- const struct iio_chan_spec *chan,
- const char *buf, size_t len)
-{
- struct stm32_lptim_cnt *priv = iio_priv(indio_dev);
-
- return stm32_lptim_cnt_set_ceiling(priv, buf, len);
-}
-
-/* LP timer with encoder */
-static const struct iio_chan_spec_ext_info stm32_lptim_enc_ext_info[] = {
- {
- .name = "preset",
- .shared = IIO_SEPARATE,
- .read = stm32_lptim_cnt_get_preset_iio,
- .write = stm32_lptim_cnt_set_preset_iio,
- },
- IIO_ENUM("polarity", IIO_SEPARATE, &stm32_lptim_cnt_polarity_en),
- IIO_ENUM_AVAILABLE("polarity", &stm32_lptim_cnt_polarity_en),
- IIO_ENUM("quadrature_mode", IIO_SEPARATE,
- &stm32_lptim_quadrature_mode_en),
- IIO_ENUM_AVAILABLE("quadrature_mode", &stm32_lptim_quadrature_mode_en),
- {}
-};
-
-static const struct iio_chan_spec stm32_lptim_enc_channels = {
- .type = IIO_COUNT,
- .channel = 0,
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
- BIT(IIO_CHAN_INFO_ENABLE) |
- BIT(IIO_CHAN_INFO_SCALE),
- .ext_info = stm32_lptim_enc_ext_info,
- .indexed = 1,
-};
-
-/* LP timer without encoder (counter only) */
-static const struct iio_chan_spec_ext_info stm32_lptim_cnt_ext_info[] = {
- {
- .name = "preset",
- .shared = IIO_SEPARATE,
- .read = stm32_lptim_cnt_get_preset_iio,
- .write = stm32_lptim_cnt_set_preset_iio,
- },
- IIO_ENUM("polarity", IIO_SEPARATE, &stm32_lptim_cnt_polarity_en),
- IIO_ENUM_AVAILABLE("polarity", &stm32_lptim_cnt_polarity_en),
- {}
-};
-
-static const struct iio_chan_spec stm32_lptim_cnt_channels = {
- .type = IIO_COUNT,
- .channel = 0,
- .info_mask_separate = BIT(IIO_CHAN_INFO_RAW) |
- BIT(IIO_CHAN_INFO_ENABLE) |
- BIT(IIO_CHAN_INFO_SCALE),
- .ext_info = stm32_lptim_cnt_ext_info,
- .indexed = 1,
-};
-
/**
* enum stm32_lptim_cnt_function - enumerates LPTimer counter & encoder modes
* @STM32_LPTIM_COUNTER_INCREASE: up count on IN1 rising, falling or both edges
* @STM32_LPTIM_ENCODER_BOTH_EDGE: count on both edges (IN1 & IN2 quadrature)
+ *
+ * In non-quadrature mode, device counts up on active edge.
+ * In quadrature mode, encoder counting scenarios are as follows:
+ * +---------+----------+--------------------+--------------------+
+ * | Active | Level on | IN1 signal | IN2 signal |
+ * | edge | opposite +----------+---------+----------+---------+
+ * | | signal | Rising | Falling | Rising | Falling |
+ * +---------+----------+----------+---------+----------+---------+
+ * | Rising | High -> | Down | - | Up | - |
+ * | edge | Low -> | Up | - | Down | - |
+ * +---------+----------+----------+---------+----------+---------+
+ * | Falling | High -> | - | Up | - | Down |
+ * | edge | Low -> | - | Down | - | Up |
+ * +---------+----------+----------+---------+----------+---------+
+ * | Both | High -> | Down | Up | Up | Down |
+ * | edges | Low -> | Up | Down | Down | Up |
+ * +---------+----------+----------+---------+----------+---------+
*/
enum stm32_lptim_cnt_function {
STM32_LPTIM_COUNTER_INCREASE,
@@ -484,7 +262,7 @@ static ssize_t stm32_lptim_cnt_ceiling_read(struct counter_device *counter,
{
struct stm32_lptim_cnt *const priv = counter->priv;
- return stm32_lptim_cnt_get_ceiling(priv, buf);
+ return snprintf(buf, PAGE_SIZE, "%u\n", priv->ceiling);
}
static ssize_t stm32_lptim_cnt_ceiling_write(struct counter_device *counter,
@@ -493,8 +271,22 @@ static ssize_t stm32_lptim_cnt_ceiling_write(struct counter_device *counter,
const char *buf, size_t len)
{
struct stm32_lptim_cnt *const priv = counter->priv;
+ unsigned int ceiling;
+ int ret;
+
+ if (stm32_lptim_is_enabled(priv))
+ return -EBUSY;
+
+ ret = kstrtouint(buf, 0, &ceiling);
+ if (ret)
+ return ret;
+
+ if (ceiling > STM32_LPTIM_MAX_ARR)
+ return -EINVAL;
+
+ priv->ceiling = ceiling;
- return stm32_lptim_cnt_set_ceiling(priv, buf, len);
+ return len;
}
static const struct counter_count_ext stm32_lptim_cnt_ext[] = {
@@ -630,32 +422,19 @@ static int stm32_lptim_cnt_probe(struct platform_device *pdev)
{
struct stm32_lptimer *ddata = dev_get_drvdata(pdev->dev.parent);
struct stm32_lptim_cnt *priv;
- struct iio_dev *indio_dev;
- int ret;
if (IS_ERR_OR_NULL(ddata))
return -EINVAL;
- indio_dev = devm_iio_device_alloc(&pdev->dev, sizeof(*priv));
- if (!indio_dev)
+ priv = devm_kzalloc(&pdev->dev, sizeof(*priv), GFP_KERNEL);
+ if (!priv)
return -ENOMEM;
- priv = iio_priv(indio_dev);
priv->dev = &pdev->dev;
priv->regmap = ddata->regmap;
priv->clk = ddata->clk;
priv->ceiling = STM32_LPTIM_MAX_ARR;
- /* Initialize IIO device */
- indio_dev->name = dev_name(&pdev->dev);
- indio_dev->dev.of_node = pdev->dev.of_node;
- indio_dev->info = &stm32_lptim_cnt_iio_info;
- if (ddata->has_encoder)
- indio_dev->channels = &stm32_lptim_enc_channels;
- else
- indio_dev->channels = &stm32_lptim_cnt_channels;
- indio_dev->num_channels = 1;
-
/* Initialize Counter device */
priv->counter.name = dev_name(&pdev->dev);
priv->counter.parent = &pdev->dev;
@@ -673,10 +452,6 @@ static int stm32_lptim_cnt_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, priv);
- ret = devm_iio_device_register(&pdev->dev, indio_dev);
- if (ret)
- return ret;
-
return devm_counter_register(&pdev->dev, &priv->counter);
}
diff --git a/drivers/counter/stm32-timer-cnt.c b/drivers/counter/stm32-timer-cnt.c
index ef2a974a2f10..75bc401fdd18 100644
--- a/drivers/counter/stm32-timer-cnt.c
+++ b/drivers/counter/stm32-timer-cnt.c
@@ -31,7 +31,7 @@ struct stm32_timer_cnt {
struct counter_device counter;
struct regmap *regmap;
struct clk *clk;
- u32 ceiling;
+ u32 max_arr;
bool enabled;
struct stm32_timer_regs bak;
};
@@ -44,13 +44,14 @@ struct stm32_timer_cnt {
* @STM32_COUNT_ENCODER_MODE_3: counts on both TI1FP1 and TI2FP2 edges
*/
enum stm32_count_function {
- STM32_COUNT_SLAVE_MODE_DISABLED = -1,
+ STM32_COUNT_SLAVE_MODE_DISABLED,
STM32_COUNT_ENCODER_MODE_1,
STM32_COUNT_ENCODER_MODE_2,
STM32_COUNT_ENCODER_MODE_3,
};
static enum counter_count_function stm32_count_functions[] = {
+ [STM32_COUNT_SLAVE_MODE_DISABLED] = COUNTER_COUNT_FUNCTION_INCREASE,
[STM32_COUNT_ENCODER_MODE_1] = COUNTER_COUNT_FUNCTION_QUADRATURE_X2_A,
[STM32_COUNT_ENCODER_MODE_2] = COUNTER_COUNT_FUNCTION_QUADRATURE_X2_B,
[STM32_COUNT_ENCODER_MODE_3] = COUNTER_COUNT_FUNCTION_QUADRATURE_X4,
@@ -73,8 +74,10 @@ static int stm32_count_write(struct counter_device *counter,
const unsigned long val)
{
struct stm32_timer_cnt *const priv = counter->priv;
+ u32 ceiling;
- if (val > priv->ceiling)
+ regmap_read(priv->regmap, TIM_ARR, &ceiling);
+ if (val > ceiling)
return -EINVAL;
return regmap_write(priv->regmap, TIM_CNT, val);
@@ -90,6 +93,9 @@ static int stm32_count_function_get(struct counter_device *counter,
regmap_read(priv->regmap, TIM_SMCR, &smcr);
switch (smcr & TIM_SMCR_SMS) {
+ case 0:
+ *function = STM32_COUNT_SLAVE_MODE_DISABLED;
+ return 0;
case 1:
*function = STM32_COUNT_ENCODER_MODE_1;
return 0;
@@ -99,9 +105,9 @@ static int stm32_count_function_get(struct counter_device *counter,
case 3:
*function = STM32_COUNT_ENCODER_MODE_3;
return 0;
+ default:
+ return -EINVAL;
}
-
- return -EINVAL;
}
static int stm32_count_function_set(struct counter_device *counter,
@@ -112,6 +118,9 @@ static int stm32_count_function_set(struct counter_device *counter,
u32 cr1, sms;
switch (function) {
+ case STM32_COUNT_SLAVE_MODE_DISABLED:
+ sms = 0;
+ break;
case STM32_COUNT_ENCODER_MODE_1:
sms = 1;
break;
@@ -122,8 +131,7 @@ static int stm32_count_function_set(struct counter_device *counter,
sms = 3;
break;
default:
- sms = 0;
- break;
+ return -EINVAL;
}
/* Store enable status */
@@ -131,10 +139,6 @@ static int stm32_count_function_set(struct counter_device *counter,
regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_CEN, 0);
- /* TIMx_ARR register shouldn't be buffered (ARPE=0) */
- regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_ARPE, 0);
- regmap_write(priv->regmap, TIM_ARR, priv->ceiling);
-
regmap_update_bits(priv->regmap, TIM_SMCR, TIM_SMCR_SMS, sms);
/* Make sure that registers are updated */
@@ -185,11 +189,13 @@ static ssize_t stm32_count_ceiling_write(struct counter_device *counter,
if (ret)
return ret;
+ if (ceiling > priv->max_arr)
+ return -ERANGE;
+
/* TIMx_ARR register shouldn't be buffered (ARPE=0) */
regmap_update_bits(priv->regmap, TIM_CR1, TIM_CR1_ARPE, 0);
regmap_write(priv->regmap, TIM_ARR, ceiling);
- priv->ceiling = ceiling;
return len;
}
@@ -274,31 +280,36 @@ static int stm32_action_get(struct counter_device *counter,
size_t function;
int err;
- /* Default action mode (e.g. STM32_COUNT_SLAVE_MODE_DISABLED) */
- *action = STM32_SYNAPSE_ACTION_NONE;
-
err = stm32_count_function_get(counter, count, &function);
if (err)
- return 0;
+ return err;
switch (function) {
+ case STM32_COUNT_SLAVE_MODE_DISABLED:
+ /* counts on internal clock when CEN=1 */
+ *action = STM32_SYNAPSE_ACTION_NONE;
+ return 0;
case STM32_COUNT_ENCODER_MODE_1:
/* counts up/down on TI1FP1 edge depending on TI2FP2 level */
if (synapse->signal->id == count->synapses[0].signal->id)
*action = STM32_SYNAPSE_ACTION_BOTH_EDGES;
- break;
+ else
+ *action = STM32_SYNAPSE_ACTION_NONE;
+ return 0;
case STM32_COUNT_ENCODER_MODE_2:
/* counts up/down on TI2FP2 edge depending on TI1FP1 level */
if (synapse->signal->id == count->synapses[1].signal->id)
*action = STM32_SYNAPSE_ACTION_BOTH_EDGES;
- break;
+ else
+ *action = STM32_SYNAPSE_ACTION_NONE;
+ return 0;
case STM32_COUNT_ENCODER_MODE_3:
/* counts up/down on both TI1FP1 and TI2FP2 edges */
*action = STM32_SYNAPSE_ACTION_BOTH_EDGES;
- break;
+ return 0;
+ default:
+ return -EINVAL;
}
-
- return 0;
}
static const struct counter_ops stm32_timer_cnt_ops = {
@@ -359,7 +370,7 @@ static int stm32_timer_cnt_probe(struct platform_device *pdev)
priv->regmap = ddata->regmap;
priv->clk = ddata->clk;
- priv->ceiling = ddata->max_arr;
+ priv->max_arr = ddata->max_arr;
priv->counter.name = dev_name(dev);
priv->counter.parent = dev;
diff --git a/drivers/cpufreq/Kconfig b/drivers/cpufreq/Kconfig
index 85de313ddec2..c3038cdc6865 100644
--- a/drivers/cpufreq/Kconfig
+++ b/drivers/cpufreq/Kconfig
@@ -13,7 +13,8 @@ config CPU_FREQ
clock speed, you need to either enable a dynamic cpufreq governor
(see below) after boot, or use a userspace tool.
- For details, take a look at <file:Documentation/cpu-freq>.
+ For details, take a look at
+ <file:Documentation/admin-guide/pm/cpufreq.rst>.
If in doubt, say N.
@@ -140,8 +141,6 @@ config CPU_FREQ_GOV_USERSPACE
To compile this driver as a module, choose M here: the
module will be called cpufreq_userspace.
- For details, take a look at <file:Documentation/cpu-freq/>.
-
If in doubt, say Y.
config CPU_FREQ_GOV_ONDEMAND
@@ -158,7 +157,8 @@ config CPU_FREQ_GOV_ONDEMAND
To compile this driver as a module, choose M here: the
module will be called cpufreq_ondemand.
- For details, take a look at linux/Documentation/cpu-freq.
+ For details, take a look at
+ <file:Documentation/admin-guide/pm/cpufreq.rst>.
If in doubt, say N.
@@ -182,7 +182,8 @@ config CPU_FREQ_GOV_CONSERVATIVE
To compile this driver as a module, choose M here: the
module will be called cpufreq_conservative.
- For details, take a look at linux/Documentation/cpu-freq.
+ For details, take a look at
+ <file:Documentation/admin-guide/pm/cpufreq.rst>.
If in doubt, say N.
@@ -246,8 +247,6 @@ config IA64_ACPI_CPUFREQ
This driver adds a CPUFreq driver which utilizes the ACPI
Processor Performance States.
- For details, take a look at <file:Documentation/cpu-freq/>.
-
If in doubt, say N.
endif
@@ -271,8 +270,6 @@ config LOONGSON2_CPUFREQ
Loongson2F and it's successors support this feature.
- For details, take a look at <file:Documentation/cpu-freq/>.
-
If in doubt, say N.
config LOONGSON1_CPUFREQ
@@ -282,8 +279,6 @@ config LOONGSON1_CPUFREQ
This option adds a CPUFreq driver for loongson1 processors which
support software configurable cpu frequency.
- For details, take a look at <file:Documentation/cpu-freq/>.
-
If in doubt, say N.
endif
@@ -293,8 +288,6 @@ config SPARC_US3_CPUFREQ
help
This adds the CPUFreq driver for UltraSPARC-III processors.
- For details, take a look at <file:Documentation/cpu-freq>.
-
If in doubt, say N.
config SPARC_US2E_CPUFREQ
@@ -302,8 +295,6 @@ config SPARC_US2E_CPUFREQ
help
This adds the CPUFreq driver for UltraSPARC-IIe processors.
- For details, take a look at <file:Documentation/cpu-freq>.
-
If in doubt, say N.
endif
@@ -318,8 +309,6 @@ config SH_CPU_FREQ
will also generate a notice in the boot log before disabling
itself if the CPU in question is not capable of rate rounding.
- For details, take a look at <file:Documentation/cpu-freq>.
-
If unsure, say N.
endif
diff --git a/drivers/cpufreq/Kconfig.arm b/drivers/cpufreq/Kconfig.arm
index e65e0a43be64..a5c5f70acfc9 100644
--- a/drivers/cpufreq/Kconfig.arm
+++ b/drivers/cpufreq/Kconfig.arm
@@ -19,6 +19,16 @@ config ACPI_CPPC_CPUFREQ
If in doubt, say N.
+config ACPI_CPPC_CPUFREQ_FIE
+ bool "Frequency Invariance support for CPPC cpufreq driver"
+ depends on ACPI_CPPC_CPUFREQ && GENERIC_ARCH_TOPOLOGY
+ default y
+ help
+ This extends frequency invariance support in the CPPC cpufreq driver,
+ by using CPPC delivered and reference performance counters.
+
+ If in doubt, say N.
+
config ARM_ALLWINNER_SUN50I_CPUFREQ_NVMEM
tristate "Allwinner nvmem based SUN50I CPUFreq driver"
depends on ARCH_SUNXI
diff --git a/drivers/cpufreq/acpi-cpufreq.c b/drivers/cpufreq/acpi-cpufreq.c
index d1bbc16fba4b..7e7450453714 100644
--- a/drivers/cpufreq/acpi-cpufreq.c
+++ b/drivers/cpufreq/acpi-cpufreq.c
@@ -646,7 +646,11 @@ static u64 get_max_boost_ratio(unsigned int cpu)
return 0;
}
- highest_perf = perf_caps.highest_perf;
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
+ highest_perf = amd_get_highest_perf();
+ else
+ highest_perf = perf_caps.highest_perf;
+
nominal_perf = perf_caps.nominal_perf;
if (!highest_perf || !nominal_perf) {
diff --git a/drivers/cpufreq/armada-37xx-cpufreq.c b/drivers/cpufreq/armada-37xx-cpufreq.c
index b4af4094309b..3fc98a3ffd91 100644
--- a/drivers/cpufreq/armada-37xx-cpufreq.c
+++ b/drivers/cpufreq/armada-37xx-cpufreq.c
@@ -25,6 +25,10 @@
#include "cpufreq-dt.h"
+/* Clk register set */
+#define ARMADA_37XX_CLK_TBG_SEL 0
+#define ARMADA_37XX_CLK_TBG_SEL_CPU_OFF 22
+
/* Power management in North Bridge register set */
#define ARMADA_37XX_NB_L0L1 0x18
#define ARMADA_37XX_NB_L2L3 0x1C
@@ -69,6 +73,8 @@
#define LOAD_LEVEL_NR 4
#define MIN_VOLT_MV 1000
+#define MIN_VOLT_MV_FOR_L1_1000MHZ 1108
+#define MIN_VOLT_MV_FOR_L1_1200MHZ 1155
/* AVS value for the corresponding voltage (in mV) */
static int avs_map[] = {
@@ -80,6 +86,8 @@ static int avs_map[] = {
};
struct armada37xx_cpufreq_state {
+ struct platform_device *pdev;
+ struct device *cpu_dev;
struct regmap *regmap;
u32 nb_l0l1;
u32 nb_l2l3;
@@ -120,10 +128,15 @@ static struct armada_37xx_dvfs *armada_37xx_cpu_freq_info_get(u32 freq)
* will be configured then the DVFS will be enabled.
*/
static void __init armada37xx_cpufreq_dvfs_setup(struct regmap *base,
- struct clk *clk, u8 *divider)
+ struct regmap *clk_base, u8 *divider)
{
+ u32 cpu_tbg_sel;
int load_lvl;
- struct clk *parent;
+
+ /* Determine to which TBG clock is CPU connected */
+ regmap_read(clk_base, ARMADA_37XX_CLK_TBG_SEL, &cpu_tbg_sel);
+ cpu_tbg_sel >>= ARMADA_37XX_CLK_TBG_SEL_CPU_OFF;
+ cpu_tbg_sel &= ARMADA_37XX_NB_TBG_SEL_MASK;
for (load_lvl = 0; load_lvl < LOAD_LEVEL_NR; load_lvl++) {
unsigned int reg, mask, val, offset = 0;
@@ -142,6 +155,11 @@ static void __init armada37xx_cpufreq_dvfs_setup(struct regmap *base,
mask = (ARMADA_37XX_NB_CLK_SEL_MASK
<< ARMADA_37XX_NB_CLK_SEL_OFF);
+ /* Set TBG index, for all levels we use the same TBG */
+ val = cpu_tbg_sel << ARMADA_37XX_NB_TBG_SEL_OFF;
+ mask = (ARMADA_37XX_NB_TBG_SEL_MASK
+ << ARMADA_37XX_NB_TBG_SEL_OFF);
+
/*
* Set cpu divider based on the pre-computed array in
* order to have balanced step.
@@ -160,14 +178,6 @@ static void __init armada37xx_cpufreq_dvfs_setup(struct regmap *base,
regmap_update_bits(base, reg, mask, val);
}
-
- /*
- * Set cpu clock source, for all the level we keep the same
- * clock source that the one already configured. For this one
- * we need to use the clock framework
- */
- parent = clk_get_parent(clk);
- clk_set_parent(clk, parent);
}
/*
@@ -202,6 +212,8 @@ static u32 armada_37xx_avs_val_match(int target_vm)
* - L2 & L3 voltage should be about 150mv smaller than L0 voltage.
* This function calculates L1 & L2 & L3 AVS values dynamically based
* on L0 voltage and fill all AVS values to the AVS value table.
+ * When base CPU frequency is 1000 or 1200 MHz then there is additional
+ * minimal avs value for load L1.
*/
static void __init armada37xx_cpufreq_avs_configure(struct regmap *base,
struct armada_37xx_dvfs *dvfs)
@@ -233,6 +245,19 @@ static void __init armada37xx_cpufreq_avs_configure(struct regmap *base,
for (load_level = 1; load_level < LOAD_LEVEL_NR; load_level++)
dvfs->avs[load_level] = avs_min;
+ /*
+ * Set the avs values for load L0 and L1 when base CPU frequency
+ * is 1000/1200 MHz to its typical initial values according to
+ * the Armada 3700 Hardware Specifications.
+ */
+ if (dvfs->cpu_freq_max >= 1000*1000*1000) {
+ if (dvfs->cpu_freq_max >= 1200*1000*1000)
+ avs_min = armada_37xx_avs_val_match(MIN_VOLT_MV_FOR_L1_1200MHZ);
+ else
+ avs_min = armada_37xx_avs_val_match(MIN_VOLT_MV_FOR_L1_1000MHZ);
+ dvfs->avs[0] = dvfs->avs[1] = avs_min;
+ }
+
return;
}
@@ -252,6 +277,26 @@ static void __init armada37xx_cpufreq_avs_configure(struct regmap *base,
target_vm = avs_map[l0_vdd_min] - 150;
target_vm = target_vm > MIN_VOLT_MV ? target_vm : MIN_VOLT_MV;
dvfs->avs[2] = dvfs->avs[3] = armada_37xx_avs_val_match(target_vm);
+
+ /*
+ * Fix the avs value for load L1 when base CPU frequency is 1000/1200 MHz,
+ * otherwise the CPU gets stuck when switching from load L1 to load L0.
+ * Also ensure that avs value for load L1 is not higher than for L0.
+ */
+ if (dvfs->cpu_freq_max >= 1000*1000*1000) {
+ u32 avs_min_l1;
+
+ if (dvfs->cpu_freq_max >= 1200*1000*1000)
+ avs_min_l1 = armada_37xx_avs_val_match(MIN_VOLT_MV_FOR_L1_1200MHZ);
+ else
+ avs_min_l1 = armada_37xx_avs_val_match(MIN_VOLT_MV_FOR_L1_1000MHZ);
+
+ if (avs_min_l1 > dvfs->avs[0])
+ avs_min_l1 = dvfs->avs[0];
+
+ if (dvfs->avs[1] < avs_min_l1)
+ dvfs->avs[1] = avs_min_l1;
+ }
}
static void __init armada37xx_cpufreq_avs_setup(struct regmap *base,
@@ -357,12 +402,17 @@ static int __init armada37xx_cpufreq_driver_init(void)
struct armada_37xx_dvfs *dvfs;
struct platform_device *pdev;
unsigned long freq;
- unsigned int cur_frequency, base_frequency;
- struct regmap *nb_pm_base, *avs_base;
+ unsigned int base_frequency;
+ struct regmap *nb_clk_base, *nb_pm_base, *avs_base;
struct device *cpu_dev;
int load_lvl, ret;
struct clk *clk, *parent;
+ nb_clk_base =
+ syscon_regmap_lookup_by_compatible("marvell,armada-3700-periph-clock-nb");
+ if (IS_ERR(nb_clk_base))
+ return -ENODEV;
+
nb_pm_base =
syscon_regmap_lookup_by_compatible("marvell,armada-3700-nb-pm");
@@ -413,15 +463,7 @@ static int __init armada37xx_cpufreq_driver_init(void)
return -EINVAL;
}
- /* Get nominal (current) CPU frequency */
- cur_frequency = clk_get_rate(clk);
- if (!cur_frequency) {
- dev_err(cpu_dev, "Failed to get clock rate for CPU\n");
- clk_put(clk);
- return -EINVAL;
- }
-
- dvfs = armada_37xx_cpu_freq_info_get(cur_frequency);
+ dvfs = armada_37xx_cpu_freq_info_get(base_frequency);
if (!dvfs) {
clk_put(clk);
return -EINVAL;
@@ -439,7 +481,7 @@ static int __init armada37xx_cpufreq_driver_init(void)
armada37xx_cpufreq_avs_configure(avs_base, dvfs);
armada37xx_cpufreq_avs_setup(avs_base, dvfs);
- armada37xx_cpufreq_dvfs_setup(nb_pm_base, clk, dvfs->divider);
+ armada37xx_cpufreq_dvfs_setup(nb_pm_base, nb_clk_base, dvfs->divider);
clk_put(clk);
for (load_lvl = ARMADA_37XX_DVFS_LOAD_0; load_lvl < LOAD_LEVEL_NR;
@@ -466,6 +508,9 @@ static int __init armada37xx_cpufreq_driver_init(void)
if (ret)
goto disable_dvfs;
+ armada37xx_cpufreq_state->cpu_dev = cpu_dev;
+ armada37xx_cpufreq_state->pdev = pdev;
+ platform_set_drvdata(pdev, dvfs);
return 0;
disable_dvfs:
@@ -473,7 +518,7 @@ disable_dvfs:
remove_opp:
/* clean-up the already added opp before leaving */
while (load_lvl-- > ARMADA_37XX_DVFS_LOAD_0) {
- freq = cur_frequency / dvfs->divider[load_lvl];
+ freq = base_frequency / dvfs->divider[load_lvl];
dev_pm_opp_remove(cpu_dev, freq);
}
@@ -484,6 +529,26 @@ remove_opp:
/* late_initcall, to guarantee the driver is loaded after A37xx clock driver */
late_initcall(armada37xx_cpufreq_driver_init);
+static void __exit armada37xx_cpufreq_driver_exit(void)
+{
+ struct platform_device *pdev = armada37xx_cpufreq_state->pdev;
+ struct armada_37xx_dvfs *dvfs = platform_get_drvdata(pdev);
+ unsigned long freq;
+ int load_lvl;
+
+ platform_device_unregister(pdev);
+
+ armada37xx_cpufreq_disable_dvfs(armada37xx_cpufreq_state->regmap);
+
+ for (load_lvl = ARMADA_37XX_DVFS_LOAD_0; load_lvl < LOAD_LEVEL_NR; load_lvl++) {
+ freq = dvfs->cpu_freq_max / dvfs->divider[load_lvl];
+ dev_pm_opp_remove(armada37xx_cpufreq_state->cpu_dev, freq);
+ }
+
+ kfree(armada37xx_cpufreq_state);
+}
+module_exit(armada37xx_cpufreq_driver_exit);
+
static const struct of_device_id __maybe_unused armada37xx_cpufreq_of_match[] = {
{ .compatible = "marvell,armada-3700-nb-pm" },
{ },
diff --git a/drivers/cpufreq/cppc_cpufreq.c b/drivers/cpufreq/cppc_cpufreq.c
index 8a482c434ea6..3848b4c222e1 100644
--- a/drivers/cpufreq/cppc_cpufreq.c
+++ b/drivers/cpufreq/cppc_cpufreq.c
@@ -10,14 +10,18 @@
#define pr_fmt(fmt) "CPPC Cpufreq:" fmt
+#include <linux/arch_topology.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/delay.h>
#include <linux/cpu.h>
#include <linux/cpufreq.h>
#include <linux/dmi.h>
+#include <linux/irq_work.h>
+#include <linux/kthread.h>
#include <linux/time.h>
#include <linux/vmalloc.h>
+#include <uapi/linux/sched/types.h>
#include <asm/unaligned.h>
@@ -57,6 +61,204 @@ static struct cppc_workaround_oem_info wa_info[] = {
}
};
+#ifdef CONFIG_ACPI_CPPC_CPUFREQ_FIE
+
+/* Frequency invariance support */
+struct cppc_freq_invariance {
+ int cpu;
+ struct irq_work irq_work;
+ struct kthread_work work;
+ struct cppc_perf_fb_ctrs prev_perf_fb_ctrs;
+ struct cppc_cpudata *cpu_data;
+};
+
+static DEFINE_PER_CPU(struct cppc_freq_invariance, cppc_freq_inv);
+static struct kthread_worker *kworker_fie;
+static bool fie_disabled;
+
+static struct cpufreq_driver cppc_cpufreq_driver;
+static unsigned int hisi_cppc_cpufreq_get_rate(unsigned int cpu);
+static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
+ struct cppc_perf_fb_ctrs fb_ctrs_t0,
+ struct cppc_perf_fb_ctrs fb_ctrs_t1);
+
+/**
+ * cppc_scale_freq_workfn - CPPC arch_freq_scale updater for frequency invariance
+ * @work: The work item.
+ *
+ * The CPPC driver register itself with the topology core to provide its own
+ * implementation (cppc_scale_freq_tick()) of topology_scale_freq_tick() which
+ * gets called by the scheduler on every tick.
+ *
+ * Note that the arch specific counters have higher priority than CPPC counters,
+ * if available, though the CPPC driver doesn't need to have any special
+ * handling for that.
+ *
+ * On an invocation of cppc_scale_freq_tick(), we schedule an irq work (since we
+ * reach here from hard-irq context), which then schedules a normal work item
+ * and cppc_scale_freq_workfn() updates the per_cpu arch_freq_scale variable
+ * based on the counter updates since the last tick.
+ */
+static void cppc_scale_freq_workfn(struct kthread_work *work)
+{
+ struct cppc_freq_invariance *cppc_fi;
+ struct cppc_perf_fb_ctrs fb_ctrs = {0};
+ struct cppc_cpudata *cpu_data;
+ unsigned long local_freq_scale;
+ u64 perf;
+
+ cppc_fi = container_of(work, struct cppc_freq_invariance, work);
+ cpu_data = cppc_fi->cpu_data;
+
+ if (cppc_get_perf_ctrs(cppc_fi->cpu, &fb_ctrs)) {
+ pr_warn("%s: failed to read perf counters\n", __func__);
+ return;
+ }
+
+ cppc_fi->prev_perf_fb_ctrs = fb_ctrs;
+ perf = cppc_perf_from_fbctrs(cpu_data, cppc_fi->prev_perf_fb_ctrs,
+ fb_ctrs);
+
+ perf <<= SCHED_CAPACITY_SHIFT;
+ local_freq_scale = div64_u64(perf, cpu_data->perf_caps.highest_perf);
+ if (WARN_ON(local_freq_scale > 1024))
+ local_freq_scale = 1024;
+
+ per_cpu(arch_freq_scale, cppc_fi->cpu) = local_freq_scale;
+}
+
+static void cppc_irq_work(struct irq_work *irq_work)
+{
+ struct cppc_freq_invariance *cppc_fi;
+
+ cppc_fi = container_of(irq_work, struct cppc_freq_invariance, irq_work);
+ kthread_queue_work(kworker_fie, &cppc_fi->work);
+}
+
+static void cppc_scale_freq_tick(void)
+{
+ struct cppc_freq_invariance *cppc_fi = &per_cpu(cppc_freq_inv, smp_processor_id());
+
+ /*
+ * cppc_get_perf_ctrs() can potentially sleep, call that from the right
+ * context.
+ */
+ irq_work_queue(&cppc_fi->irq_work);
+}
+
+static struct scale_freq_data cppc_sftd = {
+ .source = SCALE_FREQ_SOURCE_CPPC,
+ .set_freq_scale = cppc_scale_freq_tick,
+};
+
+static void cppc_freq_invariance_policy_init(struct cpufreq_policy *policy,
+ struct cppc_cpudata *cpu_data)
+{
+ struct cppc_perf_fb_ctrs fb_ctrs = {0};
+ struct cppc_freq_invariance *cppc_fi;
+ int i, ret;
+
+ if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
+ return;
+
+ if (fie_disabled)
+ return;
+
+ for_each_cpu(i, policy->cpus) {
+ cppc_fi = &per_cpu(cppc_freq_inv, i);
+ cppc_fi->cpu = i;
+ cppc_fi->cpu_data = cpu_data;
+ kthread_init_work(&cppc_fi->work, cppc_scale_freq_workfn);
+ init_irq_work(&cppc_fi->irq_work, cppc_irq_work);
+
+ ret = cppc_get_perf_ctrs(i, &fb_ctrs);
+ if (ret) {
+ pr_warn("%s: failed to read perf counters: %d\n",
+ __func__, ret);
+ fie_disabled = true;
+ } else {
+ cppc_fi->prev_perf_fb_ctrs = fb_ctrs;
+ }
+ }
+}
+
+static void __init cppc_freq_invariance_init(void)
+{
+ struct sched_attr attr = {
+ .size = sizeof(struct sched_attr),
+ .sched_policy = SCHED_DEADLINE,
+ .sched_nice = 0,
+ .sched_priority = 0,
+ /*
+ * Fake (unused) bandwidth; workaround to "fix"
+ * priority inheritance.
+ */
+ .sched_runtime = 1000000,
+ .sched_deadline = 10000000,
+ .sched_period = 10000000,
+ };
+ int ret;
+
+ if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
+ return;
+
+ if (fie_disabled)
+ return;
+
+ kworker_fie = kthread_create_worker(0, "cppc_fie");
+ if (IS_ERR(kworker_fie))
+ return;
+
+ ret = sched_setattr_nocheck(kworker_fie->task, &attr);
+ if (ret) {
+ pr_warn("%s: failed to set SCHED_DEADLINE: %d\n", __func__,
+ ret);
+ kthread_destroy_worker(kworker_fie);
+ return;
+ }
+
+ /* Register for freq-invariance */
+ topology_set_scale_freq_source(&cppc_sftd, cpu_present_mask);
+}
+
+static void cppc_freq_invariance_exit(void)
+{
+ struct cppc_freq_invariance *cppc_fi;
+ int i;
+
+ if (cppc_cpufreq_driver.get == hisi_cppc_cpufreq_get_rate)
+ return;
+
+ if (fie_disabled)
+ return;
+
+ topology_clear_scale_freq_source(SCALE_FREQ_SOURCE_CPPC, cpu_present_mask);
+
+ for_each_possible_cpu(i) {
+ cppc_fi = &per_cpu(cppc_freq_inv, i);
+ irq_work_sync(&cppc_fi->irq_work);
+ }
+
+ kthread_destroy_worker(kworker_fie);
+ kworker_fie = NULL;
+}
+
+#else
+static inline void
+cppc_freq_invariance_policy_init(struct cpufreq_policy *policy,
+ struct cppc_cpudata *cpu_data)
+{
+}
+
+static inline void cppc_freq_invariance_init(void)
+{
+}
+
+static inline void cppc_freq_invariance_exit(void)
+{
+}
+#endif /* CONFIG_ACPI_CPPC_CPUFREQ_FIE */
+
/* Callback function used to retrieve the max frequency from DMI */
static void cppc_find_dmi_mhz(const struct dmi_header *dm, void *private)
{
@@ -216,26 +418,16 @@ static unsigned int cppc_cpufreq_get_transition_delay_us(unsigned int cpu)
{
unsigned long implementor = read_cpuid_implementor();
unsigned long part_num = read_cpuid_part_number();
- unsigned int delay_us = 0;
switch (implementor) {
case ARM_CPU_IMP_QCOM:
switch (part_num) {
case QCOM_CPU_PART_FALKOR_V1:
case QCOM_CPU_PART_FALKOR:
- delay_us = 10000;
- break;
- default:
- delay_us = cppc_get_transition_latency(cpu) / NSEC_PER_USEC;
- break;
+ return 10000;
}
- break;
- default:
- delay_us = cppc_get_transition_latency(cpu) / NSEC_PER_USEC;
- break;
}
-
- return delay_us;
+ return cppc_get_transition_latency(cpu) / NSEC_PER_USEC;
}
#else
@@ -355,9 +547,12 @@ static int cppc_cpufreq_cpu_init(struct cpufreq_policy *policy)
cpu_data->perf_ctrls.desired_perf = caps->highest_perf;
ret = cppc_set_perf(cpu, &cpu_data->perf_ctrls);
- if (ret)
+ if (ret) {
pr_debug("Err setting perf value:%d on CPU:%d. ret:%d\n",
caps->highest_perf, cpu, ret);
+ } else {
+ cppc_freq_invariance_policy_init(policy, cpu_data);
+ }
return ret;
}
@@ -370,12 +565,12 @@ static inline u64 get_delta(u64 t1, u64 t0)
return (u32)t1 - (u32)t0;
}
-static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
- struct cppc_perf_fb_ctrs fb_ctrs_t0,
- struct cppc_perf_fb_ctrs fb_ctrs_t1)
+static int cppc_perf_from_fbctrs(struct cppc_cpudata *cpu_data,
+ struct cppc_perf_fb_ctrs fb_ctrs_t0,
+ struct cppc_perf_fb_ctrs fb_ctrs_t1)
{
u64 delta_reference, delta_delivered;
- u64 reference_perf, delivered_perf;
+ u64 reference_perf;
reference_perf = fb_ctrs_t0.reference_perf;
@@ -384,12 +579,21 @@ static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
delta_delivered = get_delta(fb_ctrs_t1.delivered,
fb_ctrs_t0.delivered);
- /* Check to avoid divide-by zero */
- if (delta_reference || delta_delivered)
- delivered_perf = (reference_perf * delta_delivered) /
- delta_reference;
- else
- delivered_perf = cpu_data->perf_ctrls.desired_perf;
+ /* Check to avoid divide-by zero and invalid delivered_perf */
+ if (!delta_reference || !delta_delivered)
+ return cpu_data->perf_ctrls.desired_perf;
+
+ return (reference_perf * delta_delivered) / delta_reference;
+}
+
+static int cppc_get_rate_from_fbctrs(struct cppc_cpudata *cpu_data,
+ struct cppc_perf_fb_ctrs fb_ctrs_t0,
+ struct cppc_perf_fb_ctrs fb_ctrs_t1)
+{
+ u64 delivered_perf;
+
+ delivered_perf = cppc_perf_from_fbctrs(cpu_data, fb_ctrs_t0,
+ fb_ctrs_t1);
return cppc_cpufreq_perf_to_khz(cpu_data, delivered_perf);
}
@@ -514,6 +718,8 @@ static void cppc_check_hisi_workaround(void)
static int __init cppc_cpufreq_init(void)
{
+ int ret;
+
if ((acpi_disabled) || !acpi_cpc_valid())
return -ENODEV;
@@ -521,7 +727,11 @@ static int __init cppc_cpufreq_init(void)
cppc_check_hisi_workaround();
- return cpufreq_register_driver(&cppc_cpufreq_driver);
+ ret = cpufreq_register_driver(&cppc_cpufreq_driver);
+ if (!ret)
+ cppc_freq_invariance_init();
+
+ return ret;
}
static inline void free_cpu_data(void)
@@ -538,6 +748,7 @@ static inline void free_cpu_data(void)
static void __exit cppc_cpufreq_exit(void)
{
+ cppc_freq_invariance_exit();
cpufreq_unregister_driver(&cppc_cpufreq_driver);
free_cpu_data();
diff --git a/drivers/cpufreq/cpufreq-dt-platdev.c b/drivers/cpufreq/cpufreq-dt-platdev.c
index 3ba2f716fe97..5e07065ec22f 100644
--- a/drivers/cpufreq/cpufreq-dt-platdev.c
+++ b/drivers/cpufreq/cpufreq-dt-platdev.c
@@ -103,6 +103,8 @@ static const struct of_device_id whitelist[] __initconst = {
static const struct of_device_id blacklist[] __initconst = {
{ .compatible = "allwinner,sun50i-h6", },
+ { .compatible = "arm,vexpress", },
+
{ .compatible = "calxeda,highbank", },
{ .compatible = "calxeda,ecx-2000", },
diff --git a/drivers/cpufreq/cpufreq-dt.c b/drivers/cpufreq/cpufreq-dt.c
index b1e1bdc63b01..ece52863ba62 100644
--- a/drivers/cpufreq/cpufreq-dt.c
+++ b/drivers/cpufreq/cpufreq-dt.c
@@ -255,10 +255,15 @@ static int dt_cpufreq_early_init(struct device *dev, int cpu)
* before updating priv->cpus. Otherwise, we will end up creating
* duplicate OPPs for the CPUs.
*
- * OPPs might be populated at runtime, don't check for error here.
+ * OPPs might be populated at runtime, don't fail for error here unless
+ * it is -EPROBE_DEFER.
*/
- if (!dev_pm_opp_of_cpumask_add_table(priv->cpus))
+ ret = dev_pm_opp_of_cpumask_add_table(priv->cpus);
+ if (!ret) {
priv->have_static_opps = true;
+ } else if (ret == -EPROBE_DEFER) {
+ goto out;
+ }
/*
* The OPP table must be initialized, statically or dynamically, by this
diff --git a/drivers/cpufreq/cpufreq.c b/drivers/cpufreq/cpufreq.c
index 1d1b563cea4b..802abc925b2a 100644
--- a/drivers/cpufreq/cpufreq.c
+++ b/drivers/cpufreq/cpufreq.c
@@ -42,9 +42,6 @@ static LIST_HEAD(cpufreq_policy_list);
#define for_each_inactive_policy(__policy) \
for_each_suitable_policy(__policy, false)
-#define for_each_policy(__policy) \
- list_for_each_entry(__policy, &cpufreq_policy_list, policy_list)
-
/* Iterate over governors */
static LIST_HEAD(cpufreq_governor_list);
#define for_each_governor(__governor) \
diff --git a/drivers/cpufreq/freq_table.c b/drivers/cpufreq/freq_table.c
index d3f756f7b5a0..67e56cf638ef 100644
--- a/drivers/cpufreq/freq_table.c
+++ b/drivers/cpufreq/freq_table.c
@@ -267,7 +267,7 @@ struct freq_attr cpufreq_freq_attr_##_name##_freqs = \
__ATTR_RO(_name##_frequencies)
/*
- * show_scaling_available_frequencies - show available normal frequencies for
+ * scaling_available_frequencies_show - show available normal frequencies for
* the specified CPU
*/
static ssize_t scaling_available_frequencies_show(struct cpufreq_policy *policy,
@@ -279,7 +279,7 @@ cpufreq_attr_available_freq(scaling_available);
EXPORT_SYMBOL_GPL(cpufreq_freq_attr_scaling_available_freqs);
/*
- * show_available_boost_freqs - show available boost frequencies for
+ * scaling_boost_frequencies_show - show available boost frequencies for
* the specified CPU
*/
static ssize_t scaling_boost_frequencies_show(struct cpufreq_policy *policy,
diff --git a/drivers/cpufreq/ia64-acpi-cpufreq.c b/drivers/cpufreq/ia64-acpi-cpufreq.c
index 2efe7189ccc4..c6bdc455517f 100644
--- a/drivers/cpufreq/ia64-acpi-cpufreq.c
+++ b/drivers/cpufreq/ia64-acpi-cpufreq.c
@@ -54,7 +54,7 @@ processor_set_pstate (
retval = ia64_pal_set_pstate((u64)value);
if (retval) {
- pr_debug("Failed to set freq to 0x%x, with error 0x%lx\n",
+ pr_debug("Failed to set freq to 0x%x, with error 0x%llx\n",
value, retval);
return -ENODEV;
}
@@ -77,7 +77,7 @@ processor_get_pstate (
if (retval)
pr_debug("Failed to get current freq with "
- "error 0x%lx, idx 0x%x\n", retval, *value);
+ "error 0x%llx, idx 0x%x\n", retval, *value);
return (int)retval;
}
diff --git a/drivers/cpufreq/intel_pstate.c b/drivers/cpufreq/intel_pstate.c
index 5175ae3cac44..0e69dffd5a76 100644
--- a/drivers/cpufreq/intel_pstate.c
+++ b/drivers/cpufreq/intel_pstate.c
@@ -819,19 +819,21 @@ static struct freq_attr *hwp_cpufreq_attrs[] = {
NULL,
};
-static void intel_pstate_get_hwp_max(struct cpudata *cpu, int *phy_max,
- int *current_max)
+static void __intel_pstate_get_hwp_cap(struct cpudata *cpu)
{
u64 cap;
rdmsrl_on_cpu(cpu->cpu, MSR_HWP_CAPABILITIES, &cap);
WRITE_ONCE(cpu->hwp_cap_cached, cap);
- if (global.no_turbo || global.turbo_disabled)
- *current_max = HWP_GUARANTEED_PERF(cap);
- else
- *current_max = HWP_HIGHEST_PERF(cap);
+ cpu->pstate.max_pstate = HWP_GUARANTEED_PERF(cap);
+ cpu->pstate.turbo_pstate = HWP_HIGHEST_PERF(cap);
+}
- *phy_max = HWP_HIGHEST_PERF(cap);
+static void intel_pstate_get_hwp_cap(struct cpudata *cpu)
+{
+ __intel_pstate_get_hwp_cap(cpu);
+ cpu->pstate.max_freq = cpu->pstate.max_pstate * cpu->pstate.scaling;
+ cpu->pstate.turbo_freq = cpu->pstate.turbo_pstate * cpu->pstate.scaling;
}
static void intel_pstate_hwp_set(unsigned int cpu)
@@ -1195,12 +1197,13 @@ static ssize_t store_no_turbo(struct kobject *a, struct kobj_attribute *b,
static void update_qos_request(enum freq_qos_req_type type)
{
- int max_state, turbo_max, freq, i, perf_pct;
struct freq_qos_request *req;
struct cpufreq_policy *policy;
+ int i;
for_each_possible_cpu(i) {
struct cpudata *cpu = all_cpu_data[i];
+ unsigned int freq, perf_pct;
policy = cpufreq_cpu_get(i);
if (!policy)
@@ -1213,9 +1216,7 @@ static void update_qos_request(enum freq_qos_req_type type)
continue;
if (hwp_active)
- intel_pstate_get_hwp_max(cpu, &turbo_max, &max_state);
- else
- turbo_max = cpu->pstate.turbo_pstate;
+ intel_pstate_get_hwp_cap(cpu);
if (type == FREQ_QOS_MIN) {
perf_pct = global.min_perf_pct;
@@ -1224,8 +1225,7 @@ static void update_qos_request(enum freq_qos_req_type type)
perf_pct = global.max_perf_pct;
}
- freq = DIV_ROUND_UP(turbo_max * perf_pct, 100);
- freq *= cpu->pstate.scaling;
+ freq = DIV_ROUND_UP(cpu->pstate.turbo_freq * perf_pct, 100);
if (freq_qos_update_request(req, freq) < 0)
pr_warn("Failed to update freq constraint: CPU%d\n", i);
@@ -1715,21 +1715,17 @@ static void intel_pstate_get_cpu_pstates(struct cpudata *cpu)
{
cpu->pstate.min_pstate = pstate_funcs.get_min();
cpu->pstate.max_pstate_physical = pstate_funcs.get_max_physical();
- cpu->pstate.turbo_pstate = pstate_funcs.get_turbo();
cpu->pstate.scaling = pstate_funcs.get_scaling();
if (hwp_active && !hwp_mode_bdw) {
- unsigned int phy_max, current_max;
-
- intel_pstate_get_hwp_max(cpu, &phy_max, &current_max);
- cpu->pstate.turbo_freq = phy_max * cpu->pstate.scaling;
- cpu->pstate.turbo_pstate = phy_max;
- cpu->pstate.max_pstate = HWP_GUARANTEED_PERF(READ_ONCE(cpu->hwp_cap_cached));
+ __intel_pstate_get_hwp_cap(cpu);
} else {
- cpu->pstate.turbo_freq = cpu->pstate.turbo_pstate * cpu->pstate.scaling;
cpu->pstate.max_pstate = pstate_funcs.get_max();
+ cpu->pstate.turbo_pstate = pstate_funcs.get_turbo();
}
+
cpu->pstate.max_freq = cpu->pstate.max_pstate * cpu->pstate.scaling;
+ cpu->pstate.turbo_freq = cpu->pstate.turbo_pstate * cpu->pstate.scaling;
if (pstate_funcs.get_aperf_mperf_shift)
cpu->aperf_mperf_shift = pstate_funcs.get_aperf_mperf_shift();
@@ -2199,41 +2195,34 @@ static void intel_pstate_update_perf_limits(struct cpudata *cpu,
unsigned int policy_min,
unsigned int policy_max)
{
+ int scaling = cpu->pstate.scaling;
int32_t max_policy_perf, min_policy_perf;
- int max_state, turbo_max;
- int max_freq;
/*
- * HWP needs some special consideration, because on BDX the
- * HWP_REQUEST uses abstract value to represent performance
- * rather than pure ratios.
+ * HWP needs some special consideration, because HWP_REQUEST uses
+ * abstract values to represent performance rather than pure ratios.
*/
- if (hwp_active) {
- intel_pstate_get_hwp_max(cpu, &turbo_max, &max_state);
- } else {
- max_state = global.no_turbo || global.turbo_disabled ?
- cpu->pstate.max_pstate : cpu->pstate.turbo_pstate;
- turbo_max = cpu->pstate.turbo_pstate;
- }
- max_freq = max_state * cpu->pstate.scaling;
+ if (hwp_active)
+ intel_pstate_get_hwp_cap(cpu);
- max_policy_perf = max_state * policy_max / max_freq;
+ max_policy_perf = policy_max / scaling;
if (policy_max == policy_min) {
min_policy_perf = max_policy_perf;
} else {
- min_policy_perf = max_state * policy_min / max_freq;
+ min_policy_perf = policy_min / scaling;
min_policy_perf = clamp_t(int32_t, min_policy_perf,
0, max_policy_perf);
}
- pr_debug("cpu:%d max_state %d min_policy_perf:%d max_policy_perf:%d\n",
- cpu->cpu, max_state, min_policy_perf, max_policy_perf);
+ pr_debug("cpu:%d min_policy_perf:%d max_policy_perf:%d\n",
+ cpu->cpu, min_policy_perf, max_policy_perf);
/* Normalize user input to [min_perf, max_perf] */
if (per_cpu_limits) {
cpu->min_perf_ratio = min_policy_perf;
cpu->max_perf_ratio = max_policy_perf;
} else {
+ int turbo_max = cpu->pstate.turbo_pstate;
int32_t global_min, global_max;
/* Global limits are in percent of the maximum turbo P-state. */
@@ -2322,10 +2311,9 @@ static void intel_pstate_verify_cpu_policy(struct cpudata *cpu,
update_turbo_state();
if (hwp_active) {
- int max_state, turbo_max;
-
- intel_pstate_get_hwp_max(cpu, &turbo_max, &max_state);
- max_freq = max_state * cpu->pstate.scaling;
+ intel_pstate_get_hwp_cap(cpu);
+ max_freq = global.no_turbo || global.turbo_disabled ?
+ cpu->pstate.max_freq : cpu->pstate.turbo_freq;
} else {
max_freq = intel_pstate_get_max_freq(cpu);
}
@@ -2416,25 +2404,15 @@ static int __intel_pstate_cpu_init(struct cpufreq_policy *policy)
cpu->max_perf_ratio = 0xFF;
cpu->min_perf_ratio = 0;
- policy->min = cpu->pstate.min_pstate * cpu->pstate.scaling;
- policy->max = cpu->pstate.turbo_pstate * cpu->pstate.scaling;
-
/* cpuinfo and default policy values */
policy->cpuinfo.min_freq = cpu->pstate.min_pstate * cpu->pstate.scaling;
update_turbo_state();
global.turbo_disabled_mf = global.turbo_disabled;
policy->cpuinfo.max_freq = global.turbo_disabled ?
- cpu->pstate.max_pstate : cpu->pstate.turbo_pstate;
- policy->cpuinfo.max_freq *= cpu->pstate.scaling;
-
- if (hwp_active) {
- unsigned int max_freq;
-
- max_freq = global.turbo_disabled ?
cpu->pstate.max_freq : cpu->pstate.turbo_freq;
- if (max_freq < policy->cpuinfo.max_freq)
- policy->cpuinfo.max_freq = max_freq;
- }
+
+ policy->min = policy->cpuinfo.min_freq;
+ policy->max = policy->cpuinfo.max_freq;
intel_pstate_init_acpi_perf_limits(policy);
@@ -2683,10 +2661,10 @@ static void intel_cpufreq_adjust_perf(unsigned int cpunum,
static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy)
{
- int max_state, turbo_max, min_freq, max_freq, ret;
struct freq_qos_request *req;
struct cpudata *cpu;
struct device *dev;
+ int ret, freq;
dev = get_cpu_device(policy->cpu);
if (!dev)
@@ -2711,30 +2689,31 @@ static int intel_cpufreq_cpu_init(struct cpufreq_policy *policy)
if (hwp_active) {
u64 value;
- intel_pstate_get_hwp_max(cpu, &turbo_max, &max_state);
policy->transition_delay_us = INTEL_CPUFREQ_TRANSITION_DELAY_HWP;
+
+ intel_pstate_get_hwp_cap(cpu);
+
rdmsrl_on_cpu(cpu->cpu, MSR_HWP_REQUEST, &value);
WRITE_ONCE(cpu->hwp_req_cached, value);
+
cpu->epp_cached = intel_pstate_get_epp(cpu, value);
} else {
- turbo_max = cpu->pstate.turbo_pstate;
policy->transition_delay_us = INTEL_CPUFREQ_TRANSITION_DELAY;
}
- min_freq = DIV_ROUND_UP(turbo_max * global.min_perf_pct, 100);
- min_freq *= cpu->pstate.scaling;
- max_freq = DIV_ROUND_UP(turbo_max * global.max_perf_pct, 100);
- max_freq *= cpu->pstate.scaling;
+ freq = DIV_ROUND_UP(cpu->pstate.turbo_freq * global.min_perf_pct, 100);
ret = freq_qos_add_request(&policy->constraints, req, FREQ_QOS_MIN,
- min_freq);
+ freq);
if (ret < 0) {
dev_err(dev, "Failed to add min-freq constraint (%d)\n", ret);
goto free_req;
}
+ freq = DIV_ROUND_UP(cpu->pstate.turbo_freq * global.max_perf_pct, 100);
+
ret = freq_qos_add_request(&policy->constraints, req + 1, FREQ_QOS_MAX,
- max_freq);
+ freq);
if (ret < 0) {
dev_err(dev, "Failed to add max-freq constraint (%d)\n", ret);
goto remove_min_req;
@@ -3054,6 +3033,14 @@ static const struct x86_cpu_id hwp_support_ids[] __initconst = {
{}
};
+static bool intel_pstate_hwp_is_enabled(void)
+{
+ u64 value;
+
+ rdmsrl(MSR_PM_ENABLE, value);
+ return !!(value & 0x1);
+}
+
static int __init intel_pstate_init(void)
{
const struct x86_cpu_id *id;
@@ -3072,8 +3059,12 @@ static int __init intel_pstate_init(void)
* Avoid enabling HWP for processors without EPP support,
* because that means incomplete HWP implementation which is a
* corner case and supporting it is generally problematic.
+ *
+ * If HWP is enabled already, though, there is no choice but to
+ * deal with it.
*/
- if (!no_hwp && boot_cpu_has(X86_FEATURE_HWP_EPP)) {
+ if ((!no_hwp && boot_cpu_has(X86_FEATURE_HWP_EPP)) ||
+ intel_pstate_hwp_is_enabled()) {
hwp_active++;
hwp_mode_bdw = id->driver_data;
intel_pstate.attr = hwp_cpufreq_attrs;
diff --git a/drivers/cpufreq/qcom-cpufreq-hw.c b/drivers/cpufreq/qcom-cpufreq-hw.c
index d3c23447b892..f86859bf76f1 100644
--- a/drivers/cpufreq/qcom-cpufreq-hw.c
+++ b/drivers/cpufreq/qcom-cpufreq-hw.c
@@ -317,9 +317,9 @@ static int qcom_cpufreq_hw_cpu_init(struct cpufreq_policy *policy)
}
base = ioremap(res->start, resource_size(res));
- if (IS_ERR(base)) {
+ if (!base) {
dev_err(dev, "failed to map resource %pR\n", res);
- ret = PTR_ERR(base);
+ ret = -ENOMEM;
goto release_region;
}
@@ -374,7 +374,7 @@ static int qcom_cpufreq_hw_cpu_init(struct cpufreq_policy *policy)
error:
kfree(data);
unmap_base:
- iounmap(data->base);
+ iounmap(base);
release_region:
release_mem_region(res->start, resource_size(res));
return ret;
diff --git a/drivers/cpufreq/s5pv210-cpufreq.c b/drivers/cpufreq/s5pv210-cpufreq.c
index 69786e5bbf05..ad7d4f272ddc 100644
--- a/drivers/cpufreq/s5pv210-cpufreq.c
+++ b/drivers/cpufreq/s5pv210-cpufreq.c
@@ -91,7 +91,7 @@ static DEFINE_MUTEX(set_freq_lock);
/* Use 800MHz when entering sleep mode */
#define SLEEP_FREQ (800 * 1000)
-/* Tracks if cpu freqency can be updated anymore */
+/* Tracks if CPU frequency can be updated anymore */
static bool no_cpufreq_access;
/*
@@ -190,7 +190,7 @@ static u32 clkdiv_val[5][11] = {
/*
* This function set DRAM refresh counter
- * accoriding to operating frequency of DRAM
+ * according to operating frequency of DRAM
* ch: DMC port number 0 or 1
* freq: Operating frequency of DRAM(KHz)
*/
@@ -320,7 +320,7 @@ static int s5pv210_target(struct cpufreq_policy *policy, unsigned int index)
/*
* 3. DMC1 refresh count for 133Mhz if (index == L4) is
- * true refresh counter is already programed in upper
+ * true refresh counter is already programmed in upper
* code. 0x287@83Mhz
*/
if (!bus_speed_changing)
@@ -378,7 +378,7 @@ static int s5pv210_target(struct cpufreq_policy *policy, unsigned int index)
/*
* 6. Turn on APLL
* 6-1. Set PMS values
- * 6-2. Wait untile the PLL is locked
+ * 6-2. Wait until the PLL is locked
*/
if (index == L0)
writel_relaxed(APLL_VAL_1000, S5P_APLL_CON);
@@ -390,7 +390,7 @@ static int s5pv210_target(struct cpufreq_policy *policy, unsigned int index)
} while (!(reg & (0x1 << 29)));
/*
- * 7. Change souce clock from SCLKMPLL(667Mhz)
+ * 7. Change source clock from SCLKMPLL(667Mhz)
* to SCLKA2M(200Mhz) in MFC_MUX and G3D MUX
* (667/4=166)->(200/4=50)Mhz
*/
@@ -439,8 +439,8 @@ static int s5pv210_target(struct cpufreq_policy *policy, unsigned int index)
}
/*
- * L4 level need to change memory bus speed, hence onedram clock divier
- * and memory refresh parameter should be changed
+ * L4 level needs to change memory bus speed, hence ONEDRAM clock
+ * divider and memory refresh parameter should be changed
*/
if (bus_speed_changing) {
reg = readl_relaxed(S5P_CLK_DIV6);
diff --git a/drivers/cpufreq/scmi-cpufreq.c b/drivers/cpufreq/scmi-cpufreq.c
index 5bd03b59887f..c8a4364ad3c2 100644
--- a/drivers/cpufreq/scmi-cpufreq.c
+++ b/drivers/cpufreq/scmi-cpufreq.c
@@ -2,7 +2,7 @@
/*
* System Control and Power Interface (SCMI) based CPUFreq Interface driver
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
* Sudeep Holla <sudeep.holla@arm.com>
*/
@@ -25,17 +25,17 @@ struct scmi_data {
struct device *cpu_dev;
};
-static const struct scmi_handle *handle;
+static struct scmi_protocol_handle *ph;
+static const struct scmi_perf_proto_ops *perf_ops;
static unsigned int scmi_cpufreq_get_rate(unsigned int cpu)
{
struct cpufreq_policy *policy = cpufreq_cpu_get_raw(cpu);
- const struct scmi_perf_ops *perf_ops = handle->perf_ops;
struct scmi_data *priv = policy->driver_data;
unsigned long rate;
int ret;
- ret = perf_ops->freq_get(handle, priv->domain_id, &rate, false);
+ ret = perf_ops->freq_get(ph, priv->domain_id, &rate, false);
if (ret)
return 0;
return rate / 1000;
@@ -50,19 +50,17 @@ static int
scmi_cpufreq_set_target(struct cpufreq_policy *policy, unsigned int index)
{
struct scmi_data *priv = policy->driver_data;
- const struct scmi_perf_ops *perf_ops = handle->perf_ops;
u64 freq = policy->freq_table[index].frequency;
- return perf_ops->freq_set(handle, priv->domain_id, freq * 1000, false);
+ return perf_ops->freq_set(ph, priv->domain_id, freq * 1000, false);
}
static unsigned int scmi_cpufreq_fast_switch(struct cpufreq_policy *policy,
unsigned int target_freq)
{
struct scmi_data *priv = policy->driver_data;
- const struct scmi_perf_ops *perf_ops = handle->perf_ops;
- if (!perf_ops->freq_set(handle, priv->domain_id,
+ if (!perf_ops->freq_set(ph, priv->domain_id,
target_freq * 1000, true))
return target_freq;
@@ -75,7 +73,7 @@ scmi_get_sharing_cpus(struct device *cpu_dev, struct cpumask *cpumask)
int cpu, domain, tdomain;
struct device *tcpu_dev;
- domain = handle->perf_ops->device_domain_id(cpu_dev);
+ domain = perf_ops->device_domain_id(cpu_dev);
if (domain < 0)
return domain;
@@ -87,7 +85,7 @@ scmi_get_sharing_cpus(struct device *cpu_dev, struct cpumask *cpumask)
if (!tcpu_dev)
continue;
- tdomain = handle->perf_ops->device_domain_id(tcpu_dev);
+ tdomain = perf_ops->device_domain_id(tcpu_dev);
if (tdomain == domain)
cpumask_set_cpu(cpu, cpumask);
}
@@ -102,13 +100,13 @@ scmi_get_cpu_power(unsigned long *power, unsigned long *KHz,
unsigned long Hz;
int ret, domain;
- domain = handle->perf_ops->device_domain_id(cpu_dev);
+ domain = perf_ops->device_domain_id(cpu_dev);
if (domain < 0)
return domain;
/* Get the power cost of the performance domain. */
Hz = *KHz * 1000;
- ret = handle->perf_ops->est_power_get(handle, domain, &Hz, power);
+ ret = perf_ops->est_power_get(ph, domain, &Hz, power);
if (ret)
return ret;
@@ -126,6 +124,7 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
struct scmi_data *priv;
struct cpufreq_frequency_table *freq_table;
struct em_data_callback em_cb = EM_DATA_CB(scmi_get_cpu_power);
+ cpumask_var_t opp_shared_cpus;
bool power_scale_mw;
cpu_dev = get_cpu_device(policy->cpu);
@@ -134,30 +133,64 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
return -ENODEV;
}
- ret = handle->perf_ops->device_opps_add(handle, cpu_dev);
- if (ret) {
- dev_warn(cpu_dev, "failed to add opps to the device\n");
- return ret;
- }
+ if (!zalloc_cpumask_var(&opp_shared_cpus, GFP_KERNEL))
+ ret = -ENOMEM;
+ /* Obtain CPUs that share SCMI performance controls */
ret = scmi_get_sharing_cpus(cpu_dev, policy->cpus);
if (ret) {
dev_warn(cpu_dev, "failed to get sharing cpumask\n");
- return ret;
+ goto out_free_cpumask;
}
- ret = dev_pm_opp_set_sharing_cpus(cpu_dev, policy->cpus);
- if (ret) {
- dev_err(cpu_dev, "%s: failed to mark OPPs as shared: %d\n",
- __func__, ret);
- return ret;
+ /*
+ * Obtain CPUs that share performance levels.
+ * The OPP 'sharing cpus' info may come from DT through an empty opp
+ * table and opp-shared.
+ */
+ ret = dev_pm_opp_of_get_sharing_cpus(cpu_dev, opp_shared_cpus);
+ if (ret || !cpumask_weight(opp_shared_cpus)) {
+ /*
+ * Either opp-table is not set or no opp-shared was found.
+ * Use the CPU mask from SCMI to designate CPUs sharing an OPP
+ * table.
+ */
+ cpumask_copy(opp_shared_cpus, policy->cpus);
}
+ /*
+ * A previous CPU may have marked OPPs as shared for a few CPUs, based on
+ * what OPP core provided. If the current CPU is part of those few, then
+ * there is no need to add OPPs again.
+ */
nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
if (nr_opp <= 0) {
- dev_dbg(cpu_dev, "OPP table is not ready, deferring probe\n");
- ret = -EPROBE_DEFER;
- goto out_free_opp;
+ ret = perf_ops->device_opps_add(ph, cpu_dev);
+ if (ret) {
+ dev_warn(cpu_dev, "failed to add opps to the device\n");
+ goto out_free_cpumask;
+ }
+
+ nr_opp = dev_pm_opp_get_opp_count(cpu_dev);
+ if (nr_opp <= 0) {
+ dev_err(cpu_dev, "%s: No OPPs for this device: %d\n",
+ __func__, ret);
+
+ ret = -ENODEV;
+ goto out_free_opp;
+ }
+
+ ret = dev_pm_opp_set_sharing_cpus(cpu_dev, opp_shared_cpus);
+ if (ret) {
+ dev_err(cpu_dev, "%s: failed to mark OPPs as shared: %d\n",
+ __func__, ret);
+
+ goto out_free_opp;
+ }
+
+ power_scale_mw = perf_ops->power_scale_mw_get(ph);
+ em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb,
+ opp_shared_cpus, power_scale_mw);
}
priv = kzalloc(sizeof(*priv), GFP_KERNEL);
@@ -173,7 +206,7 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
}
priv->cpu_dev = cpu_dev;
- priv->domain_id = handle->perf_ops->device_domain_id(cpu_dev);
+ priv->domain_id = perf_ops->device_domain_id(cpu_dev);
policy->driver_data = priv;
policy->freq_table = freq_table;
@@ -181,26 +214,27 @@ static int scmi_cpufreq_init(struct cpufreq_policy *policy)
/* SCMI allows DVFS request for any domain from any CPU */
policy->dvfs_possible_from_any_cpu = true;
- latency = handle->perf_ops->transition_latency_get(handle, cpu_dev);
+ latency = perf_ops->transition_latency_get(ph, cpu_dev);
if (!latency)
latency = CPUFREQ_ETERNAL;
policy->cpuinfo.transition_latency = latency;
policy->fast_switch_possible =
- handle->perf_ops->fast_switch_possible(handle, cpu_dev);
-
- power_scale_mw = handle->perf_ops->power_scale_mw_get(handle);
- em_dev_register_perf_domain(cpu_dev, nr_opp, &em_cb, policy->cpus,
- power_scale_mw);
+ perf_ops->fast_switch_possible(ph, cpu_dev);
+ free_cpumask_var(opp_shared_cpus);
return 0;
out_free_priv:
kfree(priv);
+
out_free_opp:
dev_pm_opp_remove_all_dynamic(cpu_dev);
+out_free_cpumask:
+ free_cpumask_var(opp_shared_cpus);
+
return ret;
}
@@ -233,12 +267,17 @@ static int scmi_cpufreq_probe(struct scmi_device *sdev)
{
int ret;
struct device *dev = &sdev->dev;
+ const struct scmi_handle *handle;
handle = sdev->handle;
- if (!handle || !handle->perf_ops)
+ if (!handle)
return -ENODEV;
+ perf_ops = handle->devm_protocol_get(sdev, SCMI_PROTOCOL_PERF, &ph);
+ if (IS_ERR(perf_ops))
+ return PTR_ERR(perf_ops);
+
#ifdef CONFIG_COMMON_CLK
/* dummy clock provider as needed by OPP if clocks property is used */
if (of_find_property(dev->of_node, "#clock-cells", NULL))
diff --git a/drivers/cpuidle/Kconfig.arm b/drivers/cpuidle/Kconfig.arm
index 0844fadc4be8..334f83e56120 100644
--- a/drivers/cpuidle/Kconfig.arm
+++ b/drivers/cpuidle/Kconfig.arm
@@ -107,7 +107,7 @@ config ARM_TEGRA_CPUIDLE
config ARM_QCOM_SPM_CPUIDLE
bool "CPU Idle Driver for Qualcomm Subsystem Power Manager (SPM)"
- depends on (ARCH_QCOM || COMPILE_TEST) && !ARM64
+ depends on (ARCH_QCOM || COMPILE_TEST) && !ARM64 && MMU
select ARM_CPU_SUSPEND
select CPU_IDLE_MULTIPLE_DRIVERS
select DT_IDLE_STATES
diff --git a/drivers/cpuidle/cpuidle-tegra.c b/drivers/cpuidle/cpuidle-tegra.c
index 191966dc8d02..508bd9f23792 100644
--- a/drivers/cpuidle/cpuidle-tegra.c
+++ b/drivers/cpuidle/cpuidle-tegra.c
@@ -48,11 +48,6 @@ enum tegra_state {
static atomic_t tegra_idle_barrier;
static atomic_t tegra_abort_flag;
-static inline bool tegra_cpuidle_using_firmware(void)
-{
- return firmware_ops->prepare_idle && firmware_ops->do_idle;
-}
-
static void tegra_cpuidle_report_cpus_state(void)
{
unsigned long cpu, lcpu, csr;
@@ -135,13 +130,9 @@ static int tegra_cpuidle_c7_enter(void)
{
int err;
- if (tegra_cpuidle_using_firmware()) {
- err = call_firmware_op(prepare_idle, TF_PM_MODE_LP2_NOFLUSH_L2);
- if (err)
- return err;
-
- return call_firmware_op(do_idle, 0);
- }
+ err = call_firmware_op(prepare_idle, TF_PM_MODE_LP2_NOFLUSH_L2);
+ if (err && err != -ENOSYS)
+ return err;
return cpu_suspend(0, tegra30_pm_secondary_cpu_suspend);
}
@@ -356,9 +347,7 @@ static int tegra_cpuidle_probe(struct platform_device *pdev)
* is disabled.
*/
if (!IS_ENABLED(CONFIG_PM_SLEEP)) {
- if (!tegra_cpuidle_using_firmware())
- tegra_cpuidle_disable_state(TEGRA_C7);
-
+ tegra_cpuidle_disable_state(TEGRA_C7);
tegra_cpuidle_disable_state(TEGRA_CC6);
}
diff --git a/drivers/cpuidle/driver.c b/drivers/cpuidle/driver.c
index 4070e573bf43..f70aa17e2a8e 100644
--- a/drivers/cpuidle/driver.c
+++ b/drivers/cpuidle/driver.c
@@ -181,9 +181,13 @@ static void __cpuidle_driver_init(struct cpuidle_driver *drv)
*/
if (s->target_residency > 0)
s->target_residency_ns = s->target_residency * NSEC_PER_USEC;
+ else if (s->target_residency_ns < 0)
+ s->target_residency_ns = 0;
if (s->exit_latency > 0)
s->exit_latency_ns = s->exit_latency * NSEC_PER_USEC;
+ else if (s->exit_latency_ns < 0)
+ s->exit_latency_ns = 0;
}
}
diff --git a/drivers/cpuidle/governors/menu.c b/drivers/cpuidle/governors/menu.c
index b0a7ad566081..c3aa8d6ccee3 100644
--- a/drivers/cpuidle/governors/menu.c
+++ b/drivers/cpuidle/governors/menu.c
@@ -271,7 +271,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
u64 predicted_ns;
u64 interactivity_req;
unsigned long nr_iowaiters;
- ktime_t delta_next;
+ ktime_t delta, delta_tick;
int i, idx;
if (data->needs_update) {
@@ -280,7 +280,12 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
}
/* determine the expected residency time, round up */
- data->next_timer_ns = tick_nohz_get_sleep_length(&delta_next);
+ delta = tick_nohz_get_sleep_length(&delta_tick);
+ if (unlikely(delta < 0)) {
+ delta = 0;
+ delta_tick = 0;
+ }
+ data->next_timer_ns = delta;
nr_iowaiters = nr_iowait_cpu(dev->cpu);
data->bucket = which_bucket(data->next_timer_ns, nr_iowaiters);
@@ -318,7 +323,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
* state selection.
*/
if (predicted_ns < TICK_NSEC)
- predicted_ns = delta_next;
+ predicted_ns = data->next_timer_ns;
} else {
/*
* Use the performance multiplier and the user-configurable
@@ -377,7 +382,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
* stuck in the shallow one for too long.
*/
if (drv->states[idx].target_residency_ns < TICK_NSEC &&
- s->target_residency_ns <= delta_next)
+ s->target_residency_ns <= delta_tick)
idx = i;
return idx;
@@ -399,7 +404,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
predicted_ns < TICK_NSEC) && !tick_nohz_tick_stopped()) {
*stop_tick = false;
- if (idx > 0 && drv->states[idx].target_residency_ns > delta_next) {
+ if (idx > 0 && drv->states[idx].target_residency_ns > delta_tick) {
/*
* The tick is not going to be stopped and the target
* residency of the state to be returned is not within
@@ -411,7 +416,7 @@ static int menu_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
continue;
idx = i;
- if (drv->states[i].target_residency_ns <= delta_next)
+ if (drv->states[i].target_residency_ns <= delta_tick)
break;
}
}
diff --git a/drivers/cpuidle/governors/teo.c b/drivers/cpuidle/governors/teo.c
index 6deaaf5f05b5..ac4bb27d69b0 100644
--- a/drivers/cpuidle/governors/teo.c
+++ b/drivers/cpuidle/governors/teo.c
@@ -100,8 +100,8 @@ struct teo_idle_state {
* @intervals: Saved idle duration values.
*/
struct teo_cpu {
- u64 time_span_ns;
- u64 sleep_length_ns;
+ s64 time_span_ns;
+ s64 sleep_length_ns;
struct teo_idle_state states[CPUIDLE_STATE_MAX];
int interval_idx;
u64 intervals[INTERVALS];
@@ -117,7 +117,8 @@ static DEFINE_PER_CPU(struct teo_cpu, teo_cpus);
static void teo_update(struct cpuidle_driver *drv, struct cpuidle_device *dev)
{
struct teo_cpu *cpu_data = per_cpu_ptr(&teo_cpus, dev->cpu);
- int i, idx_hit = -1, idx_timer = -1;
+ int i, idx_hit = 0, idx_timer = 0;
+ unsigned int hits, misses;
u64 measured_ns;
if (cpu_data->time_span_ns >= cpu_data->sleep_length_ns) {
@@ -174,25 +175,22 @@ static void teo_update(struct cpuidle_driver *drv, struct cpuidle_device *dev)
* also increase the "early hits" metric for the state that actually
* matches the measured idle duration.
*/
- if (idx_timer >= 0) {
- unsigned int hits = cpu_data->states[idx_timer].hits;
- unsigned int misses = cpu_data->states[idx_timer].misses;
-
- hits -= hits >> DECAY_SHIFT;
- misses -= misses >> DECAY_SHIFT;
-
- if (idx_timer > idx_hit) {
- misses += PULSE;
- if (idx_hit >= 0)
- cpu_data->states[idx_hit].early_hits += PULSE;
- } else {
- hits += PULSE;
- }
+ hits = cpu_data->states[idx_timer].hits;
+ hits -= hits >> DECAY_SHIFT;
+
+ misses = cpu_data->states[idx_timer].misses;
+ misses -= misses >> DECAY_SHIFT;
- cpu_data->states[idx_timer].misses = misses;
- cpu_data->states[idx_timer].hits = hits;
+ if (idx_timer == idx_hit) {
+ hits += PULSE;
+ } else {
+ misses += PULSE;
+ cpu_data->states[idx_hit].early_hits += PULSE;
}
+ cpu_data->states[idx_timer].misses = misses;
+ cpu_data->states[idx_timer].hits = hits;
+
/*
* Save idle duration values corresponding to non-timer wakeups for
* pattern detection.
@@ -216,7 +214,7 @@ static bool teo_time_ok(u64 interval_ns)
*/
static int teo_find_shallower_state(struct cpuidle_driver *drv,
struct cpuidle_device *dev, int state_idx,
- u64 duration_ns)
+ s64 duration_ns)
{
int i;
@@ -242,10 +240,10 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
{
struct teo_cpu *cpu_data = per_cpu_ptr(&teo_cpus, dev->cpu);
s64 latency_req = cpuidle_governor_latency_req(dev->cpu);
- u64 duration_ns;
+ int max_early_idx, prev_max_early_idx, constraint_idx, idx0, idx, i;
unsigned int hits, misses, early_hits;
- int max_early_idx, prev_max_early_idx, constraint_idx, idx, i;
ktime_t delta_tick;
+ s64 duration_ns;
if (dev->last_state_idx >= 0) {
teo_update(drv, dev);
@@ -264,6 +262,7 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
prev_max_early_idx = -1;
constraint_idx = drv->state_count;
idx = -1;
+ idx0 = idx;
for (i = 0; i < drv->state_count; i++) {
struct cpuidle_state *s = &drv->states[i];
@@ -324,6 +323,7 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
idx = i; /* first enabled state */
hits = cpu_data->states[i].hits;
misses = cpu_data->states[i].misses;
+ idx0 = i;
}
if (s->target_residency_ns > duration_ns)
@@ -376,11 +376,16 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
if (idx < 0) {
idx = 0; /* No states enabled. Must use 0. */
- } else if (idx > 0) {
+ } else if (idx > idx0) {
unsigned int count = 0;
u64 sum = 0;
/*
+ * The target residencies of at least two different enabled idle
+ * states are less than or equal to the current expected idle
+ * duration. Try to refine the selection using the most recent
+ * measured idle duration values.
+ *
* Count and sum the most recent idle duration values less than
* the current expected idle duration value.
*/
@@ -428,7 +433,8 @@ static int teo_select(struct cpuidle_driver *drv, struct cpuidle_device *dev,
* till the closest timer including the tick, try to correct
* that.
*/
- if (idx > 0 && drv->states[idx].target_residency_ns > delta_tick)
+ if (idx > idx0 &&
+ drv->states[idx].target_residency_ns > delta_tick)
idx = teo_find_shallower_state(drv, dev, idx, delta_tick);
}
diff --git a/drivers/crypto/allwinner/Kconfig b/drivers/crypto/allwinner/Kconfig
index 856fb2045656..b8e75210a0e3 100644
--- a/drivers/crypto/allwinner/Kconfig
+++ b/drivers/crypto/allwinner/Kconfig
@@ -71,10 +71,10 @@ config CRYPTO_DEV_SUN8I_CE_DEBUG
config CRYPTO_DEV_SUN8I_CE_HASH
bool "Enable support for hash on sun8i-ce"
depends on CRYPTO_DEV_SUN8I_CE
- select MD5
- select SHA1
- select SHA256
- select SHA512
+ select CRYPTO_MD5
+ select CRYPTO_SHA1
+ select CRYPTO_SHA256
+ select CRYPTO_SHA512
help
Say y to enable support for hash algorithms.
@@ -132,8 +132,8 @@ config CRYPTO_DEV_SUN8I_SS_PRNG
config CRYPTO_DEV_SUN8I_SS_HASH
bool "Enable support for hash on sun8i-ss"
depends on CRYPTO_DEV_SUN8I_SS
- select MD5
- select SHA1
- select SHA256
+ select CRYPTO_MD5
+ select CRYPTO_SHA1
+ select CRYPTO_SHA256
help
Say y to enable support for hash algorithms.
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c
index c2e6f5ed1d79..dec79fa3ebaf 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-cipher.c
@@ -561,7 +561,7 @@ int sun4i_ss_cipher_init(struct crypto_tfm *tfm)
sizeof(struct sun4i_cipher_req_ctx) +
crypto_skcipher_reqsize(op->fallback_tfm));
- err = pm_runtime_get_sync(op->ss->dev);
+ err = pm_runtime_resume_and_get(op->ss->dev);
if (err < 0)
goto error_pm;
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c
index 709905ec4680..44b8fc4b786d 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-core.c
@@ -288,8 +288,7 @@ static int sun4i_ss_pm_suspend(struct device *dev)
{
struct sun4i_ss_ctx *ss = dev_get_drvdata(dev);
- if (ss->reset)
- reset_control_assert(ss->reset);
+ reset_control_assert(ss->reset);
clk_disable_unprepare(ss->ssclk);
clk_disable_unprepare(ss->busclk);
@@ -314,12 +313,10 @@ static int sun4i_ss_pm_resume(struct device *dev)
goto err_enable;
}
- if (ss->reset) {
- err = reset_control_deassert(ss->reset);
- if (err) {
- dev_err(ss->dev, "Cannot deassert reset control\n");
- goto err_enable;
- }
+ err = reset_control_deassert(ss->reset);
+ if (err) {
+ dev_err(ss->dev, "Cannot deassert reset control\n");
+ goto err_enable;
}
return err;
@@ -401,12 +398,10 @@ static int sun4i_ss_probe(struct platform_device *pdev)
dev_dbg(&pdev->dev, "clock ahb_ss acquired\n");
ss->reset = devm_reset_control_get_optional(&pdev->dev, "ahb");
- if (IS_ERR(ss->reset)) {
- if (PTR_ERR(ss->reset) == -EPROBE_DEFER)
- return PTR_ERR(ss->reset);
+ if (IS_ERR(ss->reset))
+ return PTR_ERR(ss->reset);
+ if (!ss->reset)
dev_info(&pdev->dev, "no reset control found\n");
- ss->reset = NULL;
- }
/*
* Check that clock have the correct rates given in the datasheet
@@ -459,7 +454,7 @@ static int sun4i_ss_probe(struct platform_device *pdev)
* this info could be useful
*/
- err = pm_runtime_get_sync(ss->dev);
+ err = pm_runtime_resume_and_get(ss->dev);
if (err < 0)
goto error_pm;
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-hash.c b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-hash.c
index c1b4585e9bbc..d28292762b32 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-hash.c
@@ -27,7 +27,7 @@ int sun4i_hash_crainit(struct crypto_tfm *tfm)
algt = container_of(alg, struct sun4i_ss_alg_template, alg.hash);
op->ss = algt->ss;
- err = pm_runtime_get_sync(op->ss->dev);
+ err = pm_runtime_resume_and_get(op->ss->dev);
if (err < 0)
return err;
diff --git a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c
index 443160a114bb..491fcb7b81b4 100644
--- a/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c
+++ b/drivers/crypto/allwinner/sun4i-ss/sun4i-ss-prng.c
@@ -29,7 +29,7 @@ int sun4i_ss_prng_generate(struct crypto_rng *tfm, const u8 *src,
algt = container_of(alg, struct sun4i_ss_alg_template, alg.rng);
ss = algt->ss;
- err = pm_runtime_get_sync(ss->dev);
+ err = pm_runtime_resume_and_get(ss->dev);
if (err < 0)
return err;
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
index 33707a2e55ff..54ae8d16e493 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-cipher.c
@@ -240,11 +240,14 @@ static int sun8i_ce_cipher_prepare(struct crypto_engine *engine, void *async_req
theend_sgs:
if (areq->src == areq->dst) {
- dma_unmap_sg(ce->dev, areq->src, nr_sgs, DMA_BIDIRECTIONAL);
+ dma_unmap_sg(ce->dev, areq->src, sg_nents(areq->src),
+ DMA_BIDIRECTIONAL);
} else {
if (nr_sgs > 0)
- dma_unmap_sg(ce->dev, areq->src, nr_sgs, DMA_TO_DEVICE);
- dma_unmap_sg(ce->dev, areq->dst, nr_sgd, DMA_FROM_DEVICE);
+ dma_unmap_sg(ce->dev, areq->src, sg_nents(areq->src),
+ DMA_TO_DEVICE);
+ dma_unmap_sg(ce->dev, areq->dst, sg_nents(areq->dst),
+ DMA_FROM_DEVICE);
}
theend_iv:
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
index 158422ff5695..00194d1d9ae6 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-core.c
@@ -932,7 +932,7 @@ static int sun8i_ce_probe(struct platform_device *pdev)
if (err)
goto error_alg;
- err = pm_runtime_get_sync(ce->dev);
+ err = pm_runtime_resume_and_get(ce->dev);
if (err < 0)
goto error_alg;
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
index 2f09a37306e2..88194718a806 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c
@@ -405,7 +405,8 @@ int sun8i_ce_hash_run(struct crypto_engine *engine, void *breq)
err = sun8i_ce_run_task(ce, flow, crypto_tfm_alg_name(areq->base.tfm));
dma_unmap_single(ce->dev, addr_pad, j * 4, DMA_TO_DEVICE);
- dma_unmap_sg(ce->dev, areq->src, nr_sgs, DMA_TO_DEVICE);
+ dma_unmap_sg(ce->dev, areq->src, sg_nents(areq->src),
+ DMA_TO_DEVICE);
dma_unmap_single(ce->dev, addr_res, digestsize, DMA_FROM_DEVICE);
diff --git a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c
index cfde9ee4356b..cd1baee424a1 100644
--- a/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c
+++ b/drivers/crypto/allwinner/sun8i-ce/sun8i-ce-prng.c
@@ -99,6 +99,7 @@ int sun8i_ce_prng_generate(struct crypto_rng *tfm, const u8 *src,
dma_iv = dma_map_single(ce->dev, ctx->seed, ctx->slen, DMA_TO_DEVICE);
if (dma_mapping_error(ce->dev, dma_iv)) {
dev_err(ce->dev, "Cannot DMA MAP IV\n");
+ err = -EFAULT;
goto err_iv;
}
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
index ed2a69f82e1c..9ef1c85c4aaa 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-cipher.c
@@ -232,10 +232,13 @@ sgd_next:
theend_sgs:
if (areq->src == areq->dst) {
- dma_unmap_sg(ss->dev, areq->src, nr_sgs, DMA_BIDIRECTIONAL);
+ dma_unmap_sg(ss->dev, areq->src, sg_nents(areq->src),
+ DMA_BIDIRECTIONAL);
} else {
- dma_unmap_sg(ss->dev, areq->src, nr_sgs, DMA_TO_DEVICE);
- dma_unmap_sg(ss->dev, areq->dst, nr_sgd, DMA_FROM_DEVICE);
+ dma_unmap_sg(ss->dev, areq->src, sg_nents(areq->src),
+ DMA_TO_DEVICE);
+ dma_unmap_sg(ss->dev, areq->dst, sg_nents(areq->dst),
+ DMA_FROM_DEVICE);
}
theend_iv:
@@ -351,7 +354,7 @@ int sun8i_ss_cipher_init(struct crypto_tfm *tfm)
op->enginectx.op.prepare_request = NULL;
op->enginectx.op.unprepare_request = NULL;
- err = pm_runtime_get_sync(op->ss->dev);
+ err = pm_runtime_resume_and_get(op->ss->dev);
if (err < 0) {
dev_err(op->ss->dev, "pm error %d\n", err);
goto error_pm;
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
index e0ddc684798d..80e89066dbd1 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-core.c
@@ -753,7 +753,7 @@ static int sun8i_ss_probe(struct platform_device *pdev)
if (err)
goto error_alg;
- err = pm_runtime_get_sync(ss->dev);
+ err = pm_runtime_resume_and_get(ss->dev);
if (err < 0)
goto error_alg;
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
index 11cbcbc83a7b..3c073eb3db03 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-hash.c
@@ -348,8 +348,10 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
bf = (__le32 *)pad;
result = kzalloc(digestsize, GFP_KERNEL | GFP_DMA);
- if (!result)
+ if (!result) {
+ kfree(pad);
return -ENOMEM;
+ }
for (i = 0; i < MAX_SG; i++) {
rctx->t_dst[i].addr = 0;
@@ -432,14 +434,14 @@ int sun8i_ss_hash_run(struct crypto_engine *engine, void *breq)
err = sun8i_ss_run_hash_task(ss, rctx, crypto_tfm_alg_name(areq->base.tfm));
dma_unmap_single(ss->dev, addr_pad, j * 4, DMA_TO_DEVICE);
- dma_unmap_sg(ss->dev, areq->src, nr_sgs, DMA_TO_DEVICE);
+ dma_unmap_sg(ss->dev, areq->src, sg_nents(areq->src),
+ DMA_TO_DEVICE);
dma_unmap_single(ss->dev, addr_res, digestsize, DMA_FROM_DEVICE);
- kfree(pad);
-
memcpy(areq->result, result, algt->alg.hash.halg.digestsize);
- kfree(result);
theend:
+ kfree(pad);
+ kfree(result);
crypto_finalize_hash_request(engine, breq, err);
return 0;
}
diff --git a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-prng.c b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-prng.c
index 08a1473b2145..3191527928e4 100644
--- a/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-prng.c
+++ b/drivers/crypto/allwinner/sun8i-ss/sun8i-ss-prng.c
@@ -103,7 +103,8 @@ int sun8i_ss_prng_generate(struct crypto_rng *tfm, const u8 *src,
dma_iv = dma_map_single(ss->dev, ctx->seed, ctx->slen, DMA_TO_DEVICE);
if (dma_mapping_error(ss->dev, dma_iv)) {
dev_err(ss->dev, "Cannot DMA MAP IV\n");
- return -EFAULT;
+ err = -EFAULT;
+ goto err_free;
}
dma_dst = dma_map_single(ss->dev, d, todo, DMA_FROM_DEVICE);
@@ -167,6 +168,7 @@ err_iv:
memcpy(ctx->seed, d + dlen, ctx->slen);
}
memzero_explicit(d, todo);
+err_free:
kfree(d);
return err;
diff --git a/drivers/crypto/amcc/crypto4xx_alg.c b/drivers/crypto/amcc/crypto4xx_alg.c
index a3fa849b139a..ded732242732 100644
--- a/drivers/crypto/amcc/crypto4xx_alg.c
+++ b/drivers/crypto/amcc/crypto4xx_alg.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/**
+/*
* AMCC SoC PPC4xx Crypto Driver
*
* Copyright (c) 2008 Applied Micro Circuits Corporation.
@@ -115,7 +115,7 @@ int crypto4xx_decrypt_iv_block(struct skcipher_request *req)
return crypto4xx_crypt(req, AES_IV_SIZE, true, true);
}
-/**
+/*
* AES Functions
*/
static int crypto4xx_setkey_aes(struct crypto_skcipher *cipher,
@@ -374,7 +374,7 @@ static int crypto4xx_aead_setup_fallback(struct crypto4xx_ctx *ctx,
return crypto_aead_setkey(ctx->sw_cipher.aead, key, keylen);
}
-/**
+/*
* AES-CCM Functions
*/
@@ -489,7 +489,7 @@ int crypto4xx_setauthsize_aead(struct crypto_aead *cipher,
return crypto_aead_setauthsize(ctx->sw_cipher.aead, authsize);
}
-/**
+/*
* AES-GCM Functions
*/
@@ -617,7 +617,7 @@ int crypto4xx_decrypt_aes_gcm(struct aead_request *req)
return crypto4xx_crypt_aes_gcm(req, true);
}
-/**
+/*
* HASH SHA1 Functions
*/
static int crypto4xx_hash_alg_init(struct crypto_tfm *tfm,
@@ -711,7 +711,7 @@ int crypto4xx_hash_digest(struct ahash_request *req)
ctx->sa_len, 0, NULL);
}
-/**
+/*
* SHA1 Algorithm
*/
int crypto4xx_sha1_alg_init(struct crypto_tfm *tfm)
diff --git a/drivers/crypto/amcc/crypto4xx_core.c b/drivers/crypto/amcc/crypto4xx_core.c
index 8d1b918a0533..8278d98074e9 100644
--- a/drivers/crypto/amcc/crypto4xx_core.c
+++ b/drivers/crypto/amcc/crypto4xx_core.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-or-later
-/**
+/*
* AMCC SoC PPC4xx Crypto Driver
*
* Copyright (c) 2008 Applied Micro Circuits Corporation.
@@ -44,7 +44,7 @@
#define PPC4XX_SEC_VERSION_STR "0.5"
-/**
+/*
* PPC4xx Crypto Engine Initialization Routine
*/
static void crypto4xx_hw_init(struct crypto4xx_device *dev)
@@ -159,7 +159,7 @@ void crypto4xx_free_sa(struct crypto4xx_ctx *ctx)
ctx->sa_len = 0;
}
-/**
+/*
* alloc memory for the gather ring
* no need to alloc buf for the ring
* gdr_tail, gdr_head and gdr_count are initialized by this function
@@ -268,7 +268,7 @@ static u32 crypto4xx_put_pd_to_pdr(struct crypto4xx_device *dev, u32 idx)
return tail;
}
-/**
+/*
* alloc memory for the gather ring
* no need to alloc buf for the ring
* gdr_tail, gdr_head and gdr_count are initialized by this function
@@ -346,7 +346,7 @@ static inline struct ce_gd *crypto4xx_get_gdp(struct crypto4xx_device *dev,
return &dev->gdr[idx];
}
-/**
+/*
* alloc memory for the scatter ring
* need to alloc buf for the ring
* sdr_tail, sdr_head and sdr_count are initialized by this function
@@ -930,7 +930,7 @@ int crypto4xx_build_pd(struct crypto_async_request *req,
return is_busy ? -EBUSY : -EINPROGRESS;
}
-/**
+/*
* Algorithm Registration Functions
*/
static void crypto4xx_ctx_init(struct crypto4xx_alg *amcc_alg,
@@ -1097,7 +1097,7 @@ static void crypto4xx_bh_tasklet_cb(unsigned long data)
} while (head != tail);
}
-/**
+/*
* Top Half of isr.
*/
static inline irqreturn_t crypto4xx_interrupt_handler(int irq, void *data,
@@ -1186,7 +1186,7 @@ static int crypto4xx_prng_seed(struct crypto_rng *tfm, const u8 *seed,
return 0;
}
-/**
+/*
* Supported Crypto Algorithms
*/
static struct crypto4xx_alg_common crypto4xx_alg[] = {
@@ -1369,7 +1369,7 @@ static struct crypto4xx_alg_common crypto4xx_alg[] = {
} },
};
-/**
+/*
* Module Initialization Routine
*/
static int crypto4xx_probe(struct platform_device *ofdev)
diff --git a/drivers/crypto/amcc/crypto4xx_core.h b/drivers/crypto/amcc/crypto4xx_core.h
index a4e25b46cd0a..56c10668c0ab 100644
--- a/drivers/crypto/amcc/crypto4xx_core.h
+++ b/drivers/crypto/amcc/crypto4xx_core.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
-/**
+/*
* AMCC SoC PPC4xx Crypto Driver
*
* Copyright (c) 2008 Applied Micro Circuits Corporation.
@@ -188,7 +188,7 @@ int crypto4xx_hash_final(struct ahash_request *req);
int crypto4xx_hash_update(struct ahash_request *req);
int crypto4xx_hash_init(struct ahash_request *req);
-/**
+/*
* Note: Only use this function to copy items that is word aligned.
*/
static inline void crypto4xx_memcpy_swab32(u32 *dst, const void *buf,
diff --git a/drivers/crypto/amcc/crypto4xx_reg_def.h b/drivers/crypto/amcc/crypto4xx_reg_def.h
index c4c0a1a75941..1038061224da 100644
--- a/drivers/crypto/amcc/crypto4xx_reg_def.h
+++ b/drivers/crypto/amcc/crypto4xx_reg_def.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
-/**
+/*
* AMCC SoC PPC4xx Crypto Driver
*
* Copyright (c) 2008 Applied Micro Circuits Corporation.
@@ -104,7 +104,7 @@
#define CRYPTO4XX_PRNG_LFSR_L 0x00070030
#define CRYPTO4XX_PRNG_LFSR_H 0x00070034
-/**
+/*
* Initialize CRYPTO ENGINE registers, and memory bases.
*/
#define PPC4XX_PDR_POLL 0x3ff
@@ -123,7 +123,7 @@
#define PPC4XX_INT_TIMEOUT_CNT 0
#define PPC4XX_INT_TIMEOUT_CNT_REVB 0x3FF
#define PPC4XX_INT_CFG 1
-/**
+/*
* all follow define are ad hoc
*/
#define PPC4XX_RING_RETRY 100
@@ -131,7 +131,7 @@
#define PPC4XX_SDR_SIZE PPC4XX_NUM_SD
#define PPC4XX_GDR_SIZE PPC4XX_NUM_GD
-/**
+/*
* Generic Security Association (SA) with all possible fields. These will
* never likely used except for reference purpose. These structure format
* can be not changed as the hardware expects them to be layout as defined.
diff --git a/drivers/crypto/amcc/crypto4xx_sa.h b/drivers/crypto/amcc/crypto4xx_sa.h
index fe756abfc19f..e98e4e7abbad 100644
--- a/drivers/crypto/amcc/crypto4xx_sa.h
+++ b/drivers/crypto/amcc/crypto4xx_sa.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
-/**
+/*
* AMCC SoC PPC4xx Crypto Driver
*
* Copyright (c) 2008 Applied Micro Circuits Corporation.
@@ -14,7 +14,7 @@
#define AES_IV_SIZE 16
-/**
+/*
* Contents of Dynamic Security Association (SA) with all possible fields
*/
union dynamic_sa_contents {
@@ -122,7 +122,7 @@ union sa_command_0 {
#define SA_AES_KEY_LEN_256 4
#define SA_REV2 1
-/**
+/*
* The follow defines bits sa_command_1
* In Basic hash mode this bit define simple hash or hmac.
* In IPsec mode, this bit define muting control.
@@ -172,7 +172,7 @@ struct dynamic_sa_ctl {
union sa_command_1 sa_command_1;
} __attribute__((packed));
-/**
+/*
* State Record for Security Association (SA)
*/
struct sa_state_record {
@@ -184,7 +184,7 @@ struct sa_state_record {
};
} __attribute__((packed));
-/**
+/*
* Security Association (SA) for AES128
*
*/
@@ -213,7 +213,7 @@ struct dynamic_sa_aes192 {
#define SA_AES192_LEN (sizeof(struct dynamic_sa_aes192)/4)
#define SA_AES192_CONTENTS 0x3e000062
-/**
+/*
* Security Association (SA) for AES256
*/
struct dynamic_sa_aes256 {
@@ -228,7 +228,7 @@ struct dynamic_sa_aes256 {
#define SA_AES256_CONTENTS 0x3e000082
#define SA_AES_CONTENTS 0x3e000002
-/**
+/*
* Security Association (SA) for AES128 CCM
*/
struct dynamic_sa_aes128_ccm {
@@ -242,7 +242,7 @@ struct dynamic_sa_aes128_ccm {
#define SA_AES128_CCM_CONTENTS 0x3e000042
#define SA_AES_CCM_CONTENTS 0x3e000002
-/**
+/*
* Security Association (SA) for AES128_GCM
*/
struct dynamic_sa_aes128_gcm {
@@ -258,7 +258,7 @@ struct dynamic_sa_aes128_gcm {
#define SA_AES128_GCM_CONTENTS 0x3e000442
#define SA_AES_GCM_CONTENTS 0x3e000402
-/**
+/*
* Security Association (SA) for HASH160: HMAC-SHA1
*/
struct dynamic_sa_hash160 {
diff --git a/drivers/crypto/amcc/crypto4xx_trng.h b/drivers/crypto/amcc/crypto4xx_trng.h
index 3af732f25c1c..7356716274cb 100644
--- a/drivers/crypto/amcc/crypto4xx_trng.h
+++ b/drivers/crypto/amcc/crypto4xx_trng.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
-/**
+/*
* AMCC SoC PPC4xx Crypto Driver
*
* Copyright (c) 2008 Applied Micro Circuits Corporation.
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 8b5e07316352..c6865cbd334b 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -236,10 +236,10 @@ static int meson_cipher(struct skcipher_request *areq)
dma_unmap_single(mc->dev, phykeyiv, keyivlen, DMA_TO_DEVICE);
if (areq->src == areq->dst) {
- dma_unmap_sg(mc->dev, areq->src, nr_sgs, DMA_BIDIRECTIONAL);
+ dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_BIDIRECTIONAL);
} else {
- dma_unmap_sg(mc->dev, areq->src, nr_sgs, DMA_TO_DEVICE);
- dma_unmap_sg(mc->dev, areq->dst, nr_sgd, DMA_FROM_DEVICE);
+ dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
+ dma_unmap_sg(mc->dev, areq->dst, sg_nents(areq->dst), DMA_FROM_DEVICE);
}
if (areq->iv && ivsize > 0) {
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 5bbeff433c8c..6e7ae896717c 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -217,9 +217,6 @@ static int meson_crypto_probe(struct platform_device *pdev)
struct meson_dev *mc;
int err, i;
- if (!pdev->dev.of_node)
- return -ENODEV;
-
mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL);
if (!mc)
return -ENOMEM;
diff --git a/drivers/crypto/atmel-ecc.c b/drivers/crypto/atmel-ecc.c
index 9bd8e5167be3..333fbefbbccb 100644
--- a/drivers/crypto/atmel-ecc.c
+++ b/drivers/crypto/atmel-ecc.c
@@ -26,7 +26,7 @@
static struct atmel_ecc_driver_data driver_data;
/**
- * atmel_ecdh_ctx - transformation context
+ * struct atmel_ecdh_ctx - transformation context
* @client : pointer to i2c client device
* @fallback : used for unsupported curves or when user wants to use its own
* private key.
@@ -34,7 +34,6 @@ static struct atmel_ecc_driver_data driver_data;
* of the user to not call set_secret() while
* generate_public_key() or compute_shared_secret() are in flight.
* @curve_id : elliptic curve id
- * @n_sz : size in bytes of the n prime
* @do_fallback: true when the device doesn't support the curve or when the user
* wants to use its own private key.
*/
@@ -43,7 +42,6 @@ struct atmel_ecdh_ctx {
struct crypto_kpp *fallback;
const u8 *public_key;
unsigned int curve_id;
- size_t n_sz;
bool do_fallback;
};
@@ -51,7 +49,6 @@ static void atmel_ecdh_done(struct atmel_i2c_work_data *work_data, void *areq,
int status)
{
struct kpp_request *req = areq;
- struct atmel_ecdh_ctx *ctx = work_data->ctx;
struct atmel_i2c_cmd *cmd = &work_data->cmd;
size_t copied, n_sz;
@@ -59,7 +56,7 @@ static void atmel_ecdh_done(struct atmel_i2c_work_data *work_data, void *areq,
goto free_work_data;
/* might want less than we've got */
- n_sz = min_t(size_t, ctx->n_sz, req->dst_len);
+ n_sz = min_t(size_t, ATMEL_ECC_NIST_P256_N_SIZE, req->dst_len);
/* copy the shared secret */
copied = sg_copy_from_buffer(req->dst, sg_nents_for_len(req->dst, n_sz),
@@ -73,14 +70,6 @@ free_work_data:
kpp_request_complete(req, status);
}
-static unsigned int atmel_ecdh_supported_curve(unsigned int curve_id)
-{
- if (curve_id == ECC_CURVE_NIST_P256)
- return ATMEL_ECC_NIST_P256_N_SIZE;
-
- return 0;
-}
-
/*
* A random private key is generated and stored in the device. The device
* returns the pair public key.
@@ -104,8 +93,7 @@ static int atmel_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
return -EINVAL;
}
- ctx->n_sz = atmel_ecdh_supported_curve(params.curve_id);
- if (!ctx->n_sz || params.key_size) {
+ if (params.key_size) {
/* fallback to ecdh software implementation */
ctx->do_fallback = true;
return crypto_kpp_set_secret(ctx->fallback, buf, len);
@@ -125,7 +113,6 @@ static int atmel_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
goto free_cmd;
ctx->do_fallback = false;
- ctx->curve_id = params.curve_id;
atmel_i2c_init_genkey_cmd(cmd, DATA_SLOT_2);
@@ -263,6 +250,7 @@ static int atmel_ecdh_init_tfm(struct crypto_kpp *tfm)
struct crypto_kpp *fallback;
struct atmel_ecdh_ctx *ctx = kpp_tfm_ctx(tfm);
+ ctx->curve_id = ECC_CURVE_NIST_P256;
ctx->client = atmel_ecc_i2c_client_alloc();
if (IS_ERR(ctx->client)) {
pr_err("tfm - i2c_client binding failed\n");
@@ -306,7 +294,7 @@ static unsigned int atmel_ecdh_max_size(struct crypto_kpp *tfm)
return ATMEL_ECC_PUBKEY_SIZE;
}
-static struct kpp_alg atmel_ecdh = {
+static struct kpp_alg atmel_ecdh_nist_p256 = {
.set_secret = atmel_ecdh_set_secret,
.generate_public_key = atmel_ecdh_generate_public_key,
.compute_shared_secret = atmel_ecdh_compute_shared_secret,
@@ -315,7 +303,7 @@ static struct kpp_alg atmel_ecdh = {
.max_size = atmel_ecdh_max_size,
.base = {
.cra_flags = CRYPTO_ALG_NEED_FALLBACK,
- .cra_name = "ecdh",
+ .cra_name = "ecdh-nist-p256",
.cra_driver_name = "atmel-ecdh",
.cra_priority = ATMEL_ECC_PRIORITY,
.cra_module = THIS_MODULE,
@@ -340,14 +328,14 @@ static int atmel_ecc_probe(struct i2c_client *client,
&driver_data.i2c_client_list);
spin_unlock(&driver_data.i2c_list_lock);
- ret = crypto_register_kpp(&atmel_ecdh);
+ ret = crypto_register_kpp(&atmel_ecdh_nist_p256);
if (ret) {
spin_lock(&driver_data.i2c_list_lock);
list_del(&i2c_priv->i2c_client_list_node);
spin_unlock(&driver_data.i2c_list_lock);
dev_err(&client->dev, "%s alg registration failed\n",
- atmel_ecdh.base.cra_driver_name);
+ atmel_ecdh_nist_p256.base.cra_driver_name);
} else {
dev_info(&client->dev, "atmel ecc algorithms registered in /proc/crypto\n");
}
@@ -365,7 +353,7 @@ static int atmel_ecc_remove(struct i2c_client *client)
return -EBUSY;
}
- crypto_unregister_kpp(&atmel_ecdh);
+ crypto_unregister_kpp(&atmel_ecdh_nist_p256);
spin_lock(&driver_data.i2c_list_lock);
list_del(&i2c_priv->i2c_client_list_node);
diff --git a/drivers/crypto/atmel-i2c.c b/drivers/crypto/atmel-i2c.c
index e8e8281e027d..6fd3e969211d 100644
--- a/drivers/crypto/atmel-i2c.c
+++ b/drivers/crypto/atmel-i2c.c
@@ -339,7 +339,7 @@ int atmel_i2c_probe(struct i2c_client *client, const struct i2c_device_id *id)
}
if (bus_clk_rate > 1000000L) {
- dev_err(dev, "%d exceeds maximum supported clock frequency (1MHz)\n",
+ dev_err(dev, "%u exceeds maximum supported clock frequency (1MHz)\n",
bus_clk_rate);
return -EINVAL;
}
diff --git a/drivers/crypto/atmel-sha.c b/drivers/crypto/atmel-sha.c
index 352d80cb5ae9..1b13f601fd95 100644
--- a/drivers/crypto/atmel-sha.c
+++ b/drivers/crypto/atmel-sha.c
@@ -434,7 +434,7 @@ static int atmel_sha_init(struct ahash_request *req)
ctx->flags = 0;
- dev_dbg(dd->dev, "init: digest size: %d\n",
+ dev_dbg(dd->dev, "init: digest size: %u\n",
crypto_ahash_digestsize(tfm));
switch (crypto_ahash_digestsize(tfm)) {
@@ -1102,7 +1102,7 @@ static int atmel_sha_start(struct atmel_sha_dev *dd)
struct atmel_sha_reqctx *ctx = ahash_request_ctx(req);
int err;
- dev_dbg(dd->dev, "handling new req, op: %lu, nbytes: %d\n",
+ dev_dbg(dd->dev, "handling new req, op: %lu, nbytes: %u\n",
ctx->op, req->nbytes);
err = atmel_sha_hw_init(dd);
diff --git a/drivers/crypto/atmel-tdes.c b/drivers/crypto/atmel-tdes.c
index 4d63cb13a54f..6f01c51e3c37 100644
--- a/drivers/crypto/atmel-tdes.c
+++ b/drivers/crypto/atmel-tdes.c
@@ -1217,7 +1217,6 @@ static int atmel_tdes_probe(struct platform_device *pdev)
tdes_dd->io_base = devm_ioremap_resource(&pdev->dev, tdes_res);
if (IS_ERR(tdes_dd->io_base)) {
- dev_err(dev, "can't ioremap\n");
err = PTR_ERR(tdes_dd->io_base);
goto err_tasklet_kill;
}
diff --git a/drivers/crypto/bcm/cipher.c b/drivers/crypto/bcm/cipher.c
index 851b149f7170..053315e260c2 100644
--- a/drivers/crypto/bcm/cipher.c
+++ b/drivers/crypto/bcm/cipher.c
@@ -1019,6 +1019,7 @@ static void handle_ahash_resp(struct iproc_reqctx_s *rctx)
* a SPU response message for an AEAD request. Includes buffers to catch SPU
* message headers and the response data.
* @mssg: mailbox message containing the receive sg
+ * @req: Crypto API request
* @rctx: crypto request context
* @rx_frag_num: number of scatterlist elements required to hold the
* SPU response message
@@ -2952,9 +2953,9 @@ static int aead_gcm_esp_setkey(struct crypto_aead *cipher,
/**
* rfc4543_gcm_esp_setkey() - setkey operation for RFC4543 variant of GCM/GMAC.
- * cipher: AEAD structure
- * key: Key followed by 4 bytes of salt
- * keylen: Length of key plus salt, in bytes
+ * @cipher: AEAD structure
+ * @key: Key followed by 4 bytes of salt
+ * @keylen: Length of key plus salt, in bytes
*
* Extracts salt from key and stores it to be prepended to IV on each request.
* Digest is always 16 bytes
diff --git a/drivers/crypto/bcm/spu.c b/drivers/crypto/bcm/spu.c
index 007abf92cc05..6283e8c6d51d 100644
--- a/drivers/crypto/bcm/spu.c
+++ b/drivers/crypto/bcm/spu.c
@@ -457,7 +457,7 @@ u16 spum_hash_pad_len(enum hash_alg hash_alg, enum hash_mode hash_mode,
* @cipher_mode: Algo type
* @data_size: Length of plaintext (bytes)
*
- * @Return: Length of padding, in bytes
+ * Return: Length of padding, in bytes
*/
u32 spum_gcm_ccm_pad_len(enum spu_cipher_mode cipher_mode,
unsigned int data_size)
@@ -510,10 +510,10 @@ u32 spum_assoc_resp_len(enum spu_cipher_mode cipher_mode,
}
/**
- * spu_aead_ivlen() - Calculate the length of the AEAD IV to be included
+ * spum_aead_ivlen() - Calculate the length of the AEAD IV to be included
* in a SPU request after the AAD and before the payload.
* @cipher_mode: cipher mode
- * @iv_ctr_len: initialization vector length in bytes
+ * @iv_len: initialization vector length in bytes
*
* In Linux ~4.2 and later, the assoc_data sg includes the IV. So no need
* to include the IV as a separate field in the SPU request msg.
@@ -543,9 +543,9 @@ enum hash_type spum_hash_type(u32 src_sent)
/**
* spum_digest_size() - Determine the size of a hash digest to expect the SPU to
* return.
- * alg_digest_size: Number of bytes in the final digest for the given algo
- * alg: The hash algorithm
- * htype: Type of hash operation (init, update, full, etc)
+ * @alg_digest_size: Number of bytes in the final digest for the given algo
+ * @alg: The hash algorithm
+ * @htype: Type of hash operation (init, update, full, etc)
*
* When doing incremental hashing for an algorithm with a truncated hash
* (e.g., SHA224), the SPU returns the full digest so that it can be fed back as
@@ -580,7 +580,7 @@ u32 spum_digest_size(u32 alg_digest_size, enum hash_alg alg,
* @aead_parms: Parameters related to AEAD operation
* @data_size: Length of data to be encrypted or authenticated. If AEAD, does
* not include length of AAD.
-
+ *
* Return: the length of the SPU header in bytes. 0 if an error occurs.
*/
u32 spum_create_request(u8 *spu_hdr,
@@ -911,7 +911,7 @@ u16 spum_cipher_req_init(u8 *spu_hdr, struct spu_cipher_parms *cipher_parms)
* setkey() time in spu_cipher_req_init().
* @spu_hdr: Start of the request message header (MH field)
* @spu_req_hdr_len: Length in bytes of the SPU request header
- * @isInbound: 0 encrypt, 1 decrypt
+ * @is_inbound: 0 encrypt, 1 decrypt
* @cipher_parms: Parameters describing cipher operation to be performed
* @data_size: Length of the data in the BD field
*
diff --git a/drivers/crypto/bcm/spu2.c b/drivers/crypto/bcm/spu2.c
index 2db35b5ccaa2..07989bb8c220 100644
--- a/drivers/crypto/bcm/spu2.c
+++ b/drivers/crypto/bcm/spu2.c
@@ -543,7 +543,8 @@ void spu2_dump_msg_hdr(u8 *buf, unsigned int buf_len)
/**
* spu2_fmd_init() - At setkey time, initialize the fixed meta data for
* subsequent skcipher requests for this context.
- * @spu2_cipher_type: Cipher algorithm
+ * @fmd: Start of FMD field to be written
+ * @spu2_type: Cipher algorithm
* @spu2_mode: Cipher mode
* @cipher_key_len: Length of cipher key, in bytes
* @cipher_iv_len: Length of cipher initialization vector, in bytes
@@ -598,7 +599,7 @@ static int spu2_fmd_init(struct SPU2_FMD *fmd,
* SPU request packet.
* @fmd: Start of FMD field to be written
* @is_inbound: true if decrypting. false if encrypting.
- * @authFirst: true if alg authenticates before encrypting
+ * @auth_first: true if alg authenticates before encrypting
* @protocol: protocol selector
* @cipher_type: cipher algorithm
* @cipher_mode: cipher mode
@@ -640,6 +641,7 @@ static void spu2_fmd_ctrl0_write(struct SPU2_FMD *fmd,
* spu2_fmd_ctrl1_write() - Write ctrl1 field in fixed metadata (FMD) field of
* SPU request packet.
* @fmd: Start of FMD field to be written
+ * @is_inbound: true if decrypting. false if encrypting.
* @assoc_size: Length of additional associated data, in bytes
* @auth_key_len: Length of authentication key, in bytes
* @cipher_key_len: Length of cipher key, in bytes
@@ -793,7 +795,7 @@ u32 spu2_ctx_max_payload(enum spu_cipher_alg cipher_alg,
}
/**
- * spu_payload_length() - Given a SPU2 message header, extract the payload
+ * spu2_payload_length() - Given a SPU2 message header, extract the payload
* length.
* @spu_hdr: Start of SPU message header (FMD)
*
@@ -812,10 +814,11 @@ u32 spu2_payload_length(u8 *spu_hdr)
}
/**
- * spu_response_hdr_len() - Determine the expected length of a SPU response
+ * spu2_response_hdr_len() - Determine the expected length of a SPU response
* header.
* @auth_key_len: Length of authentication key, in bytes
* @enc_key_len: Length of encryption key, in bytes
+ * @is_hash: Unused
*
* For SPU2, includes just FMD. OMD is never requested.
*
@@ -827,7 +830,7 @@ u16 spu2_response_hdr_len(u16 auth_key_len, u16 enc_key_len, bool is_hash)
}
/**
- * spu_hash_pad_len() - Calculate the length of hash padding required to extend
+ * spu2_hash_pad_len() - Calculate the length of hash padding required to extend
* data to a full block size.
* @hash_alg: hash algorithm
* @hash_mode: hash mode
@@ -845,8 +848,10 @@ u16 spu2_hash_pad_len(enum hash_alg hash_alg, enum hash_mode hash_mode,
}
/**
- * spu2_gcm_ccm_padlen() - Determine the length of GCM/CCM padding for either
+ * spu2_gcm_ccm_pad_len() - Determine the length of GCM/CCM padding for either
* the AAD field or the data.
+ * @cipher_mode: Unused
+ * @data_size: Unused
*
* Return: 0. Unlike SPU-M, SPU2 hardware does any GCM/CCM padding required.
*/
@@ -857,7 +862,7 @@ u32 spu2_gcm_ccm_pad_len(enum spu_cipher_mode cipher_mode,
}
/**
- * spu_assoc_resp_len() - Determine the size of the AAD2 buffer needed to catch
+ * spu2_assoc_resp_len() - Determine the size of the AAD2 buffer needed to catch
* associated data in a SPU2 output packet.
* @cipher_mode: cipher mode
* @assoc_len: length of additional associated data, in bytes
@@ -878,11 +883,11 @@ u32 spu2_assoc_resp_len(enum spu_cipher_mode cipher_mode,
return resp_len;
}
-/*
- * spu_aead_ivlen() - Calculate the length of the AEAD IV to be included
+/**
+ * spu2_aead_ivlen() - Calculate the length of the AEAD IV to be included
* in a SPU request after the AAD and before the payload.
* @cipher_mode: cipher mode
- * @iv_ctr_len: initialization vector length in bytes
+ * @iv_len: initialization vector length in bytes
*
* For SPU2, AEAD IV is included in OMD and does not need to be repeated
* prior to the payload.
@@ -909,9 +914,9 @@ enum hash_type spu2_hash_type(u32 src_sent)
/**
* spu2_digest_size() - Determine the size of a hash digest to expect the SPU to
* return.
- * alg_digest_size: Number of bytes in the final digest for the given algo
- * alg: The hash algorithm
- * htype: Type of hash operation (init, update, full, etc)
+ * @alg_digest_size: Number of bytes in the final digest for the given algo
+ * @alg: The hash algorithm
+ * @htype: Type of hash operation (init, update, full, etc)
*
*/
u32 spu2_digest_size(u32 alg_digest_size, enum hash_alg alg,
@@ -921,7 +926,7 @@ u32 spu2_digest_size(u32 alg_digest_size, enum hash_alg alg,
}
/**
- * spu_create_request() - Build a SPU2 request message header, includint FMD and
+ * spu2_create_request() - Build a SPU2 request message header, includint FMD and
* OMD.
* @spu_hdr: Start of buffer where SPU request header is to be written
* @req_opts: SPU request message options
@@ -1105,7 +1110,7 @@ u32 spu2_create_request(u8 *spu_hdr,
}
/**
- * spu_cipher_req_init() - Build an skcipher SPU2 request message header,
+ * spu2_cipher_req_init() - Build an skcipher SPU2 request message header,
* including FMD and OMD.
* @spu_hdr: Location of start of SPU request (FMD field)
* @cipher_parms: Parameters describing cipher request
@@ -1162,11 +1167,11 @@ u16 spu2_cipher_req_init(u8 *spu_hdr, struct spu_cipher_parms *cipher_parms)
}
/**
- * spu_cipher_req_finish() - Finish building a SPU request message header for a
+ * spu2_cipher_req_finish() - Finish building a SPU request message header for a
* block cipher request.
* @spu_hdr: Start of the request message header (MH field)
* @spu_req_hdr_len: Length in bytes of the SPU request header
- * @isInbound: 0 encrypt, 1 decrypt
+ * @is_inbound: 0 encrypt, 1 decrypt
* @cipher_parms: Parameters describing cipher operation to be performed
* @data_size: Length of the data in the BD field
*
@@ -1222,7 +1227,7 @@ void spu2_cipher_req_finish(u8 *spu_hdr,
}
/**
- * spu_request_pad() - Create pad bytes at the end of the data.
+ * spu2_request_pad() - Create pad bytes at the end of the data.
* @pad_start: Start of buffer where pad bytes are to be written
* @gcm_padding: Length of GCM padding, in bytes
* @hash_pad_len: Number of bytes of padding extend data to full block
@@ -1311,7 +1316,7 @@ u8 spu2_rx_status_len(void)
}
/**
- * spu_status_process() - Process the status from a SPU response message.
+ * spu2_status_process() - Process the status from a SPU response message.
* @statp: start of STATUS word
*
* Return: 0 - if status is good and response should be processed
diff --git a/drivers/crypto/bcm/util.c b/drivers/crypto/bcm/util.c
index c4669a96eaec..d5d9cabea55a 100644
--- a/drivers/crypto/bcm/util.c
+++ b/drivers/crypto/bcm/util.c
@@ -119,8 +119,8 @@ int spu_sg_count(struct scatterlist *sg_list, unsigned int skip, int nbytes)
* @from_skip: number of bytes to skip in from_sg. Non-zero when previous
* request included part of the buffer in entry in from_sg.
* Assumes from_skip < from_sg->length.
- * @from_nents number of entries in from_sg
- * @length number of bytes to copy. may reach this limit before exhausting
+ * @from_nents: number of entries in from_sg
+ * @length: number of bytes to copy. may reach this limit before exhausting
* from_sg.
*
* Copies the entries themselves, not the data in the entries. Assumes to_sg has
diff --git a/drivers/crypto/caam/caamalg_qi2.c b/drivers/crypto/caam/caamalg_qi2.c
index a780e627838a..8b8ed77d8715 100644
--- a/drivers/crypto/caam/caamalg_qi2.c
+++ b/drivers/crypto/caam/caamalg_qi2.c
@@ -71,6 +71,9 @@ struct caam_skcipher_alg {
* @adata: authentication algorithm details
* @cdata: encryption algorithm details
* @authsize: authentication tag (a.k.a. ICV / MAC) size
+ * @xts_key_fallback: true if fallback tfm needs to be used due
+ * to unsupported xts key lengths
+ * @fallback: xts fallback tfm
*/
struct caam_ctx {
struct caam_flc flc[NUM_OP];
diff --git a/drivers/crypto/caam/caampkc.c b/drivers/crypto/caam/caampkc.c
index dd5f101e43f8..e313233ec6de 100644
--- a/drivers/crypto/caam/caampkc.c
+++ b/drivers/crypto/caam/caampkc.c
@@ -187,7 +187,8 @@ static void rsa_priv_f_done(struct device *dev, u32 *desc, u32 err,
}
/**
- * Count leading zeros, need it to strip, from a given scatterlist
+ * caam_rsa_count_leading_zeros - Count leading zeros, need it to strip,
+ * from a given scatterlist
*
* @sgl : scatterlist to count zeros from
* @nbytes: number of zeros, in bytes, to strip
diff --git a/drivers/crypto/cavium/cpt/cptpf_main.c b/drivers/crypto/cavium/cpt/cptpf_main.c
index 711b1acdd4e0..06ee42e8a245 100644
--- a/drivers/crypto/cavium/cpt/cptpf_main.c
+++ b/drivers/crypto/cavium/cpt/cptpf_main.c
@@ -10,7 +10,6 @@
#include <linux/moduleparam.h>
#include <linux/pci.h>
#include <linux/printk.h>
-#include <linux/version.h>
#include "cptpf.h"
diff --git a/drivers/crypto/cavium/nitrox/nitrox_isr.c b/drivers/crypto/cavium/nitrox/nitrox_isr.c
index 99b053094f5a..c288c4b51783 100644
--- a/drivers/crypto/cavium/nitrox/nitrox_isr.c
+++ b/drivers/crypto/cavium/nitrox/nitrox_isr.c
@@ -10,7 +10,7 @@
#include "nitrox_isr.h"
#include "nitrox_mbx.h"
-/**
+/*
* One vector for each type of ring
* - NPS packet ring, AQMQ ring and ZQMQ ring
*/
@@ -216,7 +216,7 @@ static void nps_core_int_tasklet(unsigned long data)
}
}
-/**
+/*
* nps_core_int_isr - interrupt handler for NITROX errors and
* mailbox communication
*/
diff --git a/drivers/crypto/cavium/nitrox/nitrox_reqmgr.c b/drivers/crypto/cavium/nitrox/nitrox_reqmgr.c
index 53ef06792133..df95ba26b414 100644
--- a/drivers/crypto/cavium/nitrox/nitrox_reqmgr.c
+++ b/drivers/crypto/cavium/nitrox/nitrox_reqmgr.c
@@ -58,14 +58,15 @@ static void softreq_unmap_sgbufs(struct nitrox_softreq *sr)
struct device *dev = DEV(ndev);
- dma_unmap_sg(dev, sr->in.sg, sr->in.sgmap_cnt, DMA_BIDIRECTIONAL);
+ dma_unmap_sg(dev, sr->in.sg, sg_nents(sr->in.sg),
+ DMA_BIDIRECTIONAL);
dma_unmap_single(dev, sr->in.sgcomp_dma, sr->in.sgcomp_len,
DMA_TO_DEVICE);
kfree(sr->in.sgcomp);
sr->in.sg = NULL;
sr->in.sgmap_cnt = 0;
- dma_unmap_sg(dev, sr->out.sg, sr->out.sgmap_cnt,
+ dma_unmap_sg(dev, sr->out.sg, sg_nents(sr->out.sg),
DMA_BIDIRECTIONAL);
dma_unmap_single(dev, sr->out.sgcomp_dma, sr->out.sgcomp_len,
DMA_TO_DEVICE);
@@ -178,7 +179,7 @@ static int dma_map_inbufs(struct nitrox_softreq *sr,
return 0;
incomp_err:
- dma_unmap_sg(dev, req->src, nents, DMA_BIDIRECTIONAL);
+ dma_unmap_sg(dev, req->src, sg_nents(req->src), DMA_BIDIRECTIONAL);
sr->in.sgmap_cnt = 0;
return ret;
}
@@ -203,7 +204,7 @@ static int dma_map_outbufs(struct nitrox_softreq *sr,
return 0;
outcomp_map_err:
- dma_unmap_sg(dev, req->dst, nents, DMA_BIDIRECTIONAL);
+ dma_unmap_sg(dev, req->dst, sg_nents(req->dst), DMA_BIDIRECTIONAL);
sr->out.sgmap_cnt = 0;
sr->out.sg = NULL;
return ret;
diff --git a/drivers/crypto/cavium/zip/common.h b/drivers/crypto/cavium/zip/common.h
index 58fb3ed6e644..54f6fb054119 100644
--- a/drivers/crypto/cavium/zip/common.h
+++ b/drivers/crypto/cavium/zip/common.h
@@ -56,7 +56,6 @@
#include <linux/seq_file.h>
#include <linux/string.h>
#include <linux/types.h>
-#include <linux/version.h>
/* Device specific zlib function definitions */
#include "zip_device.h"
diff --git a/drivers/crypto/ccp/ccp-crypto-main.c b/drivers/crypto/ccp/ccp-crypto-main.c
index 88275b4867ea..5976530c00a8 100644
--- a/drivers/crypto/ccp/ccp-crypto-main.c
+++ b/drivers/crypto/ccp/ccp-crypto-main.c
@@ -59,7 +59,7 @@ struct ccp_crypto_queue {
#define CCP_CRYPTO_MAX_QLEN 100
static struct ccp_crypto_queue req_queue;
-static spinlock_t req_queue_lock;
+static DEFINE_SPINLOCK(req_queue_lock);
struct ccp_crypto_cmd {
struct list_head entry;
@@ -410,7 +410,6 @@ static int ccp_crypto_init(void)
return ret;
}
- spin_lock_init(&req_queue_lock);
INIT_LIST_HEAD(&req_queue.cmds);
req_queue.backlog = &req_queue.cmds;
req_queue.cmd_count = 0;
diff --git a/drivers/crypto/ccp/ccp-dev.c b/drivers/crypto/ccp/ccp-dev.c
index 0971ee60f840..6777582aa1ce 100644
--- a/drivers/crypto/ccp/ccp-dev.c
+++ b/drivers/crypto/ccp/ccp-dev.c
@@ -548,7 +548,7 @@ bool ccp_queues_suspended(struct ccp_device *ccp)
return ccp->cmd_q_count == suspended;
}
-int ccp_dev_suspend(struct sp_device *sp)
+void ccp_dev_suspend(struct sp_device *sp)
{
struct ccp_device *ccp = sp->ccp_data;
unsigned long flags;
@@ -556,7 +556,7 @@ int ccp_dev_suspend(struct sp_device *sp)
/* If there's no device there's nothing to do */
if (!ccp)
- return 0;
+ return;
spin_lock_irqsave(&ccp->cmd_lock, flags);
@@ -572,11 +572,9 @@ int ccp_dev_suspend(struct sp_device *sp)
while (!ccp_queues_suspended(ccp))
wait_event_interruptible(ccp->suspend_queue,
ccp_queues_suspended(ccp));
-
- return 0;
}
-int ccp_dev_resume(struct sp_device *sp)
+void ccp_dev_resume(struct sp_device *sp)
{
struct ccp_device *ccp = sp->ccp_data;
unsigned long flags;
@@ -584,7 +582,7 @@ int ccp_dev_resume(struct sp_device *sp)
/* If there's no device there's nothing to do */
if (!ccp)
- return 0;
+ return;
spin_lock_irqsave(&ccp->cmd_lock, flags);
@@ -597,8 +595,6 @@ int ccp_dev_resume(struct sp_device *sp)
}
spin_unlock_irqrestore(&ccp->cmd_lock, flags);
-
- return 0;
}
int ccp_dev_init(struct sp_device *sp)
diff --git a/drivers/crypto/ccp/ccp-ops.c b/drivers/crypto/ccp/ccp-ops.c
index d6a8f4e4b14a..bb88198c874e 100644
--- a/drivers/crypto/ccp/ccp-ops.c
+++ b/drivers/crypto/ccp/ccp-ops.c
@@ -2418,7 +2418,6 @@ static int ccp_run_ecc_pm_cmd(struct ccp_cmd_queue *cmd_q, struct ccp_cmd *cmd)
dst.address += CCP_ECC_OUTPUT_SIZE;
ccp_reverse_get_dm_area(&dst, 0, ecc->u.pm.result.y, 0,
CCP_ECC_MODULUS_BYTES);
- dst.address += CCP_ECC_OUTPUT_SIZE;
/* Restore the workarea address */
dst.address = save;
diff --git a/drivers/crypto/ccp/sev-dev.c b/drivers/crypto/ccp/sev-dev.c
index cb9b4c4e371e..3506b2050fb8 100644
--- a/drivers/crypto/ccp/sev-dev.c
+++ b/drivers/crypto/ccp/sev-dev.c
@@ -21,6 +21,7 @@
#include <linux/ccp.h>
#include <linux/firmware.h>
#include <linux/gfp.h>
+#include <linux/cpufeature.h>
#include <asm/smp.h>
@@ -129,6 +130,7 @@ static int sev_cmd_buffer_len(int cmd)
case SEV_CMD_DOWNLOAD_FIRMWARE: return sizeof(struct sev_data_download_firmware);
case SEV_CMD_GET_ID: return sizeof(struct sev_data_get_id);
case SEV_CMD_ATTESTATION_REPORT: return sizeof(struct sev_data_attestation_report);
+ case SEV_CMD_SEND_CANCEL: return sizeof(struct sev_data_send_cancel);
default: return 0;
}
@@ -141,6 +143,7 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
struct sev_device *sev;
unsigned int phys_lsb, phys_msb;
unsigned int reg, ret = 0;
+ int buf_len;
if (!psp || !psp->sev_data)
return -ENODEV;
@@ -150,15 +153,27 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
sev = psp->sev_data;
+ buf_len = sev_cmd_buffer_len(cmd);
+ if (WARN_ON_ONCE(!data != !buf_len))
+ return -EINVAL;
+
+ /*
+ * Copy the incoming data to driver's scratch buffer as __pa() will not
+ * work for some memory, e.g. vmalloc'd addresses, and @data may not be
+ * physically contiguous.
+ */
+ if (data)
+ memcpy(sev->cmd_buf, data, buf_len);
+
/* Get the physical address of the command buffer */
- phys_lsb = data ? lower_32_bits(__psp_pa(data)) : 0;
- phys_msb = data ? upper_32_bits(__psp_pa(data)) : 0;
+ phys_lsb = data ? lower_32_bits(__psp_pa(sev->cmd_buf)) : 0;
+ phys_msb = data ? upper_32_bits(__psp_pa(sev->cmd_buf)) : 0;
dev_dbg(sev->dev, "sev command id %#x buffer 0x%08x%08x timeout %us\n",
cmd, phys_msb, phys_lsb, psp_timeout);
print_hex_dump_debug("(in): ", DUMP_PREFIX_OFFSET, 16, 2, data,
- sev_cmd_buffer_len(cmd), false);
+ buf_len, false);
iowrite32(phys_lsb, sev->io_regs + sev->vdata->cmdbuff_addr_lo_reg);
iowrite32(phys_msb, sev->io_regs + sev->vdata->cmdbuff_addr_hi_reg);
@@ -194,7 +209,14 @@ static int __sev_do_cmd_locked(int cmd, void *data, int *psp_ret)
}
print_hex_dump_debug("(out): ", DUMP_PREFIX_OFFSET, 16, 2, data,
- sev_cmd_buffer_len(cmd), false);
+ buf_len, false);
+
+ /*
+ * Copy potential output from the PSP back to data. Do this even on
+ * failure in case the caller wants to glean something from the error.
+ */
+ if (data)
+ memcpy(data, sev->cmd_buf, buf_len);
return ret;
}
@@ -213,6 +235,7 @@ static int sev_do_cmd(int cmd, void *data, int *psp_ret)
static int __sev_platform_init_locked(int *error)
{
struct psp_device *psp = psp_master;
+ struct sev_data_init data;
struct sev_device *sev;
int rc = 0;
@@ -224,6 +247,7 @@ static int __sev_platform_init_locked(int *error)
if (sev->state == SEV_STATE_INIT)
return 0;
+ memset(&data, 0, sizeof(data));
if (sev_es_tmr) {
u64 tmr_pa;
@@ -233,12 +257,12 @@ static int __sev_platform_init_locked(int *error)
*/
tmr_pa = __pa(sev_es_tmr);
- sev->init_cmd_buf.flags |= SEV_INIT_FLAGS_SEV_ES;
- sev->init_cmd_buf.tmr_address = tmr_pa;
- sev->init_cmd_buf.tmr_len = SEV_ES_TMR_SIZE;
+ data.flags |= SEV_INIT_FLAGS_SEV_ES;
+ data.tmr_address = tmr_pa;
+ data.tmr_len = SEV_ES_TMR_SIZE;
}
- rc = __sev_do_cmd_locked(SEV_CMD_INIT, &sev->init_cmd_buf, error);
+ rc = __sev_do_cmd_locked(SEV_CMD_INIT, &data, error);
if (rc)
return rc;
@@ -295,15 +319,14 @@ static int sev_platform_shutdown(int *error)
static int sev_get_platform_state(int *state, int *error)
{
- struct sev_device *sev = psp_master->sev_data;
+ struct sev_user_data_status data;
int rc;
- rc = __sev_do_cmd_locked(SEV_CMD_PLATFORM_STATUS,
- &sev->status_cmd_buf, error);
+ rc = __sev_do_cmd_locked(SEV_CMD_PLATFORM_STATUS, &data, error);
if (rc)
return rc;
- *state = sev->status_cmd_buf.state;
+ *state = data.state;
return rc;
}
@@ -341,15 +364,14 @@ static int sev_ioctl_do_reset(struct sev_issue_cmd *argp, bool writable)
static int sev_ioctl_do_platform_status(struct sev_issue_cmd *argp)
{
- struct sev_device *sev = psp_master->sev_data;
- struct sev_user_data_status *data = &sev->status_cmd_buf;
+ struct sev_user_data_status data;
int ret;
- ret = __sev_do_cmd_locked(SEV_CMD_PLATFORM_STATUS, data, &argp->error);
+ ret = __sev_do_cmd_locked(SEV_CMD_PLATFORM_STATUS, &data, &argp->error);
if (ret)
return ret;
- if (copy_to_user((void __user *)argp->data, data, sizeof(*data)))
+ if (copy_to_user((void __user *)argp->data, &data, sizeof(data)))
ret = -EFAULT;
return ret;
@@ -376,7 +398,7 @@ static int sev_ioctl_do_pek_csr(struct sev_issue_cmd *argp, bool writable)
{
struct sev_device *sev = psp_master->sev_data;
struct sev_user_data_pek_csr input;
- struct sev_data_pek_csr *data;
+ struct sev_data_pek_csr data;
void __user *input_address;
void *blob = NULL;
int ret;
@@ -387,9 +409,7 @@ static int sev_ioctl_do_pek_csr(struct sev_issue_cmd *argp, bool writable)
if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
return -EFAULT;
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data)
- return -ENOMEM;
+ memset(&data, 0, sizeof(data));
/* userspace wants to query CSR length */
if (!input.address || !input.length)
@@ -397,19 +417,15 @@ static int sev_ioctl_do_pek_csr(struct sev_issue_cmd *argp, bool writable)
/* allocate a physically contiguous buffer to store the CSR blob */
input_address = (void __user *)input.address;
- if (input.length > SEV_FW_BLOB_MAX_SIZE) {
- ret = -EFAULT;
- goto e_free;
- }
+ if (input.length > SEV_FW_BLOB_MAX_SIZE)
+ return -EFAULT;
blob = kmalloc(input.length, GFP_KERNEL);
- if (!blob) {
- ret = -ENOMEM;
- goto e_free;
- }
+ if (!blob)
+ return -ENOMEM;
- data->address = __psp_pa(blob);
- data->len = input.length;
+ data.address = __psp_pa(blob);
+ data.len = input.length;
cmd:
if (sev->state == SEV_STATE_UNINIT) {
@@ -418,10 +434,10 @@ cmd:
goto e_free_blob;
}
- ret = __sev_do_cmd_locked(SEV_CMD_PEK_CSR, data, &argp->error);
+ ret = __sev_do_cmd_locked(SEV_CMD_PEK_CSR, &data, &argp->error);
/* If we query the CSR length, FW responded with expected data. */
- input.length = data->len;
+ input.length = data.len;
if (copy_to_user((void __user *)argp->data, &input, sizeof(input))) {
ret = -EFAULT;
@@ -435,8 +451,6 @@ cmd:
e_free_blob:
kfree(blob);
-e_free:
- kfree(data);
return ret;
}
@@ -456,21 +470,20 @@ EXPORT_SYMBOL_GPL(psp_copy_user_blob);
static int sev_get_api_version(void)
{
struct sev_device *sev = psp_master->sev_data;
- struct sev_user_data_status *status;
+ struct sev_user_data_status status;
int error = 0, ret;
- status = &sev->status_cmd_buf;
- ret = sev_platform_status(status, &error);
+ ret = sev_platform_status(&status, &error);
if (ret) {
dev_err(sev->dev,
"SEV: failed to get status. Error: %#x\n", error);
return 1;
}
- sev->api_major = status->api_major;
- sev->api_minor = status->api_minor;
- sev->build = status->build;
- sev->state = status->state;
+ sev->api_major = status.api_major;
+ sev->api_minor = status.api_minor;
+ sev->build = status.build;
+ sev->state = status.state;
return 0;
}
@@ -568,7 +581,7 @@ static int sev_ioctl_do_pek_import(struct sev_issue_cmd *argp, bool writable)
{
struct sev_device *sev = psp_master->sev_data;
struct sev_user_data_pek_cert_import input;
- struct sev_data_pek_cert_import *data;
+ struct sev_data_pek_cert_import data;
void *pek_blob, *oca_blob;
int ret;
@@ -578,19 +591,14 @@ static int sev_ioctl_do_pek_import(struct sev_issue_cmd *argp, bool writable)
if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
return -EFAULT;
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data)
- return -ENOMEM;
-
/* copy PEK certificate blobs from userspace */
pek_blob = psp_copy_user_blob(input.pek_cert_address, input.pek_cert_len);
- if (IS_ERR(pek_blob)) {
- ret = PTR_ERR(pek_blob);
- goto e_free;
- }
+ if (IS_ERR(pek_blob))
+ return PTR_ERR(pek_blob);
- data->pek_cert_address = __psp_pa(pek_blob);
- data->pek_cert_len = input.pek_cert_len;
+ data.reserved = 0;
+ data.pek_cert_address = __psp_pa(pek_blob);
+ data.pek_cert_len = input.pek_cert_len;
/* copy PEK certificate blobs from userspace */
oca_blob = psp_copy_user_blob(input.oca_cert_address, input.oca_cert_len);
@@ -599,8 +607,8 @@ static int sev_ioctl_do_pek_import(struct sev_issue_cmd *argp, bool writable)
goto e_free_pek;
}
- data->oca_cert_address = __psp_pa(oca_blob);
- data->oca_cert_len = input.oca_cert_len;
+ data.oca_cert_address = __psp_pa(oca_blob);
+ data.oca_cert_len = input.oca_cert_len;
/* If platform is not in INIT state then transition it to INIT */
if (sev->state != SEV_STATE_INIT) {
@@ -609,21 +617,19 @@ static int sev_ioctl_do_pek_import(struct sev_issue_cmd *argp, bool writable)
goto e_free_oca;
}
- ret = __sev_do_cmd_locked(SEV_CMD_PEK_CERT_IMPORT, data, &argp->error);
+ ret = __sev_do_cmd_locked(SEV_CMD_PEK_CERT_IMPORT, &data, &argp->error);
e_free_oca:
kfree(oca_blob);
e_free_pek:
kfree(pek_blob);
-e_free:
- kfree(data);
return ret;
}
static int sev_ioctl_do_get_id2(struct sev_issue_cmd *argp)
{
struct sev_user_data_get_id2 input;
- struct sev_data_get_id *data;
+ struct sev_data_get_id data;
void __user *input_address;
void *id_blob = NULL;
int ret;
@@ -637,28 +643,25 @@ static int sev_ioctl_do_get_id2(struct sev_issue_cmd *argp)
input_address = (void __user *)input.address;
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data)
- return -ENOMEM;
-
if (input.address && input.length) {
id_blob = kmalloc(input.length, GFP_KERNEL);
- if (!id_blob) {
- kfree(data);
+ if (!id_blob)
return -ENOMEM;
- }
- data->address = __psp_pa(id_blob);
- data->len = input.length;
+ data.address = __psp_pa(id_blob);
+ data.len = input.length;
+ } else {
+ data.address = 0;
+ data.len = 0;
}
- ret = __sev_do_cmd_locked(SEV_CMD_GET_ID, data, &argp->error);
+ ret = __sev_do_cmd_locked(SEV_CMD_GET_ID, &data, &argp->error);
/*
* Firmware will return the length of the ID value (either the minimum
* required length or the actual length written), return it to the user.
*/
- input.length = data->len;
+ input.length = data.len;
if (copy_to_user((void __user *)argp->data, &input, sizeof(input))) {
ret = -EFAULT;
@@ -666,7 +669,7 @@ static int sev_ioctl_do_get_id2(struct sev_issue_cmd *argp)
}
if (id_blob) {
- if (copy_to_user(input_address, id_blob, data->len)) {
+ if (copy_to_user(input_address, id_blob, data.len)) {
ret = -EFAULT;
goto e_free;
}
@@ -674,7 +677,6 @@ static int sev_ioctl_do_get_id2(struct sev_issue_cmd *argp)
e_free:
kfree(id_blob);
- kfree(data);
return ret;
}
@@ -724,7 +726,7 @@ static int sev_ioctl_do_pdh_export(struct sev_issue_cmd *argp, bool writable)
struct sev_device *sev = psp_master->sev_data;
struct sev_user_data_pdh_cert_export input;
void *pdh_blob = NULL, *cert_blob = NULL;
- struct sev_data_pdh_cert_export *data;
+ struct sev_data_pdh_cert_export data;
void __user *input_cert_chain_address;
void __user *input_pdh_cert_address;
int ret;
@@ -742,9 +744,7 @@ static int sev_ioctl_do_pdh_export(struct sev_issue_cmd *argp, bool writable)
if (copy_from_user(&input, (void __user *)argp->data, sizeof(input)))
return -EFAULT;
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data)
- return -ENOMEM;
+ memset(&data, 0, sizeof(data));
/* Userspace wants to query the certificate length. */
if (!input.pdh_cert_address ||
@@ -756,25 +756,19 @@ static int sev_ioctl_do_pdh_export(struct sev_issue_cmd *argp, bool writable)
input_cert_chain_address = (void __user *)input.cert_chain_address;
/* Allocate a physically contiguous buffer to store the PDH blob. */
- if (input.pdh_cert_len > SEV_FW_BLOB_MAX_SIZE) {
- ret = -EFAULT;
- goto e_free;
- }
+ if (input.pdh_cert_len > SEV_FW_BLOB_MAX_SIZE)
+ return -EFAULT;
/* Allocate a physically contiguous buffer to store the cert chain blob. */
- if (input.cert_chain_len > SEV_FW_BLOB_MAX_SIZE) {
- ret = -EFAULT;
- goto e_free;
- }
+ if (input.cert_chain_len > SEV_FW_BLOB_MAX_SIZE)
+ return -EFAULT;
pdh_blob = kmalloc(input.pdh_cert_len, GFP_KERNEL);
- if (!pdh_blob) {
- ret = -ENOMEM;
- goto e_free;
- }
+ if (!pdh_blob)
+ return -ENOMEM;
- data->pdh_cert_address = __psp_pa(pdh_blob);
- data->pdh_cert_len = input.pdh_cert_len;
+ data.pdh_cert_address = __psp_pa(pdh_blob);
+ data.pdh_cert_len = input.pdh_cert_len;
cert_blob = kmalloc(input.cert_chain_len, GFP_KERNEL);
if (!cert_blob) {
@@ -782,15 +776,15 @@ static int sev_ioctl_do_pdh_export(struct sev_issue_cmd *argp, bool writable)
goto e_free_pdh;
}
- data->cert_chain_address = __psp_pa(cert_blob);
- data->cert_chain_len = input.cert_chain_len;
+ data.cert_chain_address = __psp_pa(cert_blob);
+ data.cert_chain_len = input.cert_chain_len;
cmd:
- ret = __sev_do_cmd_locked(SEV_CMD_PDH_CERT_EXPORT, data, &argp->error);
+ ret = __sev_do_cmd_locked(SEV_CMD_PDH_CERT_EXPORT, &data, &argp->error);
/* If we query the length, FW responded with expected data. */
- input.cert_chain_len = data->cert_chain_len;
- input.pdh_cert_len = data->pdh_cert_len;
+ input.cert_chain_len = data.cert_chain_len;
+ input.pdh_cert_len = data.pdh_cert_len;
if (copy_to_user((void __user *)argp->data, &input, sizeof(input))) {
ret = -EFAULT;
@@ -815,8 +809,6 @@ e_free_cert:
kfree(cert_blob);
e_free_pdh:
kfree(pdh_blob);
-e_free:
- kfree(data);
return ret;
}
@@ -972,10 +964,19 @@ int sev_dev_init(struct psp_device *psp)
struct sev_device *sev;
int ret = -ENOMEM;
+ if (!boot_cpu_has(X86_FEATURE_SEV)) {
+ dev_info_once(dev, "SEV: memory encryption not enabled by BIOS\n");
+ return 0;
+ }
+
sev = devm_kzalloc(dev, sizeof(*sev), GFP_KERNEL);
if (!sev)
goto e_err;
+ sev->cmd_buf = (void *)devm_get_free_pages(dev, GFP_KERNEL, 0);
+ if (!sev->cmd_buf)
+ goto e_sev;
+
psp->sev_data = sev;
sev->dev = dev;
@@ -987,7 +988,7 @@ int sev_dev_init(struct psp_device *psp)
if (!sev->vdata) {
ret = -ENODEV;
dev_err(dev, "sev: missing driver data\n");
- goto e_err;
+ goto e_buf;
}
psp_set_sev_irq_handler(psp, sev_irq_handler, sev);
@@ -1002,6 +1003,10 @@ int sev_dev_init(struct psp_device *psp)
e_irq:
psp_clear_sev_irq_handler(psp);
+e_buf:
+ devm_free_pages(dev, (unsigned long)sev->cmd_buf);
+e_sev:
+ devm_kfree(dev, sev);
e_err:
psp->sev_data = NULL;
diff --git a/drivers/crypto/ccp/sev-dev.h b/drivers/crypto/ccp/sev-dev.h
index dd5c4fe82914..666c21eb81ab 100644
--- a/drivers/crypto/ccp/sev-dev.h
+++ b/drivers/crypto/ccp/sev-dev.h
@@ -46,12 +46,12 @@ struct sev_device {
unsigned int int_rcvd;
wait_queue_head_t int_queue;
struct sev_misc_dev *misc;
- struct sev_user_data_status status_cmd_buf;
- struct sev_data_init init_cmd_buf;
u8 api_major;
u8 api_minor;
u8 build;
+
+ void *cmd_buf;
};
int sev_dev_init(struct psp_device *psp);
diff --git a/drivers/crypto/ccp/sp-dev.c b/drivers/crypto/ccp/sp-dev.c
index 6284a15e5047..7eb3e4668286 100644
--- a/drivers/crypto/ccp/sp-dev.c
+++ b/drivers/crypto/ccp/sp-dev.c
@@ -213,12 +213,8 @@ void sp_destroy(struct sp_device *sp)
int sp_suspend(struct sp_device *sp)
{
- int ret;
-
if (sp->dev_vdata->ccp_vdata) {
- ret = ccp_dev_suspend(sp);
- if (ret)
- return ret;
+ ccp_dev_suspend(sp);
}
return 0;
@@ -226,12 +222,8 @@ int sp_suspend(struct sp_device *sp)
int sp_resume(struct sp_device *sp)
{
- int ret;
-
if (sp->dev_vdata->ccp_vdata) {
- ret = ccp_dev_resume(sp);
- if (ret)
- return ret;
+ ccp_dev_resume(sp);
}
return 0;
diff --git a/drivers/crypto/ccp/sp-dev.h b/drivers/crypto/ccp/sp-dev.h
index 0218d0670eee..20377e67f65d 100644
--- a/drivers/crypto/ccp/sp-dev.h
+++ b/drivers/crypto/ccp/sp-dev.h
@@ -134,8 +134,8 @@ struct sp_device *sp_get_psp_master_device(void);
int ccp_dev_init(struct sp_device *sp);
void ccp_dev_destroy(struct sp_device *sp);
-int ccp_dev_suspend(struct sp_device *sp);
-int ccp_dev_resume(struct sp_device *sp);
+void ccp_dev_suspend(struct sp_device *sp);
+void ccp_dev_resume(struct sp_device *sp);
#else /* !CONFIG_CRYPTO_DEV_SP_CCP */
@@ -144,15 +144,8 @@ static inline int ccp_dev_init(struct sp_device *sp)
return 0;
}
static inline void ccp_dev_destroy(struct sp_device *sp) { }
-
-static inline int ccp_dev_suspend(struct sp_device *sp)
-{
- return 0;
-}
-static inline int ccp_dev_resume(struct sp_device *sp)
-{
- return 0;
-}
+static inline void ccp_dev_suspend(struct sp_device *sp) { }
+static inline void ccp_dev_resume(struct sp_device *sp) { }
#endif /* CONFIG_CRYPTO_DEV_SP_CCP */
#ifdef CONFIG_CRYPTO_DEV_SP_PSP
diff --git a/drivers/crypto/ccp/sp-pci.c b/drivers/crypto/ccp/sp-pci.c
index f471dbaef1fb..f468594ef8af 100644
--- a/drivers/crypto/ccp/sp-pci.c
+++ b/drivers/crypto/ccp/sp-pci.c
@@ -356,6 +356,7 @@ static const struct pci_device_id sp_pci_table[] = {
{ PCI_VDEVICE(AMD, 0x1468), (kernel_ulong_t)&dev_vdata[2] },
{ PCI_VDEVICE(AMD, 0x1486), (kernel_ulong_t)&dev_vdata[3] },
{ PCI_VDEVICE(AMD, 0x15DF), (kernel_ulong_t)&dev_vdata[4] },
+ { PCI_VDEVICE(AMD, 0x1649), (kernel_ulong_t)&dev_vdata[4] },
/* Last entry must be zero */
{ 0, }
};
diff --git a/drivers/crypto/ccp/tee-dev.c b/drivers/crypto/ccp/tee-dev.c
index 5e697a90ea7f..5c9d47f3be37 100644
--- a/drivers/crypto/ccp/tee-dev.c
+++ b/drivers/crypto/ccp/tee-dev.c
@@ -5,7 +5,7 @@
* Author: Rijo Thomas <Rijo-john.Thomas@amd.com>
* Author: Devaraj Rangasamy <Devaraj.Rangasamy@amd.com>
*
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
*/
#include <linux/types.h>
@@ -36,6 +36,7 @@ static int tee_alloc_ring(struct psp_tee_device *tee, int ring_size)
if (!start_addr)
return -ENOMEM;
+ memset(start_addr, 0x0, ring_size);
rb_mgr->ring_start = start_addr;
rb_mgr->ring_size = ring_size;
rb_mgr->ring_pa = __psp_pa(start_addr);
@@ -244,41 +245,54 @@ static int tee_submit_cmd(struct psp_tee_device *tee, enum tee_cmd_id cmd_id,
void *buf, size_t len, struct tee_ring_cmd **resp)
{
struct tee_ring_cmd *cmd;
- u32 rptr, wptr;
int nloop = 1000, ret = 0;
+ u32 rptr;
*resp = NULL;
mutex_lock(&tee->rb_mgr.mutex);
- wptr = tee->rb_mgr.wptr;
-
- /* Check if ring buffer is full */
+ /* Loop until empty entry found in ring buffer */
do {
+ /* Get pointer to ring buffer command entry */
+ cmd = (struct tee_ring_cmd *)
+ (tee->rb_mgr.ring_start + tee->rb_mgr.wptr);
+
rptr = ioread32(tee->io_regs + tee->vdata->ring_rptr_reg);
- if (!(wptr + sizeof(struct tee_ring_cmd) == rptr))
+ /* Check if ring buffer is full or command entry is waiting
+ * for response from TEE
+ */
+ if (!(tee->rb_mgr.wptr + sizeof(struct tee_ring_cmd) == rptr ||
+ cmd->flag == CMD_WAITING_FOR_RESPONSE))
break;
- dev_info(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u\n",
- rptr, wptr);
+ dev_dbg(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u\n",
+ rptr, tee->rb_mgr.wptr);
- /* Wait if ring buffer is full */
+ /* Wait if ring buffer is full or TEE is processing data */
mutex_unlock(&tee->rb_mgr.mutex);
schedule_timeout_interruptible(msecs_to_jiffies(10));
mutex_lock(&tee->rb_mgr.mutex);
} while (--nloop);
- if (!nloop && (wptr + sizeof(struct tee_ring_cmd) == rptr)) {
- dev_err(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u\n",
- rptr, wptr);
+ if (!nloop &&
+ (tee->rb_mgr.wptr + sizeof(struct tee_ring_cmd) == rptr ||
+ cmd->flag == CMD_WAITING_FOR_RESPONSE)) {
+ dev_err(tee->dev, "tee: ring buffer full. rptr = %u wptr = %u response flag %u\n",
+ rptr, tee->rb_mgr.wptr, cmd->flag);
ret = -EBUSY;
goto unlock;
}
- /* Pointer to empty data entry in ring buffer */
- cmd = (struct tee_ring_cmd *)(tee->rb_mgr.ring_start + wptr);
+ /* Do not submit command if PSP got disabled while processing any
+ * command in another thread
+ */
+ if (psp_dead) {
+ ret = -EBUSY;
+ goto unlock;
+ }
/* Write command data into ring buffer */
cmd->cmd_id = cmd_id;
@@ -286,6 +300,9 @@ static int tee_submit_cmd(struct psp_tee_device *tee, enum tee_cmd_id cmd_id,
memset(&cmd->buf[0], 0, sizeof(cmd->buf));
memcpy(&cmd->buf[0], buf, len);
+ /* Indicate driver is waiting for response */
+ cmd->flag = CMD_WAITING_FOR_RESPONSE;
+
/* Update local copy of write pointer */
tee->rb_mgr.wptr += sizeof(struct tee_ring_cmd);
if (tee->rb_mgr.wptr >= tee->rb_mgr.ring_size)
@@ -309,14 +326,14 @@ static int tee_wait_cmd_completion(struct psp_tee_device *tee,
struct tee_ring_cmd *resp,
unsigned int timeout)
{
- /* ~5ms sleep per loop => nloop = timeout * 200 */
- int nloop = timeout * 200;
+ /* ~1ms sleep per loop => nloop = timeout * 1000 */
+ int nloop = timeout * 1000;
while (--nloop) {
if (resp->cmd_state == TEE_CMD_STATE_COMPLETED)
return 0;
- usleep_range(5000, 5100);
+ usleep_range(1000, 1100);
}
dev_err(tee->dev, "tee: command 0x%x timed out, disabling PSP\n",
@@ -353,12 +370,16 @@ int psp_tee_process_cmd(enum tee_cmd_id cmd_id, void *buf, size_t len,
return ret;
ret = tee_wait_cmd_completion(tee, resp, TEE_DEFAULT_TIMEOUT);
- if (ret)
+ if (ret) {
+ resp->flag = CMD_RESPONSE_TIMEDOUT;
return ret;
+ }
memcpy(buf, &resp->buf[0], len);
*status = resp->status;
+ resp->flag = CMD_RESPONSE_COPIED;
+
return 0;
}
EXPORT_SYMBOL(psp_tee_process_cmd);
diff --git a/drivers/crypto/ccp/tee-dev.h b/drivers/crypto/ccp/tee-dev.h
index f09960112115..49d26158b71e 100644
--- a/drivers/crypto/ccp/tee-dev.h
+++ b/drivers/crypto/ccp/tee-dev.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: MIT */
/*
- * Copyright 2019 Advanced Micro Devices, Inc.
+ * Copyright (C) 2019,2021 Advanced Micro Devices, Inc.
*
* Author: Rijo Thomas <Rijo-john.Thomas@amd.com>
* Author: Devaraj Rangasamy <Devaraj.Rangasamy@amd.com>
@@ -18,7 +18,7 @@
#include <linux/mutex.h>
#define TEE_DEFAULT_TIMEOUT 10
-#define MAX_BUFFER_SIZE 992
+#define MAX_BUFFER_SIZE 988
/**
* enum tee_ring_cmd_id - TEE interface commands for ring buffer configuration
@@ -82,6 +82,20 @@ enum tee_cmd_state {
};
/**
+ * enum cmd_resp_state - TEE command's response status maintained by driver
+ * @CMD_RESPONSE_INVALID: initial state when no command is written to ring
+ * @CMD_WAITING_FOR_RESPONSE: driver waiting for response from TEE
+ * @CMD_RESPONSE_TIMEDOUT: failed to get response from TEE
+ * @CMD_RESPONSE_COPIED: driver has copied response from TEE
+ */
+enum cmd_resp_state {
+ CMD_RESPONSE_INVALID,
+ CMD_WAITING_FOR_RESPONSE,
+ CMD_RESPONSE_TIMEDOUT,
+ CMD_RESPONSE_COPIED,
+};
+
+/**
* struct tee_ring_cmd - Structure of the command buffer in TEE ring
* @cmd_id: refers to &enum tee_cmd_id. Command id for the ring buffer
* interface
@@ -91,6 +105,7 @@ enum tee_cmd_state {
* @pdata: private data (currently unused)
* @res1: reserved region
* @buf: TEE command specific buffer
+ * @flag: refers to &enum cmd_resp_state
*/
struct tee_ring_cmd {
u32 cmd_id;
@@ -100,6 +115,7 @@ struct tee_ring_cmd {
u64 pdata;
u32 res1[2];
u8 buf[MAX_BUFFER_SIZE];
+ u32 flag;
/* Total size: 1024 bytes */
} __packed;
diff --git a/drivers/crypto/ccree/cc_driver.c b/drivers/crypto/ccree/cc_driver.c
index d0e59e942568..e599ac6dc162 100644
--- a/drivers/crypto/ccree/cc_driver.c
+++ b/drivers/crypto/ccree/cc_driver.c
@@ -352,10 +352,8 @@ static int init_cc_resources(struct platform_device *plat_dev)
req_mem_cc_regs = platform_get_resource(plat_dev, IORESOURCE_MEM, 0);
/* Map registers space */
new_drvdata->cc_base = devm_ioremap_resource(dev, req_mem_cc_regs);
- if (IS_ERR(new_drvdata->cc_base)) {
- dev_err(dev, "Failed to ioremap registers");
+ if (IS_ERR(new_drvdata->cc_base))
return PTR_ERR(new_drvdata->cc_base);
- }
dev_dbg(dev, "Got MEM resource (%s): %pR\n", req_mem_cc_regs->name,
req_mem_cc_regs);
diff --git a/drivers/crypto/chelsio/chcr_algo.c b/drivers/crypto/chelsio/chcr_algo.c
index f5a336634daa..6933546f87b1 100644
--- a/drivers/crypto/chelsio/chcr_algo.c
+++ b/drivers/crypto/chelsio/chcr_algo.c
@@ -126,11 +126,6 @@ static inline struct uld_ctx *ULD_CTX(struct chcr_context *ctx)
return container_of(ctx->dev, struct uld_ctx, dev);
}
-static inline int is_ofld_imm(const struct sk_buff *skb)
-{
- return (skb->len <= SGE_MAX_WR_LEN);
-}
-
static inline void chcr_init_hctx_per_wr(struct chcr_ahash_req_ctx *reqctx)
{
memset(&reqctx->hctx_wr, 0, sizeof(struct chcr_hctx_per_wr));
@@ -769,13 +764,14 @@ static inline void create_wreq(struct chcr_context *ctx,
struct uld_ctx *u_ctx = ULD_CTX(ctx);
unsigned int tx_channel_id, rx_channel_id;
unsigned int txqidx = 0, rxqidx = 0;
- unsigned int qid, fid;
+ unsigned int qid, fid, portno;
get_qidxs(req, &txqidx, &rxqidx);
qid = u_ctx->lldi.rxq_ids[rxqidx];
fid = u_ctx->lldi.rxq_ids[0];
+ portno = rxqidx / ctx->rxq_perchan;
tx_channel_id = txqidx / ctx->txq_perchan;
- rx_channel_id = rxqidx / ctx->rxq_perchan;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[portno]);
chcr_req->wreq.op_to_cctx_size = FILL_WR_OP_CCTX_SIZE;
@@ -797,15 +793,13 @@ static inline void create_wreq(struct chcr_context *ctx,
/**
* create_cipher_wr - form the WR for cipher operations
- * @req: cipher req.
- * @ctx: crypto driver context of the request.
- * @qid: ingress qid where response of this WR should be received.
- * @op_type: encryption or decryption
+ * @wrparam: Container for create_cipher_wr()'s parameters
*/
static struct sk_buff *create_cipher_wr(struct cipher_wr_param *wrparam)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(wrparam->req);
struct chcr_context *ctx = c_ctx(tfm);
+ struct uld_ctx *u_ctx = ULD_CTX(ctx);
struct ablk_ctx *ablkctx = ABLK_CTX(ctx);
struct sk_buff *skb = NULL;
struct chcr_wr *chcr_req;
@@ -822,6 +816,7 @@ static struct sk_buff *create_cipher_wr(struct cipher_wr_param *wrparam)
struct adapter *adap = padap(ctx->dev);
unsigned int rx_channel_id = reqctx->rxqidx / ctx->rxq_perchan;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[rx_channel_id]);
nents = sg_nents_xlen(reqctx->dstsg, wrparam->bytes, CHCR_DST_SG_SIZE,
reqctx->dst_ofst);
dst_size = get_space_for_phys_dsgl(nents);
@@ -1559,7 +1554,8 @@ static inline void chcr_free_shash(struct crypto_shash *base_hash)
/**
* create_hash_wr - Create hash work request
- * @req - Cipher req base
+ * @req: Cipher req base
+ * @param: Container for create_hash_wr()'s parameters
*/
static struct sk_buff *create_hash_wr(struct ahash_request *req,
struct hash_wr_param *param)
@@ -1580,6 +1576,7 @@ static struct sk_buff *create_hash_wr(struct ahash_request *req,
int error = 0;
unsigned int rx_channel_id = req_ctx->rxqidx / ctx->rxq_perchan;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[rx_channel_id]);
transhdr_len = HASH_TRANSHDR_SIZE(param->kctx_len);
req_ctx->hctx_wr.imm = (transhdr_len + param->bfr_len +
param->sg_len) <= SGE_MAX_WR_LEN;
@@ -2438,6 +2435,7 @@ static struct sk_buff *create_authenc_wr(struct aead_request *req,
{
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct chcr_context *ctx = a_ctx(tfm);
+ struct uld_ctx *u_ctx = ULD_CTX(ctx);
struct chcr_aead_ctx *aeadctx = AEAD_CTX(ctx);
struct chcr_authenc_ctx *actx = AUTHENC_CTX(aeadctx);
struct chcr_aead_reqctx *reqctx = aead_request_ctx(req);
@@ -2457,6 +2455,7 @@ static struct sk_buff *create_authenc_wr(struct aead_request *req,
struct adapter *adap = padap(ctx->dev);
unsigned int rx_channel_id = reqctx->rxqidx / ctx->rxq_perchan;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[rx_channel_id]);
if (req->cryptlen == 0)
return NULL;
@@ -2710,9 +2709,11 @@ void chcr_add_aead_dst_ent(struct aead_request *req,
struct dsgl_walk dsgl_walk;
unsigned int authsize = crypto_aead_authsize(tfm);
struct chcr_context *ctx = a_ctx(tfm);
+ struct uld_ctx *u_ctx = ULD_CTX(ctx);
u32 temp;
unsigned int rx_channel_id = reqctx->rxqidx / ctx->rxq_perchan;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[rx_channel_id]);
dsgl_walk_init(&dsgl_walk, phys_cpl);
dsgl_walk_add_page(&dsgl_walk, IV + reqctx->b0_len, reqctx->iv_dma);
temp = req->assoclen + req->cryptlen +
@@ -2752,9 +2753,11 @@ void chcr_add_cipher_dst_ent(struct skcipher_request *req,
struct chcr_skcipher_req_ctx *reqctx = skcipher_request_ctx(req);
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(wrparam->req);
struct chcr_context *ctx = c_ctx(tfm);
+ struct uld_ctx *u_ctx = ULD_CTX(ctx);
struct dsgl_walk dsgl_walk;
unsigned int rx_channel_id = reqctx->rxqidx / ctx->rxq_perchan;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[rx_channel_id]);
dsgl_walk_init(&dsgl_walk, phys_cpl);
dsgl_walk_add_sg(&dsgl_walk, reqctx->dstsg, wrparam->bytes,
reqctx->dst_ofst);
@@ -2958,6 +2961,7 @@ static void fill_sec_cpl_for_aead(struct cpl_tx_sec_pdu *sec_cpl,
{
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct chcr_context *ctx = a_ctx(tfm);
+ struct uld_ctx *u_ctx = ULD_CTX(ctx);
struct chcr_aead_ctx *aeadctx = AEAD_CTX(ctx);
struct chcr_aead_reqctx *reqctx = aead_request_ctx(req);
unsigned int cipher_mode = CHCR_SCMD_CIPHER_MODE_AES_CCM;
@@ -2967,6 +2971,8 @@ static void fill_sec_cpl_for_aead(struct cpl_tx_sec_pdu *sec_cpl,
unsigned int tag_offset = 0, auth_offset = 0;
unsigned int assoclen;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[rx_channel_id]);
+
if (get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_RFC4309)
assoclen = req->assoclen - 8;
else
@@ -3127,6 +3133,7 @@ static struct sk_buff *create_gcm_wr(struct aead_request *req,
{
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
struct chcr_context *ctx = a_ctx(tfm);
+ struct uld_ctx *u_ctx = ULD_CTX(ctx);
struct chcr_aead_ctx *aeadctx = AEAD_CTX(ctx);
struct chcr_aead_reqctx *reqctx = aead_request_ctx(req);
struct sk_buff *skb = NULL;
@@ -3143,6 +3150,7 @@ static struct sk_buff *create_gcm_wr(struct aead_request *req,
struct adapter *adap = padap(ctx->dev);
unsigned int rx_channel_id = reqctx->rxqidx / ctx->rxq_perchan;
+ rx_channel_id = cxgb4_port_e2cchan(u_ctx->lldi.ports[rx_channel_id]);
if (get_aead_subtype(tfm) == CRYPTO_ALG_SUB_TYPE_AEAD_RFC4106)
assoclen = req->assoclen - 8;
diff --git a/drivers/crypto/chelsio/chcr_core.c b/drivers/crypto/chelsio/chcr_core.c
index f91f9d762a45..39c70e6255f9 100644
--- a/drivers/crypto/chelsio/chcr_core.c
+++ b/drivers/crypto/chelsio/chcr_core.c
@@ -1,4 +1,4 @@
-/**
+/*
* This file is part of the Chelsio T4/T5/T6 Ethernet driver for Linux.
*
* Copyright (C) 2011-2016 Chelsio Communications. All rights reserved.
@@ -184,7 +184,7 @@ static void *chcr_uld_add(const struct cxgb4_lld_info *lld)
struct uld_ctx *u_ctx;
/* Create the device and add it in the device list */
- pr_info_once("%s - version %s\n", DRV_DESC, DRV_VERSION);
+ pr_info_once("%s\n", DRV_DESC);
if (!(lld->ulp_crypto & ULP_CRYPTO_LOOKASIDE))
return ERR_PTR(-EOPNOTSUPP);
@@ -309,4 +309,3 @@ module_exit(chcr_crypto_exit);
MODULE_DESCRIPTION("Crypto Co-processor for Chelsio Terminator cards.");
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Chelsio Communications");
-MODULE_VERSION(DRV_VERSION);
diff --git a/drivers/crypto/chelsio/chcr_core.h b/drivers/crypto/chelsio/chcr_core.h
index b02f981e7c32..f7c8bb95a71b 100644
--- a/drivers/crypto/chelsio/chcr_core.h
+++ b/drivers/crypto/chelsio/chcr_core.h
@@ -44,7 +44,6 @@
#include "cxgb4_uld.h"
#define DRV_MODULE_NAME "chcr"
-#define DRV_VERSION "1.0.0.0-ko"
#define DRV_DESC "Chelsio T6 Crypto Co-processor Driver"
#define MAX_PENDING_REQ_TO_HW 20
diff --git a/drivers/crypto/geode-aes.c b/drivers/crypto/geode-aes.c
index 4ee010f39912..fa5a9f207bc9 100644
--- a/drivers/crypto/geode-aes.c
+++ b/drivers/crypto/geode-aes.c
@@ -21,7 +21,7 @@
/* Static structures */
static void __iomem *_iobase;
-static spinlock_t lock;
+static DEFINE_SPINLOCK(lock);
/* Write a 128 bit field (either a writable key or IV) */
static inline void
@@ -383,8 +383,6 @@ static int geode_aes_probe(struct pci_dev *dev, const struct pci_device_id *id)
goto erequest;
}
- spin_lock_init(&lock);
-
/* Clear any pending activity */
iowrite32(AES_INTR_PENDING | AES_INTR_MASK, _iobase + AES_INTR_REG);
diff --git a/drivers/crypto/hisilicon/Kconfig b/drivers/crypto/hisilicon/Kconfig
index 843192666dc3..e572f9982d4e 100644
--- a/drivers/crypto/hisilicon/Kconfig
+++ b/drivers/crypto/hisilicon/Kconfig
@@ -68,6 +68,8 @@ config CRYPTO_DEV_HISI_HPRE
select CRYPTO_DEV_HISI_QM
select CRYPTO_DH
select CRYPTO_RSA
+ select CRYPTO_CURVE25519
+ select CRYPTO_ECDH
help
Support for HiSilicon HPRE(High Performance RSA Engine)
accelerator, which can accelerate RSA and DH algorithms.
diff --git a/drivers/crypto/hisilicon/hpre/hpre.h b/drivers/crypto/hisilicon/hpre/hpre.h
index 181c109b19f7..e0b4a1982ee9 100644
--- a/drivers/crypto/hisilicon/hpre/hpre.h
+++ b/drivers/crypto/hisilicon/hpre/hpre.h
@@ -10,6 +10,14 @@
#define HPRE_PF_DEF_Q_NUM 64
#define HPRE_PF_DEF_Q_BASE 0
+/*
+ * type used in qm sqc DW6.
+ * 0 - Algorithm which has been supported in V2, like RSA, DH and so on;
+ * 1 - ECC algorithm in V3.
+ */
+#define HPRE_V2_ALG_TYPE 0
+#define HPRE_V3_ECC_ALG_TYPE 1
+
enum {
HPRE_CLUSTER0,
HPRE_CLUSTER1,
@@ -18,7 +26,6 @@ enum {
};
enum hpre_ctrl_dbgfs_file {
- HPRE_CURRENT_QM,
HPRE_CLEAR_ENABLE,
HPRE_CLUSTER_CTRL,
HPRE_DEBUG_FILE_NUM,
@@ -75,6 +82,9 @@ enum hpre_alg_type {
HPRE_ALG_KG_CRT = 0x3,
HPRE_ALG_DH_G2 = 0x4,
HPRE_ALG_DH = 0x5,
+ HPRE_ALG_ECC_MUL = 0xD,
+ /* shared by x25519 and x448, but x448 is not supported now */
+ HPRE_ALG_CURVE25519_MUL = 0x10,
};
struct hpre_sqe {
@@ -92,8 +102,8 @@ struct hpre_sqe {
__le32 rsvd1[_HPRE_SQE_ALIGN_EXT];
};
-struct hisi_qp *hpre_create_qp(void);
-int hpre_algs_register(void);
-void hpre_algs_unregister(void);
+struct hisi_qp *hpre_create_qp(u8 type);
+int hpre_algs_register(struct hisi_qm *qm);
+void hpre_algs_unregister(struct hisi_qm *qm);
#endif
diff --git a/drivers/crypto/hisilicon/hpre/hpre_crypto.c b/drivers/crypto/hisilicon/hpre/hpre_crypto.c
index a87f9904087a..a380087c83f7 100644
--- a/drivers/crypto/hisilicon/hpre/hpre_crypto.c
+++ b/drivers/crypto/hisilicon/hpre/hpre_crypto.c
@@ -1,7 +1,10 @@
// SPDX-License-Identifier: GPL-2.0
/* Copyright (c) 2019 HiSilicon Limited. */
#include <crypto/akcipher.h>
+#include <crypto/curve25519.h>
#include <crypto/dh.h>
+#include <crypto/ecc_curve.h>
+#include <crypto/ecdh.h>
#include <crypto/internal/akcipher.h>
#include <crypto/internal/kpp.h>
#include <crypto/internal/rsa.h>
@@ -36,6 +39,13 @@ struct hpre_ctx;
#define HPRE_DFX_SEC_TO_US 1000000
#define HPRE_DFX_US_TO_NS 1000
+/* size in bytes of the n prime */
+#define HPRE_ECC_NIST_P192_N_SIZE 24
+#define HPRE_ECC_NIST_P256_N_SIZE 32
+
+/* size in bytes */
+#define HPRE_ECC_HW256_KSZ_B 32
+
typedef void (*hpre_cb)(struct hpre_ctx *ctx, void *sqe);
struct hpre_rsa_ctx {
@@ -61,14 +71,35 @@ struct hpre_dh_ctx {
* else if base if the counterpart public key we
* compute the shared secret
* ZZ = yb^xa mod p; [RFC2631 sec 2.1.1]
+ * low address: d--->n, please refer to Hisilicon HPRE UM
*/
- char *xa_p; /* low address: d--->n, please refer to Hisilicon HPRE UM */
+ char *xa_p;
dma_addr_t dma_xa_p;
char *g; /* m */
dma_addr_t dma_g;
};
+struct hpre_ecdh_ctx {
+ /* low address: p->a->k->b */
+ unsigned char *p;
+ dma_addr_t dma_p;
+
+ /* low address: x->y */
+ unsigned char *g;
+ dma_addr_t dma_g;
+};
+
+struct hpre_curve25519_ctx {
+ /* low address: p->a->k */
+ unsigned char *p;
+ dma_addr_t dma_p;
+
+ /* gx coordinate */
+ unsigned char *g;
+ dma_addr_t dma_g;
+};
+
struct hpre_ctx {
struct hisi_qp *qp;
struct hpre_asym_request **req_list;
@@ -80,7 +111,11 @@ struct hpre_ctx {
union {
struct hpre_rsa_ctx rsa;
struct hpre_dh_ctx dh;
+ struct hpre_ecdh_ctx ecdh;
+ struct hpre_curve25519_ctx curve25519;
};
+ /* for ecc algorithms */
+ unsigned int curve_id;
};
struct hpre_asym_request {
@@ -91,6 +126,8 @@ struct hpre_asym_request {
union {
struct akcipher_request *rsa;
struct kpp_request *dh;
+ struct kpp_request *ecdh;
+ struct kpp_request *curve25519;
} areq;
int err;
int req_id;
@@ -152,12 +189,12 @@ static void hpre_rm_req_from_ctx(struct hpre_asym_request *hpre_req)
}
}
-static struct hisi_qp *hpre_get_qp_and_start(void)
+static struct hisi_qp *hpre_get_qp_and_start(u8 type)
{
struct hisi_qp *qp;
int ret;
- qp = hpre_create_qp();
+ qp = hpre_create_qp(type);
if (!qp) {
pr_err("Can not create hpre qp!\n");
return ERR_PTR(-ENODEV);
@@ -261,8 +298,6 @@ static void hpre_hw_data_clr_all(struct hpre_ctx *ctx,
dma_addr_t tmp;
tmp = le64_to_cpu(sqe->in);
- if (unlikely(!tmp))
- return;
if (src) {
if (req->src)
@@ -272,8 +307,6 @@ static void hpre_hw_data_clr_all(struct hpre_ctx *ctx,
}
tmp = le64_to_cpu(sqe->out);
- if (unlikely(!tmp))
- return;
if (req->dst) {
if (dst)
@@ -288,13 +321,16 @@ static void hpre_hw_data_clr_all(struct hpre_ctx *ctx,
static int hpre_alg_res_post_hf(struct hpre_ctx *ctx, struct hpre_sqe *sqe,
void **kreq)
{
+ struct device *dev = HPRE_DEV(ctx);
struct hpre_asym_request *req;
- int err, id, done;
+ unsigned int err, done, alg;
+ int id;
#define HPRE_NO_HW_ERR 0
#define HPRE_HW_TASK_DONE 3
#define HREE_HW_ERR_MASK 0x7ff
#define HREE_SQE_DONE_MASK 0x3
+#define HREE_ALG_TYPE_MASK 0x1f
id = (int)le16_to_cpu(sqe->tag);
req = ctx->req_list[id];
hpre_rm_req_from_ctx(req);
@@ -307,7 +343,11 @@ static int hpre_alg_res_post_hf(struct hpre_ctx *ctx, struct hpre_sqe *sqe,
HREE_SQE_DONE_MASK;
if (likely(err == HPRE_NO_HW_ERR && done == HPRE_HW_TASK_DONE))
- return 0;
+ return 0;
+
+ alg = le32_to_cpu(sqe->dw0) & HREE_ALG_TYPE_MASK;
+ dev_err_ratelimited(dev, "alg[0x%x] error: done[0x%x], etype[0x%x]\n",
+ alg, done, err);
return -EINVAL;
}
@@ -413,7 +453,6 @@ static void hpre_alg_cb(struct hisi_qp *qp, void *resp)
struct hpre_sqe *sqe = resp;
struct hpre_asym_request *req = ctx->req_list[le16_to_cpu(sqe->tag)];
-
if (unlikely(!req)) {
atomic64_inc(&dfx[HPRE_INVALID_REQ_CNT].value);
return;
@@ -422,18 +461,29 @@ static void hpre_alg_cb(struct hisi_qp *qp, void *resp)
req->cb(ctx, resp);
}
-static int hpre_ctx_init(struct hpre_ctx *ctx)
+static void hpre_stop_qp_and_put(struct hisi_qp *qp)
+{
+ hisi_qm_stop_qp(qp);
+ hisi_qm_free_qps(&qp, 1);
+}
+
+static int hpre_ctx_init(struct hpre_ctx *ctx, u8 type)
{
struct hisi_qp *qp;
+ int ret;
- qp = hpre_get_qp_and_start();
+ qp = hpre_get_qp_and_start(type);
if (IS_ERR(qp))
return PTR_ERR(qp);
qp->qp_ctx = ctx;
qp->req_cb = hpre_alg_cb;
- return hpre_ctx_set(ctx, qp, QM_Q_DEPTH);
+ ret = hpre_ctx_set(ctx, qp, QM_Q_DEPTH);
+ if (ret)
+ hpre_stop_qp_and_put(qp);
+
+ return ret;
}
static int hpre_msg_request_set(struct hpre_ctx *ctx, void *req, bool is_rsa)
@@ -510,7 +560,6 @@ static int hpre_send(struct hpre_ctx *ctx, struct hpre_sqe *msg)
return ret;
}
-#ifdef CONFIG_CRYPTO_DH
static int hpre_dh_compute_value(struct kpp_request *req)
{
struct crypto_kpp *tfm = crypto_kpp_reqtfm(req);
@@ -674,7 +723,7 @@ static int hpre_dh_init_tfm(struct crypto_kpp *tfm)
{
struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
- return hpre_ctx_init(ctx);
+ return hpre_ctx_init(ctx, HPRE_V2_ALG_TYPE);
}
static void hpre_dh_exit_tfm(struct crypto_kpp *tfm)
@@ -683,7 +732,6 @@ static void hpre_dh_exit_tfm(struct crypto_kpp *tfm)
hpre_dh_clear_ctx(ctx, true);
}
-#endif
static void hpre_rsa_drop_leading_zeros(const char **ptr, size_t *len)
{
@@ -1100,7 +1148,7 @@ static int hpre_rsa_init_tfm(struct crypto_akcipher *tfm)
return PTR_ERR(ctx->rsa.soft_tfm);
}
- ret = hpre_ctx_init(ctx);
+ ret = hpre_ctx_init(ctx, HPRE_V2_ALG_TYPE);
if (ret)
crypto_free_akcipher(ctx->rsa.soft_tfm);
@@ -1115,6 +1163,734 @@ static void hpre_rsa_exit_tfm(struct crypto_akcipher *tfm)
crypto_free_akcipher(ctx->rsa.soft_tfm);
}
+static void hpre_key_to_big_end(u8 *data, int len)
+{
+ int i, j;
+ u8 tmp;
+
+ for (i = 0; i < len / 2; i++) {
+ j = len - i - 1;
+ tmp = data[j];
+ data[j] = data[i];
+ data[i] = tmp;
+ }
+}
+
+static void hpre_ecc_clear_ctx(struct hpre_ctx *ctx, bool is_clear_all,
+ bool is_ecdh)
+{
+ struct device *dev = HPRE_DEV(ctx);
+ unsigned int sz = ctx->key_sz;
+ unsigned int shift = sz << 1;
+
+ if (is_clear_all)
+ hisi_qm_stop_qp(ctx->qp);
+
+ if (is_ecdh && ctx->ecdh.p) {
+ /* ecdh: p->a->k->b */
+ memzero_explicit(ctx->ecdh.p + shift, sz);
+ dma_free_coherent(dev, sz << 3, ctx->ecdh.p, ctx->ecdh.dma_p);
+ ctx->ecdh.p = NULL;
+ } else if (!is_ecdh && ctx->curve25519.p) {
+ /* curve25519: p->a->k */
+ memzero_explicit(ctx->curve25519.p + shift, sz);
+ dma_free_coherent(dev, sz << 2, ctx->curve25519.p,
+ ctx->curve25519.dma_p);
+ ctx->curve25519.p = NULL;
+ }
+
+ hpre_ctx_clear(ctx, is_clear_all);
+}
+
+static unsigned int hpre_ecdh_supported_curve(unsigned short id)
+{
+ switch (id) {
+ case ECC_CURVE_NIST_P192:
+ case ECC_CURVE_NIST_P256:
+ return HPRE_ECC_HW256_KSZ_B;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+static void fill_curve_param(void *addr, u64 *param, unsigned int cur_sz, u8 ndigits)
+{
+ unsigned int sz = cur_sz - (ndigits - 1) * sizeof(u64);
+ u8 i = 0;
+
+ while (i < ndigits - 1) {
+ memcpy(addr + sizeof(u64) * i, &param[i], sizeof(u64));
+ i++;
+ }
+
+ memcpy(addr + sizeof(u64) * i, &param[ndigits - 1], sz);
+ hpre_key_to_big_end((u8 *)addr, cur_sz);
+}
+
+static int hpre_ecdh_fill_curve(struct hpre_ctx *ctx, struct ecdh *params,
+ unsigned int cur_sz)
+{
+ unsigned int shifta = ctx->key_sz << 1;
+ unsigned int shiftb = ctx->key_sz << 2;
+ void *p = ctx->ecdh.p + ctx->key_sz - cur_sz;
+ void *a = ctx->ecdh.p + shifta - cur_sz;
+ void *b = ctx->ecdh.p + shiftb - cur_sz;
+ void *x = ctx->ecdh.g + ctx->key_sz - cur_sz;
+ void *y = ctx->ecdh.g + shifta - cur_sz;
+ const struct ecc_curve *curve = ecc_get_curve(ctx->curve_id);
+ char *n;
+
+ if (unlikely(!curve))
+ return -EINVAL;
+
+ n = kzalloc(ctx->key_sz, GFP_KERNEL);
+ if (!n)
+ return -ENOMEM;
+
+ fill_curve_param(p, curve->p, cur_sz, curve->g.ndigits);
+ fill_curve_param(a, curve->a, cur_sz, curve->g.ndigits);
+ fill_curve_param(b, curve->b, cur_sz, curve->g.ndigits);
+ fill_curve_param(x, curve->g.x, cur_sz, curve->g.ndigits);
+ fill_curve_param(y, curve->g.y, cur_sz, curve->g.ndigits);
+ fill_curve_param(n, curve->n, cur_sz, curve->g.ndigits);
+
+ if (params->key_size == cur_sz && memcmp(params->key, n, cur_sz) >= 0) {
+ kfree(n);
+ return -EINVAL;
+ }
+
+ kfree(n);
+ return 0;
+}
+
+static unsigned int hpre_ecdh_get_curvesz(unsigned short id)
+{
+ switch (id) {
+ case ECC_CURVE_NIST_P192:
+ return HPRE_ECC_NIST_P192_N_SIZE;
+ case ECC_CURVE_NIST_P256:
+ return HPRE_ECC_NIST_P256_N_SIZE;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
+static int hpre_ecdh_set_param(struct hpre_ctx *ctx, struct ecdh *params)
+{
+ struct device *dev = HPRE_DEV(ctx);
+ unsigned int sz, shift, curve_sz;
+ int ret;
+
+ ctx->key_sz = hpre_ecdh_supported_curve(ctx->curve_id);
+ if (!ctx->key_sz)
+ return -EINVAL;
+
+ curve_sz = hpre_ecdh_get_curvesz(ctx->curve_id);
+ if (!curve_sz || params->key_size > curve_sz)
+ return -EINVAL;
+
+ sz = ctx->key_sz;
+
+ if (!ctx->ecdh.p) {
+ ctx->ecdh.p = dma_alloc_coherent(dev, sz << 3, &ctx->ecdh.dma_p,
+ GFP_KERNEL);
+ if (!ctx->ecdh.p)
+ return -ENOMEM;
+ }
+
+ shift = sz << 2;
+ ctx->ecdh.g = ctx->ecdh.p + shift;
+ ctx->ecdh.dma_g = ctx->ecdh.dma_p + shift;
+
+ ret = hpre_ecdh_fill_curve(ctx, params, curve_sz);
+ if (ret) {
+ dev_err(dev, "failed to fill curve_param, ret = %d!\n", ret);
+ dma_free_coherent(dev, sz << 3, ctx->ecdh.p, ctx->ecdh.dma_p);
+ ctx->ecdh.p = NULL;
+ return ret;
+ }
+
+ return 0;
+}
+
+static bool hpre_key_is_zero(char *key, unsigned short key_sz)
+{
+ int i;
+
+ for (i = 0; i < key_sz; i++)
+ if (key[i])
+ return false;
+
+ return true;
+}
+
+static int hpre_ecdh_set_secret(struct crypto_kpp *tfm, const void *buf,
+ unsigned int len)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+ struct device *dev = HPRE_DEV(ctx);
+ unsigned int sz, sz_shift;
+ struct ecdh params;
+ int ret;
+
+ if (crypto_ecdh_decode_key(buf, len, &params) < 0) {
+ dev_err(dev, "failed to decode ecdh key!\n");
+ return -EINVAL;
+ }
+
+ if (hpre_key_is_zero(params.key, params.key_size)) {
+ dev_err(dev, "Invalid hpre key!\n");
+ return -EINVAL;
+ }
+
+ hpre_ecc_clear_ctx(ctx, false, true);
+
+ ret = hpre_ecdh_set_param(ctx, &params);
+ if (ret < 0) {
+ dev_err(dev, "failed to set hpre param, ret = %d!\n", ret);
+ return ret;
+ }
+
+ sz = ctx->key_sz;
+ sz_shift = (sz << 1) + sz - params.key_size;
+ memcpy(ctx->ecdh.p + sz_shift, params.key, params.key_size);
+
+ return 0;
+}
+
+static void hpre_ecdh_hw_data_clr_all(struct hpre_ctx *ctx,
+ struct hpre_asym_request *req,
+ struct scatterlist *dst,
+ struct scatterlist *src)
+{
+ struct device *dev = HPRE_DEV(ctx);
+ struct hpre_sqe *sqe = &req->req;
+ dma_addr_t dma;
+
+ dma = le64_to_cpu(sqe->in);
+
+ if (src && req->src)
+ dma_free_coherent(dev, ctx->key_sz << 2, req->src, dma);
+
+ dma = le64_to_cpu(sqe->out);
+
+ if (req->dst)
+ dma_free_coherent(dev, ctx->key_sz << 1, req->dst, dma);
+ if (dst)
+ dma_unmap_single(dev, dma, ctx->key_sz << 1, DMA_FROM_DEVICE);
+}
+
+static void hpre_ecdh_cb(struct hpre_ctx *ctx, void *resp)
+{
+ unsigned int curve_sz = hpre_ecdh_get_curvesz(ctx->curve_id);
+ struct hpre_dfx *dfx = ctx->hpre->debug.dfx;
+ struct hpre_asym_request *req = NULL;
+ struct kpp_request *areq;
+ u64 overtime_thrhld;
+ char *p;
+ int ret;
+
+ ret = hpre_alg_res_post_hf(ctx, resp, (void **)&req);
+ areq = req->areq.ecdh;
+ areq->dst_len = ctx->key_sz << 1;
+
+ overtime_thrhld = atomic64_read(&dfx[HPRE_OVERTIME_THRHLD].value);
+ if (overtime_thrhld && hpre_is_bd_timeout(req, overtime_thrhld))
+ atomic64_inc(&dfx[HPRE_OVER_THRHLD_CNT].value);
+
+ p = sg_virt(areq->dst);
+ memmove(p, p + ctx->key_sz - curve_sz, curve_sz);
+ memmove(p + curve_sz, p + areq->dst_len - curve_sz, curve_sz);
+
+ hpre_ecdh_hw_data_clr_all(ctx, req, areq->dst, areq->src);
+ kpp_request_complete(areq, ret);
+
+ atomic64_inc(&dfx[HPRE_RECV_CNT].value);
+}
+
+static int hpre_ecdh_msg_request_set(struct hpre_ctx *ctx,
+ struct kpp_request *req)
+{
+ struct hpre_asym_request *h_req;
+ struct hpre_sqe *msg;
+ int req_id;
+ void *tmp;
+
+ if (req->dst_len < ctx->key_sz << 1) {
+ req->dst_len = ctx->key_sz << 1;
+ return -EINVAL;
+ }
+
+ tmp = kpp_request_ctx(req);
+ h_req = PTR_ALIGN(tmp, HPRE_ALIGN_SZ);
+ h_req->cb = hpre_ecdh_cb;
+ h_req->areq.ecdh = req;
+ msg = &h_req->req;
+ memset(msg, 0, sizeof(*msg));
+ msg->key = cpu_to_le64(ctx->ecdh.dma_p);
+
+ msg->dw0 |= cpu_to_le32(0x1U << HPRE_SQE_DONE_SHIFT);
+ msg->task_len1 = (ctx->key_sz >> HPRE_BITS_2_BYTES_SHIFT) - 1;
+ h_req->ctx = ctx;
+
+ req_id = hpre_add_req_to_ctx(h_req);
+ if (req_id < 0)
+ return -EBUSY;
+
+ msg->tag = cpu_to_le16((u16)req_id);
+ return 0;
+}
+
+static int hpre_ecdh_src_data_init(struct hpre_asym_request *hpre_req,
+ struct scatterlist *data, unsigned int len)
+{
+ struct hpre_sqe *msg = &hpre_req->req;
+ struct hpre_ctx *ctx = hpre_req->ctx;
+ struct device *dev = HPRE_DEV(ctx);
+ unsigned int tmpshift;
+ dma_addr_t dma = 0;
+ void *ptr;
+ int shift;
+
+ /* Src_data include gx and gy. */
+ shift = ctx->key_sz - (len >> 1);
+ if (unlikely(shift < 0))
+ return -EINVAL;
+
+ ptr = dma_alloc_coherent(dev, ctx->key_sz << 2, &dma, GFP_KERNEL);
+ if (unlikely(!ptr))
+ return -ENOMEM;
+
+ tmpshift = ctx->key_sz << 1;
+ scatterwalk_map_and_copy(ptr + tmpshift, data, 0, len, 0);
+ memcpy(ptr + shift, ptr + tmpshift, len >> 1);
+ memcpy(ptr + ctx->key_sz + shift, ptr + tmpshift + (len >> 1), len >> 1);
+
+ hpre_req->src = ptr;
+ msg->in = cpu_to_le64(dma);
+ return 0;
+}
+
+static int hpre_ecdh_dst_data_init(struct hpre_asym_request *hpre_req,
+ struct scatterlist *data, unsigned int len)
+{
+ struct hpre_sqe *msg = &hpre_req->req;
+ struct hpre_ctx *ctx = hpre_req->ctx;
+ struct device *dev = HPRE_DEV(ctx);
+ dma_addr_t dma = 0;
+
+ if (unlikely(!data || !sg_is_last(data) || len != ctx->key_sz << 1)) {
+ dev_err(dev, "data or data length is illegal!\n");
+ return -EINVAL;
+ }
+
+ hpre_req->dst = NULL;
+ dma = dma_map_single(dev, sg_virt(data), len, DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(dev, dma))) {
+ dev_err(dev, "dma map data err!\n");
+ return -ENOMEM;
+ }
+
+ msg->out = cpu_to_le64(dma);
+ return 0;
+}
+
+static int hpre_ecdh_compute_value(struct kpp_request *req)
+{
+ struct crypto_kpp *tfm = crypto_kpp_reqtfm(req);
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+ struct device *dev = HPRE_DEV(ctx);
+ void *tmp = kpp_request_ctx(req);
+ struct hpre_asym_request *hpre_req = PTR_ALIGN(tmp, HPRE_ALIGN_SZ);
+ struct hpre_sqe *msg = &hpre_req->req;
+ int ret;
+
+ ret = hpre_ecdh_msg_request_set(ctx, req);
+ if (unlikely(ret)) {
+ dev_err(dev, "failed to set ecdh request, ret = %d!\n", ret);
+ return ret;
+ }
+
+ if (req->src) {
+ ret = hpre_ecdh_src_data_init(hpre_req, req->src, req->src_len);
+ if (unlikely(ret)) {
+ dev_err(dev, "failed to init src data, ret = %d!\n", ret);
+ goto clear_all;
+ }
+ } else {
+ msg->in = cpu_to_le64(ctx->ecdh.dma_g);
+ }
+
+ ret = hpre_ecdh_dst_data_init(hpre_req, req->dst, req->dst_len);
+ if (unlikely(ret)) {
+ dev_err(dev, "failed to init dst data, ret = %d!\n", ret);
+ goto clear_all;
+ }
+
+ msg->dw0 = cpu_to_le32(le32_to_cpu(msg->dw0) | HPRE_ALG_ECC_MUL);
+ ret = hpre_send(ctx, msg);
+ if (likely(!ret))
+ return -EINPROGRESS;
+
+clear_all:
+ hpre_rm_req_from_ctx(hpre_req);
+ hpre_ecdh_hw_data_clr_all(ctx, hpre_req, req->dst, req->src);
+ return ret;
+}
+
+static unsigned int hpre_ecdh_max_size(struct crypto_kpp *tfm)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+
+ /* max size is the pub_key_size, include x and y */
+ return ctx->key_sz << 1;
+}
+
+static int hpre_ecdh_nist_p192_init_tfm(struct crypto_kpp *tfm)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+
+ ctx->curve_id = ECC_CURVE_NIST_P192;
+
+ return hpre_ctx_init(ctx, HPRE_V3_ECC_ALG_TYPE);
+}
+
+static int hpre_ecdh_nist_p256_init_tfm(struct crypto_kpp *tfm)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+
+ ctx->curve_id = ECC_CURVE_NIST_P256;
+
+ return hpre_ctx_init(ctx, HPRE_V3_ECC_ALG_TYPE);
+}
+
+static void hpre_ecdh_exit_tfm(struct crypto_kpp *tfm)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+
+ hpre_ecc_clear_ctx(ctx, true, true);
+}
+
+static void hpre_curve25519_fill_curve(struct hpre_ctx *ctx, const void *buf,
+ unsigned int len)
+{
+ u8 secret[CURVE25519_KEY_SIZE] = { 0 };
+ unsigned int sz = ctx->key_sz;
+ const struct ecc_curve *curve;
+ unsigned int shift = sz << 1;
+ void *p;
+
+ /*
+ * The key from 'buf' is in little-endian, we should preprocess it as
+ * the description in rfc7748: "k[0] &= 248, k[31] &= 127, k[31] |= 64",
+ * then convert it to big endian. Only in this way, the result can be
+ * the same as the software curve-25519 that exists in crypto.
+ */
+ memcpy(secret, buf, len);
+ curve25519_clamp_secret(secret);
+ hpre_key_to_big_end(secret, CURVE25519_KEY_SIZE);
+
+ p = ctx->curve25519.p + sz - len;
+
+ curve = ecc_get_curve25519();
+
+ /* fill curve parameters */
+ fill_curve_param(p, curve->p, len, curve->g.ndigits);
+ fill_curve_param(p + sz, curve->a, len, curve->g.ndigits);
+ memcpy(p + shift, secret, len);
+ fill_curve_param(p + shift + sz, curve->g.x, len, curve->g.ndigits);
+ memzero_explicit(secret, CURVE25519_KEY_SIZE);
+}
+
+static int hpre_curve25519_set_param(struct hpre_ctx *ctx, const void *buf,
+ unsigned int len)
+{
+ struct device *dev = HPRE_DEV(ctx);
+ unsigned int sz = ctx->key_sz;
+ unsigned int shift = sz << 1;
+
+ /* p->a->k->gx */
+ if (!ctx->curve25519.p) {
+ ctx->curve25519.p = dma_alloc_coherent(dev, sz << 2,
+ &ctx->curve25519.dma_p,
+ GFP_KERNEL);
+ if (!ctx->curve25519.p)
+ return -ENOMEM;
+ }
+
+ ctx->curve25519.g = ctx->curve25519.p + shift + sz;
+ ctx->curve25519.dma_g = ctx->curve25519.dma_p + shift + sz;
+
+ hpre_curve25519_fill_curve(ctx, buf, len);
+
+ return 0;
+}
+
+static int hpre_curve25519_set_secret(struct crypto_kpp *tfm, const void *buf,
+ unsigned int len)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+ struct device *dev = HPRE_DEV(ctx);
+ int ret = -EINVAL;
+
+ if (len != CURVE25519_KEY_SIZE ||
+ !crypto_memneq(buf, curve25519_null_point, CURVE25519_KEY_SIZE)) {
+ dev_err(dev, "key is null or key len is not 32bytes!\n");
+ return ret;
+ }
+
+ /* Free old secret if any */
+ hpre_ecc_clear_ctx(ctx, false, false);
+
+ ctx->key_sz = CURVE25519_KEY_SIZE;
+ ret = hpre_curve25519_set_param(ctx, buf, CURVE25519_KEY_SIZE);
+ if (ret) {
+ dev_err(dev, "failed to set curve25519 param, ret = %d!\n", ret);
+ hpre_ecc_clear_ctx(ctx, false, false);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void hpre_curve25519_hw_data_clr_all(struct hpre_ctx *ctx,
+ struct hpre_asym_request *req,
+ struct scatterlist *dst,
+ struct scatterlist *src)
+{
+ struct device *dev = HPRE_DEV(ctx);
+ struct hpre_sqe *sqe = &req->req;
+ dma_addr_t dma;
+
+ dma = le64_to_cpu(sqe->in);
+
+ if (src && req->src)
+ dma_free_coherent(dev, ctx->key_sz, req->src, dma);
+
+ dma = le64_to_cpu(sqe->out);
+
+ if (req->dst)
+ dma_free_coherent(dev, ctx->key_sz, req->dst, dma);
+ if (dst)
+ dma_unmap_single(dev, dma, ctx->key_sz, DMA_FROM_DEVICE);
+}
+
+static void hpre_curve25519_cb(struct hpre_ctx *ctx, void *resp)
+{
+ struct hpre_dfx *dfx = ctx->hpre->debug.dfx;
+ struct hpre_asym_request *req = NULL;
+ struct kpp_request *areq;
+ u64 overtime_thrhld;
+ int ret;
+
+ ret = hpre_alg_res_post_hf(ctx, resp, (void **)&req);
+ areq = req->areq.curve25519;
+ areq->dst_len = ctx->key_sz;
+
+ overtime_thrhld = atomic64_read(&dfx[HPRE_OVERTIME_THRHLD].value);
+ if (overtime_thrhld && hpre_is_bd_timeout(req, overtime_thrhld))
+ atomic64_inc(&dfx[HPRE_OVER_THRHLD_CNT].value);
+
+ hpre_key_to_big_end(sg_virt(areq->dst), CURVE25519_KEY_SIZE);
+
+ hpre_curve25519_hw_data_clr_all(ctx, req, areq->dst, areq->src);
+ kpp_request_complete(areq, ret);
+
+ atomic64_inc(&dfx[HPRE_RECV_CNT].value);
+}
+
+static int hpre_curve25519_msg_request_set(struct hpre_ctx *ctx,
+ struct kpp_request *req)
+{
+ struct hpre_asym_request *h_req;
+ struct hpre_sqe *msg;
+ int req_id;
+ void *tmp;
+
+ if (unlikely(req->dst_len < ctx->key_sz)) {
+ req->dst_len = ctx->key_sz;
+ return -EINVAL;
+ }
+
+ tmp = kpp_request_ctx(req);
+ h_req = PTR_ALIGN(tmp, HPRE_ALIGN_SZ);
+ h_req->cb = hpre_curve25519_cb;
+ h_req->areq.curve25519 = req;
+ msg = &h_req->req;
+ memset(msg, 0, sizeof(*msg));
+ msg->key = cpu_to_le64(ctx->curve25519.dma_p);
+
+ msg->dw0 |= cpu_to_le32(0x1U << HPRE_SQE_DONE_SHIFT);
+ msg->task_len1 = (ctx->key_sz >> HPRE_BITS_2_BYTES_SHIFT) - 1;
+ h_req->ctx = ctx;
+
+ req_id = hpre_add_req_to_ctx(h_req);
+ if (req_id < 0)
+ return -EBUSY;
+
+ msg->tag = cpu_to_le16((u16)req_id);
+ return 0;
+}
+
+static void hpre_curve25519_src_modulo_p(u8 *ptr)
+{
+ int i;
+
+ for (i = 0; i < CURVE25519_KEY_SIZE - 1; i++)
+ ptr[i] = 0;
+
+ /* The modulus is ptr's last byte minus '0xed'(last byte of p) */
+ ptr[i] -= 0xed;
+}
+
+static int hpre_curve25519_src_init(struct hpre_asym_request *hpre_req,
+ struct scatterlist *data, unsigned int len)
+{
+ struct hpre_sqe *msg = &hpre_req->req;
+ struct hpre_ctx *ctx = hpre_req->ctx;
+ struct device *dev = HPRE_DEV(ctx);
+ u8 p[CURVE25519_KEY_SIZE] = { 0 };
+ const struct ecc_curve *curve;
+ dma_addr_t dma = 0;
+ u8 *ptr;
+
+ if (len != CURVE25519_KEY_SIZE) {
+ dev_err(dev, "sourc_data len is not 32bytes, len = %u!\n", len);
+ return -EINVAL;
+ }
+
+ ptr = dma_alloc_coherent(dev, ctx->key_sz, &dma, GFP_KERNEL);
+ if (unlikely(!ptr))
+ return -ENOMEM;
+
+ scatterwalk_map_and_copy(ptr, data, 0, len, 0);
+
+ if (!crypto_memneq(ptr, curve25519_null_point, CURVE25519_KEY_SIZE)) {
+ dev_err(dev, "gx is null!\n");
+ goto err;
+ }
+
+ /*
+ * Src_data(gx) is in little-endian order, MSB in the final byte should
+ * be masked as described in RFC7748, then transform it to big-endian
+ * form, then hisi_hpre can use the data.
+ */
+ ptr[31] &= 0x7f;
+ hpre_key_to_big_end(ptr, CURVE25519_KEY_SIZE);
+
+ curve = ecc_get_curve25519();
+
+ fill_curve_param(p, curve->p, CURVE25519_KEY_SIZE, curve->g.ndigits);
+
+ /*
+ * When src_data equals (2^255 - 19) ~ (2^255 - 1), it is out of p,
+ * we get its modulus to p, and then use it.
+ */
+ if (memcmp(ptr, p, ctx->key_sz) >= 0)
+ hpre_curve25519_src_modulo_p(ptr);
+
+ hpre_req->src = ptr;
+ msg->in = cpu_to_le64(dma);
+ return 0;
+
+err:
+ dma_free_coherent(dev, ctx->key_sz, ptr, dma);
+ return -EINVAL;
+}
+
+static int hpre_curve25519_dst_init(struct hpre_asym_request *hpre_req,
+ struct scatterlist *data, unsigned int len)
+{
+ struct hpre_sqe *msg = &hpre_req->req;
+ struct hpre_ctx *ctx = hpre_req->ctx;
+ struct device *dev = HPRE_DEV(ctx);
+ dma_addr_t dma = 0;
+
+ if (!data || !sg_is_last(data) || len != ctx->key_sz) {
+ dev_err(dev, "data or data length is illegal!\n");
+ return -EINVAL;
+ }
+
+ hpre_req->dst = NULL;
+ dma = dma_map_single(dev, sg_virt(data), len, DMA_FROM_DEVICE);
+ if (unlikely(dma_mapping_error(dev, dma))) {
+ dev_err(dev, "dma map data err!\n");
+ return -ENOMEM;
+ }
+
+ msg->out = cpu_to_le64(dma);
+ return 0;
+}
+
+static int hpre_curve25519_compute_value(struct kpp_request *req)
+{
+ struct crypto_kpp *tfm = crypto_kpp_reqtfm(req);
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+ struct device *dev = HPRE_DEV(ctx);
+ void *tmp = kpp_request_ctx(req);
+ struct hpre_asym_request *hpre_req = PTR_ALIGN(tmp, HPRE_ALIGN_SZ);
+ struct hpre_sqe *msg = &hpre_req->req;
+ int ret;
+
+ ret = hpre_curve25519_msg_request_set(ctx, req);
+ if (unlikely(ret)) {
+ dev_err(dev, "failed to set curve25519 request, ret = %d!\n", ret);
+ return ret;
+ }
+
+ if (req->src) {
+ ret = hpre_curve25519_src_init(hpre_req, req->src, req->src_len);
+ if (unlikely(ret)) {
+ dev_err(dev, "failed to init src data, ret = %d!\n",
+ ret);
+ goto clear_all;
+ }
+ } else {
+ msg->in = cpu_to_le64(ctx->curve25519.dma_g);
+ }
+
+ ret = hpre_curve25519_dst_init(hpre_req, req->dst, req->dst_len);
+ if (unlikely(ret)) {
+ dev_err(dev, "failed to init dst data, ret = %d!\n", ret);
+ goto clear_all;
+ }
+
+ msg->dw0 = cpu_to_le32(le32_to_cpu(msg->dw0) | HPRE_ALG_CURVE25519_MUL);
+ ret = hpre_send(ctx, msg);
+ if (likely(!ret))
+ return -EINPROGRESS;
+
+clear_all:
+ hpre_rm_req_from_ctx(hpre_req);
+ hpre_curve25519_hw_data_clr_all(ctx, hpre_req, req->dst, req->src);
+ return ret;
+}
+
+static unsigned int hpre_curve25519_max_size(struct crypto_kpp *tfm)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+
+ return ctx->key_sz;
+}
+
+static int hpre_curve25519_init_tfm(struct crypto_kpp *tfm)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+
+ return hpre_ctx_init(ctx, HPRE_V3_ECC_ALG_TYPE);
+}
+
+static void hpre_curve25519_exit_tfm(struct crypto_kpp *tfm)
+{
+ struct hpre_ctx *ctx = kpp_tfm_ctx(tfm);
+
+ hpre_ecc_clear_ctx(ctx, true, false);
+}
+
static struct akcipher_alg rsa = {
.sign = hpre_rsa_dec,
.verify = hpre_rsa_enc,
@@ -1135,7 +1911,6 @@ static struct akcipher_alg rsa = {
},
};
-#ifdef CONFIG_CRYPTO_DH
static struct kpp_alg dh = {
.set_secret = hpre_dh_set_secret,
.generate_public_key = hpre_dh_compute_value,
@@ -1152,9 +1927,83 @@ static struct kpp_alg dh = {
.cra_module = THIS_MODULE,
},
};
-#endif
-int hpre_algs_register(void)
+static struct kpp_alg ecdh_nist_p192 = {
+ .set_secret = hpre_ecdh_set_secret,
+ .generate_public_key = hpre_ecdh_compute_value,
+ .compute_shared_secret = hpre_ecdh_compute_value,
+ .max_size = hpre_ecdh_max_size,
+ .init = hpre_ecdh_nist_p192_init_tfm,
+ .exit = hpre_ecdh_exit_tfm,
+ .reqsize = sizeof(struct hpre_asym_request) + HPRE_ALIGN_SZ,
+ .base = {
+ .cra_ctxsize = sizeof(struct hpre_ctx),
+ .cra_priority = HPRE_CRYPTO_ALG_PRI,
+ .cra_name = "ecdh-nist-p192",
+ .cra_driver_name = "hpre-ecdh",
+ .cra_module = THIS_MODULE,
+ },
+};
+
+static struct kpp_alg ecdh_nist_p256 = {
+ .set_secret = hpre_ecdh_set_secret,
+ .generate_public_key = hpre_ecdh_compute_value,
+ .compute_shared_secret = hpre_ecdh_compute_value,
+ .max_size = hpre_ecdh_max_size,
+ .init = hpre_ecdh_nist_p256_init_tfm,
+ .exit = hpre_ecdh_exit_tfm,
+ .reqsize = sizeof(struct hpre_asym_request) + HPRE_ALIGN_SZ,
+ .base = {
+ .cra_ctxsize = sizeof(struct hpre_ctx),
+ .cra_priority = HPRE_CRYPTO_ALG_PRI,
+ .cra_name = "ecdh-nist-p256",
+ .cra_driver_name = "hpre-ecdh",
+ .cra_module = THIS_MODULE,
+ },
+};
+
+static struct kpp_alg curve25519_alg = {
+ .set_secret = hpre_curve25519_set_secret,
+ .generate_public_key = hpre_curve25519_compute_value,
+ .compute_shared_secret = hpre_curve25519_compute_value,
+ .max_size = hpre_curve25519_max_size,
+ .init = hpre_curve25519_init_tfm,
+ .exit = hpre_curve25519_exit_tfm,
+ .reqsize = sizeof(struct hpre_asym_request) + HPRE_ALIGN_SZ,
+ .base = {
+ .cra_ctxsize = sizeof(struct hpre_ctx),
+ .cra_priority = HPRE_CRYPTO_ALG_PRI,
+ .cra_name = "curve25519",
+ .cra_driver_name = "hpre-curve25519",
+ .cra_module = THIS_MODULE,
+ },
+};
+
+
+static int hpre_register_ecdh(void)
+{
+ int ret;
+
+ ret = crypto_register_kpp(&ecdh_nist_p192);
+ if (ret)
+ return ret;
+
+ ret = crypto_register_kpp(&ecdh_nist_p256);
+ if (ret) {
+ crypto_unregister_kpp(&ecdh_nist_p192);
+ return ret;
+ }
+
+ return 0;
+}
+
+static void hpre_unregister_ecdh(void)
+{
+ crypto_unregister_kpp(&ecdh_nist_p256);
+ crypto_unregister_kpp(&ecdh_nist_p192);
+}
+
+int hpre_algs_register(struct hisi_qm *qm)
{
int ret;
@@ -1162,19 +2011,37 @@ int hpre_algs_register(void)
ret = crypto_register_akcipher(&rsa);
if (ret)
return ret;
-#ifdef CONFIG_CRYPTO_DH
+
ret = crypto_register_kpp(&dh);
if (ret)
- crypto_unregister_akcipher(&rsa);
-#endif
+ goto unreg_rsa;
+
+ if (qm->ver >= QM_HW_V3) {
+ ret = hpre_register_ecdh();
+ if (ret)
+ goto unreg_dh;
+ ret = crypto_register_kpp(&curve25519_alg);
+ if (ret)
+ goto unreg_ecdh;
+ }
+ return 0;
+unreg_ecdh:
+ hpre_unregister_ecdh();
+unreg_dh:
+ crypto_unregister_kpp(&dh);
+unreg_rsa:
+ crypto_unregister_akcipher(&rsa);
return ret;
}
-void hpre_algs_unregister(void)
+void hpre_algs_unregister(struct hisi_qm *qm)
{
- crypto_unregister_akcipher(&rsa);
-#ifdef CONFIG_CRYPTO_DH
+ if (qm->ver >= QM_HW_V3) {
+ crypto_unregister_kpp(&curve25519_alg);
+ hpre_unregister_ecdh();
+ }
+
crypto_unregister_kpp(&dh);
-#endif
+ crypto_unregister_akcipher(&rsa);
}
diff --git a/drivers/crypto/hisilicon/hpre/hpre_main.c b/drivers/crypto/hisilicon/hpre/hpre_main.c
index e7a2c70eb9cf..046bc962c8b2 100644
--- a/drivers/crypto/hisilicon/hpre/hpre_main.c
+++ b/drivers/crypto/hisilicon/hpre/hpre_main.c
@@ -13,7 +13,6 @@
#include <linux/uacce.h>
#include "hpre.h"
-#define HPRE_QUEUE_NUM_V2 1024
#define HPRE_QM_ABNML_INT_MASK 0x100004
#define HPRE_CTRL_CNT_CLR_CE_BIT BIT(0)
#define HPRE_COMM_CNT_CLR_CE 0x0
@@ -119,7 +118,6 @@ static struct hisi_qm_list hpre_devices = {
};
static const char * const hpre_debug_file_name[] = {
- [HPRE_CURRENT_QM] = "current_qm",
[HPRE_CLEAR_ENABLE] = "rdclr_en",
[HPRE_CLUSTER_CTRL] = "cluster_ctrl",
};
@@ -226,41 +224,44 @@ static u32 vfs_num;
module_param_cb(vfs_num, &vfs_num_ops, &vfs_num, 0444);
MODULE_PARM_DESC(vfs_num, "Number of VFs to enable(1-63), 0(default)");
-struct hisi_qp *hpre_create_qp(void)
+struct hisi_qp *hpre_create_qp(u8 type)
{
int node = cpu_to_node(smp_processor_id());
struct hisi_qp *qp = NULL;
int ret;
- ret = hisi_qm_alloc_qps_node(&hpre_devices, 1, 0, node, &qp);
+ if (type != HPRE_V2_ALG_TYPE && type != HPRE_V3_ECC_ALG_TYPE)
+ return NULL;
+
+ /*
+ * type: 0 - RSA/DH. algorithm supported in V2,
+ * 1 - ECC algorithm in V3.
+ */
+ ret = hisi_qm_alloc_qps_node(&hpre_devices, 1, type, node, &qp);
if (!ret)
return qp;
return NULL;
}
-static void hpre_pasid_enable(struct hisi_qm *qm)
+static void hpre_config_pasid(struct hisi_qm *qm)
{
- u32 val;
-
- val = readl_relaxed(qm->io_base + HPRE_DATA_RUSER_CFG);
- val |= BIT(HPRE_PASID_EN_BIT);
- writel_relaxed(val, qm->io_base + HPRE_DATA_RUSER_CFG);
- val = readl_relaxed(qm->io_base + HPRE_DATA_WUSER_CFG);
- val |= BIT(HPRE_PASID_EN_BIT);
- writel_relaxed(val, qm->io_base + HPRE_DATA_WUSER_CFG);
-}
+ u32 val1, val2;
-static void hpre_pasid_disable(struct hisi_qm *qm)
-{
- u32 val;
+ if (qm->ver >= QM_HW_V3)
+ return;
- val = readl_relaxed(qm->io_base + HPRE_DATA_RUSER_CFG);
- val &= ~BIT(HPRE_PASID_EN_BIT);
- writel_relaxed(val, qm->io_base + HPRE_DATA_RUSER_CFG);
- val = readl_relaxed(qm->io_base + HPRE_DATA_WUSER_CFG);
- val &= ~BIT(HPRE_PASID_EN_BIT);
- writel_relaxed(val, qm->io_base + HPRE_DATA_WUSER_CFG);
+ val1 = readl_relaxed(qm->io_base + HPRE_DATA_RUSER_CFG);
+ val2 = readl_relaxed(qm->io_base + HPRE_DATA_WUSER_CFG);
+ if (qm->use_sva) {
+ val1 |= BIT(HPRE_PASID_EN_BIT);
+ val2 |= BIT(HPRE_PASID_EN_BIT);
+ } else {
+ val1 &= ~BIT(HPRE_PASID_EN_BIT);
+ val2 &= ~BIT(HPRE_PASID_EN_BIT);
+ }
+ writel_relaxed(val1, qm->io_base + HPRE_DATA_RUSER_CFG);
+ writel_relaxed(val2, qm->io_base + HPRE_DATA_WUSER_CFG);
}
static int hpre_cfg_by_dsm(struct hisi_qm *qm)
@@ -320,7 +321,7 @@ static int hpre_set_cluster(struct hisi_qm *qm)
}
/*
- * For Kunpeng 920, we shoul disable FLR triggered by hardware (BME/PM/SRIOV).
+ * For Kunpeng 920, we should disable FLR triggered by hardware (BME/PM/SRIOV).
* Or it may stay in D3 state when we bind and unbind hpre quickly,
* as it does FLR triggered by hardware.
*/
@@ -383,15 +384,14 @@ static int hpre_set_user_domain_and_cache(struct hisi_qm *qm)
if (qm->ver == QM_HW_V2) {
ret = hpre_cfg_by_dsm(qm);
if (ret)
- dev_err(dev, "acpi_evaluate_dsm err.\n");
+ return ret;
disable_flr_of_bme(qm);
-
- /* Enable data buffer pasid */
- if (qm->use_sva)
- hpre_pasid_enable(qm);
}
+ /* Config data buffer pasid needed by Kunpeng 920 */
+ hpre_config_pasid(qm);
+
return ret;
}
@@ -401,10 +401,6 @@ static void hpre_cnt_regs_clear(struct hisi_qm *qm)
unsigned long offset;
int i;
- /* clear current_qm */
- writel(0x0, qm->io_base + QM_DFX_MB_CNT_VF);
- writel(0x0, qm->io_base + QM_DFX_DB_CNT_VF);
-
/* clear clusterX/cluster_ctrl */
for (i = 0; i < clusters_num; i++) {
offset = HPRE_CLSTR_BASE + i * HPRE_CLSTR_ADDR_INTRVL;
@@ -456,49 +452,6 @@ static inline struct hisi_qm *hpre_file_to_qm(struct hpre_debugfs_file *file)
return &hpre->qm;
}
-static u32 hpre_current_qm_read(struct hpre_debugfs_file *file)
-{
- struct hisi_qm *qm = hpre_file_to_qm(file);
-
- return readl(qm->io_base + QM_DFX_MB_CNT_VF);
-}
-
-static int hpre_current_qm_write(struct hpre_debugfs_file *file, u32 val)
-{
- struct hisi_qm *qm = hpre_file_to_qm(file);
- u32 num_vfs = qm->vfs_num;
- u32 vfq_num, tmp;
-
- if (val > num_vfs)
- return -EINVAL;
-
- /* According PF or VF Dev ID to calculation curr_qm_qp_num and store */
- if (val == 0) {
- qm->debug.curr_qm_qp_num = qm->qp_num;
- } else {
- vfq_num = (qm->ctrl_qp_num - qm->qp_num) / num_vfs;
- if (val == num_vfs) {
- qm->debug.curr_qm_qp_num =
- qm->ctrl_qp_num - qm->qp_num - (num_vfs - 1) * vfq_num;
- } else {
- qm->debug.curr_qm_qp_num = vfq_num;
- }
- }
-
- writel(val, qm->io_base + QM_DFX_MB_CNT_VF);
- writel(val, qm->io_base + QM_DFX_DB_CNT_VF);
-
- tmp = val |
- (readl(qm->io_base + QM_DFX_SQE_CNT_VF_SQN) & CURRENT_Q_MASK);
- writel(tmp, qm->io_base + QM_DFX_SQE_CNT_VF_SQN);
-
- tmp = val |
- (readl(qm->io_base + QM_DFX_CQE_CNT_VF_CQN) & CURRENT_Q_MASK);
- writel(tmp, qm->io_base + QM_DFX_CQE_CNT_VF_CQN);
-
- return 0;
-}
-
static u32 hpre_clear_enable_read(struct hpre_debugfs_file *file)
{
struct hisi_qm *qm = hpre_file_to_qm(file);
@@ -519,7 +472,7 @@ static int hpre_clear_enable_write(struct hpre_debugfs_file *file, u32 val)
~HPRE_CTRL_CNT_CLR_CE_BIT) | val;
writel(tmp, qm->io_base + HPRE_CTRL_CNT_CLR_CE);
- return 0;
+ return 0;
}
static u32 hpre_cluster_inqry_read(struct hpre_debugfs_file *file)
@@ -541,7 +494,7 @@ static int hpre_cluster_inqry_write(struct hpre_debugfs_file *file, u32 val)
writel(val, qm->io_base + offset + HPRE_CLUSTER_INQURY);
- return 0;
+ return 0;
}
static ssize_t hpre_ctrl_debug_read(struct file *filp, char __user *buf,
@@ -554,9 +507,6 @@ static ssize_t hpre_ctrl_debug_read(struct file *filp, char __user *buf,
spin_lock_irq(&file->lock);
switch (file->type) {
- case HPRE_CURRENT_QM:
- val = hpre_current_qm_read(file);
- break;
case HPRE_CLEAR_ENABLE:
val = hpre_clear_enable_read(file);
break;
@@ -597,11 +547,6 @@ static ssize_t hpre_ctrl_debug_write(struct file *filp, const char __user *buf,
spin_lock_irq(&file->lock);
switch (file->type) {
- case HPRE_CURRENT_QM:
- ret = hpre_current_qm_write(file, val);
- if (ret)
- goto err_input;
- break;
case HPRE_CLEAR_ENABLE:
ret = hpre_clear_enable_write(file, val);
if (ret)
@@ -740,11 +685,6 @@ static int hpre_ctrl_debug_init(struct hisi_qm *qm)
{
int ret;
- ret = hpre_create_debugfs_file(qm, NULL, HPRE_CURRENT_QM,
- HPRE_CURRENT_QM);
- if (ret)
- return ret;
-
ret = hpre_create_debugfs_file(qm, NULL, HPRE_CLEAR_ENABLE,
HPRE_CLEAR_ENABLE);
if (ret)
@@ -812,9 +752,9 @@ static int hpre_qm_init(struct hisi_qm *qm, struct pci_dev *pdev)
}
if (pdev->revision >= QM_HW_V3)
- qm->algs = "rsa\ndh\necdh\nx25519\nx448\necdsa\nsm2\n";
+ qm->algs = "rsa\ndh\necdh\nx25519\nx448\necdsa\nsm2";
else
- qm->algs = "rsa\ndh\n";
+ qm->algs = "rsa\ndh";
qm->mode = uacce_mode;
qm->pdev = pdev;
qm->ver = pdev->revision;
@@ -867,6 +807,20 @@ static void hpre_open_axi_master_ooo(struct hisi_qm *qm)
HPRE_ADDR(qm, HPRE_AM_OOO_SHUTDOWN_ENB));
}
+static void hpre_err_info_init(struct hisi_qm *qm)
+{
+ struct hisi_qm_err_info *err_info = &qm->err_info;
+
+ err_info->ce = QM_BASE_CE;
+ err_info->fe = 0;
+ err_info->ecc_2bits_mask = HPRE_CORE_ECC_2BIT_ERR |
+ HPRE_OOO_ECC_2BIT_ERR;
+ err_info->dev_ce_mask = HPRE_HAC_RAS_CE_ENABLE;
+ err_info->msi_wr_port = HPRE_WR_MSI_PORT;
+ err_info->acpi_rst = "HRST";
+ err_info->nfe = QM_BASE_NFE | QM_ACC_DO_TASK_TIMEOUT;
+}
+
static const struct hisi_qm_err_ini hpre_err_ini = {
.hw_init = hpre_set_user_domain_and_cache,
.hw_err_enable = hpre_hw_error_enable,
@@ -875,16 +829,7 @@ static const struct hisi_qm_err_ini hpre_err_ini = {
.clear_dev_hw_err_status = hpre_clear_hw_err_status,
.log_dev_hw_err = hpre_log_hw_error,
.open_axi_master_ooo = hpre_open_axi_master_ooo,
- .err_info = {
- .ce = QM_BASE_CE,
- .nfe = QM_BASE_NFE | QM_ACC_DO_TASK_TIMEOUT,
- .fe = 0,
- .ecc_2bits_mask = HPRE_CORE_ECC_2BIT_ERR |
- HPRE_OOO_ECC_2BIT_ERR,
- .dev_ce_mask = HPRE_HAC_RAS_CE_ENABLE,
- .msi_wr_port = HPRE_WR_MSI_PORT,
- .acpi_rst = "HRST",
- }
+ .err_info_init = hpre_err_info_init,
};
static int hpre_pf_probe_init(struct hpre *hpre)
@@ -892,13 +837,12 @@ static int hpre_pf_probe_init(struct hpre *hpre)
struct hisi_qm *qm = &hpre->qm;
int ret;
- qm->ctrl_qp_num = HPRE_QUEUE_NUM_V2;
-
ret = hpre_set_user_domain_and_cache(qm);
if (ret)
return ret;
qm->err_ini = &hpre_err_ini;
+ qm->err_ini->err_info_init(qm);
hisi_qm_dev_err_init(qm);
return 0;
@@ -1006,8 +950,6 @@ static void hpre_remove(struct pci_dev *pdev)
hisi_qm_stop(qm, QM_NORMAL);
if (qm->fun_type == QM_HW_PF) {
- if (qm->use_sva && qm->ver == QM_HW_V2)
- hpre_pasid_disable(qm);
hpre_cnt_regs_clear(qm);
qm->debug.curr_qm_qp_num = 0;
hisi_qm_dev_err_uninit(qm);
@@ -1016,7 +958,6 @@ static void hpre_remove(struct pci_dev *pdev)
hisi_qm_uninit(qm);
}
-
static const struct pci_error_handlers hpre_err_handler = {
.error_detected = hisi_qm_dev_err_detected,
.slot_reset = hisi_qm_dev_slot_reset,
@@ -1075,4 +1016,5 @@ module_exit(hpre_exit);
MODULE_LICENSE("GPL v2");
MODULE_AUTHOR("Zaibo Xu <xuzaibo@huawei.com>");
+MODULE_AUTHOR("Meng Yu <yumeng18@huawei.com>");
MODULE_DESCRIPTION("Driver for HiSilicon HPRE accelerator");
diff --git a/drivers/crypto/hisilicon/qm.c b/drivers/crypto/hisilicon/qm.c
index 13cb4216561a..ce439a0c66c9 100644
--- a/drivers/crypto/hisilicon/qm.c
+++ b/drivers/crypto/hisilicon/qm.c
@@ -38,6 +38,7 @@
#define QM_MB_CMD_SQC_BT 0x4
#define QM_MB_CMD_CQC_BT 0x5
#define QM_MB_CMD_SQC_VFT_V2 0x6
+#define QM_MB_CMD_STOP_QP 0x8
#define QM_MB_CMD_SEND_BASE 0x300
#define QM_MB_EVENT_SHIFT 8
@@ -93,6 +94,12 @@
#define QM_DB_PRIORITY_SHIFT_V1 48
#define QM_DOORBELL_SQ_CQ_BASE_V2 0x1000
#define QM_DOORBELL_EQ_AEQ_BASE_V2 0x2000
+#define QM_QUE_ISO_CFG_V 0x0030
+#define QM_QUE_ISO_EN 0x100154
+#define QM_CAPBILITY 0x100158
+#define QM_QP_NUN_MASK GENMASK(10, 0)
+#define QM_QP_DB_INTERVAL 0x10000
+#define QM_QP_MAX_NUM_SHIFT 11
#define QM_DB_CMD_SHIFT_V2 12
#define QM_DB_RAND_SHIFT_V2 16
#define QM_DB_INDEX_SHIFT_V2 32
@@ -129,9 +136,9 @@
#define QM_DFX_CNT_CLR_CE 0x100118
#define QM_ABNORMAL_INT_SOURCE 0x100000
-#define QM_ABNORMAL_INT_SOURCE_CLR GENMASK(12, 0)
+#define QM_ABNORMAL_INT_SOURCE_CLR GENMASK(14, 0)
#define QM_ABNORMAL_INT_MASK 0x100004
-#define QM_ABNORMAL_INT_MASK_VALUE 0x1fff
+#define QM_ABNORMAL_INT_MASK_VALUE 0x7fff
#define QM_ABNORMAL_INT_STATUS 0x100008
#define QM_ABNORMAL_INT_SET 0x10000c
#define QM_ABNORMAL_INF00 0x100010
@@ -164,6 +171,14 @@
#define ACC_AM_ROB_ECC_INT_STS 0x300104
#define ACC_ROB_ECC_ERR_MULTPL BIT(1)
+#define QM_DFX_MB_CNT_VF 0x104010
+#define QM_DFX_DB_CNT_VF 0x104020
+#define QM_DFX_SQE_CNT_VF_SQN 0x104030
+#define QM_DFX_CQE_CNT_VF_CQN 0x104040
+#define QM_DFX_QN_SHIFT 16
+#define CURRENT_FUN_MASK GENMASK(5, 0)
+#define CURRENT_Q_MASK GENMASK(31, 16)
+
#define POLL_PERIOD 10
#define POLL_TIMEOUT 1000
#define WAIT_PERIOD_US_MAX 200
@@ -173,6 +188,7 @@
#define QM_CACHE_WB_DONE 0x208
#define PCI_BAR_2 2
+#define PCI_BAR_4 4
#define QM_SQE_DATA_ALIGN_MASK GENMASK(6, 0)
#define QMC_ALIGN(sz) ALIGN(sz, 32)
@@ -334,6 +350,7 @@ struct hisi_qm_hw_ops {
void (*hw_error_init)(struct hisi_qm *qm, u32 ce, u32 nfe, u32 fe);
void (*hw_error_uninit)(struct hisi_qm *qm);
enum acc_err_result (*hw_error_handle)(struct hisi_qm *qm);
+ int (*stop_qp)(struct hisi_qp *qp);
};
struct qm_dfx_item {
@@ -350,6 +367,7 @@ static struct qm_dfx_item qm_dfx_files[] = {
};
static const char * const qm_debug_file_name[] = {
+ [CURRENT_QM] = "current_qm",
[CURRENT_Q] = "current_q",
[CLEAR_ENABLE] = "clear_enable",
};
@@ -373,6 +391,8 @@ static const struct hisi_qm_hw_error qm_hw_error[] = {
{ .int_msk = BIT(10), .msg = "qm_db_timeout" },
{ .int_msk = BIT(11), .msg = "qm_of_fifo_of" },
{ .int_msk = BIT(12), .msg = "qm_db_random_invalid" },
+ { .int_msk = BIT(13), .msg = "qm_mailbox_timeout" },
+ { .int_msk = BIT(14), .msg = "qm_flr_timeout" },
{ /* sentinel */ }
};
@@ -557,21 +577,22 @@ static void qm_db_v1(struct hisi_qm *qm, u16 qn, u8 cmd, u16 index, u8 priority)
static void qm_db_v2(struct hisi_qm *qm, u16 qn, u8 cmd, u16 index, u8 priority)
{
- u64 doorbell;
- u64 dbase;
+ void __iomem *io_base = qm->io_base;
u16 randata = 0;
+ u64 doorbell;
if (cmd == QM_DOORBELL_CMD_SQ || cmd == QM_DOORBELL_CMD_CQ)
- dbase = QM_DOORBELL_SQ_CQ_BASE_V2;
+ io_base = qm->db_io_base + (u64)qn * qm->db_interval +
+ QM_DOORBELL_SQ_CQ_BASE_V2;
else
- dbase = QM_DOORBELL_EQ_AEQ_BASE_V2;
+ io_base += QM_DOORBELL_EQ_AEQ_BASE_V2;
doorbell = qn | ((u64)cmd << QM_DB_CMD_SHIFT_V2) |
((u64)randata << QM_DB_RAND_SHIFT_V2) |
((u64)index << QM_DB_INDEX_SHIFT_V2) |
((u64)priority << QM_DB_PRIORITY_SHIFT_V2);
- writeq(doorbell, qm->io_base + dbase);
+ writeq(doorbell, io_base);
}
static void qm_db(struct hisi_qm *qm, u16 qn, u8 cmd, u16 index, u8 priority)
@@ -865,6 +886,26 @@ static int qm_get_vft_v2(struct hisi_qm *qm, u32 *base, u32 *number)
return 0;
}
+static int qm_get_vf_qp_num(struct hisi_qm *qm, u32 fun_num)
+{
+ u32 remain_q_num, vfq_num;
+ u32 num_vfs = qm->vfs_num;
+
+ vfq_num = (qm->ctrl_qp_num - qm->qp_num) / num_vfs;
+ if (vfq_num >= qm->max_qp_num)
+ return qm->max_qp_num;
+
+ remain_q_num = (qm->ctrl_qp_num - qm->qp_num) % num_vfs;
+ if (vfq_num + remain_q_num <= qm->max_qp_num)
+ return fun_num == num_vfs ? vfq_num + remain_q_num : vfq_num;
+
+ /*
+ * if vfq_num + remain_q_num > max_qp_num, the last VFs,
+ * each with one more queue.
+ */
+ return fun_num + remain_q_num > num_vfs ? vfq_num + 1 : vfq_num;
+}
+
static struct hisi_qm *file_to_qm(struct debugfs_file *file)
{
struct qm_debug *debug = file->debug;
@@ -918,6 +959,41 @@ static int clear_enable_write(struct debugfs_file *file, u32 rd_clr_ctrl)
return 0;
}
+static u32 current_qm_read(struct debugfs_file *file)
+{
+ struct hisi_qm *qm = file_to_qm(file);
+
+ return readl(qm->io_base + QM_DFX_MB_CNT_VF);
+}
+
+static int current_qm_write(struct debugfs_file *file, u32 val)
+{
+ struct hisi_qm *qm = file_to_qm(file);
+ u32 tmp;
+
+ if (val > qm->vfs_num)
+ return -EINVAL;
+
+ /* According PF or VF Dev ID to calculation curr_qm_qp_num and store */
+ if (!val)
+ qm->debug.curr_qm_qp_num = qm->qp_num;
+ else
+ qm->debug.curr_qm_qp_num = qm_get_vf_qp_num(qm, val);
+
+ writel(val, qm->io_base + QM_DFX_MB_CNT_VF);
+ writel(val, qm->io_base + QM_DFX_DB_CNT_VF);
+
+ tmp = val |
+ (readl(qm->io_base + QM_DFX_SQE_CNT_VF_SQN) & CURRENT_Q_MASK);
+ writel(tmp, qm->io_base + QM_DFX_SQE_CNT_VF_SQN);
+
+ tmp = val |
+ (readl(qm->io_base + QM_DFX_CQE_CNT_VF_CQN) & CURRENT_Q_MASK);
+ writel(tmp, qm->io_base + QM_DFX_CQE_CNT_VF_CQN);
+
+ return 0;
+}
+
static ssize_t qm_debug_read(struct file *filp, char __user *buf,
size_t count, loff_t *pos)
{
@@ -929,6 +1005,9 @@ static ssize_t qm_debug_read(struct file *filp, char __user *buf,
mutex_lock(&file->lock);
switch (index) {
+ case CURRENT_QM:
+ val = current_qm_read(file);
+ break;
case CURRENT_Q:
val = current_q_read(file);
break;
@@ -971,27 +1050,24 @@ static ssize_t qm_debug_write(struct file *filp, const char __user *buf,
mutex_lock(&file->lock);
switch (index) {
+ case CURRENT_QM:
+ ret = current_qm_write(file, val);
+ break;
case CURRENT_Q:
ret = current_q_write(file, val);
- if (ret)
- goto err_input;
break;
case CLEAR_ENABLE:
ret = clear_enable_write(file, val);
- if (ret)
- goto err_input;
break;
default:
ret = -EINVAL;
- goto err_input;
}
mutex_unlock(&file->lock);
- return count;
+ if (ret)
+ return ret;
-err_input:
- mutex_unlock(&file->lock);
- return ret;
+ return count;
}
static const struct file_operations qm_debug_fops = {
@@ -1529,12 +1605,12 @@ static const struct file_operations qm_cmd_fops = {
.write = qm_cmd_write,
};
-static void qm_create_debugfs_file(struct hisi_qm *qm, enum qm_debug_file index)
+static void qm_create_debugfs_file(struct hisi_qm *qm, struct dentry *dir,
+ enum qm_debug_file index)
{
- struct dentry *qm_d = qm->debug.qm_d;
struct debugfs_file *file = qm->debug.files + index;
- debugfs_create_file(qm_debug_file_name[index], 0600, qm_d, file,
+ debugfs_create_file(qm_debug_file_name[index], 0600, dir, file,
&qm_debug_fops);
file->index = index;
@@ -1628,7 +1704,7 @@ static enum acc_err_result qm_hw_error_handle_v2(struct hisi_qm *qm)
if (val == (QM_DB_RANDOM_INVALID | QM_BASE_CE)) {
writel(error_status, qm->io_base +
QM_ABNORMAL_INT_SOURCE);
- writel(qm->err_ini->err_info.nfe,
+ writel(qm->err_info.nfe,
qm->io_base + QM_RAS_NFE_ENABLE);
return ACC_ERR_RECOVERED;
}
@@ -1639,6 +1715,11 @@ static enum acc_err_result qm_hw_error_handle_v2(struct hisi_qm *qm)
return ACC_ERR_RECOVERED;
}
+static int qm_stop_qp(struct hisi_qp *qp)
+{
+ return qm_mb(qp->qm, QM_MB_CMD_STOP_QP, 0, qp->qp_id, 0);
+}
+
static const struct hisi_qm_hw_ops qm_hw_ops_v1 = {
.qm_db = qm_db_v1,
.get_irq_num = qm_get_irq_num_v1,
@@ -1654,6 +1735,16 @@ static const struct hisi_qm_hw_ops qm_hw_ops_v2 = {
.hw_error_handle = qm_hw_error_handle_v2,
};
+static const struct hisi_qm_hw_ops qm_hw_ops_v3 = {
+ .get_vft = qm_get_vft_v2,
+ .qm_db = qm_db_v2,
+ .get_irq_num = qm_get_irq_num_v2,
+ .hw_error_init = qm_hw_error_init_v2,
+ .hw_error_uninit = qm_hw_error_uninit_v2,
+ .hw_error_handle = qm_hw_error_handle_v2,
+ .stop_qp = qm_stop_qp,
+};
+
static void *qm_get_avail_sqe(struct hisi_qp *qp)
{
struct hisi_qp_status *qp_status = &qp->qp_status;
@@ -1933,6 +2024,14 @@ static int qm_drain_qp(struct hisi_qp *qp)
if (qm->err_status.is_qm_ecc_mbit || qm->err_status.is_dev_ecc_mbit)
return 0;
+ /* Kunpeng930 supports drain qp by device */
+ if (qm->ops->stop_qp) {
+ ret = qm->ops->stop_qp(qp);
+ if (ret)
+ dev_err(dev, "Failed to stop qp(%u)!\n", qp->qp_id);
+ return ret;
+ }
+
addr = qm_ctx_alloc(qm, size, &dma_addr);
if (IS_ERR(addr)) {
dev_err(dev, "Failed to alloc ctx for sqc and cqc!\n");
@@ -2132,6 +2231,8 @@ static int hisi_qm_uacce_mmap(struct uacce_queue *q,
{
struct hisi_qp *qp = q->priv;
struct hisi_qm *qm = qp->qm;
+ resource_size_t phys_base = qm->db_phys_base +
+ qp->qp_id * qm->db_interval;
size_t sz = vma->vm_end - vma->vm_start;
struct pci_dev *pdev = qm->pdev;
struct device *dev = &pdev->dev;
@@ -2143,16 +2244,19 @@ static int hisi_qm_uacce_mmap(struct uacce_queue *q,
if (qm->ver == QM_HW_V1) {
if (sz > PAGE_SIZE * QM_DOORBELL_PAGE_NR)
return -EINVAL;
- } else {
+ } else if (qm->ver == QM_HW_V2 || !qm->use_db_isolation) {
if (sz > PAGE_SIZE * (QM_DOORBELL_PAGE_NR +
QM_DOORBELL_SQ_CQ_BASE_V2 / PAGE_SIZE))
return -EINVAL;
+ } else {
+ if (sz > qm->db_interval)
+ return -EINVAL;
}
vma->vm_flags |= VM_IO;
return remap_pfn_range(vma, vma->vm_start,
- qm->phys_base >> PAGE_SHIFT,
+ phys_base >> PAGE_SHIFT,
sz, pgprot_noncached(vma->vm_page_prot));
case UACCE_QFRT_DUS:
if (sz != qp->qdma.size)
@@ -2267,14 +2371,20 @@ static int qm_alloc_uacce(struct hisi_qm *qm)
uacce->priv = qm;
uacce->algs = qm->algs;
- if (qm->ver == QM_HW_V1) {
- mmio_page_nr = QM_DOORBELL_PAGE_NR;
+ if (qm->ver == QM_HW_V1)
uacce->api_ver = HISI_QM_API_VER_BASE;
- } else {
+ else if (qm->ver == QM_HW_V2)
+ uacce->api_ver = HISI_QM_API_VER2_BASE;
+ else
+ uacce->api_ver = HISI_QM_API_VER3_BASE;
+
+ if (qm->ver == QM_HW_V1)
+ mmio_page_nr = QM_DOORBELL_PAGE_NR;
+ else if (qm->ver == QM_HW_V2 || !qm->use_db_isolation)
mmio_page_nr = QM_DOORBELL_PAGE_NR +
QM_DOORBELL_SQ_CQ_BASE_V2 / PAGE_SIZE;
- uacce->api_ver = HISI_QM_API_VER2_BASE;
- }
+ else
+ mmio_page_nr = qm->db_interval / PAGE_SIZE;
dus_page_nr = (PAGE_SIZE - 1 + qm->sqe_size * QM_Q_DEPTH +
sizeof(struct qm_cqe) * QM_Q_DEPTH) >> PAGE_SHIFT;
@@ -2482,8 +2592,10 @@ static void hisi_qm_pre_init(struct hisi_qm *qm)
if (qm->ver == QM_HW_V1)
qm->ops = &qm_hw_ops_v1;
- else
+ else if (qm->ver == QM_HW_V2)
qm->ops = &qm_hw_ops_v2;
+ else
+ qm->ops = &qm_hw_ops_v3;
pci_set_drvdata(pdev, qm);
mutex_init(&qm->mailbox_lock);
@@ -2492,13 +2604,23 @@ static void hisi_qm_pre_init(struct hisi_qm *qm)
qm->misc_ctl = false;
}
-static void hisi_qm_pci_uninit(struct hisi_qm *qm)
+static void qm_put_pci_res(struct hisi_qm *qm)
{
struct pci_dev *pdev = qm->pdev;
- pci_free_irq_vectors(pdev);
+ if (qm->use_db_isolation)
+ iounmap(qm->db_io_base);
+
iounmap(qm->io_base);
pci_release_mem_regions(pdev);
+}
+
+static void hisi_qm_pci_uninit(struct hisi_qm *qm)
+{
+ struct pci_dev *pdev = qm->pdev;
+
+ pci_free_irq_vectors(pdev);
+ qm_put_pci_res(qm);
pci_disable_device(pdev);
}
@@ -2527,7 +2649,6 @@ void hisi_qm_uninit(struct hisi_qm *qm)
hisi_qm_cache_wb(qm);
dma_free_coherent(dev, qm->qdma.size,
qm->qdma.va, qm->qdma.dma);
- memset(&qm->qdma, 0, sizeof(qm->qdma));
}
qm_irq_unregister(qm);
@@ -2681,7 +2802,7 @@ static int __hisi_qm_start(struct hisi_qm *qm)
{
int ret;
- WARN_ON(!qm->qdma.dma);
+ WARN_ON(!qm->qdma.va);
if (qm->fun_type == QM_HW_PF) {
ret = qm_dev_mem_reset(qm);
@@ -2930,9 +3051,11 @@ void hisi_qm_debug_init(struct hisi_qm *qm)
qm->debug.qm_d = qm_d;
/* only show this in PF */
- if (qm->fun_type == QM_HW_PF)
+ if (qm->fun_type == QM_HW_PF) {
+ qm_create_debugfs_file(qm, qm->debug.debug_root, CURRENT_QM);
for (i = CURRENT_Q; i < DEBUG_FILE_NUM; i++)
- qm_create_debugfs_file(qm, i);
+ qm_create_debugfs_file(qm, qm_d, i);
+ }
debugfs_create_file("regs", 0444, qm->debug.qm_d, qm, &qm_regs_fops);
@@ -2960,6 +3083,10 @@ void hisi_qm_debug_regs_clear(struct hisi_qm *qm)
struct qm_dfx_registers *regs;
int i;
+ /* clear current_qm */
+ writel(0x0, qm->io_base + QM_DFX_MB_CNT_VF);
+ writel(0x0, qm->io_base + QM_DFX_DB_CNT_VF);
+
/* clear current_q */
writel(0x0, qm->io_base + QM_DFX_SQE_CNT_VF_SQN);
writel(0x0, qm->io_base + QM_DFX_CQE_CNT_VF_CQN);
@@ -2982,7 +3109,7 @@ EXPORT_SYMBOL_GPL(hisi_qm_debug_regs_clear);
static void qm_hw_error_init(struct hisi_qm *qm)
{
- const struct hisi_qm_err_info *err_info = &qm->err_ini->err_info;
+ struct hisi_qm_err_info *err_info = &qm->err_info;
if (!qm->ops->hw_error_init) {
dev_err(&qm->pdev->dev, "QM doesn't support hw error handling!\n");
@@ -3175,30 +3302,46 @@ EXPORT_SYMBOL_GPL(hisi_qm_alloc_qps_node);
static int qm_vf_q_assign(struct hisi_qm *qm, u32 num_vfs)
{
- u32 remain_q_num, q_num, i, j;
+ u32 remain_q_num, vfs_q_num, act_q_num, q_num, i, j;
+ u32 max_qp_num = qm->max_qp_num;
u32 q_base = qm->qp_num;
int ret;
if (!num_vfs)
return -EINVAL;
- remain_q_num = qm->ctrl_qp_num - qm->qp_num;
+ vfs_q_num = qm->ctrl_qp_num - qm->qp_num;
- /* If remain queues not enough, return error. */
- if (qm->ctrl_qp_num < qm->qp_num || remain_q_num < num_vfs)
+ /* If vfs_q_num is less than num_vfs, return error. */
+ if (vfs_q_num < num_vfs)
return -EINVAL;
- q_num = remain_q_num / num_vfs;
- for (i = 1; i <= num_vfs; i++) {
- if (i == num_vfs)
- q_num += remain_q_num % num_vfs;
- ret = hisi_qm_set_vft(qm, i, q_base, q_num);
+ q_num = vfs_q_num / num_vfs;
+ remain_q_num = vfs_q_num % num_vfs;
+
+ for (i = num_vfs; i > 0; i--) {
+ /*
+ * if q_num + remain_q_num > max_qp_num in last vf, divide the
+ * remaining queues equally.
+ */
+ if (i == num_vfs && q_num + remain_q_num <= max_qp_num) {
+ act_q_num = q_num + remain_q_num;
+ remain_q_num = 0;
+ } else if (remain_q_num > 0) {
+ act_q_num = q_num + 1;
+ remain_q_num--;
+ } else {
+ act_q_num = q_num;
+ }
+
+ act_q_num = min_t(int, act_q_num, max_qp_num);
+ ret = hisi_qm_set_vft(qm, i, q_base, act_q_num);
if (ret) {
- for (j = i; j > 0; j--)
+ for (j = num_vfs; j > i; j--)
hisi_qm_set_vft(qm, j, 0, 0);
return ret;
}
- q_base += q_num;
+ q_base += act_q_num;
}
return 0;
@@ -3318,15 +3461,15 @@ static enum acc_err_result qm_dev_err_handle(struct hisi_qm *qm)
/* get device hardware error status */
err_sts = qm->err_ini->get_dev_hw_err_status(qm);
if (err_sts) {
- if (err_sts & qm->err_ini->err_info.ecc_2bits_mask)
+ if (err_sts & qm->err_info.ecc_2bits_mask)
qm->err_status.is_dev_ecc_mbit = true;
if (qm->err_ini->log_dev_hw_err)
qm->err_ini->log_dev_hw_err(qm, err_sts);
/* ce error does not need to be reset */
- if ((err_sts | qm->err_ini->err_info.dev_ce_mask) ==
- qm->err_ini->err_info.dev_ce_mask) {
+ if ((err_sts | qm->err_info.dev_ce_mask) ==
+ qm->err_info.dev_ce_mask) {
if (qm->err_ini->clear_dev_hw_err_status)
qm->err_ini->clear_dev_hw_err_status(qm,
err_sts);
@@ -3639,7 +3782,7 @@ static int qm_soft_reset(struct hisi_qm *qm)
acpi_status s;
s = acpi_evaluate_integer(ACPI_HANDLE(&pdev->dev),
- qm->err_ini->err_info.acpi_rst,
+ qm->err_info.acpi_rst,
NULL, &value);
if (ACPI_FAILURE(s)) {
pci_err(pdev, "NO controller reset method!\n");
@@ -3707,12 +3850,11 @@ static void qm_restart_prepare(struct hisi_qm *qm)
/* temporarily close the OOO port used for PEH to write out MSI */
value = readl(qm->io_base + ACC_AM_CFG_PORT_WR_EN);
- writel(value & ~qm->err_ini->err_info.msi_wr_port,
+ writel(value & ~qm->err_info.msi_wr_port,
qm->io_base + ACC_AM_CFG_PORT_WR_EN);
/* clear dev ecc 2bit error source if having */
- value = qm_get_dev_err_status(qm) &
- qm->err_ini->err_info.ecc_2bits_mask;
+ value = qm_get_dev_err_status(qm) & qm->err_info.ecc_2bits_mask;
if (value && qm->err_ini->clear_dev_hw_err_status)
qm->err_ini->clear_dev_hw_err_status(qm, value);
@@ -3736,7 +3878,7 @@ static void qm_restart_done(struct hisi_qm *qm)
/* open the OOO port for PEH to write out MSI */
value = readl(qm->io_base + ACC_AM_CFG_PORT_WR_EN);
- value |= qm->err_ini->err_info.msi_wr_port;
+ value |= qm->err_info.msi_wr_port;
writel(value, qm->io_base + ACC_AM_CFG_PORT_WR_EN);
qm->err_status.is_qm_ecc_mbit = false;
@@ -3875,8 +4017,7 @@ static int qm_check_dev_error(struct hisi_qm *qm)
if (ret)
return ret;
- return (qm_get_dev_err_status(qm) &
- qm->err_ini->err_info.ecc_2bits_mask);
+ return (qm_get_dev_err_status(qm) & qm->err_info.ecc_2bits_mask);
}
void hisi_qm_reset_prepare(struct pci_dev *pdev)
@@ -4084,7 +4225,7 @@ int hisi_qm_alg_register(struct hisi_qm *qm, struct hisi_qm_list *qm_list)
mutex_unlock(&qm_list->lock);
if (flag) {
- ret = qm_list->register_to_crypto();
+ ret = qm_list->register_to_crypto(qm);
if (ret) {
mutex_lock(&qm_list->lock);
list_del(&qm->list);
@@ -4115,59 +4256,134 @@ void hisi_qm_alg_unregister(struct hisi_qm *qm, struct hisi_qm_list *qm_list)
mutex_unlock(&qm_list->lock);
if (list_empty(&qm_list->list))
- qm_list->unregister_from_crypto();
+ qm_list->unregister_from_crypto(qm);
}
EXPORT_SYMBOL_GPL(hisi_qm_alg_unregister);
-static int hisi_qm_pci_init(struct hisi_qm *qm)
+static int qm_get_qp_num(struct hisi_qm *qm)
+{
+ if (qm->ver == QM_HW_V1)
+ qm->ctrl_qp_num = QM_QNUM_V1;
+ else if (qm->ver == QM_HW_V2)
+ qm->ctrl_qp_num = QM_QNUM_V2;
+ else
+ qm->ctrl_qp_num = readl(qm->io_base + QM_CAPBILITY) &
+ QM_QP_NUN_MASK;
+
+ if (qm->use_db_isolation)
+ qm->max_qp_num = (readl(qm->io_base + QM_CAPBILITY) >>
+ QM_QP_MAX_NUM_SHIFT) & QM_QP_NUN_MASK;
+ else
+ qm->max_qp_num = qm->ctrl_qp_num;
+
+ /* check if qp number is valid */
+ if (qm->qp_num > qm->max_qp_num) {
+ dev_err(&qm->pdev->dev, "qp num(%u) is more than max qp num(%u)!\n",
+ qm->qp_num, qm->max_qp_num);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+static int qm_get_pci_res(struct hisi_qm *qm)
{
struct pci_dev *pdev = qm->pdev;
struct device *dev = &pdev->dev;
- unsigned int num_vec;
int ret;
- ret = pci_enable_device_mem(pdev);
- if (ret < 0) {
- dev_err(dev, "Failed to enable device mem!\n");
- return ret;
- }
-
ret = pci_request_mem_regions(pdev, qm->dev_name);
if (ret < 0) {
dev_err(dev, "Failed to request mem regions!\n");
- goto err_disable_pcidev;
+ return ret;
}
qm->phys_base = pci_resource_start(pdev, PCI_BAR_2);
- qm->phys_size = pci_resource_len(qm->pdev, PCI_BAR_2);
- qm->io_base = ioremap(qm->phys_base, qm->phys_size);
+ qm->io_base = ioremap(qm->phys_base, pci_resource_len(pdev, PCI_BAR_2));
if (!qm->io_base) {
ret = -EIO;
- goto err_release_mem_regions;
+ goto err_request_mem_regions;
+ }
+
+ if (qm->ver > QM_HW_V2) {
+ if (qm->fun_type == QM_HW_PF)
+ qm->use_db_isolation = readl(qm->io_base +
+ QM_QUE_ISO_EN) & BIT(0);
+ else
+ qm->use_db_isolation = readl(qm->io_base +
+ QM_QUE_ISO_CFG_V) & BIT(0);
+ }
+
+ if (qm->use_db_isolation) {
+ qm->db_interval = QM_QP_DB_INTERVAL;
+ qm->db_phys_base = pci_resource_start(pdev, PCI_BAR_4);
+ qm->db_io_base = ioremap(qm->db_phys_base,
+ pci_resource_len(pdev, PCI_BAR_4));
+ if (!qm->db_io_base) {
+ ret = -EIO;
+ goto err_ioremap;
+ }
+ } else {
+ qm->db_phys_base = qm->phys_base;
+ qm->db_io_base = qm->io_base;
+ qm->db_interval = 0;
}
+ if (qm->fun_type == QM_HW_PF) {
+ ret = qm_get_qp_num(qm);
+ if (ret)
+ goto err_db_ioremap;
+ }
+
+ return 0;
+
+err_db_ioremap:
+ if (qm->use_db_isolation)
+ iounmap(qm->db_io_base);
+err_ioremap:
+ iounmap(qm->io_base);
+err_request_mem_regions:
+ pci_release_mem_regions(pdev);
+ return ret;
+}
+
+static int hisi_qm_pci_init(struct hisi_qm *qm)
+{
+ struct pci_dev *pdev = qm->pdev;
+ struct device *dev = &pdev->dev;
+ unsigned int num_vec;
+ int ret;
+
+ ret = pci_enable_device_mem(pdev);
+ if (ret < 0) {
+ dev_err(dev, "Failed to enable device mem!\n");
+ return ret;
+ }
+
+ ret = qm_get_pci_res(qm);
+ if (ret)
+ goto err_disable_pcidev;
+
ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(64));
if (ret < 0)
- goto err_iounmap;
+ goto err_get_pci_res;
pci_set_master(pdev);
if (!qm->ops->get_irq_num) {
ret = -EOPNOTSUPP;
- goto err_iounmap;
+ goto err_get_pci_res;
}
num_vec = qm->ops->get_irq_num(qm);
ret = pci_alloc_irq_vectors(pdev, num_vec, num_vec, PCI_IRQ_MSI);
if (ret < 0) {
dev_err(dev, "Failed to enable MSI vectors!\n");
- goto err_iounmap;
+ goto err_get_pci_res;
}
return 0;
-err_iounmap:
- iounmap(qm->io_base);
-err_release_mem_regions:
- pci_release_mem_regions(pdev);
+err_get_pci_res:
+ qm_put_pci_res(qm);
err_disable_pcidev:
pci_disable_device(pdev);
return ret;
@@ -4187,28 +4403,28 @@ int hisi_qm_init(struct hisi_qm *qm)
hisi_qm_pre_init(qm);
- ret = qm_alloc_uacce(qm);
- if (ret < 0)
- dev_warn(dev, "fail to alloc uacce (%d)\n", ret);
-
ret = hisi_qm_pci_init(qm);
if (ret)
- goto err_remove_uacce;
+ return ret;
ret = qm_irq_register(qm);
if (ret)
- goto err_pci_uninit;
+ goto err_pci_init;
if (qm->fun_type == QM_HW_VF && qm->ver != QM_HW_V1) {
/* v2 starts to support get vft by mailbox */
ret = hisi_qm_get_vft(qm, &qm->qp_base, &qm->qp_num);
if (ret)
- goto err_irq_unregister;
+ goto err_irq_register;
}
+ ret = qm_alloc_uacce(qm);
+ if (ret < 0)
+ dev_warn(dev, "fail to alloc uacce (%d)\n", ret);
+
ret = hisi_qm_memory_init(qm);
if (ret)
- goto err_irq_unregister;
+ goto err_alloc_uacce;
INIT_WORK(&qm->work, qm_work_process);
if (qm->fun_type == QM_HW_PF)
@@ -4218,13 +4434,13 @@ int hisi_qm_init(struct hisi_qm *qm)
return 0;
-err_irq_unregister:
- qm_irq_unregister(qm);
-err_pci_uninit:
- hisi_qm_pci_uninit(qm);
-err_remove_uacce:
+err_alloc_uacce:
uacce_remove(qm->uacce);
qm->uacce = NULL;
+err_irq_register:
+ qm_irq_unregister(qm);
+err_pci_init:
+ hisi_qm_pci_uninit(qm);
return ret;
}
EXPORT_SYMBOL_GPL(hisi_qm_init);
diff --git a/drivers/crypto/hisilicon/qm.h b/drivers/crypto/hisilicon/qm.h
index 54967c6b9c78..acefdf8b3a50 100644
--- a/drivers/crypto/hisilicon/qm.h
+++ b/drivers/crypto/hisilicon/qm.h
@@ -51,14 +51,6 @@
#define PEH_AXUSER_CFG 0x401001
#define PEH_AXUSER_CFG_ENABLE 0xffffffff
-#define QM_DFX_MB_CNT_VF 0x104010
-#define QM_DFX_DB_CNT_VF 0x104020
-#define QM_DFX_SQE_CNT_VF_SQN 0x104030
-#define QM_DFX_CQE_CNT_VF_CQN 0x104040
-#define QM_DFX_QN_SHIFT 16
-#define CURRENT_FUN_MASK GENMASK(5, 0)
-#define CURRENT_Q_MASK GENMASK(31, 16)
-
#define QM_AXI_RRESP BIT(0)
#define QM_AXI_BRESP BIT(1)
#define QM_ECC_MBIT BIT(2)
@@ -72,10 +64,13 @@
#define QM_DB_TIMEOUT BIT(10)
#define QM_OF_FIFO_OF BIT(11)
#define QM_DB_RANDOM_INVALID BIT(12)
+#define QM_MAILBOX_TIMEOUT BIT(13)
+#define QM_FLR_TIMEOUT BIT(14)
#define QM_BASE_NFE (QM_AXI_RRESP | QM_AXI_BRESP | QM_ECC_MBIT | \
QM_ACC_GET_TASK_TIMEOUT | QM_DB_TIMEOUT | \
- QM_OF_FIFO_OF | QM_DB_RANDOM_INVALID)
+ QM_OF_FIFO_OF | QM_DB_RANDOM_INVALID | \
+ QM_MAILBOX_TIMEOUT | QM_FLR_TIMEOUT)
#define QM_BASE_CE QM_ECC_1BIT
#define QM_Q_DEPTH 1024
@@ -123,6 +118,7 @@ enum qm_fun_type {
};
enum qm_debug_file {
+ CURRENT_QM,
CURRENT_Q,
CLEAR_ENABLE,
DEBUG_FILE_NUM,
@@ -193,14 +189,14 @@ struct hisi_qm_err_ini {
void (*open_axi_master_ooo)(struct hisi_qm *qm);
void (*close_axi_master_ooo)(struct hisi_qm *qm);
void (*log_dev_hw_err)(struct hisi_qm *qm, u32 err_sts);
- struct hisi_qm_err_info err_info;
+ void (*err_info_init)(struct hisi_qm *qm);
};
struct hisi_qm_list {
struct mutex lock;
struct list_head list;
- int (*register_to_crypto)(void);
- void (*unregister_from_crypto)(void);
+ int (*register_to_crypto)(struct hisi_qm *qm);
+ void (*unregister_from_crypto)(struct hisi_qm *qm);
};
struct hisi_qm {
@@ -209,12 +205,15 @@ struct hisi_qm {
const char *dev_name;
struct pci_dev *pdev;
void __iomem *io_base;
+ void __iomem *db_io_base;
u32 sqe_size;
u32 qp_base;
u32 qp_num;
u32 qp_in_used;
u32 ctrl_qp_num;
+ u32 max_qp_num;
u32 vfs_num;
+ u32 db_interval;
struct list_head list;
struct hisi_qm_list *qm_list;
@@ -230,6 +229,7 @@ struct hisi_qm {
struct hisi_qm_status status;
const struct hisi_qm_err_ini *err_ini;
+ struct hisi_qm_err_info err_info;
struct hisi_qm_err_status err_status;
unsigned long misc_ctl; /* driver removing and reset sched */
@@ -252,8 +252,11 @@ struct hisi_qm {
const char *algs;
bool use_sva;
bool is_frozen;
+
+ /* doorbell isolation enable */
+ bool use_db_isolation;
resource_size_t phys_base;
- resource_size_t phys_size;
+ resource_size_t db_phys_base;
struct uacce_device *uacce;
int mode;
};
diff --git a/drivers/crypto/hisilicon/sec/sec_algs.c b/drivers/crypto/hisilicon/sec/sec_algs.c
index 8ca945ac297e..0a3c8f019b02 100644
--- a/drivers/crypto/hisilicon/sec/sec_algs.c
+++ b/drivers/crypto/hisilicon/sec/sec_algs.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
-/* Copyright (c) 2016-2017 Hisilicon Limited. */
+/* Copyright (c) 2016-2017 HiSilicon Limited. */
#include <linux/crypto.h>
#include <linux/dma-mapping.h>
#include <linux/dmapool.h>
diff --git a/drivers/crypto/hisilicon/sec/sec_drv.c b/drivers/crypto/hisilicon/sec/sec_drv.c
index 91ee2bb575df..c8de1b51c843 100644
--- a/drivers/crypto/hisilicon/sec/sec_drv.c
+++ b/drivers/crypto/hisilicon/sec/sec_drv.c
@@ -1,8 +1,8 @@
// SPDX-License-Identifier: GPL-2.0
/*
- * Driver for the Hisilicon SEC units found on Hip06 Hip07
+ * Driver for the HiSilicon SEC units found on Hip06 Hip07
*
- * Copyright (c) 2016-2017 Hisilicon Limited.
+ * Copyright (c) 2016-2017 HiSilicon Limited.
*/
#include <linux/acpi.h>
#include <linux/atomic.h>
@@ -233,7 +233,7 @@ static int sec_queue_map_io(struct sec_queue *queue)
IORESOURCE_MEM,
2 + queue->queue_id);
if (!res) {
- dev_err(dev, "Failed to get queue %d memory resource\n",
+ dev_err(dev, "Failed to get queue %u memory resource\n",
queue->queue_id);
return -ENOMEM;
}
@@ -653,12 +653,12 @@ static int sec_queue_free(struct sec_queue *queue)
struct sec_dev_info *info = queue->dev_info;
if (queue->queue_id >= SEC_Q_NUM) {
- dev_err(info->dev, "No queue %d\n", queue->queue_id);
+ dev_err(info->dev, "No queue %u\n", queue->queue_id);
return -ENODEV;
}
if (!queue->in_use) {
- dev_err(info->dev, "Queue %d is idle\n", queue->queue_id);
+ dev_err(info->dev, "Queue %u is idle\n", queue->queue_id);
return -ENODEV;
}
@@ -834,6 +834,7 @@ int sec_queue_stop_release(struct sec_queue *queue)
/**
* sec_queue_empty() - Is this hardware queue currently empty.
+ * @queue: The queue to test
*
* We need to know if we have an empty queue for some of the chaining modes
* as if it is not empty we may need to hold the message in a software queue
@@ -1315,6 +1316,6 @@ static struct platform_driver sec_driver = {
module_platform_driver(sec_driver);
MODULE_LICENSE("GPL");
-MODULE_DESCRIPTION("Hisilicon Security Accelerators");
+MODULE_DESCRIPTION("HiSilicon Security Accelerators");
MODULE_AUTHOR("Zaibo Xu <xuzaibo@huawei.com");
MODULE_AUTHOR("Jonathan Cameron <jonathan.cameron@huawei.com>");
diff --git a/drivers/crypto/hisilicon/sec/sec_drv.h b/drivers/crypto/hisilicon/sec/sec_drv.h
index 4d9063a8b10b..179a8250d691 100644
--- a/drivers/crypto/hisilicon/sec/sec_drv.h
+++ b/drivers/crypto/hisilicon/sec/sec_drv.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0 */
-/* Copyright (c) 2016-2017 Hisilicon Limited. */
+/* Copyright (c) 2016-2017 HiSilicon Limited. */
#ifndef _SEC_DRV_H_
#define _SEC_DRV_H_
diff --git a/drivers/crypto/hisilicon/sec2/sec.h b/drivers/crypto/hisilicon/sec2/sec.h
index 08491912afd5..dfdce2f21e65 100644
--- a/drivers/crypto/hisilicon/sec2/sec.h
+++ b/drivers/crypto/hisilicon/sec2/sec.h
@@ -4,8 +4,6 @@
#ifndef __HISI_SEC_V2_H
#define __HISI_SEC_V2_H
-#include <linux/list.h>
-
#include "../qm.h"
#include "sec_crypto.h"
@@ -50,7 +48,7 @@ struct sec_req {
int err_type;
int req_id;
- int flag;
+ u32 flag;
/* Status of the SEC request */
bool fake_busy;
@@ -139,6 +137,7 @@ struct sec_ctx {
bool pbuf_supported;
struct sec_cipher_ctx c_ctx;
struct sec_auth_ctx a_ctx;
+ struct device *dev;
};
enum sec_endian {
@@ -148,7 +147,6 @@ enum sec_endian {
};
enum sec_debug_file_index {
- SEC_CURRENT_QM,
SEC_CLEAR_ENABLE,
SEC_DEBUG_FILE_NUM,
};
@@ -183,6 +181,6 @@ struct sec_dev {
void sec_destroy_qps(struct hisi_qp **qps, int qp_num);
struct hisi_qp **sec_create_qps(void);
-int sec_register_to_crypto(void);
-void sec_unregister_from_crypto(void);
+int sec_register_to_crypto(struct hisi_qm *qm);
+void sec_unregister_from_crypto(struct hisi_qm *qm);
#endif
diff --git a/drivers/crypto/hisilicon/sec2/sec_crypto.c b/drivers/crypto/hisilicon/sec2/sec_crypto.c
index 2eaa516b3231..133aede8bf07 100644
--- a/drivers/crypto/hisilicon/sec2/sec_crypto.c
+++ b/drivers/crypto/hisilicon/sec2/sec_crypto.c
@@ -7,6 +7,7 @@
#include <crypto/des.h>
#include <crypto/hash.h>
#include <crypto/internal/aead.h>
+#include <crypto/internal/des.h>
#include <crypto/sha1.h>
#include <crypto/sha2.h>
#include <crypto/skcipher.h>
@@ -43,7 +44,6 @@
#define SEC_TOTAL_IV_SZ (SEC_IV_SIZE * QM_Q_DEPTH)
#define SEC_SGL_SGE_NR 128
-#define SEC_CTX_DEV(ctx) (&(ctx)->sec->qm.pdev->dev)
#define SEC_CIPHER_AUTH 0xfe
#define SEC_AUTH_CIPHER 0x1
#define SEC_MAX_MAC_LEN 64
@@ -96,7 +96,7 @@ static int sec_alloc_req_id(struct sec_req *req, struct sec_qp_ctx *qp_ctx)
0, QM_Q_DEPTH, GFP_ATOMIC);
mutex_unlock(&qp_ctx->req_lock);
if (unlikely(req_id < 0)) {
- dev_err(SEC_CTX_DEV(req->ctx), "alloc req id fail!\n");
+ dev_err(req->ctx->dev, "alloc req id fail!\n");
return req_id;
}
@@ -112,7 +112,7 @@ static void sec_free_req_id(struct sec_req *req)
int req_id = req->req_id;
if (unlikely(req_id < 0 || req_id >= QM_Q_DEPTH)) {
- dev_err(SEC_CTX_DEV(req->ctx), "free request id invalid!\n");
+ dev_err(req->ctx->dev, "free request id invalid!\n");
return;
}
@@ -138,7 +138,7 @@ static int sec_aead_verify(struct sec_req *req)
aead_req->cryptlen + aead_req->assoclen -
authsize);
if (unlikely(sz != authsize || memcmp(mac_out, mac, sz))) {
- dev_err(SEC_CTX_DEV(req->ctx), "aead verify failure!\n");
+ dev_err(req->ctx->dev, "aead verify failure!\n");
return -EBADMSG;
}
@@ -177,7 +177,7 @@ static void sec_req_cb(struct hisi_qp *qp, void *resp)
if (unlikely(req->err_type || done != SEC_SQE_DONE ||
(ctx->alg_type == SEC_SKCIPHER && flag != SEC_SQE_CFLAG) ||
(ctx->alg_type == SEC_AEAD && flag != SEC_SQE_AEAD_FLAG))) {
- dev_err(SEC_CTX_DEV(ctx),
+ dev_err_ratelimited(ctx->dev,
"err_type[%d],done[%d],flag[%d]\n",
req->err_type, done, flag);
err = -EIO;
@@ -326,8 +326,8 @@ static int sec_alloc_pbuf_resource(struct device *dev, struct sec_alg_res *res)
static int sec_alg_resource_alloc(struct sec_ctx *ctx,
struct sec_qp_ctx *qp_ctx)
{
- struct device *dev = SEC_CTX_DEV(ctx);
struct sec_alg_res *res = qp_ctx->res;
+ struct device *dev = ctx->dev;
int ret;
ret = sec_alloc_civ_resource(dev, res);
@@ -360,7 +360,7 @@ alloc_fail:
static void sec_alg_resource_free(struct sec_ctx *ctx,
struct sec_qp_ctx *qp_ctx)
{
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
sec_free_civ_resource(dev, qp_ctx->res);
@@ -373,7 +373,7 @@ static void sec_alg_resource_free(struct sec_ctx *ctx,
static int sec_create_qp_ctx(struct hisi_qm *qm, struct sec_ctx *ctx,
int qp_ctx_id, int alg_type)
{
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
struct sec_qp_ctx *qp_ctx;
struct hisi_qp *qp;
int ret = -ENOMEM;
@@ -428,7 +428,7 @@ err_destroy_idr:
static void sec_release_qp_ctx(struct sec_ctx *ctx,
struct sec_qp_ctx *qp_ctx)
{
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
hisi_qm_stop_qp(qp_ctx->qp);
sec_alg_resource_free(ctx, qp_ctx);
@@ -452,6 +452,7 @@ static int sec_ctx_base_init(struct sec_ctx *ctx)
sec = container_of(ctx->qps[0]->qm, struct sec_dev, qm);
ctx->sec = sec;
+ ctx->dev = &sec->qm.pdev->dev;
ctx->hlf_q_num = sec->ctx_q_num >> 1;
ctx->pbuf_supported = ctx->sec->iommu_used;
@@ -476,11 +477,9 @@ static int sec_ctx_base_init(struct sec_ctx *ctx)
err_sec_release_qp_ctx:
for (i = i - 1; i >= 0; i--)
sec_release_qp_ctx(ctx, &ctx->qp_ctx[i]);
-
kfree(ctx->qp_ctx);
err_destroy_qps:
sec_destroy_qps(ctx->qps, sec->ctx_q_num);
-
return ret;
}
@@ -499,7 +498,7 @@ static int sec_cipher_init(struct sec_ctx *ctx)
{
struct sec_cipher_ctx *c_ctx = &ctx->c_ctx;
- c_ctx->c_key = dma_alloc_coherent(SEC_CTX_DEV(ctx), SEC_MAX_KEY_SIZE,
+ c_ctx->c_key = dma_alloc_coherent(ctx->dev, SEC_MAX_KEY_SIZE,
&c_ctx->c_key_dma, GFP_KERNEL);
if (!c_ctx->c_key)
return -ENOMEM;
@@ -512,7 +511,7 @@ static void sec_cipher_uninit(struct sec_ctx *ctx)
struct sec_cipher_ctx *c_ctx = &ctx->c_ctx;
memzero_explicit(c_ctx->c_key, SEC_MAX_KEY_SIZE);
- dma_free_coherent(SEC_CTX_DEV(ctx), SEC_MAX_KEY_SIZE,
+ dma_free_coherent(ctx->dev, SEC_MAX_KEY_SIZE,
c_ctx->c_key, c_ctx->c_key_dma);
}
@@ -520,7 +519,7 @@ static int sec_auth_init(struct sec_ctx *ctx)
{
struct sec_auth_ctx *a_ctx = &ctx->a_ctx;
- a_ctx->a_key = dma_alloc_coherent(SEC_CTX_DEV(ctx), SEC_MAX_KEY_SIZE,
+ a_ctx->a_key = dma_alloc_coherent(ctx->dev, SEC_MAX_KEY_SIZE,
&a_ctx->a_key_dma, GFP_KERNEL);
if (!a_ctx->a_key)
return -ENOMEM;
@@ -533,7 +532,7 @@ static void sec_auth_uninit(struct sec_ctx *ctx)
struct sec_auth_ctx *a_ctx = &ctx->a_ctx;
memzero_explicit(a_ctx->a_key, SEC_MAX_KEY_SIZE);
- dma_free_coherent(SEC_CTX_DEV(ctx), SEC_MAX_KEY_SIZE,
+ dma_free_coherent(ctx->dev, SEC_MAX_KEY_SIZE,
a_ctx->a_key, a_ctx->a_key_dma);
}
@@ -546,7 +545,7 @@ static int sec_skcipher_init(struct crypto_skcipher *tfm)
crypto_skcipher_set_reqsize(tfm, sizeof(struct sec_req));
ctx->c_ctx.ivsize = crypto_skcipher_ivsize(tfm);
if (ctx->c_ctx.ivsize > SEC_IV_SIZE) {
- dev_err(SEC_CTX_DEV(ctx), "get error skcipher iv size!\n");
+ pr_err("get error skcipher iv size!\n");
return -EINVAL;
}
@@ -573,10 +572,18 @@ static void sec_skcipher_uninit(struct crypto_skcipher *tfm)
sec_ctx_base_uninit(ctx);
}
-static int sec_skcipher_3des_setkey(struct sec_cipher_ctx *c_ctx,
+static int sec_skcipher_3des_setkey(struct crypto_skcipher *tfm, const u8 *key,
const u32 keylen,
const enum sec_cmode c_mode)
{
+ struct sec_ctx *ctx = crypto_skcipher_ctx(tfm);
+ struct sec_cipher_ctx *c_ctx = &ctx->c_ctx;
+ int ret;
+
+ ret = verify_skcipher_des3_key(tfm, key);
+ if (ret)
+ return ret;
+
switch (keylen) {
case SEC_DES3_2KEY_SIZE:
c_ctx->c_key_len = SEC_CKEY_3DES_2KEY;
@@ -633,12 +640,13 @@ static int sec_skcipher_setkey(struct crypto_skcipher *tfm, const u8 *key,
{
struct sec_ctx *ctx = crypto_skcipher_ctx(tfm);
struct sec_cipher_ctx *c_ctx = &ctx->c_ctx;
+ struct device *dev = ctx->dev;
int ret;
if (c_mode == SEC_CMODE_XTS) {
ret = xts_verify_key(tfm, key, keylen);
if (ret) {
- dev_err(SEC_CTX_DEV(ctx), "xts mode key err!\n");
+ dev_err(dev, "xts mode key err!\n");
return ret;
}
}
@@ -648,7 +656,7 @@ static int sec_skcipher_setkey(struct crypto_skcipher *tfm, const u8 *key,
switch (c_alg) {
case SEC_CALG_3DES:
- ret = sec_skcipher_3des_setkey(c_ctx, keylen, c_mode);
+ ret = sec_skcipher_3des_setkey(tfm, key, keylen, c_mode);
break;
case SEC_CALG_AES:
case SEC_CALG_SM4:
@@ -659,7 +667,7 @@ static int sec_skcipher_setkey(struct crypto_skcipher *tfm, const u8 *key,
}
if (ret) {
- dev_err(SEC_CTX_DEV(ctx), "set sec key err!\n");
+ dev_err(dev, "set sec key err!\n");
return ret;
}
@@ -691,7 +699,7 @@ static int sec_cipher_pbuf_map(struct sec_ctx *ctx, struct sec_req *req,
struct aead_request *aead_req = req->aead_req.aead_req;
struct sec_cipher_req *c_req = &req->c_req;
struct sec_qp_ctx *qp_ctx = req->qp_ctx;
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
int copy_size, pbuf_length;
int req_id = req->req_id;
@@ -701,21 +709,14 @@ static int sec_cipher_pbuf_map(struct sec_ctx *ctx, struct sec_req *req,
copy_size = c_req->c_len;
pbuf_length = sg_copy_to_buffer(src, sg_nents(src),
- qp_ctx->res[req_id].pbuf,
- copy_size);
-
+ qp_ctx->res[req_id].pbuf,
+ copy_size);
if (unlikely(pbuf_length != copy_size)) {
dev_err(dev, "copy src data to pbuf error!\n");
return -EINVAL;
}
c_req->c_in_dma = qp_ctx->res[req_id].pbuf_dma;
-
- if (!c_req->c_in_dma) {
- dev_err(dev, "fail to set pbuffer address!\n");
- return -ENOMEM;
- }
-
c_req->c_out_dma = c_req->c_in_dma;
return 0;
@@ -727,7 +728,7 @@ static void sec_cipher_pbuf_unmap(struct sec_ctx *ctx, struct sec_req *req,
struct aead_request *aead_req = req->aead_req.aead_req;
struct sec_cipher_req *c_req = &req->c_req;
struct sec_qp_ctx *qp_ctx = req->qp_ctx;
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
int copy_size, pbuf_length;
int req_id = req->req_id;
@@ -739,7 +740,6 @@ static void sec_cipher_pbuf_unmap(struct sec_ctx *ctx, struct sec_req *req,
pbuf_length = sg_copy_from_buffer(dst, sg_nents(dst),
qp_ctx->res[req_id].pbuf,
copy_size);
-
if (unlikely(pbuf_length != copy_size))
dev_err(dev, "copy pbuf data to dst error!\n");
}
@@ -751,7 +751,7 @@ static int sec_cipher_map(struct sec_ctx *ctx, struct sec_req *req,
struct sec_aead_req *a_req = &req->aead_req;
struct sec_qp_ctx *qp_ctx = req->qp_ctx;
struct sec_alg_res *res = &qp_ctx->res[req->req_id];
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
int ret;
if (req->use_pbuf) {
@@ -806,7 +806,7 @@ static void sec_cipher_unmap(struct sec_ctx *ctx, struct sec_req *req,
struct scatterlist *src, struct scatterlist *dst)
{
struct sec_cipher_req *c_req = &req->c_req;
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
if (req->use_pbuf) {
sec_cipher_pbuf_unmap(ctx, req, dst);
@@ -891,6 +891,7 @@ static int sec_aead_setkey(struct crypto_aead *tfm, const u8 *key,
{
struct sec_ctx *ctx = crypto_aead_ctx(tfm);
struct sec_cipher_ctx *c_ctx = &ctx->c_ctx;
+ struct device *dev = ctx->dev;
struct crypto_authenc_keys keys;
int ret;
@@ -904,13 +905,13 @@ static int sec_aead_setkey(struct crypto_aead *tfm, const u8 *key,
ret = sec_aead_aes_set_key(c_ctx, &keys);
if (ret) {
- dev_err(SEC_CTX_DEV(ctx), "set sec cipher key err!\n");
+ dev_err(dev, "set sec cipher key err!\n");
goto bad_key;
}
ret = sec_aead_auth_set_key(&ctx->a_ctx, &keys);
if (ret) {
- dev_err(SEC_CTX_DEV(ctx), "set sec auth key err!\n");
+ dev_err(dev, "set sec auth key err!\n");
goto bad_key;
}
@@ -1062,7 +1063,7 @@ static void sec_update_iv(struct sec_req *req, enum sec_alg_type alg_type)
sz = sg_pcopy_to_buffer(sgl, sg_nents(sgl), iv, iv_size,
cryptlen - iv_size);
if (unlikely(sz != iv_size))
- dev_err(SEC_CTX_DEV(req->ctx), "copy output iv error!\n");
+ dev_err(req->ctx->dev, "copy output iv error!\n");
}
static struct sec_req *sec_back_req_clear(struct sec_ctx *ctx,
@@ -1160,7 +1161,7 @@ static int sec_aead_bd_fill(struct sec_ctx *ctx, struct sec_req *req)
ret = sec_skcipher_bd_fill(ctx, req);
if (unlikely(ret)) {
- dev_err(SEC_CTX_DEV(ctx), "skcipher bd fill is error!\n");
+ dev_err(ctx->dev, "skcipher bd fill is error!\n");
return ret;
}
@@ -1194,7 +1195,7 @@ static void sec_aead_callback(struct sec_ctx *c, struct sec_req *req, int err)
a_req->assoclen);
if (unlikely(sz != authsize)) {
- dev_err(SEC_CTX_DEV(req->ctx), "copy out mac err!\n");
+ dev_err(c->dev, "copy out mac err!\n");
err = -EINVAL;
}
}
@@ -1259,7 +1260,7 @@ static int sec_process(struct sec_ctx *ctx, struct sec_req *req)
ret = ctx->req_op->bd_send(ctx, req);
if (unlikely((ret != -EBUSY && ret != -EINPROGRESS) ||
(ret == -EBUSY && !(req->flag & CRYPTO_TFM_REQ_MAY_BACKLOG)))) {
- dev_err_ratelimited(SEC_CTX_DEV(ctx), "send sec request failed!\n");
+ dev_err_ratelimited(ctx->dev, "send sec request failed!\n");
goto err_send_req;
}
@@ -1325,7 +1326,7 @@ static int sec_aead_init(struct crypto_aead *tfm)
ctx->alg_type = SEC_AEAD;
ctx->c_ctx.ivsize = crypto_aead_ivsize(tfm);
if (ctx->c_ctx.ivsize > SEC_IV_SIZE) {
- dev_err(SEC_CTX_DEV(ctx), "get error aead iv size!\n");
+ dev_err(ctx->dev, "get error aead iv size!\n");
return -EINVAL;
}
@@ -1374,7 +1375,7 @@ static int sec_aead_ctx_init(struct crypto_aead *tfm, const char *hash_name)
auth_ctx->hash_tfm = crypto_alloc_shash(hash_name, 0, 0);
if (IS_ERR(auth_ctx->hash_tfm)) {
- dev_err(SEC_CTX_DEV(ctx), "aead alloc shash error!\n");
+ dev_err(ctx->dev, "aead alloc shash error!\n");
sec_aead_exit(tfm);
return PTR_ERR(auth_ctx->hash_tfm);
}
@@ -1405,10 +1406,40 @@ static int sec_aead_sha512_ctx_init(struct crypto_aead *tfm)
return sec_aead_ctx_init(tfm, "sha512");
}
+
+static int sec_skcipher_cryptlen_ckeck(struct sec_ctx *ctx,
+ struct sec_req *sreq)
+{
+ u32 cryptlen = sreq->c_req.sk_req->cryptlen;
+ struct device *dev = ctx->dev;
+ u8 c_mode = ctx->c_ctx.c_mode;
+ int ret = 0;
+
+ switch (c_mode) {
+ case SEC_CMODE_XTS:
+ if (unlikely(cryptlen < AES_BLOCK_SIZE)) {
+ dev_err(dev, "skcipher XTS mode input length error!\n");
+ ret = -EINVAL;
+ }
+ break;
+ case SEC_CMODE_ECB:
+ case SEC_CMODE_CBC:
+ if (unlikely(cryptlen & (AES_BLOCK_SIZE - 1))) {
+ dev_err(dev, "skcipher AES input length error!\n");
+ ret = -EINVAL;
+ }
+ break;
+ default:
+ ret = -EINVAL;
+ }
+
+ return ret;
+}
+
static int sec_skcipher_param_check(struct sec_ctx *ctx, struct sec_req *sreq)
{
struct skcipher_request *sk_req = sreq->c_req.sk_req;
- struct device *dev = SEC_CTX_DEV(ctx);
+ struct device *dev = ctx->dev;
u8 c_alg = ctx->c_ctx.c_alg;
if (unlikely(!sk_req->src || !sk_req->dst)) {
@@ -1429,12 +1460,9 @@ static int sec_skcipher_param_check(struct sec_ctx *ctx, struct sec_req *sreq)
}
return 0;
} else if (c_alg == SEC_CALG_AES || c_alg == SEC_CALG_SM4) {
- if (unlikely(sk_req->cryptlen & (AES_BLOCK_SIZE - 1))) {
- dev_err(dev, "skcipher aes input length error!\n");
- return -EINVAL;
- }
- return 0;
+ return sec_skcipher_cryptlen_ckeck(ctx, sreq);
}
+
dev_err(dev, "skcipher algorithm error!\n");
return -EINVAL;
@@ -1531,14 +1559,15 @@ static struct skcipher_alg sec_skciphers[] = {
static int sec_aead_param_check(struct sec_ctx *ctx, struct sec_req *sreq)
{
- u8 c_alg = ctx->c_ctx.c_alg;
struct aead_request *req = sreq->aead_req.aead_req;
struct crypto_aead *tfm = crypto_aead_reqtfm(req);
size_t authsize = crypto_aead_authsize(tfm);
+ struct device *dev = ctx->dev;
+ u8 c_alg = ctx->c_ctx.c_alg;
if (unlikely(!req->src || !req->dst || !req->cryptlen ||
req->assoclen > SEC_MAX_AAD_LEN)) {
- dev_err(SEC_CTX_DEV(ctx), "aead input param error!\n");
+ dev_err(dev, "aead input param error!\n");
return -EINVAL;
}
@@ -1550,7 +1579,7 @@ static int sec_aead_param_check(struct sec_ctx *ctx, struct sec_req *sreq)
/* Support AES only */
if (unlikely(c_alg != SEC_CALG_AES)) {
- dev_err(SEC_CTX_DEV(ctx), "aead crypto alg error!\n");
+ dev_err(dev, "aead crypto alg error!\n");
return -EINVAL;
}
if (sreq->c_req.encrypt)
@@ -1559,7 +1588,7 @@ static int sec_aead_param_check(struct sec_ctx *ctx, struct sec_req *sreq)
sreq->c_req.c_len = req->cryptlen - authsize;
if (unlikely(sreq->c_req.c_len & (AES_BLOCK_SIZE - 1))) {
- dev_err(SEC_CTX_DEV(ctx), "aead crypto length error!\n");
+ dev_err(dev, "aead crypto length error!\n");
return -EINVAL;
}
@@ -1634,7 +1663,7 @@ static struct aead_alg sec_aeads[] = {
AES_BLOCK_SIZE, AES_BLOCK_SIZE, SHA512_DIGEST_SIZE),
};
-int sec_register_to_crypto(void)
+int sec_register_to_crypto(struct hisi_qm *qm)
{
int ret;
@@ -1651,7 +1680,7 @@ int sec_register_to_crypto(void)
return ret;
}
-void sec_unregister_from_crypto(void)
+void sec_unregister_from_crypto(struct hisi_qm *qm)
{
crypto_unregister_skciphers(sec_skciphers,
ARRAY_SIZE(sec_skciphers));
diff --git a/drivers/crypto/hisilicon/sec2/sec_crypto.h b/drivers/crypto/hisilicon/sec2/sec_crypto.h
index b2786e17d8fe..9c78edac56a4 100644
--- a/drivers/crypto/hisilicon/sec2/sec_crypto.h
+++ b/drivers/crypto/hisilicon/sec2/sec_crypto.h
@@ -64,7 +64,6 @@ enum sec_addr_type {
};
struct sec_sqe_type2 {
-
/*
* mac_len: 0~4 bits
* a_key_len: 5~10 bits
@@ -120,7 +119,6 @@ struct sec_sqe_type2 {
/* c_pad_len_field: 0~1 bits */
__le16 c_pad_len_field;
-
__le64 long_a_data_len;
__le64 a_ivin_addr;
__le64 a_key_addr;
@@ -211,6 +209,6 @@ struct sec_sqe {
struct sec_sqe_type2 type2;
};
-int sec_register_to_crypto(void);
-void sec_unregister_from_crypto(void);
+int sec_register_to_crypto(struct hisi_qm *qm);
+void sec_unregister_from_crypto(struct hisi_qm *qm);
#endif
diff --git a/drivers/crypto/hisilicon/sec2/sec_main.c b/drivers/crypto/hisilicon/sec2/sec_main.c
index dc68ba76f65e..6f0062d4408c 100644
--- a/drivers/crypto/hisilicon/sec2/sec_main.c
+++ b/drivers/crypto/hisilicon/sec2/sec_main.c
@@ -19,7 +19,6 @@
#define SEC_VF_NUM 63
#define SEC_QUEUE_NUM_V1 4096
-#define SEC_QUEUE_NUM_V2 1024
#define SEC_PF_PCI_DEVICE_ID 0xa255
#define SEC_VF_PCI_DEVICE_ID 0xa256
@@ -35,18 +34,16 @@
#define SEC_CTX_Q_NUM_MAX 32
#define SEC_CTRL_CNT_CLR_CE 0x301120
-#define SEC_CTRL_CNT_CLR_CE_BIT BIT(0)
-#define SEC_ENGINE_PF_CFG_OFF 0x300000
-#define SEC_ACC_COMMON_REG_OFF 0x1000
+#define SEC_CTRL_CNT_CLR_CE_BIT BIT(0)
#define SEC_CORE_INT_SOURCE 0x301010
#define SEC_CORE_INT_MASK 0x301000
#define SEC_CORE_INT_STATUS 0x301008
#define SEC_CORE_SRAM_ECC_ERR_INFO 0x301C14
-#define SEC_ECC_NUM(err) (((err) >> 16) & 0xFF)
-#define SEC_ECC_ADDR(err) ((err) >> 0)
+#define SEC_ECC_NUM 16
+#define SEC_ECC_MASH 0xFF
#define SEC_CORE_INT_DISABLE 0x0
-#define SEC_CORE_INT_ENABLE 0x1ff
-#define SEC_CORE_INT_CLEAR 0x1ff
+#define SEC_CORE_INT_ENABLE 0x7c1ff
+#define SEC_CORE_INT_CLEAR 0x7c1ff
#define SEC_SAA_ENABLE 0x17f
#define SEC_RAS_CE_REG 0x301050
@@ -54,24 +51,24 @@
#define SEC_RAS_NFE_REG 0x301058
#define SEC_RAS_CE_ENB_MSK 0x88
#define SEC_RAS_FE_ENB_MSK 0x0
-#define SEC_RAS_NFE_ENB_MSK 0x177
-#define SEC_RAS_DISABLE 0x0
-#define SEC_MEM_START_INIT_REG 0x0100
-#define SEC_MEM_INIT_DONE_REG 0x0104
+#define SEC_RAS_NFE_ENB_MSK 0x7c177
+#define SEC_RAS_DISABLE 0x0
+#define SEC_MEM_START_INIT_REG 0x301100
+#define SEC_MEM_INIT_DONE_REG 0x301104
-#define SEC_CONTROL_REG 0x0200
+#define SEC_CONTROL_REG 0x301200
#define SEC_TRNG_EN_SHIFT 8
#define SEC_CLK_GATE_ENABLE BIT(3)
#define SEC_CLK_GATE_DISABLE (~BIT(3))
#define SEC_AXI_SHUTDOWN_ENABLE BIT(12)
#define SEC_AXI_SHUTDOWN_DISABLE 0xFFFFEFFF
-#define SEC_INTERFACE_USER_CTRL0_REG 0x0220
-#define SEC_INTERFACE_USER_CTRL1_REG 0x0224
-#define SEC_SAA_EN_REG 0x0270
-#define SEC_BD_ERR_CHK_EN_REG0 0x0380
-#define SEC_BD_ERR_CHK_EN_REG1 0x0384
-#define SEC_BD_ERR_CHK_EN_REG3 0x038c
+#define SEC_INTERFACE_USER_CTRL0_REG 0x301220
+#define SEC_INTERFACE_USER_CTRL1_REG 0x301224
+#define SEC_SAA_EN_REG 0x301270
+#define SEC_BD_ERR_CHK_EN_REG0 0x301380
+#define SEC_BD_ERR_CHK_EN_REG1 0x301384
+#define SEC_BD_ERR_CHK_EN_REG3 0x30138c
#define SEC_USER0_SMMU_NORMAL (BIT(23) | BIT(15))
#define SEC_USER1_SMMU_NORMAL (BIT(31) | BIT(23) | BIT(15) | BIT(7))
@@ -95,9 +92,6 @@
#define SEC_SQE_MASK_OFFSET 64
#define SEC_SQE_MASK_LEN 48
-#define SEC_ADDR(qm, offset) ((qm)->io_base + (offset) + \
- SEC_ENGINE_PF_CFG_OFF + SEC_ACC_COMMON_REG_OFF)
-
struct sec_hw_error {
u32 int_msk;
const char *msg;
@@ -117,20 +111,66 @@ static struct hisi_qm_list sec_devices = {
};
static const struct sec_hw_error sec_hw_errors[] = {
- {.int_msk = BIT(0), .msg = "sec_axi_rresp_err_rint"},
- {.int_msk = BIT(1), .msg = "sec_axi_bresp_err_rint"},
- {.int_msk = BIT(2), .msg = "sec_ecc_2bit_err_rint"},
- {.int_msk = BIT(3), .msg = "sec_ecc_1bit_err_rint"},
- {.int_msk = BIT(4), .msg = "sec_req_trng_timeout_rint"},
- {.int_msk = BIT(5), .msg = "sec_fsm_hbeat_rint"},
- {.int_msk = BIT(6), .msg = "sec_channel_req_rng_timeout_rint"},
- {.int_msk = BIT(7), .msg = "sec_bd_err_rint"},
- {.int_msk = BIT(8), .msg = "sec_chain_buff_err_rint"},
- { /* sentinel */ }
+ {
+ .int_msk = BIT(0),
+ .msg = "sec_axi_rresp_err_rint"
+ },
+ {
+ .int_msk = BIT(1),
+ .msg = "sec_axi_bresp_err_rint"
+ },
+ {
+ .int_msk = BIT(2),
+ .msg = "sec_ecc_2bit_err_rint"
+ },
+ {
+ .int_msk = BIT(3),
+ .msg = "sec_ecc_1bit_err_rint"
+ },
+ {
+ .int_msk = BIT(4),
+ .msg = "sec_req_trng_timeout_rint"
+ },
+ {
+ .int_msk = BIT(5),
+ .msg = "sec_fsm_hbeat_rint"
+ },
+ {
+ .int_msk = BIT(6),
+ .msg = "sec_channel_req_rng_timeout_rint"
+ },
+ {
+ .int_msk = BIT(7),
+ .msg = "sec_bd_err_rint"
+ },
+ {
+ .int_msk = BIT(8),
+ .msg = "sec_chain_buff_err_rint"
+ },
+ {
+ .int_msk = BIT(14),
+ .msg = "sec_no_secure_access"
+ },
+ {
+ .int_msk = BIT(15),
+ .msg = "sec_wrapping_key_auth_err"
+ },
+ {
+ .int_msk = BIT(16),
+ .msg = "sec_km_key_crc_fail"
+ },
+ {
+ .int_msk = BIT(17),
+ .msg = "sec_axi_poison_err"
+ },
+ {
+ .int_msk = BIT(18),
+ .msg = "sec_sva_err"
+ },
+ {}
};
static const char * const sec_dbg_file_name[] = {
- [SEC_CURRENT_QM] = "current_qm",
[SEC_CLEAR_ENABLE] = "clear_enable",
};
@@ -277,9 +317,7 @@ static u8 sec_get_endian(struct hisi_qm *qm)
"cannot access a register in VF!\n");
return SEC_LE;
}
- reg = readl_relaxed(qm->io_base + SEC_ENGINE_PF_CFG_OFF +
- SEC_ACC_COMMON_REG_OFF + SEC_CONTROL_REG);
-
+ reg = readl_relaxed(qm->io_base + SEC_CONTROL_REG);
/* BD little endian mode */
if (!(reg & BIT(0)))
return SEC_LE;
@@ -299,13 +337,13 @@ static int sec_engine_init(struct hisi_qm *qm)
u32 reg;
/* disable clock gate control */
- reg = readl_relaxed(SEC_ADDR(qm, SEC_CONTROL_REG));
+ reg = readl_relaxed(qm->io_base + SEC_CONTROL_REG);
reg &= SEC_CLK_GATE_DISABLE;
- writel_relaxed(reg, SEC_ADDR(qm, SEC_CONTROL_REG));
+ writel_relaxed(reg, qm->io_base + SEC_CONTROL_REG);
- writel_relaxed(0x1, SEC_ADDR(qm, SEC_MEM_START_INIT_REG));
+ writel_relaxed(0x1, qm->io_base + SEC_MEM_START_INIT_REG);
- ret = readl_relaxed_poll_timeout(SEC_ADDR(qm, SEC_MEM_INIT_DONE_REG),
+ ret = readl_relaxed_poll_timeout(qm->io_base + SEC_MEM_INIT_DONE_REG,
reg, reg & 0x1, SEC_DELAY_10_US,
SEC_POLL_TIMEOUT_US);
if (ret) {
@@ -313,40 +351,40 @@ static int sec_engine_init(struct hisi_qm *qm)
return ret;
}
- reg = readl_relaxed(SEC_ADDR(qm, SEC_CONTROL_REG));
+ reg = readl_relaxed(qm->io_base + SEC_CONTROL_REG);
reg |= (0x1 << SEC_TRNG_EN_SHIFT);
- writel_relaxed(reg, SEC_ADDR(qm, SEC_CONTROL_REG));
+ writel_relaxed(reg, qm->io_base + SEC_CONTROL_REG);
- reg = readl_relaxed(SEC_ADDR(qm, SEC_INTERFACE_USER_CTRL0_REG));
+ reg = readl_relaxed(qm->io_base + SEC_INTERFACE_USER_CTRL0_REG);
reg |= SEC_USER0_SMMU_NORMAL;
- writel_relaxed(reg, SEC_ADDR(qm, SEC_INTERFACE_USER_CTRL0_REG));
+ writel_relaxed(reg, qm->io_base + SEC_INTERFACE_USER_CTRL0_REG);
- reg = readl_relaxed(SEC_ADDR(qm, SEC_INTERFACE_USER_CTRL1_REG));
+ reg = readl_relaxed(qm->io_base + SEC_INTERFACE_USER_CTRL1_REG);
reg &= SEC_USER1_SMMU_MASK;
if (qm->use_sva && qm->ver == QM_HW_V2)
reg |= SEC_USER1_SMMU_SVA;
else
reg |= SEC_USER1_SMMU_NORMAL;
- writel_relaxed(reg, SEC_ADDR(qm, SEC_INTERFACE_USER_CTRL1_REG));
+ writel_relaxed(reg, qm->io_base + SEC_INTERFACE_USER_CTRL1_REG);
writel(SEC_SINGLE_PORT_MAX_TRANS,
qm->io_base + AM_CFG_SINGLE_PORT_MAX_TRANS);
- writel(SEC_SAA_ENABLE, SEC_ADDR(qm, SEC_SAA_EN_REG));
+ writel(SEC_SAA_ENABLE, qm->io_base + SEC_SAA_EN_REG);
/* Enable sm4 extra mode, as ctr/ecb */
writel_relaxed(SEC_BD_ERR_CHK_EN0,
- SEC_ADDR(qm, SEC_BD_ERR_CHK_EN_REG0));
+ qm->io_base + SEC_BD_ERR_CHK_EN_REG0);
/* Enable sm4 xts mode multiple iv */
writel_relaxed(SEC_BD_ERR_CHK_EN1,
- SEC_ADDR(qm, SEC_BD_ERR_CHK_EN_REG1));
+ qm->io_base + SEC_BD_ERR_CHK_EN_REG1);
writel_relaxed(SEC_BD_ERR_CHK_EN3,
- SEC_ADDR(qm, SEC_BD_ERR_CHK_EN_REG3));
+ qm->io_base + SEC_BD_ERR_CHK_EN_REG3);
/* config endian */
- reg = readl_relaxed(SEC_ADDR(qm, SEC_CONTROL_REG));
+ reg = readl_relaxed(qm->io_base + SEC_CONTROL_REG);
reg |= sec_get_endian(qm);
- writel_relaxed(reg, SEC_ADDR(qm, SEC_CONTROL_REG));
+ writel_relaxed(reg, qm->io_base + SEC_CONTROL_REG);
return 0;
}
@@ -381,10 +419,6 @@ static void sec_debug_regs_clear(struct hisi_qm *qm)
{
int i;
- /* clear current_qm */
- writel(0x0, qm->io_base + QM_DFX_MB_CNT_VF);
- writel(0x0, qm->io_base + QM_DFX_DB_CNT_VF);
-
/* clear sec dfx regs */
writel(0x1, qm->io_base + SEC_CTRL_CNT_CLR_CE);
for (i = 0; i < ARRAY_SIZE(sec_dfx_regs); i++)
@@ -406,7 +440,7 @@ static void sec_hw_error_enable(struct hisi_qm *qm)
return;
}
- val = readl(SEC_ADDR(qm, SEC_CONTROL_REG));
+ val = readl(qm->io_base + SEC_CONTROL_REG);
/* clear SEC hw error source if having */
writel(SEC_CORE_INT_CLEAR, qm->io_base + SEC_CORE_INT_SOURCE);
@@ -422,14 +456,14 @@ static void sec_hw_error_enable(struct hisi_qm *qm)
/* enable SEC block master OOO when m-bit error occur */
val = val | SEC_AXI_SHUTDOWN_ENABLE;
- writel(val, SEC_ADDR(qm, SEC_CONTROL_REG));
+ writel(val, qm->io_base + SEC_CONTROL_REG);
}
static void sec_hw_error_disable(struct hisi_qm *qm)
{
u32 val;
- val = readl(SEC_ADDR(qm, SEC_CONTROL_REG));
+ val = readl(qm->io_base + SEC_CONTROL_REG);
/* disable RAS int */
writel(SEC_RAS_DISABLE, qm->io_base + SEC_RAS_CE_REG);
@@ -442,51 +476,7 @@ static void sec_hw_error_disable(struct hisi_qm *qm)
/* disable SEC block master OOO when m-bit error occur */
val = val & SEC_AXI_SHUTDOWN_DISABLE;
- writel(val, SEC_ADDR(qm, SEC_CONTROL_REG));
-}
-
-static u32 sec_current_qm_read(struct sec_debug_file *file)
-{
- struct hisi_qm *qm = file->qm;
-
- return readl(qm->io_base + QM_DFX_MB_CNT_VF);
-}
-
-static int sec_current_qm_write(struct sec_debug_file *file, u32 val)
-{
- struct hisi_qm *qm = file->qm;
- u32 vfq_num;
- u32 tmp;
-
- if (val > qm->vfs_num)
- return -EINVAL;
-
- /* According PF or VF Dev ID to calculation curr_qm_qp_num and store */
- if (!val) {
- qm->debug.curr_qm_qp_num = qm->qp_num;
- } else {
- vfq_num = (qm->ctrl_qp_num - qm->qp_num) / qm->vfs_num;
-
- if (val == qm->vfs_num)
- qm->debug.curr_qm_qp_num =
- qm->ctrl_qp_num - qm->qp_num -
- (qm->vfs_num - 1) * vfq_num;
- else
- qm->debug.curr_qm_qp_num = vfq_num;
- }
-
- writel(val, qm->io_base + QM_DFX_MB_CNT_VF);
- writel(val, qm->io_base + QM_DFX_DB_CNT_VF);
-
- tmp = val |
- (readl(qm->io_base + QM_DFX_SQE_CNT_VF_SQN) & CURRENT_Q_MASK);
- writel(tmp, qm->io_base + QM_DFX_SQE_CNT_VF_SQN);
-
- tmp = val |
- (readl(qm->io_base + QM_DFX_CQE_CNT_VF_CQN) & CURRENT_Q_MASK);
- writel(tmp, qm->io_base + QM_DFX_CQE_CNT_VF_CQN);
-
- return 0;
+ writel(val, qm->io_base + SEC_CONTROL_REG);
}
static u32 sec_clear_enable_read(struct sec_debug_file *file)
@@ -523,9 +513,6 @@ static ssize_t sec_debug_read(struct file *filp, char __user *buf,
spin_lock_irq(&file->lock);
switch (file->index) {
- case SEC_CURRENT_QM:
- val = sec_current_qm_read(file);
- break;
case SEC_CLEAR_ENABLE:
val = sec_clear_enable_read(file);
break;
@@ -566,11 +553,6 @@ static ssize_t sec_debug_write(struct file *filp, const char __user *buf,
spin_lock_irq(&file->lock);
switch (file->index) {
- case SEC_CURRENT_QM:
- ret = sec_current_qm_write(file, val);
- if (ret)
- goto err_input;
- break;
case SEC_CLEAR_ENABLE:
ret = sec_clear_enable_write(file, val);
if (ret)
@@ -655,7 +637,7 @@ static int sec_debug_init(struct hisi_qm *qm)
int i;
if (qm->pdev->device == SEC_PF_PCI_DEVICE_ID) {
- for (i = SEC_CURRENT_QM; i < SEC_DEBUG_FILE_NUM; i++) {
+ for (i = SEC_CLEAR_ENABLE; i < SEC_DEBUG_FILE_NUM; i++) {
spin_lock_init(&sec->debug.files[i].lock);
sec->debug.files[i].index = i;
sec->debug.files[i].qm = qm;
@@ -712,7 +694,8 @@ static void sec_log_hw_error(struct hisi_qm *qm, u32 err_sts)
err_val = readl(qm->io_base +
SEC_CORE_SRAM_ECC_ERR_INFO);
dev_err(dev, "multi ecc sram num=0x%x\n",
- SEC_ECC_NUM(err_val));
+ ((err_val) >> SEC_ECC_NUM) &
+ SEC_ECC_MASH);
}
}
errs++;
@@ -733,9 +716,23 @@ static void sec_open_axi_master_ooo(struct hisi_qm *qm)
{
u32 val;
- val = readl(SEC_ADDR(qm, SEC_CONTROL_REG));
- writel(val & SEC_AXI_SHUTDOWN_DISABLE, SEC_ADDR(qm, SEC_CONTROL_REG));
- writel(val | SEC_AXI_SHUTDOWN_ENABLE, SEC_ADDR(qm, SEC_CONTROL_REG));
+ val = readl(qm->io_base + SEC_CONTROL_REG);
+ writel(val & SEC_AXI_SHUTDOWN_DISABLE, qm->io_base + SEC_CONTROL_REG);
+ writel(val | SEC_AXI_SHUTDOWN_ENABLE, qm->io_base + SEC_CONTROL_REG);
+}
+
+static void sec_err_info_init(struct hisi_qm *qm)
+{
+ struct hisi_qm_err_info *err_info = &qm->err_info;
+
+ err_info->ce = QM_BASE_CE;
+ err_info->fe = 0;
+ err_info->ecc_2bits_mask = SEC_CORE_INT_STATUS_M_ECC;
+ err_info->dev_ce_mask = SEC_RAS_CE_ENB_MSK;
+ err_info->msi_wr_port = BIT(0);
+ err_info->acpi_rst = "SRST";
+ err_info->nfe = QM_BASE_NFE | QM_ACC_DO_TASK_TIMEOUT |
+ QM_ACC_WB_NOT_READY_TIMEOUT;
}
static const struct hisi_qm_err_ini sec_err_ini = {
@@ -746,16 +743,7 @@ static const struct hisi_qm_err_ini sec_err_ini = {
.clear_dev_hw_err_status = sec_clear_hw_err_status,
.log_dev_hw_err = sec_log_hw_error,
.open_axi_master_ooo = sec_open_axi_master_ooo,
- .err_info = {
- .ce = QM_BASE_CE,
- .nfe = QM_BASE_NFE | QM_ACC_DO_TASK_TIMEOUT |
- QM_ACC_WB_NOT_READY_TIMEOUT,
- .fe = 0,
- .ecc_2bits_mask = SEC_CORE_INT_STATUS_M_ECC,
- .dev_ce_mask = SEC_RAS_CE_ENB_MSK,
- .msi_wr_port = BIT(0),
- .acpi_rst = "SRST",
- }
+ .err_info_init = sec_err_info_init,
};
static int sec_pf_probe_init(struct sec_dev *sec)
@@ -763,12 +751,8 @@ static int sec_pf_probe_init(struct sec_dev *sec)
struct hisi_qm *qm = &sec->qm;
int ret;
- if (qm->ver == QM_HW_V1)
- qm->ctrl_qp_num = SEC_QUEUE_NUM_V1;
- else
- qm->ctrl_qp_num = SEC_QUEUE_NUM_V2;
-
qm->err_ini = &sec_err_ini;
+ qm->err_ini->err_info_init(qm);
ret = sec_set_user_domain_and_cache(qm);
if (ret)
@@ -786,7 +770,7 @@ static int sec_qm_init(struct hisi_qm *qm, struct pci_dev *pdev)
qm->pdev = pdev;
qm->ver = pdev->revision;
- qm->algs = "cipher\ndigest\naead\n";
+ qm->algs = "cipher\ndigest\naead";
qm->mode = uacce_mode;
qm->sqe_size = SEC_SQE_SIZE;
qm->dev_name = sec_name;
@@ -909,10 +893,15 @@ static int sec_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (ret)
pci_warn(pdev, "Failed to init debugfs!\n");
- ret = hisi_qm_alg_register(qm, &sec_devices);
- if (ret < 0) {
- pr_err("Failed to register driver to crypto.\n");
- goto err_qm_stop;
+ if (qm->qp_num >= ctx_q_num) {
+ ret = hisi_qm_alg_register(qm, &sec_devices);
+ if (ret < 0) {
+ pr_err("Failed to register driver to crypto.\n");
+ goto err_qm_stop;
+ }
+ } else {
+ pci_warn(qm->pdev,
+ "Failed to use kernel mode, qp not enough!\n");
}
if (qm->uacce) {
@@ -948,7 +937,9 @@ static void sec_remove(struct pci_dev *pdev)
struct hisi_qm *qm = pci_get_drvdata(pdev);
hisi_qm_wait_task_finish(qm, &sec_devices);
- hisi_qm_alg_unregister(qm, &sec_devices);
+ if (qm->qp_num >= ctx_q_num)
+ hisi_qm_alg_unregister(qm, &sec_devices);
+
if (qm->fun_type == QM_HW_PF && qm->vfs_num)
hisi_qm_sriov_disable(pdev, true);
diff --git a/drivers/crypto/hisilicon/sgl.c b/drivers/crypto/hisilicon/sgl.c
index 3bff6394acaf..057273769f26 100644
--- a/drivers/crypto/hisilicon/sgl.c
+++ b/drivers/crypto/hisilicon/sgl.c
@@ -56,7 +56,7 @@ struct hisi_acc_sgl_pool {
struct hisi_acc_sgl_pool *hisi_acc_create_sgl_pool(struct device *dev,
u32 count, u32 sge_nr)
{
- u32 sgl_size, block_size, sgl_num_per_block, block_num, remain_sgl = 0;
+ u32 sgl_size, block_size, sgl_num_per_block, block_num, remain_sgl;
struct hisi_acc_sgl_pool *pool;
struct mem_block *block;
u32 i, j;
@@ -66,6 +66,11 @@ struct hisi_acc_sgl_pool *hisi_acc_create_sgl_pool(struct device *dev,
sgl_size = sizeof(struct acc_hw_sge) * sge_nr +
sizeof(struct hisi_acc_hw_sgl);
+
+ /*
+ * the pool may allocate a block of memory of size PAGE_SIZE * 2^(MAX_ORDER - 1),
+ * block size may exceed 2^31 on ia64, so the max of block size is 2^31
+ */
block_size = 1 << (PAGE_SHIFT + MAX_ORDER <= 32 ?
PAGE_SHIFT + MAX_ORDER - 1 : 31);
sgl_num_per_block = block_size / sgl_size;
@@ -85,8 +90,10 @@ struct hisi_acc_sgl_pool *hisi_acc_create_sgl_pool(struct device *dev,
block[i].sgl = dma_alloc_coherent(dev, block_size,
&block[i].sgl_dma,
GFP_KERNEL);
- if (!block[i].sgl)
+ if (!block[i].sgl) {
+ dev_err(dev, "Fail to allocate hw SG buffer!\n");
goto err_free_mem;
+ }
block[i].size = block_size;
}
@@ -95,8 +102,10 @@ struct hisi_acc_sgl_pool *hisi_acc_create_sgl_pool(struct device *dev,
block[i].sgl = dma_alloc_coherent(dev, remain_sgl * sgl_size,
&block[i].sgl_dma,
GFP_KERNEL);
- if (!block[i].sgl)
+ if (!block[i].sgl) {
+ dev_err(dev, "Fail to allocate remained hw SG buffer!\n");
goto err_free_mem;
+ }
block[i].size = remain_sgl * sgl_size;
}
@@ -167,6 +176,7 @@ static void sg_map_to_hw_sg(struct scatterlist *sgl,
{
hw_sge->buf = sg_dma_address(sgl);
hw_sge->len = cpu_to_le32(sg_dma_len(sgl));
+ hw_sge->page_ctrl = sg_virt(sgl);
}
static void inc_hw_sgl_sge(struct hisi_acc_hw_sgl *hw_sgl)
@@ -182,6 +192,18 @@ static void update_hw_sgl_sum_sge(struct hisi_acc_hw_sgl *hw_sgl, u16 sum)
hw_sgl->entry_sum_in_chain = cpu_to_le16(sum);
}
+static void clear_hw_sgl_sge(struct hisi_acc_hw_sgl *hw_sgl)
+{
+ struct acc_hw_sge *hw_sge = hw_sgl->sge_entries;
+ int i;
+
+ for (i = 0; i < le16_to_cpu(hw_sgl->entry_sum_in_sgl); i++) {
+ hw_sge[i].page_ctrl = NULL;
+ hw_sge[i].buf = 0;
+ hw_sge[i].len = 0;
+ }
+}
+
/**
* hisi_acc_sg_buf_map_to_hw_sgl - Map a scatterlist to a hw sgl.
* @dev: The device which hw sgl belongs to.
@@ -211,16 +233,19 @@ hisi_acc_sg_buf_map_to_hw_sgl(struct device *dev,
sg_n = sg_nents(sgl);
sg_n_mapped = dma_map_sg(dev, sgl, sg_n, DMA_BIDIRECTIONAL);
- if (!sg_n_mapped)
+ if (!sg_n_mapped) {
+ dev_err(dev, "DMA mapping for SG error!\n");
return ERR_PTR(-EINVAL);
+ }
if (sg_n_mapped > pool->sge_nr) {
- dma_unmap_sg(dev, sgl, sg_n, DMA_BIDIRECTIONAL);
+ dev_err(dev, "the number of entries in input scatterlist is bigger than SGL pool setting.\n");
return ERR_PTR(-EINVAL);
}
curr_hw_sgl = acc_get_sgl(pool, index, &curr_sgl_dma);
if (IS_ERR(curr_hw_sgl)) {
+ dev_err(dev, "Get SGL error!\n");
dma_unmap_sg(dev, sgl, sg_n, DMA_BIDIRECTIONAL);
return ERR_PTR(-ENOMEM);
@@ -256,7 +281,7 @@ void hisi_acc_sg_buf_unmap(struct device *dev, struct scatterlist *sgl,
return;
dma_unmap_sg(dev, sgl, sg_nents(sgl), DMA_BIDIRECTIONAL);
-
+ clear_hw_sgl_sge(hw_sgl);
hw_sgl->entry_sum_in_chain = 0;
hw_sgl->entry_sum_in_sgl = 0;
hw_sgl->entry_length_in_sgl = 0;
diff --git a/drivers/crypto/hisilicon/trng/trng.c b/drivers/crypto/hisilicon/trng/trng.c
index 29712685498a..829f2caf0f67 100644
--- a/drivers/crypto/hisilicon/trng/trng.c
+++ b/drivers/crypto/hisilicon/trng/trng.c
@@ -18,6 +18,8 @@
#define HISI_TRNG_REG 0x00F0
#define HISI_TRNG_BYTES 4
#define HISI_TRNG_QUALITY 512
+#define HISI_TRNG_VERSION 0x01B8
+#define HISI_TRNG_VER_V1 GENMASK(31, 0)
#define SLEEP_US 10
#define TIMEOUT_US 10000
#define SW_DRBG_NUM_SHIFT 2
@@ -50,6 +52,7 @@ struct hisi_trng {
struct hisi_trng_list *trng_list;
struct list_head list;
struct hwrng rng;
+ u32 ver;
bool is_used;
struct mutex mutex;
};
@@ -260,6 +263,7 @@ static int hisi_trng_probe(struct platform_device *pdev)
return PTR_ERR(trng->base);
trng->is_used = false;
+ trng->ver = readl(trng->base + HISI_TRNG_VERSION);
if (!trng_devices.is_init) {
INIT_LIST_HEAD(&trng_devices.list);
mutex_init(&trng_devices.lock);
@@ -267,7 +271,8 @@ static int hisi_trng_probe(struct platform_device *pdev)
}
hisi_trng_add_to_list(trng);
- if (atomic_inc_return(&trng_active_devs) == 1) {
+ if (trng->ver != HISI_TRNG_VER_V1 &&
+ atomic_inc_return(&trng_active_devs) == 1) {
ret = crypto_register_rng(&hisi_trng_alg);
if (ret) {
dev_err(&pdev->dev,
@@ -289,7 +294,8 @@ static int hisi_trng_probe(struct platform_device *pdev)
return ret;
err_crypto_unregister:
- if (atomic_dec_return(&trng_active_devs) == 0)
+ if (trng->ver != HISI_TRNG_VER_V1 &&
+ atomic_dec_return(&trng_active_devs) == 0)
crypto_unregister_rng(&hisi_trng_alg);
err_remove_from_list:
@@ -305,7 +311,8 @@ static int hisi_trng_remove(struct platform_device *pdev)
while (hisi_trng_del_from_list(trng))
;
- if (atomic_dec_return(&trng_active_devs) == 0)
+ if (trng->ver != HISI_TRNG_VER_V1 &&
+ atomic_dec_return(&trng_active_devs) == 0)
crypto_unregister_rng(&hisi_trng_alg);
return 0;
diff --git a/drivers/crypto/hisilicon/zip/zip.h b/drivers/crypto/hisilicon/zip/zip.h
index 92397f993e23..517fdbdff3ea 100644
--- a/drivers/crypto/hisilicon/zip/zip.h
+++ b/drivers/crypto/hisilicon/zip/zip.h
@@ -33,35 +33,55 @@ struct hisi_zip_sqe {
u32 consumed;
u32 produced;
u32 comp_data_length;
+ /*
+ * status: 0~7 bits
+ * rsvd: 8~31 bits
+ */
u32 dw3;
u32 input_data_length;
- u32 lba_l;
- u32 lba_h;
+ u32 dw5;
+ u32 dw6;
+ /*
+ * in_sge_data_offset: 0~23 bits
+ * rsvd: 24~27 bits
+ * sqe_type: 29~31 bits
+ */
u32 dw7;
+ /*
+ * out_sge_data_offset: 0~23 bits
+ * rsvd: 24~31 bits
+ */
u32 dw8;
+ /*
+ * request_type: 0~7 bits
+ * buffer_type: 8~11 bits
+ * rsvd: 13~31 bits
+ */
u32 dw9;
u32 dw10;
- u32 priv_info;
+ u32 dw11;
u32 dw12;
- u32 tag;
+ /* tag: in sqe type 0 */
+ u32 dw13;
u32 dest_avail_out;
- u32 rsvd0;
- u32 comp_head_addr_l;
- u32 comp_head_addr_h;
+ u32 dw15;
+ u32 dw16;
+ u32 dw17;
u32 source_addr_l;
u32 source_addr_h;
u32 dest_addr_l;
u32 dest_addr_h;
- u32 stream_ctx_addr_l;
- u32 stream_ctx_addr_h;
- u32 cipher_key1_addr_l;
- u32 cipher_key1_addr_h;
- u32 cipher_key2_addr_l;
- u32 cipher_key2_addr_h;
+ u32 dw22;
+ u32 dw23;
+ u32 dw24;
+ u32 dw25;
+ /* tag: in sqe type 3 */
+ u32 dw26;
+ u32 dw27;
u32 rsvd1[4];
};
int zip_create_qps(struct hisi_qp **qps, int ctx_num, int node);
-int hisi_zip_register_to_crypto(void);
-void hisi_zip_unregister_from_crypto(void);
+int hisi_zip_register_to_crypto(struct hisi_qm *qm);
+void hisi_zip_unregister_from_crypto(struct hisi_qm *qm);
#endif
diff --git a/drivers/crypto/hisilicon/zip/zip_crypto.c b/drivers/crypto/hisilicon/zip/zip_crypto.c
index 08b4660b014c..9520a4113c81 100644
--- a/drivers/crypto/hisilicon/zip/zip_crypto.c
+++ b/drivers/crypto/hisilicon/zip/zip_crypto.c
@@ -10,6 +10,7 @@
#define HZIP_BD_STATUS_M GENMASK(7, 0)
/* hisi_zip_sqe dw7 */
#define HZIP_IN_SGE_DATA_OFFSET_M GENMASK(23, 0)
+#define HZIP_SQE_TYPE_M GENMASK(31, 28)
/* hisi_zip_sqe dw8 */
#define HZIP_OUT_SGE_DATA_OFFSET_M GENMASK(23, 0)
/* hisi_zip_sqe dw9 */
@@ -91,8 +92,22 @@ struct hisi_zip_qp_ctx {
struct hisi_zip_ctx *ctx;
};
+struct hisi_zip_sqe_ops {
+ u8 sqe_type;
+ void (*fill_addr)(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req);
+ void (*fill_buf_size)(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req);
+ void (*fill_buf_type)(struct hisi_zip_sqe *sqe, u8 buf_type);
+ void (*fill_req_type)(struct hisi_zip_sqe *sqe, u8 req_type);
+ void (*fill_tag)(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req);
+ void (*fill_sqe_type)(struct hisi_zip_sqe *sqe, u8 sqe_type);
+ u32 (*get_tag)(struct hisi_zip_sqe *sqe);
+ u32 (*get_status)(struct hisi_zip_sqe *sqe);
+ u32 (*get_dstlen)(struct hisi_zip_sqe *sqe);
+};
+
struct hisi_zip_ctx {
struct hisi_zip_qp_ctx qp_ctx[HZIP_CTX_Q_NUM];
+ const struct hisi_zip_sqe_ops *ops;
};
static int sgl_sge_nr_set(const char *val, const struct kernel_param *kp)
@@ -119,35 +134,367 @@ static u16 sgl_sge_nr = HZIP_SGL_SGE_NR;
module_param_cb(sgl_sge_nr, &sgl_sge_nr_ops, &sgl_sge_nr, 0444);
MODULE_PARM_DESC(sgl_sge_nr, "Number of sge in sgl(1-255)");
-static void hisi_zip_config_buf_type(struct hisi_zip_sqe *sqe, u8 buf_type)
+static u16 get_extra_field_size(const u8 *start)
+{
+ return *((u16 *)start) + GZIP_HEAD_FEXTRA_XLEN;
+}
+
+static u32 get_name_field_size(const u8 *start)
+{
+ return strlen(start) + 1;
+}
+
+static u32 get_comment_field_size(const u8 *start)
+{
+ return strlen(start) + 1;
+}
+
+static u32 __get_gzip_head_size(const u8 *src)
+{
+ u8 head_flg = *(src + GZIP_HEAD_FLG_SHIFT);
+ u32 size = GZIP_HEAD_FEXTRA_SHIFT;
+
+ if (head_flg & GZIP_HEAD_FEXTRA_BIT)
+ size += get_extra_field_size(src + size);
+ if (head_flg & GZIP_HEAD_FNAME_BIT)
+ size += get_name_field_size(src + size);
+ if (head_flg & GZIP_HEAD_FCOMMENT_BIT)
+ size += get_comment_field_size(src + size);
+ if (head_flg & GZIP_HEAD_FHCRC_BIT)
+ size += GZIP_HEAD_FHCRC_SIZE;
+
+ return size;
+}
+
+static size_t __maybe_unused get_gzip_head_size(struct scatterlist *sgl)
+{
+ char buf[HZIP_GZIP_HEAD_BUF];
+
+ sg_copy_to_buffer(sgl, sg_nents(sgl), buf, sizeof(buf));
+
+ return __get_gzip_head_size(buf);
+}
+
+static int add_comp_head(struct scatterlist *dst, u8 req_type)
+{
+ int head_size = TO_HEAD_SIZE(req_type);
+ const u8 *head = TO_HEAD(req_type);
+ int ret;
+
+ ret = sg_copy_from_buffer(dst, sg_nents(dst), head, head_size);
+ if (ret != head_size) {
+ pr_err("the head size of buffer is wrong (%d)!\n", ret);
+ return -ENOMEM;
+ }
+
+ return head_size;
+}
+
+static int get_comp_head_size(struct acomp_req *acomp_req, u8 req_type)
+{
+ if (!acomp_req->src || !acomp_req->slen)
+ return -EINVAL;
+
+ if (req_type == HZIP_ALG_TYPE_GZIP &&
+ acomp_req->slen < GZIP_HEAD_FEXTRA_SHIFT)
+ return -EINVAL;
+
+ switch (req_type) {
+ case HZIP_ALG_TYPE_ZLIB:
+ return TO_HEAD_SIZE(HZIP_ALG_TYPE_ZLIB);
+ case HZIP_ALG_TYPE_GZIP:
+ return TO_HEAD_SIZE(HZIP_ALG_TYPE_GZIP);
+ default:
+ pr_err("request type does not support!\n");
+ return -EINVAL;
+ }
+}
+
+static struct hisi_zip_req *hisi_zip_create_req(struct acomp_req *req,
+ struct hisi_zip_qp_ctx *qp_ctx,
+ size_t head_size, bool is_comp)
+{
+ struct hisi_zip_req_q *req_q = &qp_ctx->req_q;
+ struct hisi_zip_req *q = req_q->q;
+ struct hisi_zip_req *req_cache;
+ int req_id;
+
+ write_lock(&req_q->req_lock);
+
+ req_id = find_first_zero_bit(req_q->req_bitmap, req_q->size);
+ if (req_id >= req_q->size) {
+ write_unlock(&req_q->req_lock);
+ dev_dbg(&qp_ctx->qp->qm->pdev->dev, "req cache is full!\n");
+ return ERR_PTR(-EAGAIN);
+ }
+ set_bit(req_id, req_q->req_bitmap);
+
+ req_cache = q + req_id;
+ req_cache->req_id = req_id;
+ req_cache->req = req;
+
+ if (is_comp) {
+ req_cache->sskip = 0;
+ req_cache->dskip = head_size;
+ } else {
+ req_cache->sskip = head_size;
+ req_cache->dskip = 0;
+ }
+
+ write_unlock(&req_q->req_lock);
+
+ return req_cache;
+}
+
+static void hisi_zip_remove_req(struct hisi_zip_qp_ctx *qp_ctx,
+ struct hisi_zip_req *req)
+{
+ struct hisi_zip_req_q *req_q = &qp_ctx->req_q;
+
+ write_lock(&req_q->req_lock);
+ clear_bit(req->req_id, req_q->req_bitmap);
+ memset(req, 0, sizeof(struct hisi_zip_req));
+ write_unlock(&req_q->req_lock);
+}
+
+static void hisi_zip_fill_addr(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req)
+{
+ sqe->source_addr_l = lower_32_bits(req->dma_src);
+ sqe->source_addr_h = upper_32_bits(req->dma_src);
+ sqe->dest_addr_l = lower_32_bits(req->dma_dst);
+ sqe->dest_addr_h = upper_32_bits(req->dma_dst);
+}
+
+static void hisi_zip_fill_buf_size(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req)
+{
+ struct acomp_req *a_req = req->req;
+
+ sqe->input_data_length = a_req->slen - req->sskip;
+ sqe->dest_avail_out = a_req->dlen - req->dskip;
+ sqe->dw7 = FIELD_PREP(HZIP_IN_SGE_DATA_OFFSET_M, req->sskip);
+ sqe->dw8 = FIELD_PREP(HZIP_OUT_SGE_DATA_OFFSET_M, req->dskip);
+}
+
+static void hisi_zip_fill_buf_type(struct hisi_zip_sqe *sqe, u8 buf_type)
{
u32 val;
- val = (sqe->dw9) & ~HZIP_BUF_TYPE_M;
+ val = sqe->dw9 & ~HZIP_BUF_TYPE_M;
val |= FIELD_PREP(HZIP_BUF_TYPE_M, buf_type);
sqe->dw9 = val;
}
-static void hisi_zip_config_tag(struct hisi_zip_sqe *sqe, u32 tag)
+static void hisi_zip_fill_req_type(struct hisi_zip_sqe *sqe, u8 req_type)
{
- sqe->tag = tag;
+ u32 val;
+
+ val = sqe->dw9 & ~HZIP_REQ_TYPE_M;
+ val |= FIELD_PREP(HZIP_REQ_TYPE_M, req_type);
+ sqe->dw9 = val;
}
-static void hisi_zip_fill_sqe(struct hisi_zip_sqe *sqe, u8 req_type,
- dma_addr_t s_addr, dma_addr_t d_addr, u32 slen,
- u32 dlen, u32 sskip, u32 dskip)
+static void hisi_zip_fill_tag_v1(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req)
{
+ sqe->dw13 = req->req_id;
+}
+
+static void hisi_zip_fill_tag_v2(struct hisi_zip_sqe *sqe, struct hisi_zip_req *req)
+{
+ sqe->dw26 = req->req_id;
+}
+
+static void hisi_zip_fill_sqe_type(struct hisi_zip_sqe *sqe, u8 sqe_type)
+{
+ u32 val;
+
+ val = sqe->dw7 & ~HZIP_SQE_TYPE_M;
+ val |= FIELD_PREP(HZIP_SQE_TYPE_M, sqe_type);
+ sqe->dw7 = val;
+}
+
+static void hisi_zip_fill_sqe(struct hisi_zip_ctx *ctx, struct hisi_zip_sqe *sqe,
+ u8 req_type, struct hisi_zip_req *req)
+{
+ const struct hisi_zip_sqe_ops *ops = ctx->ops;
+
memset(sqe, 0, sizeof(struct hisi_zip_sqe));
- sqe->input_data_length = slen - sskip;
- sqe->dw7 = FIELD_PREP(HZIP_IN_SGE_DATA_OFFSET_M, sskip);
- sqe->dw8 = FIELD_PREP(HZIP_OUT_SGE_DATA_OFFSET_M, dskip);
- sqe->dw9 = FIELD_PREP(HZIP_REQ_TYPE_M, req_type);
- sqe->dest_avail_out = dlen - dskip;
- sqe->source_addr_l = lower_32_bits(s_addr);
- sqe->source_addr_h = upper_32_bits(s_addr);
- sqe->dest_addr_l = lower_32_bits(d_addr);
- sqe->dest_addr_h = upper_32_bits(d_addr);
+ ops->fill_addr(sqe, req);
+ ops->fill_buf_size(sqe, req);
+ ops->fill_buf_type(sqe, HZIP_SGL);
+ ops->fill_req_type(sqe, req_type);
+ ops->fill_tag(sqe, req);
+ ops->fill_sqe_type(sqe, ops->sqe_type);
+}
+
+static int hisi_zip_do_work(struct hisi_zip_req *req,
+ struct hisi_zip_qp_ctx *qp_ctx)
+{
+ struct hisi_acc_sgl_pool *pool = qp_ctx->sgl_pool;
+ struct hisi_zip_dfx *dfx = &qp_ctx->zip_dev->dfx;
+ struct acomp_req *a_req = req->req;
+ struct hisi_qp *qp = qp_ctx->qp;
+ struct device *dev = &qp->qm->pdev->dev;
+ struct hisi_zip_sqe zip_sqe;
+ int ret;
+
+ if (!a_req->src || !a_req->slen || !a_req->dst || !a_req->dlen)
+ return -EINVAL;
+
+ req->hw_src = hisi_acc_sg_buf_map_to_hw_sgl(dev, a_req->src, pool,
+ req->req_id << 1, &req->dma_src);
+ if (IS_ERR(req->hw_src)) {
+ dev_err(dev, "failed to map the src buffer to hw sgl (%ld)!\n",
+ PTR_ERR(req->hw_src));
+ return PTR_ERR(req->hw_src);
+ }
+
+ req->hw_dst = hisi_acc_sg_buf_map_to_hw_sgl(dev, a_req->dst, pool,
+ (req->req_id << 1) + 1,
+ &req->dma_dst);
+ if (IS_ERR(req->hw_dst)) {
+ ret = PTR_ERR(req->hw_dst);
+ dev_err(dev, "failed to map the dst buffer to hw slg (%d)!\n",
+ ret);
+ goto err_unmap_input;
+ }
+
+ hisi_zip_fill_sqe(qp_ctx->ctx, &zip_sqe, qp->req_type, req);
+
+ /* send command to start a task */
+ atomic64_inc(&dfx->send_cnt);
+ ret = hisi_qp_send(qp, &zip_sqe);
+ if (ret < 0) {
+ atomic64_inc(&dfx->send_busy_cnt);
+ ret = -EAGAIN;
+ dev_dbg_ratelimited(dev, "failed to send request!\n");
+ goto err_unmap_output;
+ }
+
+ return -EINPROGRESS;
+
+err_unmap_output:
+ hisi_acc_sg_buf_unmap(dev, a_req->dst, req->hw_dst);
+err_unmap_input:
+ hisi_acc_sg_buf_unmap(dev, a_req->src, req->hw_src);
+ return ret;
+}
+
+static u32 hisi_zip_get_tag_v1(struct hisi_zip_sqe *sqe)
+{
+ return sqe->dw13;
+}
+
+static u32 hisi_zip_get_tag_v2(struct hisi_zip_sqe *sqe)
+{
+ return sqe->dw26;
+}
+
+static u32 hisi_zip_get_status(struct hisi_zip_sqe *sqe)
+{
+ return sqe->dw3 & HZIP_BD_STATUS_M;
+}
+
+static u32 hisi_zip_get_dstlen(struct hisi_zip_sqe *sqe)
+{
+ return sqe->produced;
+}
+
+static void hisi_zip_acomp_cb(struct hisi_qp *qp, void *data)
+{
+ struct hisi_zip_qp_ctx *qp_ctx = qp->qp_ctx;
+ const struct hisi_zip_sqe_ops *ops = qp_ctx->ctx->ops;
+ struct hisi_zip_dfx *dfx = &qp_ctx->zip_dev->dfx;
+ struct hisi_zip_req_q *req_q = &qp_ctx->req_q;
+ struct device *dev = &qp->qm->pdev->dev;
+ struct hisi_zip_sqe *sqe = data;
+ u32 tag = ops->get_tag(sqe);
+ struct hisi_zip_req *req = req_q->q + tag;
+ struct acomp_req *acomp_req = req->req;
+ u32 status, dlen, head_size;
+ int err = 0;
+
+ atomic64_inc(&dfx->recv_cnt);
+ status = ops->get_status(sqe);
+ if (status != 0 && status != HZIP_NC_ERR) {
+ dev_err(dev, "%scompress fail in qp%u: %u, output: %u\n",
+ (qp->alg_type == 0) ? "" : "de", qp->qp_id, status,
+ sqe->produced);
+ atomic64_inc(&dfx->err_bd_cnt);
+ err = -EIO;
+ }
+
+ dlen = ops->get_dstlen(sqe);
+
+ hisi_acc_sg_buf_unmap(dev, acomp_req->src, req->hw_src);
+ hisi_acc_sg_buf_unmap(dev, acomp_req->dst, req->hw_dst);
+
+ head_size = (qp->alg_type == 0) ? TO_HEAD_SIZE(qp->req_type) : 0;
+ acomp_req->dlen = dlen + head_size;
+
+ if (acomp_req->base.complete)
+ acomp_request_complete(acomp_req, err);
+
+ hisi_zip_remove_req(qp_ctx, req);
+}
+
+static int hisi_zip_acompress(struct acomp_req *acomp_req)
+{
+ struct hisi_zip_ctx *ctx = crypto_tfm_ctx(acomp_req->base.tfm);
+ struct hisi_zip_qp_ctx *qp_ctx = &ctx->qp_ctx[HZIP_QPC_COMP];
+ struct device *dev = &qp_ctx->qp->qm->pdev->dev;
+ struct hisi_zip_req *req;
+ int head_size;
+ int ret;
+
+ /* let's output compression head now */
+ head_size = add_comp_head(acomp_req->dst, qp_ctx->qp->req_type);
+ if (head_size < 0) {
+ dev_err_ratelimited(dev, "failed to add comp head (%d)!\n",
+ head_size);
+ return head_size;
+ }
+
+ req = hisi_zip_create_req(acomp_req, qp_ctx, head_size, true);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+
+ ret = hisi_zip_do_work(req, qp_ctx);
+ if (ret != -EINPROGRESS) {
+ dev_info_ratelimited(dev, "failed to do compress (%d)!\n", ret);
+ hisi_zip_remove_req(qp_ctx, req);
+ }
+
+ return ret;
+}
+
+static int hisi_zip_adecompress(struct acomp_req *acomp_req)
+{
+ struct hisi_zip_ctx *ctx = crypto_tfm_ctx(acomp_req->base.tfm);
+ struct hisi_zip_qp_ctx *qp_ctx = &ctx->qp_ctx[HZIP_QPC_DECOMP];
+ struct device *dev = &qp_ctx->qp->qm->pdev->dev;
+ struct hisi_zip_req *req;
+ int head_size, ret;
+
+ head_size = get_comp_head_size(acomp_req, qp_ctx->qp->req_type);
+ if (head_size < 0) {
+ dev_err_ratelimited(dev, "failed to get comp head size (%d)!\n",
+ head_size);
+ return head_size;
+ }
+
+ req = hisi_zip_create_req(acomp_req, qp_ctx, head_size, false);
+ if (IS_ERR(req))
+ return PTR_ERR(req);
+
+ ret = hisi_zip_do_work(req, qp_ctx);
+ if (ret != -EINPROGRESS) {
+ dev_info_ratelimited(dev, "failed to do decompress (%d)!\n",
+ ret);
+ hisi_zip_remove_req(qp_ctx, req);
+ }
+
+ return ret;
}
static int hisi_zip_start_qp(struct hisi_qp *qp, struct hisi_zip_qp_ctx *ctx,
@@ -177,9 +524,36 @@ static void hisi_zip_release_qp(struct hisi_zip_qp_ctx *ctx)
hisi_qm_release_qp(ctx->qp);
}
+static const struct hisi_zip_sqe_ops hisi_zip_ops_v1 = {
+ .sqe_type = 0,
+ .fill_addr = hisi_zip_fill_addr,
+ .fill_buf_size = hisi_zip_fill_buf_size,
+ .fill_buf_type = hisi_zip_fill_buf_type,
+ .fill_req_type = hisi_zip_fill_req_type,
+ .fill_tag = hisi_zip_fill_tag_v1,
+ .fill_sqe_type = hisi_zip_fill_sqe_type,
+ .get_tag = hisi_zip_get_tag_v1,
+ .get_status = hisi_zip_get_status,
+ .get_dstlen = hisi_zip_get_dstlen,
+};
+
+static const struct hisi_zip_sqe_ops hisi_zip_ops_v2 = {
+ .sqe_type = 0x3,
+ .fill_addr = hisi_zip_fill_addr,
+ .fill_buf_size = hisi_zip_fill_buf_size,
+ .fill_buf_type = hisi_zip_fill_buf_type,
+ .fill_req_type = hisi_zip_fill_req_type,
+ .fill_tag = hisi_zip_fill_tag_v2,
+ .fill_sqe_type = hisi_zip_fill_sqe_type,
+ .get_tag = hisi_zip_get_tag_v2,
+ .get_status = hisi_zip_get_status,
+ .get_dstlen = hisi_zip_get_dstlen,
+};
+
static int hisi_zip_ctx_init(struct hisi_zip_ctx *hisi_zip_ctx, u8 req_type, int node)
{
struct hisi_qp *qps[HZIP_CTX_Q_NUM] = { NULL };
+ struct hisi_zip_qp_ctx *qp_ctx;
struct hisi_zip *hisi_zip;
int ret, i, j;
@@ -193,8 +567,9 @@ static int hisi_zip_ctx_init(struct hisi_zip_ctx *hisi_zip_ctx, u8 req_type, int
for (i = 0; i < HZIP_CTX_Q_NUM; i++) {
/* alg_type = 0 for compress, 1 for decompress in hw sqe */
- ret = hisi_zip_start_qp(qps[i], &hisi_zip_ctx->qp_ctx[i], i,
- req_type);
+ qp_ctx = &hisi_zip_ctx->qp_ctx[i];
+ qp_ctx->ctx = hisi_zip_ctx;
+ ret = hisi_zip_start_qp(qps[i], qp_ctx, i, req_type);
if (ret) {
for (j = i - 1; j >= 0; j--)
hisi_qm_stop_qp(hisi_zip_ctx->qp_ctx[j].qp);
@@ -203,9 +578,14 @@ static int hisi_zip_ctx_init(struct hisi_zip_ctx *hisi_zip_ctx, u8 req_type, int
return ret;
}
- hisi_zip_ctx->qp_ctx[i].zip_dev = hisi_zip;
+ qp_ctx->zip_dev = hisi_zip;
}
+ if (hisi_zip->qm.ver < QM_HW_V3)
+ hisi_zip_ctx->ops = &hisi_zip_ops_v1;
+ else
+ hisi_zip_ctx->ops = &hisi_zip_ops_v2;
+
return 0;
}
@@ -217,38 +597,6 @@ static void hisi_zip_ctx_exit(struct hisi_zip_ctx *hisi_zip_ctx)
hisi_zip_release_qp(&hisi_zip_ctx->qp_ctx[i]);
}
-static u16 get_extra_field_size(const u8 *start)
-{
- return *((u16 *)start) + GZIP_HEAD_FEXTRA_XLEN;
-}
-
-static u32 get_name_field_size(const u8 *start)
-{
- return strlen(start) + 1;
-}
-
-static u32 get_comment_field_size(const u8 *start)
-{
- return strlen(start) + 1;
-}
-
-static u32 __get_gzip_head_size(const u8 *src)
-{
- u8 head_flg = *(src + GZIP_HEAD_FLG_SHIFT);
- u32 size = GZIP_HEAD_FEXTRA_SHIFT;
-
- if (head_flg & GZIP_HEAD_FEXTRA_BIT)
- size += get_extra_field_size(src + size);
- if (head_flg & GZIP_HEAD_FNAME_BIT)
- size += get_name_field_size(src + size);
- if (head_flg & GZIP_HEAD_FCOMMENT_BIT)
- size += get_comment_field_size(src + size);
- if (head_flg & GZIP_HEAD_FHCRC_BIT)
- size += GZIP_HEAD_FHCRC_SIZE;
-
- return size;
-}
-
static int hisi_zip_create_req_q(struct hisi_zip_ctx *ctx)
{
struct hisi_zip_req_q *req_q;
@@ -336,52 +684,6 @@ static void hisi_zip_release_sgl_pool(struct hisi_zip_ctx *ctx)
ctx->qp_ctx[i].sgl_pool);
}
-static void hisi_zip_remove_req(struct hisi_zip_qp_ctx *qp_ctx,
- struct hisi_zip_req *req)
-{
- struct hisi_zip_req_q *req_q = &qp_ctx->req_q;
-
- write_lock(&req_q->req_lock);
- clear_bit(req->req_id, req_q->req_bitmap);
- memset(req, 0, sizeof(struct hisi_zip_req));
- write_unlock(&req_q->req_lock);
-}
-
-static void hisi_zip_acomp_cb(struct hisi_qp *qp, void *data)
-{
- struct hisi_zip_sqe *sqe = data;
- struct hisi_zip_qp_ctx *qp_ctx = qp->qp_ctx;
- struct hisi_zip_dfx *dfx = &qp_ctx->zip_dev->dfx;
- struct hisi_zip_req_q *req_q = &qp_ctx->req_q;
- struct hisi_zip_req *req = req_q->q + sqe->tag;
- struct acomp_req *acomp_req = req->req;
- struct device *dev = &qp->qm->pdev->dev;
- u32 status, dlen, head_size;
- int err = 0;
-
- atomic64_inc(&dfx->recv_cnt);
- status = sqe->dw3 & HZIP_BD_STATUS_M;
- if (status != 0 && status != HZIP_NC_ERR) {
- dev_err(dev, "%scompress fail in qp%u: %u, output: %u\n",
- (qp->alg_type == 0) ? "" : "de", qp->qp_id, status,
- sqe->produced);
- atomic64_inc(&dfx->err_bd_cnt);
- err = -EIO;
- }
- dlen = sqe->produced;
-
- hisi_acc_sg_buf_unmap(dev, acomp_req->src, req->hw_src);
- hisi_acc_sg_buf_unmap(dev, acomp_req->dst, req->hw_dst);
-
- head_size = (qp->alg_type == 0) ? TO_HEAD_SIZE(qp->req_type) : 0;
- acomp_req->dlen = dlen + head_size;
-
- if (acomp_req->base.complete)
- acomp_request_complete(acomp_req, err);
-
- hisi_zip_remove_req(qp_ctx, req);
-}
-
static void hisi_zip_set_acomp_cb(struct hisi_zip_ctx *ctx,
void (*fn)(struct hisi_qp *, void *))
{
@@ -439,204 +741,6 @@ static void hisi_zip_acomp_exit(struct crypto_acomp *tfm)
hisi_zip_ctx_exit(ctx);
}
-static int add_comp_head(struct scatterlist *dst, u8 req_type)
-{
- int head_size = TO_HEAD_SIZE(req_type);
- const u8 *head = TO_HEAD(req_type);
- int ret;
-
- ret = sg_copy_from_buffer(dst, sg_nents(dst), head, head_size);
- if (ret != head_size) {
- pr_err("the head size of buffer is wrong (%d)!\n", ret);
- return -ENOMEM;
- }
-
- return head_size;
-}
-
-static size_t __maybe_unused get_gzip_head_size(struct scatterlist *sgl)
-{
- char buf[HZIP_GZIP_HEAD_BUF];
-
- sg_copy_to_buffer(sgl, sg_nents(sgl), buf, sizeof(buf));
-
- return __get_gzip_head_size(buf);
-}
-
-static int get_comp_head_size(struct acomp_req *acomp_req, u8 req_type)
-{
- if (!acomp_req->src || !acomp_req->slen)
- return -EINVAL;
-
- if ((req_type == HZIP_ALG_TYPE_GZIP) &&
- (acomp_req->slen < GZIP_HEAD_FEXTRA_SHIFT))
- return -EINVAL;
-
- switch (req_type) {
- case HZIP_ALG_TYPE_ZLIB:
- return TO_HEAD_SIZE(HZIP_ALG_TYPE_ZLIB);
- case HZIP_ALG_TYPE_GZIP:
- return TO_HEAD_SIZE(HZIP_ALG_TYPE_GZIP);
- default:
- pr_err("request type does not support!\n");
- return -EINVAL;
- }
-}
-
-static struct hisi_zip_req *hisi_zip_create_req(struct acomp_req *req,
- struct hisi_zip_qp_ctx *qp_ctx,
- size_t head_size, bool is_comp)
-{
- struct hisi_zip_req_q *req_q = &qp_ctx->req_q;
- struct hisi_zip_req *q = req_q->q;
- struct hisi_zip_req *req_cache;
- int req_id;
-
- write_lock(&req_q->req_lock);
-
- req_id = find_first_zero_bit(req_q->req_bitmap, req_q->size);
- if (req_id >= req_q->size) {
- write_unlock(&req_q->req_lock);
- dev_dbg(&qp_ctx->qp->qm->pdev->dev, "req cache is full!\n");
- return ERR_PTR(-EAGAIN);
- }
- set_bit(req_id, req_q->req_bitmap);
-
- req_cache = q + req_id;
- req_cache->req_id = req_id;
- req_cache->req = req;
-
- if (is_comp) {
- req_cache->sskip = 0;
- req_cache->dskip = head_size;
- } else {
- req_cache->sskip = head_size;
- req_cache->dskip = 0;
- }
-
- write_unlock(&req_q->req_lock);
-
- return req_cache;
-}
-
-static int hisi_zip_do_work(struct hisi_zip_req *req,
- struct hisi_zip_qp_ctx *qp_ctx)
-{
- struct acomp_req *a_req = req->req;
- struct hisi_qp *qp = qp_ctx->qp;
- struct device *dev = &qp->qm->pdev->dev;
- struct hisi_acc_sgl_pool *pool = qp_ctx->sgl_pool;
- struct hisi_zip_dfx *dfx = &qp_ctx->zip_dev->dfx;
- struct hisi_zip_sqe zip_sqe;
- dma_addr_t input, output;
- int ret;
-
- if (!a_req->src || !a_req->slen || !a_req->dst || !a_req->dlen)
- return -EINVAL;
-
- req->hw_src = hisi_acc_sg_buf_map_to_hw_sgl(dev, a_req->src, pool,
- req->req_id << 1, &input);
- if (IS_ERR(req->hw_src)) {
- dev_err(dev, "failed to map the src buffer to hw sgl (%ld)!\n",
- PTR_ERR(req->hw_src));
- return PTR_ERR(req->hw_src);
- }
- req->dma_src = input;
-
- req->hw_dst = hisi_acc_sg_buf_map_to_hw_sgl(dev, a_req->dst, pool,
- (req->req_id << 1) + 1,
- &output);
- if (IS_ERR(req->hw_dst)) {
- ret = PTR_ERR(req->hw_dst);
- dev_err(dev, "failed to map the dst buffer to hw slg (%d)!\n",
- ret);
- goto err_unmap_input;
- }
- req->dma_dst = output;
-
- hisi_zip_fill_sqe(&zip_sqe, qp->req_type, input, output, a_req->slen,
- a_req->dlen, req->sskip, req->dskip);
- hisi_zip_config_buf_type(&zip_sqe, HZIP_SGL);
- hisi_zip_config_tag(&zip_sqe, req->req_id);
-
- /* send command to start a task */
- atomic64_inc(&dfx->send_cnt);
- ret = hisi_qp_send(qp, &zip_sqe);
- if (ret < 0) {
- atomic64_inc(&dfx->send_busy_cnt);
- ret = -EAGAIN;
- dev_dbg_ratelimited(dev, "failed to send request!\n");
- goto err_unmap_output;
- }
-
- return -EINPROGRESS;
-
-err_unmap_output:
- hisi_acc_sg_buf_unmap(dev, a_req->dst, req->hw_dst);
-err_unmap_input:
- hisi_acc_sg_buf_unmap(dev, a_req->src, req->hw_src);
- return ret;
-}
-
-static int hisi_zip_acompress(struct acomp_req *acomp_req)
-{
- struct hisi_zip_ctx *ctx = crypto_tfm_ctx(acomp_req->base.tfm);
- struct hisi_zip_qp_ctx *qp_ctx = &ctx->qp_ctx[HZIP_QPC_COMP];
- struct device *dev = &qp_ctx->qp->qm->pdev->dev;
- struct hisi_zip_req *req;
- int head_size;
- int ret;
-
- /* let's output compression head now */
- head_size = add_comp_head(acomp_req->dst, qp_ctx->qp->req_type);
- if (head_size < 0) {
- dev_err_ratelimited(dev, "failed to add comp head (%d)!\n",
- head_size);
- return head_size;
- }
-
- req = hisi_zip_create_req(acomp_req, qp_ctx, head_size, true);
- if (IS_ERR(req))
- return PTR_ERR(req);
-
- ret = hisi_zip_do_work(req, qp_ctx);
- if (ret != -EINPROGRESS) {
- dev_info_ratelimited(dev, "failed to do compress (%d)!\n", ret);
- hisi_zip_remove_req(qp_ctx, req);
- }
-
- return ret;
-}
-
-static int hisi_zip_adecompress(struct acomp_req *acomp_req)
-{
- struct hisi_zip_ctx *ctx = crypto_tfm_ctx(acomp_req->base.tfm);
- struct hisi_zip_qp_ctx *qp_ctx = &ctx->qp_ctx[HZIP_QPC_DECOMP];
- struct device *dev = &qp_ctx->qp->qm->pdev->dev;
- struct hisi_zip_req *req;
- int head_size, ret;
-
- head_size = get_comp_head_size(acomp_req, qp_ctx->qp->req_type);
- if (head_size < 0) {
- dev_err_ratelimited(dev, "failed to get comp head size (%d)!\n",
- head_size);
- return head_size;
- }
-
- req = hisi_zip_create_req(acomp_req, qp_ctx, head_size, false);
- if (IS_ERR(req))
- return PTR_ERR(req);
-
- ret = hisi_zip_do_work(req, qp_ctx);
- if (ret != -EINPROGRESS) {
- dev_info_ratelimited(dev, "failed to do decompress (%d)!\n",
- ret);
- hisi_zip_remove_req(qp_ctx, req);
- }
-
- return ret;
-}
-
static struct acomp_alg hisi_zip_acomp_zlib = {
.init = hisi_zip_acomp_init,
.exit = hisi_zip_acomp_exit,
@@ -665,7 +769,7 @@ static struct acomp_alg hisi_zip_acomp_gzip = {
}
};
-int hisi_zip_register_to_crypto(void)
+int hisi_zip_register_to_crypto(struct hisi_qm *qm)
{
int ret;
@@ -684,7 +788,7 @@ int hisi_zip_register_to_crypto(void)
return ret;
}
-void hisi_zip_unregister_from_crypto(void)
+void hisi_zip_unregister_from_crypto(struct hisi_qm *qm)
{
crypto_unregister_acomp(&hisi_zip_acomp_gzip);
crypto_unregister_acomp(&hisi_zip_acomp_zlib);
diff --git a/drivers/crypto/hisilicon/zip/zip_main.c b/drivers/crypto/hisilicon/zip/zip_main.c
index 02c445722445..2178b40e9f82 100644
--- a/drivers/crypto/hisilicon/zip/zip_main.c
+++ b/drivers/crypto/hisilicon/zip/zip_main.c
@@ -18,7 +18,6 @@
#define PCI_DEVICE_ID_ZIP_VF 0xa251
#define HZIP_QUEUE_NUM_V1 4096
-#define HZIP_QUEUE_NUM_V2 1024
#define HZIP_CLOCK_GATE_CTRL 0x301004
#define COMP0_ENABLE BIT(0)
@@ -69,10 +68,10 @@
#define HZIP_CORE_INT_RAS_CE_ENABLE 0x1
#define HZIP_CORE_INT_RAS_NFE_ENB 0x301164
#define HZIP_CORE_INT_RAS_FE_ENB 0x301168
-#define HZIP_CORE_INT_RAS_NFE_ENABLE 0x7FE
+#define HZIP_CORE_INT_RAS_NFE_ENABLE 0x1FFE
#define HZIP_SRAM_ECC_ERR_NUM_SHIFT 16
#define HZIP_SRAM_ECC_ERR_ADDR_SHIFT 24
-#define HZIP_CORE_INT_MASK_ALL GENMASK(10, 0)
+#define HZIP_CORE_INT_MASK_ALL GENMASK(12, 0)
#define HZIP_COMP_CORE_NUM 2
#define HZIP_DECOMP_CORE_NUM 6
#define HZIP_CORE_NUM (HZIP_COMP_CORE_NUM + \
@@ -134,17 +133,17 @@ static const struct hisi_zip_hw_error zip_hw_error[] = {
{ .int_msk = BIT(8), .msg = "zip_com_inf_err" },
{ .int_msk = BIT(9), .msg = "zip_enc_inf_err" },
{ .int_msk = BIT(10), .msg = "zip_pre_out_err" },
+ { .int_msk = BIT(11), .msg = "zip_axi_poison_err" },
+ { .int_msk = BIT(12), .msg = "zip_sva_err" },
{ /* sentinel */ }
};
enum ctrl_debug_file_index {
- HZIP_CURRENT_QM,
HZIP_CLEAR_ENABLE,
HZIP_DEBUG_FILE_NUM,
};
static const char * const ctrl_debug_file_name[] = {
- [HZIP_CURRENT_QM] = "current_qm",
[HZIP_CLEAR_ENABLE] = "clear_enable",
};
@@ -363,48 +362,6 @@ static inline struct hisi_qm *file_to_qm(struct ctrl_debug_file *file)
return &hisi_zip->qm;
}
-static u32 current_qm_read(struct ctrl_debug_file *file)
-{
- struct hisi_qm *qm = file_to_qm(file);
-
- return readl(qm->io_base + QM_DFX_MB_CNT_VF);
-}
-
-static int current_qm_write(struct ctrl_debug_file *file, u32 val)
-{
- struct hisi_qm *qm = file_to_qm(file);
- u32 vfq_num;
- u32 tmp;
-
- if (val > qm->vfs_num)
- return -EINVAL;
-
- /* According PF or VF Dev ID to calculation curr_qm_qp_num and store */
- if (val == 0) {
- qm->debug.curr_qm_qp_num = qm->qp_num;
- } else {
- vfq_num = (qm->ctrl_qp_num - qm->qp_num) / qm->vfs_num;
- if (val == qm->vfs_num)
- qm->debug.curr_qm_qp_num = qm->ctrl_qp_num -
- qm->qp_num - (qm->vfs_num - 1) * vfq_num;
- else
- qm->debug.curr_qm_qp_num = vfq_num;
- }
-
- writel(val, qm->io_base + QM_DFX_MB_CNT_VF);
- writel(val, qm->io_base + QM_DFX_DB_CNT_VF);
-
- tmp = val |
- (readl(qm->io_base + QM_DFX_SQE_CNT_VF_SQN) & CURRENT_Q_MASK);
- writel(tmp, qm->io_base + QM_DFX_SQE_CNT_VF_SQN);
-
- tmp = val |
- (readl(qm->io_base + QM_DFX_CQE_CNT_VF_CQN) & CURRENT_Q_MASK);
- writel(tmp, qm->io_base + QM_DFX_CQE_CNT_VF_CQN);
-
- return 0;
-}
-
static u32 clear_enable_read(struct ctrl_debug_file *file)
{
struct hisi_qm *qm = file_to_qm(file);
@@ -438,9 +395,6 @@ static ssize_t hisi_zip_ctrl_debug_read(struct file *filp, char __user *buf,
spin_lock_irq(&file->lock);
switch (file->index) {
- case HZIP_CURRENT_QM:
- val = current_qm_read(file);
- break;
case HZIP_CLEAR_ENABLE:
val = clear_enable_read(file);
break;
@@ -478,11 +432,6 @@ static ssize_t hisi_zip_ctrl_debug_write(struct file *filp,
spin_lock_irq(&file->lock);
switch (file->index) {
- case HZIP_CURRENT_QM:
- ret = current_qm_write(file, val);
- if (ret)
- goto err_input;
- break;
case HZIP_CLEAR_ENABLE:
ret = clear_enable_write(file, val);
if (ret)
@@ -580,7 +529,7 @@ static int hisi_zip_ctrl_debug_init(struct hisi_qm *qm)
struct hisi_zip *zip = container_of(qm, struct hisi_zip, qm);
int i;
- for (i = HZIP_CURRENT_QM; i < HZIP_DEBUG_FILE_NUM; i++) {
+ for (i = HZIP_CLEAR_ENABLE; i < HZIP_DEBUG_FILE_NUM; i++) {
spin_lock_init(&zip->ctrl->files[i].lock);
zip->ctrl->files[i].ctrl = zip->ctrl;
zip->ctrl->files[i].index = i;
@@ -627,10 +576,6 @@ static void hisi_zip_debug_regs_clear(struct hisi_qm *qm)
{
int i, j;
- /* clear current_qm */
- writel(0x0, qm->io_base + QM_DFX_MB_CNT_VF);
- writel(0x0, qm->io_base + QM_DFX_DB_CNT_VF);
-
/* enable register read_clear bit */
writel(HZIP_RD_CNT_CLR_CE_EN, qm->io_base + HZIP_SOFT_CTRL_CNT_CLR_CE);
for (i = 0; i < ARRAY_SIZE(core_offsets); i++)
@@ -714,6 +659,22 @@ static void hisi_zip_close_axi_master_ooo(struct hisi_qm *qm)
qm->io_base + HZIP_CORE_INT_SET);
}
+static void hisi_zip_err_info_init(struct hisi_qm *qm)
+{
+ struct hisi_qm_err_info *err_info = &qm->err_info;
+
+ err_info->ce = QM_BASE_CE;
+ err_info->fe = 0;
+ err_info->ecc_2bits_mask = HZIP_CORE_INT_STATUS_M_ECC;
+ err_info->dev_ce_mask = HZIP_CORE_INT_RAS_CE_ENABLE;
+ err_info->msi_wr_port = HZIP_WR_PORT;
+ err_info->acpi_rst = "ZRST";
+ err_info->nfe = QM_BASE_NFE | QM_ACC_WB_NOT_READY_TIMEOUT;
+
+ if (qm->ver >= QM_HW_V3)
+ err_info->nfe |= QM_ACC_DO_TASK_TIMEOUT;
+}
+
static const struct hisi_qm_err_ini hisi_zip_err_ini = {
.hw_init = hisi_zip_set_user_domain_and_cache,
.hw_err_enable = hisi_zip_hw_error_enable,
@@ -723,16 +684,7 @@ static const struct hisi_qm_err_ini hisi_zip_err_ini = {
.log_dev_hw_err = hisi_zip_log_hw_error,
.open_axi_master_ooo = hisi_zip_open_axi_master_ooo,
.close_axi_master_ooo = hisi_zip_close_axi_master_ooo,
- .err_info = {
- .ce = QM_BASE_CE,
- .nfe = QM_BASE_NFE |
- QM_ACC_WB_NOT_READY_TIMEOUT,
- .fe = 0,
- .ecc_2bits_mask = HZIP_CORE_INT_STATUS_M_ECC,
- .dev_ce_mask = HZIP_CORE_INT_RAS_CE_ENABLE,
- .msi_wr_port = HZIP_WR_PORT,
- .acpi_rst = "ZRST",
- }
+ .err_info_init = hisi_zip_err_info_init,
};
static int hisi_zip_pf_probe_init(struct hisi_zip *hisi_zip)
@@ -746,13 +698,8 @@ static int hisi_zip_pf_probe_init(struct hisi_zip *hisi_zip)
hisi_zip->ctrl = ctrl;
ctrl->hisi_zip = hisi_zip;
-
- if (qm->ver == QM_HW_V1)
- qm->ctrl_qp_num = HZIP_QUEUE_NUM_V1;
- else
- qm->ctrl_qp_num = HZIP_QUEUE_NUM_V2;
-
qm->err_ini = &hisi_zip_err_ini;
+ qm->err_ini->err_info_init(qm);
hisi_zip_set_user_domain_and_cache(qm);
hisi_qm_dev_err_init(qm);
diff --git a/drivers/crypto/img-hash.c b/drivers/crypto/img-hash.c
index e813115d5432..aa4c7b2af3e2 100644
--- a/drivers/crypto/img-hash.c
+++ b/drivers/crypto/img-hash.c
@@ -963,8 +963,6 @@ static int img_hash_probe(struct platform_device *pdev)
hdev->io_base = devm_platform_ioremap_resource(pdev, 0);
if (IS_ERR(hdev->io_base)) {
err = PTR_ERR(hdev->io_base);
- dev_err(dev, "can't ioremap, returned %d\n", err);
-
goto res_err;
}
@@ -972,7 +970,6 @@ static int img_hash_probe(struct platform_device *pdev)
hash_res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
hdev->cpu_addr = devm_ioremap_resource(dev, hash_res);
if (IS_ERR(hdev->cpu_addr)) {
- dev_err(dev, "can't ioremap write port\n");
err = PTR_ERR(hdev->cpu_addr);
goto res_err;
}
diff --git a/drivers/crypto/inside-secure/safexcel.c b/drivers/crypto/inside-secure/safexcel.c
index 6364583b88b2..9ff885d50edf 100644
--- a/drivers/crypto/inside-secure/safexcel.c
+++ b/drivers/crypto/inside-secure/safexcel.c
@@ -688,7 +688,7 @@ static int safexcel_hw_init(struct safexcel_crypto_priv *priv)
/* Leave the DSE threads reset state */
writel(0, EIP197_HIA_DSE_THR(priv) + EIP197_HIA_DSE_THR_CTRL(pe));
- /* Configure the procesing engine thresholds */
+ /* Configure the processing engine thresholds */
writel(EIP197_PE_OUT_DBUF_THRES_MIN(opbuflo) |
EIP197_PE_OUT_DBUF_THRES_MAX(opbufhi),
EIP197_PE(priv) + EIP197_PE_OUT_DBUF_THRES(pe));
diff --git a/drivers/crypto/ixp4xx_crypto.c b/drivers/crypto/ixp4xx_crypto.c
index 8b0f17fc09fb..0616e369522e 100644
--- a/drivers/crypto/ixp4xx_crypto.c
+++ b/drivers/crypto/ixp4xx_crypto.c
@@ -265,7 +265,7 @@ static int setup_crypt_desc(void)
return 0;
}
-static spinlock_t desc_lock;
+static DEFINE_SPINLOCK(desc_lock);
static struct crypt_ctl *get_crypt_desc(void)
{
int i;
@@ -293,7 +293,7 @@ static struct crypt_ctl *get_crypt_desc(void)
}
}
-static spinlock_t emerg_lock;
+static DEFINE_SPINLOCK(emerg_lock);
static struct crypt_ctl *get_crypt_desc_emerg(void)
{
int i;
@@ -1379,9 +1379,6 @@ static int __init ixp_module_init(void)
if (IS_ERR(pdev))
return PTR_ERR(pdev);
- spin_lock_init(&desc_lock);
- spin_lock_init(&emerg_lock);
-
err = init_ixp_crypto(&pdev->dev);
if (err) {
platform_device_unregister(pdev);
diff --git a/drivers/crypto/keembay/keembay-ocs-aes-core.c b/drivers/crypto/keembay/keembay-ocs-aes-core.c
index b6b25d994af3..e2a39fdaf623 100644
--- a/drivers/crypto/keembay/keembay-ocs-aes-core.c
+++ b/drivers/crypto/keembay/keembay-ocs-aes-core.c
@@ -1623,10 +1623,8 @@ static int kmb_ocs_aes_probe(struct platform_device *pdev)
}
aes_dev->base_reg = devm_ioremap_resource(&pdev->dev, aes_mem);
- if (IS_ERR(aes_dev->base_reg)) {
- dev_err(dev, "Failed to get base address\n");
+ if (IS_ERR(aes_dev->base_reg))
return PTR_ERR(aes_dev->base_reg);
- }
/* Get and request IRQ */
aes_dev->irq = platform_get_irq(pdev, 0);
@@ -1649,8 +1647,10 @@ static int kmb_ocs_aes_probe(struct platform_device *pdev)
/* Initialize crypto engine */
aes_dev->engine = crypto_engine_alloc_init(dev, true);
- if (!aes_dev->engine)
+ if (!aes_dev->engine) {
+ rc = -ENOMEM;
goto list_del;
+ }
rc = crypto_engine_start(aes_dev->engine);
if (rc) {
diff --git a/drivers/crypto/keembay/keembay-ocs-hcu-core.c b/drivers/crypto/keembay/keembay-ocs-hcu-core.c
index c4b97b4160e9..0379dbf32a4c 100644
--- a/drivers/crypto/keembay/keembay-ocs-hcu-core.c
+++ b/drivers/crypto/keembay/keembay-ocs-hcu-core.c
@@ -1192,10 +1192,8 @@ static int kmb_ocs_hcu_probe(struct platform_device *pdev)
}
hcu_dev->io_base = devm_ioremap_resource(dev, hcu_mem);
- if (IS_ERR(hcu_dev->io_base)) {
- dev_err(dev, "Could not io-remap mem resource.\n");
+ if (IS_ERR(hcu_dev->io_base))
return PTR_ERR(hcu_dev->io_base);
- }
init_completion(&hcu_dev->irq_done);
@@ -1220,8 +1218,10 @@ static int kmb_ocs_hcu_probe(struct platform_device *pdev)
/* Initialize crypto engine */
hcu_dev->engine = crypto_engine_alloc_init(dev, 1);
- if (!hcu_dev->engine)
+ if (!hcu_dev->engine) {
+ rc = -ENOMEM;
goto list_del;
+ }
rc = crypto_engine_start(hcu_dev->engine);
if (rc) {
diff --git a/drivers/crypto/keembay/ocs-hcu.c b/drivers/crypto/keembay/ocs-hcu.c
index 81eecacf603a..deb9bd460ee6 100644
--- a/drivers/crypto/keembay/ocs-hcu.c
+++ b/drivers/crypto/keembay/ocs-hcu.c
@@ -93,7 +93,7 @@
#define OCS_HCU_WAIT_BUSY_TIMEOUT_US 1000000
/**
- * struct ocs_hcu_dma_list - An entry in an OCS DMA linked list.
+ * struct ocs_hcu_dma_entry - An entry in an OCS DMA linked list.
* @src_addr: Source address of the data.
* @src_len: Length of data to be fetched.
* @nxt_desc: Next descriptor to fetch.
@@ -107,7 +107,7 @@ struct ocs_hcu_dma_entry {
};
/**
- * struct ocs_dma_list - OCS-specific DMA linked list.
+ * struct ocs_hcu_dma_list - OCS-specific DMA linked list.
* @head: The head of the list (points to the array backing the list).
* @tail: The current tail of the list; NULL if the list is empty.
* @dma_addr: The DMA address of @head (i.e., the DMA address of the backing
@@ -597,7 +597,7 @@ int ocs_hcu_hash_init(struct ocs_hcu_hash_ctx *ctx, enum ocs_hcu_algo algo)
}
/**
- * ocs_hcu_digest() - Perform a hashing iteration.
+ * ocs_hcu_hash_update() - Perform a hashing iteration.
* @hcu_dev: The OCS HCU device to use.
* @ctx: The OCS HCU hashing context.
* @dma_list: The OCS DMA list mapping the input data to process.
@@ -632,7 +632,7 @@ int ocs_hcu_hash_update(struct ocs_hcu_dev *hcu_dev,
}
/**
- * ocs_hcu_hash_final() - Update and finalize hash computation.
+ * ocs_hcu_hash_finup() - Update and finalize hash computation.
* @hcu_dev: The OCS HCU device to use.
* @ctx: The OCS HCU hashing context.
* @dma_list: The OCS DMA list mapping the input data to process.
diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h
index 3518fac29834..ecedd91a8d85 100644
--- a/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h
+++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_common.h
@@ -121,14 +121,14 @@ int otx2_cpt_send_mbox_msg(struct otx2_mbox *mbox, struct pci_dev *pdev);
int otx2_cpt_send_af_reg_requests(struct otx2_mbox *mbox,
struct pci_dev *pdev);
-int otx2_cpt_add_read_af_reg(struct otx2_mbox *mbox,
- struct pci_dev *pdev, u64 reg, u64 *val);
+int otx2_cpt_add_read_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
+ u64 reg, u64 *val, int blkaddr);
int otx2_cpt_add_write_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
- u64 reg, u64 val);
+ u64 reg, u64 val, int blkaddr);
int otx2_cpt_read_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
- u64 reg, u64 *val);
+ u64 reg, u64 *val, int blkaddr);
int otx2_cpt_write_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
- u64 reg, u64 val);
+ u64 reg, u64 val, int blkaddr);
struct otx2_cptlfs_info;
int otx2_cpt_attach_rscrs_msg(struct otx2_cptlfs_info *lfs);
int otx2_cpt_detach_rsrcs_msg(struct otx2_cptlfs_info *lfs);
diff --git a/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c b/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c
index 51cb6404ded7..9074876d38e5 100644
--- a/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c
+++ b/drivers/crypto/marvell/octeontx2/otx2_cpt_mbox_common.c
@@ -43,7 +43,7 @@ int otx2_cpt_send_af_reg_requests(struct otx2_mbox *mbox, struct pci_dev *pdev)
}
int otx2_cpt_add_read_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
- u64 reg, u64 *val)
+ u64 reg, u64 *val, int blkaddr)
{
struct cpt_rd_wr_reg_msg *reg_msg;
@@ -62,12 +62,13 @@ int otx2_cpt_add_read_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
reg_msg->is_write = 0;
reg_msg->reg_offset = reg;
reg_msg->ret_val = val;
+ reg_msg->blkaddr = blkaddr;
return 0;
}
int otx2_cpt_add_write_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
- u64 reg, u64 val)
+ u64 reg, u64 val, int blkaddr)
{
struct cpt_rd_wr_reg_msg *reg_msg;
@@ -86,16 +87,17 @@ int otx2_cpt_add_write_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
reg_msg->is_write = 1;
reg_msg->reg_offset = reg;
reg_msg->val = val;
+ reg_msg->blkaddr = blkaddr;
return 0;
}
int otx2_cpt_read_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
- u64 reg, u64 *val)
+ u64 reg, u64 *val, int blkaddr)
{
int ret;
- ret = otx2_cpt_add_read_af_reg(mbox, pdev, reg, val);
+ ret = otx2_cpt_add_read_af_reg(mbox, pdev, reg, val, blkaddr);
if (ret)
return ret;
@@ -103,11 +105,11 @@ int otx2_cpt_read_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
}
int otx2_cpt_write_af_reg(struct otx2_mbox *mbox, struct pci_dev *pdev,
- u64 reg, u64 val)
+ u64 reg, u64 val, int blkaddr)
{
int ret;
- ret = otx2_cpt_add_write_af_reg(mbox, pdev, reg, val);
+ ret = otx2_cpt_add_write_af_reg(mbox, pdev, reg, val, blkaddr);
if (ret)
return ret;
diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.c b/drivers/crypto/marvell/octeontx2/otx2_cptlf.c
index 823a4571fd67..34aba1532761 100644
--- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.c
+++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.c
@@ -56,7 +56,7 @@ static int cptlf_set_pri(struct otx2_cptlf_info *lf, int pri)
ret = otx2_cpt_read_af_reg(lfs->mbox, lfs->pdev,
CPT_AF_LFX_CTL(lf->slot),
- &lf_ctrl.u);
+ &lf_ctrl.u, lfs->blkaddr);
if (ret)
return ret;
@@ -64,7 +64,7 @@ static int cptlf_set_pri(struct otx2_cptlf_info *lf, int pri)
ret = otx2_cpt_write_af_reg(lfs->mbox, lfs->pdev,
CPT_AF_LFX_CTL(lf->slot),
- lf_ctrl.u);
+ lf_ctrl.u, lfs->blkaddr);
return ret;
}
@@ -77,7 +77,7 @@ static int cptlf_set_eng_grps_mask(struct otx2_cptlf_info *lf,
ret = otx2_cpt_read_af_reg(lfs->mbox, lfs->pdev,
CPT_AF_LFX_CTL(lf->slot),
- &lf_ctrl.u);
+ &lf_ctrl.u, lfs->blkaddr);
if (ret)
return ret;
@@ -85,7 +85,7 @@ static int cptlf_set_eng_grps_mask(struct otx2_cptlf_info *lf,
ret = otx2_cpt_write_af_reg(lfs->mbox, lfs->pdev,
CPT_AF_LFX_CTL(lf->slot),
- lf_ctrl.u);
+ lf_ctrl.u, lfs->blkaddr);
return ret;
}
diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h
index 314e97354100..ab1678fc564d 100644
--- a/drivers/crypto/marvell/octeontx2/otx2_cptlf.h
+++ b/drivers/crypto/marvell/octeontx2/otx2_cptlf.h
@@ -95,6 +95,7 @@ struct otx2_cptlfs_info {
u8 kcrypto_eng_grp_num; /* Kernel crypto engine group number */
u8 kvf_limits; /* Kernel crypto limits */
atomic_t state; /* LF's state. started/reset */
+ int blkaddr; /* CPT blkaddr: BLKADDR_CPT0/BLKADDR_CPT1 */
};
static inline void otx2_cpt_free_instruction_queues(
diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf.h b/drivers/crypto/marvell/octeontx2/otx2_cptpf.h
index 8c899ad531a5..e19af1356f12 100644
--- a/drivers/crypto/marvell/octeontx2/otx2_cptpf.h
+++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf.h
@@ -51,6 +51,7 @@ struct otx2_cptpf_dev {
u8 max_vfs; /* Maximum number of VFs supported by CPT */
u8 enabled_vfs; /* Number of enabled VFs */
u8 kvf_limits; /* Kernel crypto limits */
+ bool has_cpt1;
};
irqreturn_t otx2_cptpf_afpf_mbox_intr(int irq, void *arg);
diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c
index 5277e04badd9..58f47e3ab62e 100644
--- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c
+++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_main.c
@@ -451,19 +451,19 @@ static int cpt_is_pf_usable(struct otx2_cptpf_dev *cptpf)
return 0;
}
-static int cptpf_device_reset(struct otx2_cptpf_dev *cptpf)
+static int cptx_device_reset(struct otx2_cptpf_dev *cptpf, int blkaddr)
{
int timeout = 10, ret;
u64 reg = 0;
ret = otx2_cpt_write_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
- CPT_AF_BLK_RST, 0x1);
+ CPT_AF_BLK_RST, 0x1, blkaddr);
if (ret)
return ret;
do {
ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
- CPT_AF_BLK_RST, &reg);
+ CPT_AF_BLK_RST, &reg, blkaddr);
if (ret)
return ret;
@@ -478,11 +478,35 @@ static int cptpf_device_reset(struct otx2_cptpf_dev *cptpf)
return ret;
}
+static int cptpf_device_reset(struct otx2_cptpf_dev *cptpf)
+{
+ int ret = 0;
+
+ if (cptpf->has_cpt1) {
+ ret = cptx_device_reset(cptpf, BLKADDR_CPT1);
+ if (ret)
+ return ret;
+ }
+ return cptx_device_reset(cptpf, BLKADDR_CPT0);
+}
+
+static void cptpf_check_block_implemented(struct otx2_cptpf_dev *cptpf)
+{
+ u64 cfg;
+
+ cfg = otx2_cpt_read64(cptpf->reg_base, BLKADDR_RVUM, 0,
+ RVU_PF_BLOCK_ADDRX_DISC(BLKADDR_CPT1));
+ if (cfg & BIT_ULL(11))
+ cptpf->has_cpt1 = true;
+}
+
static int cptpf_device_init(struct otx2_cptpf_dev *cptpf)
{
union otx2_cptx_af_constants1 af_cnsts1 = {0};
int ret = 0;
+ /* check if 'implemented' bit is set for block BLKADDR_CPT1 */
+ cptpf_check_block_implemented(cptpf);
/* Reset the CPT PF device */
ret = cptpf_device_reset(cptpf);
if (ret)
@@ -490,7 +514,8 @@ static int cptpf_device_init(struct otx2_cptpf_dev *cptpf)
/* Get number of SE, IE and AE engines */
ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
- CPT_AF_CONSTANTS1, &af_cnsts1.u);
+ CPT_AF_CONSTANTS1, &af_cnsts1.u,
+ BLKADDR_CPT0);
if (ret)
return ret;
diff --git a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c
index 1dc3ba298139..a531f4c8b441 100644
--- a/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c
+++ b/drivers/crypto/marvell/octeontx2/otx2_cptpf_ucode.c
@@ -153,16 +153,16 @@ static int get_ucode_type(struct device *dev,
}
static int __write_ucode_base(struct otx2_cptpf_dev *cptpf, int eng,
- dma_addr_t dma_addr)
+ dma_addr_t dma_addr, int blkaddr)
{
return otx2_cpt_write_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
CPT_AF_EXEX_UCODE_BASE(eng),
- (u64)dma_addr);
+ (u64)dma_addr, blkaddr);
}
-static int cpt_set_ucode_base(struct otx2_cpt_eng_grp_info *eng_grp, void *obj)
+static int cptx_set_ucode_base(struct otx2_cpt_eng_grp_info *eng_grp,
+ struct otx2_cptpf_dev *cptpf, int blkaddr)
{
- struct otx2_cptpf_dev *cptpf = obj;
struct otx2_cpt_engs_rsvd *engs;
dma_addr_t dma_addr;
int i, bit, ret;
@@ -170,7 +170,7 @@ static int cpt_set_ucode_base(struct otx2_cpt_eng_grp_info *eng_grp, void *obj)
/* Set PF number for microcode fetches */
ret = otx2_cpt_write_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
CPT_AF_PF_FUNC,
- cptpf->pf_id << RVU_PFVF_PF_SHIFT);
+ cptpf->pf_id << RVU_PFVF_PF_SHIFT, blkaddr);
if (ret)
return ret;
@@ -187,7 +187,8 @@ static int cpt_set_ucode_base(struct otx2_cpt_eng_grp_info *eng_grp, void *obj)
*/
for_each_set_bit(bit, engs->bmap, eng_grp->g->engs_num)
if (!eng_grp->g->eng_ref_cnt[bit]) {
- ret = __write_ucode_base(cptpf, bit, dma_addr);
+ ret = __write_ucode_base(cptpf, bit, dma_addr,
+ blkaddr);
if (ret)
return ret;
}
@@ -195,23 +196,32 @@ static int cpt_set_ucode_base(struct otx2_cpt_eng_grp_info *eng_grp, void *obj)
return 0;
}
-static int cpt_detach_and_disable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
- void *obj)
+static int cpt_set_ucode_base(struct otx2_cpt_eng_grp_info *eng_grp, void *obj)
{
struct otx2_cptpf_dev *cptpf = obj;
- struct otx2_cpt_bitmap bmap;
+ int ret;
+
+ if (cptpf->has_cpt1) {
+ ret = cptx_set_ucode_base(eng_grp, cptpf, BLKADDR_CPT1);
+ if (ret)
+ return ret;
+ }
+ return cptx_set_ucode_base(eng_grp, cptpf, BLKADDR_CPT0);
+}
+
+static int cptx_detach_and_disable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
+ struct otx2_cptpf_dev *cptpf,
+ struct otx2_cpt_bitmap bmap,
+ int blkaddr)
+{
int i, timeout = 10;
int busy, ret;
u64 reg = 0;
- bmap = get_cores_bmap(&cptpf->pdev->dev, eng_grp);
- if (!bmap.size)
- return -EINVAL;
-
/* Detach the cores from group */
for_each_set_bit(i, bmap.bits, bmap.size) {
ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
- CPT_AF_EXEX_CTL2(i), &reg);
+ CPT_AF_EXEX_CTL2(i), &reg, blkaddr);
if (ret)
return ret;
@@ -221,7 +231,8 @@ static int cpt_detach_and_disable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
ret = otx2_cpt_write_af_reg(&cptpf->afpf_mbox,
cptpf->pdev,
- CPT_AF_EXEX_CTL2(i), reg);
+ CPT_AF_EXEX_CTL2(i), reg,
+ blkaddr);
if (ret)
return ret;
}
@@ -237,7 +248,8 @@ static int cpt_detach_and_disable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
for_each_set_bit(i, bmap.bits, bmap.size) {
ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox,
cptpf->pdev,
- CPT_AF_EXEX_STS(i), &reg);
+ CPT_AF_EXEX_STS(i), &reg,
+ blkaddr);
if (ret)
return ret;
@@ -253,7 +265,8 @@ static int cpt_detach_and_disable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
if (!eng_grp->g->eng_ref_cnt[i]) {
ret = otx2_cpt_write_af_reg(&cptpf->afpf_mbox,
cptpf->pdev,
- CPT_AF_EXEX_CTL(i), 0x0);
+ CPT_AF_EXEX_CTL(i), 0x0,
+ blkaddr);
if (ret)
return ret;
}
@@ -262,22 +275,39 @@ static int cpt_detach_and_disable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
return 0;
}
-static int cpt_attach_and_enable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
- void *obj)
+static int cpt_detach_and_disable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
+ void *obj)
{
struct otx2_cptpf_dev *cptpf = obj;
struct otx2_cpt_bitmap bmap;
- u64 reg = 0;
- int i, ret;
+ int ret;
bmap = get_cores_bmap(&cptpf->pdev->dev, eng_grp);
if (!bmap.size)
return -EINVAL;
+ if (cptpf->has_cpt1) {
+ ret = cptx_detach_and_disable_cores(eng_grp, cptpf, bmap,
+ BLKADDR_CPT1);
+ if (ret)
+ return ret;
+ }
+ return cptx_detach_and_disable_cores(eng_grp, cptpf, bmap,
+ BLKADDR_CPT0);
+}
+
+static int cptx_attach_and_enable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
+ struct otx2_cptpf_dev *cptpf,
+ struct otx2_cpt_bitmap bmap,
+ int blkaddr)
+{
+ u64 reg = 0;
+ int i, ret;
+
/* Attach the cores to the group */
for_each_set_bit(i, bmap.bits, bmap.size) {
ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
- CPT_AF_EXEX_CTL2(i), &reg);
+ CPT_AF_EXEX_CTL2(i), &reg, blkaddr);
if (ret)
return ret;
@@ -287,7 +317,8 @@ static int cpt_attach_and_enable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
ret = otx2_cpt_write_af_reg(&cptpf->afpf_mbox,
cptpf->pdev,
- CPT_AF_EXEX_CTL2(i), reg);
+ CPT_AF_EXEX_CTL2(i), reg,
+ blkaddr);
if (ret)
return ret;
}
@@ -295,15 +326,33 @@ static int cpt_attach_and_enable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
/* Enable the cores */
for_each_set_bit(i, bmap.bits, bmap.size) {
- ret = otx2_cpt_add_write_af_reg(&cptpf->afpf_mbox,
- cptpf->pdev,
- CPT_AF_EXEX_CTL(i), 0x1);
+ ret = otx2_cpt_add_write_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
+ CPT_AF_EXEX_CTL(i), 0x1,
+ blkaddr);
if (ret)
return ret;
}
- ret = otx2_cpt_send_af_reg_requests(&cptpf->afpf_mbox, cptpf->pdev);
+ return otx2_cpt_send_af_reg_requests(&cptpf->afpf_mbox, cptpf->pdev);
+}
- return ret;
+static int cpt_attach_and_enable_cores(struct otx2_cpt_eng_grp_info *eng_grp,
+ void *obj)
+{
+ struct otx2_cptpf_dev *cptpf = obj;
+ struct otx2_cpt_bitmap bmap;
+ int ret;
+
+ bmap = get_cores_bmap(&cptpf->pdev->dev, eng_grp);
+ if (!bmap.size)
+ return -EINVAL;
+
+ if (cptpf->has_cpt1) {
+ ret = cptx_attach_and_enable_cores(eng_grp, cptpf, bmap,
+ BLKADDR_CPT1);
+ if (ret)
+ return ret;
+ }
+ return cptx_attach_and_enable_cores(eng_grp, cptpf, bmap, BLKADDR_CPT0);
}
static int load_fw(struct device *dev, struct fw_info_t *fw_info,
@@ -1140,20 +1189,18 @@ release_fw:
return ret;
}
-int otx2_cpt_disable_all_cores(struct otx2_cptpf_dev *cptpf)
+static int cptx_disable_all_cores(struct otx2_cptpf_dev *cptpf, int total_cores,
+ int blkaddr)
{
- int i, ret, busy, total_cores;
- int timeout = 10;
- u64 reg = 0;
-
- total_cores = cptpf->eng_grps.avail.max_se_cnt +
- cptpf->eng_grps.avail.max_ie_cnt +
- cptpf->eng_grps.avail.max_ae_cnt;
+ int timeout = 10, ret;
+ int i, busy;
+ u64 reg;
/* Disengage the cores from groups */
for (i = 0; i < total_cores; i++) {
ret = otx2_cpt_add_write_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
- CPT_AF_EXEX_CTL2(i), 0x0);
+ CPT_AF_EXEX_CTL2(i), 0x0,
+ blkaddr);
if (ret)
return ret;
@@ -1173,7 +1220,8 @@ int otx2_cpt_disable_all_cores(struct otx2_cptpf_dev *cptpf)
for (i = 0; i < total_cores; i++) {
ret = otx2_cpt_read_af_reg(&cptpf->afpf_mbox,
cptpf->pdev,
- CPT_AF_EXEX_STS(i), &reg);
+ CPT_AF_EXEX_STS(i), &reg,
+ blkaddr);
if (ret)
return ret;
@@ -1187,13 +1235,30 @@ int otx2_cpt_disable_all_cores(struct otx2_cptpf_dev *cptpf)
/* Disable the cores */
for (i = 0; i < total_cores; i++) {
ret = otx2_cpt_add_write_af_reg(&cptpf->afpf_mbox, cptpf->pdev,
- CPT_AF_EXEX_CTL(i), 0x0);
+ CPT_AF_EXEX_CTL(i), 0x0,
+ blkaddr);
if (ret)
return ret;
}
return otx2_cpt_send_af_reg_requests(&cptpf->afpf_mbox, cptpf->pdev);
}
+int otx2_cpt_disable_all_cores(struct otx2_cptpf_dev *cptpf)
+{
+ int total_cores, ret;
+
+ total_cores = cptpf->eng_grps.avail.max_se_cnt +
+ cptpf->eng_grps.avail.max_ie_cnt +
+ cptpf->eng_grps.avail.max_ae_cnt;
+
+ if (cptpf->has_cpt1) {
+ ret = cptx_disable_all_cores(cptpf, total_cores, BLKADDR_CPT1);
+ if (ret)
+ return ret;
+ }
+ return cptx_disable_all_cores(cptpf, total_cores, BLKADDR_CPT0);
+}
+
void otx2_cpt_cleanup_eng_grps(struct pci_dev *pdev,
struct otx2_cpt_eng_grps *eng_grps)
{
@@ -1354,6 +1419,7 @@ int otx2_cpt_discover_eng_capabilities(struct otx2_cptpf_dev *cptpf)
lfs->pdev = pdev;
lfs->reg_base = cptpf->reg_base;
lfs->mbox = &cptpf->afpf_mbox;
+ lfs->blkaddr = BLKADDR_CPT0;
ret = otx2_cptlf_init(&cptpf->lfs, OTX2_CPT_ALL_ENG_GRPS_MASK,
OTX2_CPT_QUEUE_HI_PRIO, 1);
if (ret)
diff --git a/drivers/crypto/nx/nx-842-pseries.c b/drivers/crypto/nx/nx-842-pseries.c
index 2de5e3672e42..cc8dd3072b8b 100644
--- a/drivers/crypto/nx/nx-842-pseries.c
+++ b/drivers/crypto/nx/nx-842-pseries.c
@@ -1042,7 +1042,7 @@ error:
return ret;
}
-static int nx842_remove(struct vio_dev *viodev)
+static void nx842_remove(struct vio_dev *viodev)
{
struct nx842_devdata *old_devdata;
unsigned long flags;
@@ -1063,8 +1063,6 @@ static int nx842_remove(struct vio_dev *viodev)
if (old_devdata)
kfree(old_devdata->counters);
kfree(old_devdata);
-
- return 0;
}
static const struct vio_device_id nx842_vio_driver_ids[] = {
diff --git a/drivers/crypto/nx/nx-aes-cbc.c b/drivers/crypto/nx/nx-aes-cbc.c
index 92e921eceed7..d6314ea9ae89 100644
--- a/drivers/crypto/nx/nx-aes-cbc.c
+++ b/drivers/crypto/nx/nx-aes-cbc.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES CBC routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx-aes-ccm.c b/drivers/crypto/nx/nx-aes-ccm.c
index 4c9362eebefd..e7384d107573 100644
--- a/drivers/crypto/nx/nx-aes-ccm.c
+++ b/drivers/crypto/nx/nx-aes-ccm.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES CCM routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx-aes-ctr.c b/drivers/crypto/nx/nx-aes-ctr.c
index 6d5ce1a66f1e..13f518802343 100644
--- a/drivers/crypto/nx/nx-aes-ctr.c
+++ b/drivers/crypto/nx/nx-aes-ctr.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES CTR routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx-aes-ecb.c b/drivers/crypto/nx/nx-aes-ecb.c
index 77e338dc33f1..7a729dc2bc17 100644
--- a/drivers/crypto/nx/nx-aes-ecb.c
+++ b/drivers/crypto/nx/nx-aes-ecb.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES ECB routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx-aes-gcm.c b/drivers/crypto/nx/nx-aes-gcm.c
index 19c6ed5baea4..fc9baca13920 100644
--- a/drivers/crypto/nx/nx-aes-gcm.c
+++ b/drivers/crypto/nx/nx-aes-gcm.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES GCM routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx-aes-xcbc.c b/drivers/crypto/nx/nx-aes-xcbc.c
index 48dc1c98ca52..eb5c8f689360 100644
--- a/drivers/crypto/nx/nx-aes-xcbc.c
+++ b/drivers/crypto/nx/nx-aes-xcbc.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES XCBC routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx-common-powernv.c b/drivers/crypto/nx/nx-common-powernv.c
index 13c65deda8e9..446f611726df 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -932,8 +932,10 @@ static int __init nx_powernv_probe_vas(struct device_node *pn)
ret = find_nx_device_tree(dn, chip_id, vasid,
NX_CT_GZIP, "ibm,p9-nx-gzip", &ct_gzip);
- if (ret)
+ if (ret) {
+ of_node_put(dn);
return ret;
+ }
}
if (!ct_842 || !ct_gzip) {
diff --git a/drivers/crypto/nx/nx-sha256.c b/drivers/crypto/nx/nx-sha256.c
index 90d9a37a57f6..b0ad665e4bda 100644
--- a/drivers/crypto/nx/nx-sha256.c
+++ b/drivers/crypto/nx/nx-sha256.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* SHA-256 routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx-sha512.c b/drivers/crypto/nx/nx-sha512.c
index eb8627a0f317..c29103a1a0b6 100644
--- a/drivers/crypto/nx/nx-sha512.c
+++ b/drivers/crypto/nx/nx-sha512.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* SHA-512 routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
diff --git a/drivers/crypto/nx/nx.c b/drivers/crypto/nx/nx.c
index 0d2dc5be7f19..010e87d9da36 100644
--- a/drivers/crypto/nx/nx.c
+++ b/drivers/crypto/nx/nx.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* Routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
@@ -200,7 +200,8 @@ struct nx_sg *nx_walk_and_build(struct nx_sg *nx_dst,
* @sg: sg list head
* @end: sg lisg end
* @delta: is the amount we need to crop in order to bound the list.
- *
+ * @nbytes: length of data in the scatterlists or data length - whichever
+ * is greater.
*/
static long int trim_sg_list(struct nx_sg *sg,
struct nx_sg *end,
@@ -783,7 +784,7 @@ static int nx_probe(struct vio_dev *viodev, const struct vio_device_id *id)
return nx_register_algs();
}
-static int nx_remove(struct vio_dev *viodev)
+static void nx_remove(struct vio_dev *viodev)
{
dev_dbg(&viodev->dev, "entering nx_remove for UA 0x%x\n",
viodev->unit_address);
@@ -811,8 +812,6 @@ static int nx_remove(struct vio_dev *viodev)
nx_unregister_skcipher(&nx_ecb_aes_alg, NX_FC_AES,
NX_MODE_AES_ECB);
}
-
- return 0;
}
diff --git a/drivers/crypto/nx/nx_debugfs.c b/drivers/crypto/nx/nx_debugfs.c
index 1975bcbee997..ee7cd88bb10a 100644
--- a/drivers/crypto/nx/nx_debugfs.c
+++ b/drivers/crypto/nx/nx_debugfs.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* debugfs routines supporting the Power 7+ Nest Accelerators driver
*
* Copyright (C) 2011-2012 International Business Machines Inc.
diff --git a/drivers/crypto/omap-aes.c b/drivers/crypto/omap-aes.c
index a45bdcf3026d..0dd4c6b157de 100644
--- a/drivers/crypto/omap-aes.c
+++ b/drivers/crypto/omap-aes.c
@@ -103,9 +103,8 @@ static int omap_aes_hw_init(struct omap_aes_dev *dd)
dd->err = 0;
}
- err = pm_runtime_get_sync(dd->dev);
+ err = pm_runtime_resume_and_get(dd->dev);
if (err < 0) {
- pm_runtime_put_noidle(dd->dev);
dev_err(dd->dev, "failed to get sync: %d\n", err);
return err;
}
@@ -1134,7 +1133,7 @@ static int omap_aes_probe(struct platform_device *pdev)
pm_runtime_set_autosuspend_delay(dev, DEFAULT_AUTOSUSPEND_DELAY);
pm_runtime_enable(dev);
- err = pm_runtime_get_sync(dev);
+ err = pm_runtime_resume_and_get(dev);
if (err < 0) {
dev_err(dev, "%s: failed to get_sync(%d)\n",
__func__, err);
@@ -1303,7 +1302,7 @@ static int omap_aes_suspend(struct device *dev)
static int omap_aes_resume(struct device *dev)
{
- pm_runtime_get_sync(dev);
+ pm_runtime_resume_and_get(dev);
return 0;
}
#endif
diff --git a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c
index 6a9be01fdf33..3524ddd48930 100644
--- a/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c
+++ b/drivers/crypto/qat/qat_4xxx/adf_4xxx_hw_data.c
@@ -224,6 +224,7 @@ void adf_init_hw_data_4xxx(struct adf_hw_device_data *hw_data)
hw_data->uof_get_name = uof_get_name;
hw_data->uof_get_ae_mask = uof_get_ae_mask;
hw_data->set_msix_rttable = set_msix_default_rttable;
+ hw_data->set_ssm_wdtimer = adf_gen4_set_ssm_wdtimer;
adf_gen4_init_hw_csr_ops(&hw_data->csr_ops);
}
diff --git a/drivers/crypto/qat/qat_c3xxx/adf_c3xxx_hw_data.c b/drivers/crypto/qat/qat_c3xxx/adf_c3xxx_hw_data.c
index f5990d042c9a..1dd64af22bea 100644
--- a/drivers/crypto/qat/qat_c3xxx/adf_c3xxx_hw_data.c
+++ b/drivers/crypto/qat/qat_c3xxx/adf_c3xxx_hw_data.c
@@ -212,6 +212,7 @@ void adf_init_hw_data_c3xxx(struct adf_hw_device_data *hw_data)
hw_data->enable_vf2pf_comms = adf_pf_enable_vf2pf_comms;
hw_data->reset_device = adf_reset_flr;
hw_data->min_iov_compat_ver = ADF_PFVF_COMPATIBILITY_VERSION;
+ hw_data->set_ssm_wdtimer = adf_gen2_set_ssm_wdtimer;
adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
}
diff --git a/drivers/crypto/qat/qat_c3xxxvf/adf_drv.c b/drivers/crypto/qat/qat_c3xxxvf/adf_drv.c
index 1d1532e8fb6d..067ca5e17d38 100644
--- a/drivers/crypto/qat/qat_c3xxxvf/adf_drv.c
+++ b/drivers/crypto/qat/qat_c3xxxvf/adf_drv.c
@@ -184,12 +184,12 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (ret)
goto out_err_free_reg;
- set_bit(ADF_STATUS_PF_RUNNING, &accel_dev->status);
-
ret = adf_dev_init(accel_dev);
if (ret)
goto out_err_dev_shutdown;
+ set_bit(ADF_STATUS_PF_RUNNING, &accel_dev->status);
+
ret = adf_dev_start(accel_dev);
if (ret)
goto out_err_dev_stop;
diff --git a/drivers/crypto/qat/qat_c62x/adf_c62x_hw_data.c b/drivers/crypto/qat/qat_c62x/adf_c62x_hw_data.c
index cadcf12884c8..30337390513c 100644
--- a/drivers/crypto/qat/qat_c62x/adf_c62x_hw_data.c
+++ b/drivers/crypto/qat/qat_c62x/adf_c62x_hw_data.c
@@ -214,6 +214,7 @@ void adf_init_hw_data_c62x(struct adf_hw_device_data *hw_data)
hw_data->enable_vf2pf_comms = adf_pf_enable_vf2pf_comms;
hw_data->reset_device = adf_reset_flr;
hw_data->min_iov_compat_ver = ADF_PFVF_COMPATIBILITY_VERSION;
+ hw_data->set_ssm_wdtimer = adf_gen2_set_ssm_wdtimer;
adf_gen2_init_hw_csr_ops(&hw_data->csr_ops);
}
diff --git a/drivers/crypto/qat/qat_c62xvf/adf_drv.c b/drivers/crypto/qat/qat_c62xvf/adf_drv.c
index 04742a6d91ca..51ea88c0b17d 100644
--- a/drivers/crypto/qat/qat_c62xvf/adf_drv.c
+++ b/drivers/crypto/qat/qat_c62xvf/adf_drv.c
@@ -184,12 +184,12 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (ret)
goto out_err_free_reg;
- set_bit(ADF_STATUS_PF_RUNNING, &accel_dev->status);
-
ret = adf_dev_init(accel_dev);
if (ret)
goto out_err_dev_shutdown;
+ set_bit(ADF_STATUS_PF_RUNNING, &accel_dev->status);
+
ret = adf_dev_start(accel_dev);
if (ret)
goto out_err_dev_stop;
diff --git a/drivers/crypto/qat/qat_common/adf_accel_devices.h b/drivers/crypto/qat/qat_common/adf_accel_devices.h
index 5527344546e5..ac435b44f1d2 100644
--- a/drivers/crypto/qat/qat_common/adf_accel_devices.h
+++ b/drivers/crypto/qat/qat_common/adf_accel_devices.h
@@ -173,6 +173,7 @@ struct adf_hw_device_data {
void (*configure_iov_threads)(struct adf_accel_dev *accel_dev,
bool enable);
void (*enable_ints)(struct adf_accel_dev *accel_dev);
+ void (*set_ssm_wdtimer)(struct adf_accel_dev *accel_dev);
int (*enable_vf2pf_comms)(struct adf_accel_dev *accel_dev);
void (*reset_device)(struct adf_accel_dev *accel_dev);
void (*set_msix_rttable)(struct adf_accel_dev *accel_dev);
diff --git a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c
index 1aa17303838d..9e560c7d4163 100644
--- a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c
+++ b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.c
@@ -179,3 +179,28 @@ u32 adf_gen2_get_accel_cap(struct adf_accel_dev *accel_dev)
return capabilities;
}
EXPORT_SYMBOL_GPL(adf_gen2_get_accel_cap);
+
+void adf_gen2_set_ssm_wdtimer(struct adf_accel_dev *accel_dev)
+{
+ struct adf_hw_device_data *hw_data = accel_dev->hw_device;
+ u32 timer_val_pke = ADF_SSM_WDT_PKE_DEFAULT_VALUE;
+ u32 timer_val = ADF_SSM_WDT_DEFAULT_VALUE;
+ unsigned long accel_mask = hw_data->accel_mask;
+ void __iomem *pmisc_addr;
+ struct adf_bar *pmisc;
+ int pmisc_id;
+ u32 i = 0;
+
+ pmisc_id = hw_data->get_misc_bar_id(hw_data);
+ pmisc = &GET_BARS(accel_dev)[pmisc_id];
+ pmisc_addr = pmisc->virt_addr;
+
+ /* Configures WDT timers */
+ for_each_set_bit(i, &accel_mask, hw_data->num_accel) {
+ /* Enable WDT for sym and dc */
+ ADF_CSR_WR(pmisc_addr, ADF_SSMWDT(i), timer_val);
+ /* Enable WDT for pke */
+ ADF_CSR_WR(pmisc_addr, ADF_SSMWDTPKE(i), timer_val_pke);
+ }
+}
+EXPORT_SYMBOL_GPL(adf_gen2_set_ssm_wdtimer);
diff --git a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.h b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.h
index 3816e6500352..756b0ddfac5e 100644
--- a/drivers/crypto/qat/qat_common/adf_gen2_hw_data.h
+++ b/drivers/crypto/qat/qat_common/adf_gen2_hw_data.h
@@ -113,11 +113,24 @@ do { \
/* Power gating */
#define ADF_POWERGATE_PKE BIT(24)
+/* WDT timers
+ *
+ * Timeout is in cycles. Clock speed may vary across products but this
+ * value should be a few milli-seconds.
+ */
+#define ADF_SSM_WDT_DEFAULT_VALUE 0x200000
+#define ADF_SSM_WDT_PKE_DEFAULT_VALUE 0x2000000
+#define ADF_SSMWDT_OFFSET 0x54
+#define ADF_SSMWDTPKE_OFFSET 0x58
+#define ADF_SSMWDT(i) (ADF_SSMWDT_OFFSET + ((i) * 0x4000))
+#define ADF_SSMWDTPKE(i) (ADF_SSMWDTPKE_OFFSET + ((i) * 0x4000))
+
void adf_gen2_cfg_iov_thds(struct adf_accel_dev *accel_dev, bool enable,
int num_a_regs, int num_b_regs);
void adf_gen2_init_hw_csr_ops(struct adf_hw_csr_ops *csr_ops);
void adf_gen2_get_admin_info(struct admin_info *admin_csrs_info);
void adf_gen2_get_arb_info(struct arb_info *arb_info);
u32 adf_gen2_get_accel_cap(struct adf_accel_dev *accel_dev);
+void adf_gen2_set_ssm_wdtimer(struct adf_accel_dev *accel_dev);
#endif
diff --git a/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c b/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c
index b72ff58e0bc7..000528327b29 100644
--- a/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c
+++ b/drivers/crypto/qat/qat_common/adf_gen4_hw_data.c
@@ -99,3 +99,43 @@ void adf_gen4_init_hw_csr_ops(struct adf_hw_csr_ops *csr_ops)
csr_ops->write_csr_ring_srv_arb_en = write_csr_ring_srv_arb_en;
}
EXPORT_SYMBOL_GPL(adf_gen4_init_hw_csr_ops);
+
+static inline void adf_gen4_unpack_ssm_wdtimer(u64 value, u32 *upper,
+ u32 *lower)
+{
+ *lower = lower_32_bits(value);
+ *upper = upper_32_bits(value);
+}
+
+void adf_gen4_set_ssm_wdtimer(struct adf_accel_dev *accel_dev)
+{
+ struct adf_hw_device_data *hw_data = accel_dev->hw_device;
+ u64 timer_val_pke = ADF_SSM_WDT_PKE_DEFAULT_VALUE;
+ u64 timer_val = ADF_SSM_WDT_DEFAULT_VALUE;
+ u32 ssm_wdt_pke_high = 0;
+ u32 ssm_wdt_pke_low = 0;
+ u32 ssm_wdt_high = 0;
+ u32 ssm_wdt_low = 0;
+ void __iomem *pmisc_addr;
+ struct adf_bar *pmisc;
+ int pmisc_id;
+
+ pmisc_id = hw_data->get_misc_bar_id(hw_data);
+ pmisc = &GET_BARS(accel_dev)[pmisc_id];
+ pmisc_addr = pmisc->virt_addr;
+
+ /* Convert 64bit WDT timer value into 32bit values for
+ * mmio write to 32bit CSRs.
+ */
+ adf_gen4_unpack_ssm_wdtimer(timer_val, &ssm_wdt_high, &ssm_wdt_low);
+ adf_gen4_unpack_ssm_wdtimer(timer_val_pke, &ssm_wdt_pke_high,
+ &ssm_wdt_pke_low);
+
+ /* Enable WDT for sym and dc */
+ ADF_CSR_WR(pmisc_addr, ADF_SSMWDTL_OFFSET, ssm_wdt_low);
+ ADF_CSR_WR(pmisc_addr, ADF_SSMWDTH_OFFSET, ssm_wdt_high);
+ /* Enable WDT for pke */
+ ADF_CSR_WR(pmisc_addr, ADF_SSMWDTPKEL_OFFSET, ssm_wdt_pke_low);
+ ADF_CSR_WR(pmisc_addr, ADF_SSMWDTPKEH_OFFSET, ssm_wdt_pke_high);
+}
+EXPORT_SYMBOL_GPL(adf_gen4_set_ssm_wdtimer);
diff --git a/drivers/crypto/qat/qat_common/adf_gen4_hw_data.h b/drivers/crypto/qat/qat_common/adf_gen4_hw_data.h
index 8ab62b2ac311..b8fca1ff7aab 100644
--- a/drivers/crypto/qat/qat_common/adf_gen4_hw_data.h
+++ b/drivers/crypto/qat/qat_common/adf_gen4_hw_data.h
@@ -94,6 +94,18 @@ do { \
ADF_RING_BUNDLE_SIZE * (bank) + \
ADF_RING_CSR_RING_SRV_ARB_EN, (value))
-void adf_gen4_init_hw_csr_ops(struct adf_hw_csr_ops *csr_ops);
+/* WDT timers
+ *
+ * Timeout is in cycles. Clock speed may vary across products but this
+ * value should be a few milli-seconds.
+ */
+#define ADF_SSM_WDT_DEFAULT_VALUE 0x200000
+#define ADF_SSM_WDT_PKE_DEFAULT_VALUE 0x8000000
+#define ADF_SSMWDTL_OFFSET 0x54
+#define ADF_SSMWDTH_OFFSET 0x5C
+#define ADF_SSMWDTPKEL_OFFSET 0x58
+#define ADF_SSMWDTPKEH_OFFSET 0x60
+void adf_gen4_set_ssm_wdtimer(struct adf_accel_dev *accel_dev);
+void adf_gen4_init_hw_csr_ops(struct adf_hw_csr_ops *csr_ops);
#endif
diff --git a/drivers/crypto/qat/qat_common/adf_init.c b/drivers/crypto/qat/qat_common/adf_init.c
index 42029153408e..744c40351428 100644
--- a/drivers/crypto/qat/qat_common/adf_init.c
+++ b/drivers/crypto/qat/qat_common/adf_init.c
@@ -162,6 +162,10 @@ int adf_dev_start(struct adf_accel_dev *accel_dev)
return -EFAULT;
}
+ /* Set ssm watch dog timer */
+ if (hw_data->set_ssm_wdtimer)
+ hw_data->set_ssm_wdtimer(accel_dev);
+
list_for_each(list_itr, &service_table) {
service = list_entry(list_itr, struct service_hndl, list);
if (service->event_hld(accel_dev, ADF_EVENT_START)) {
diff --git a/drivers/crypto/qat/qat_common/adf_isr.c b/drivers/crypto/qat/qat_common/adf_isr.c
index c45853463530..e3ad5587be49 100644
--- a/drivers/crypto/qat/qat_common/adf_isr.c
+++ b/drivers/crypto/qat/qat_common/adf_isr.c
@@ -291,19 +291,32 @@ int adf_isr_resource_alloc(struct adf_accel_dev *accel_dev)
ret = adf_isr_alloc_msix_entry_table(accel_dev);
if (ret)
- return ret;
- if (adf_enable_msix(accel_dev))
goto err_out;
- if (adf_setup_bh(accel_dev))
- goto err_out;
+ ret = adf_enable_msix(accel_dev);
+ if (ret)
+ goto err_free_msix_table;
- if (adf_request_irqs(accel_dev))
- goto err_out;
+ ret = adf_setup_bh(accel_dev);
+ if (ret)
+ goto err_disable_msix;
+
+ ret = adf_request_irqs(accel_dev);
+ if (ret)
+ goto err_cleanup_bh;
return 0;
+
+err_cleanup_bh:
+ adf_cleanup_bh(accel_dev);
+
+err_disable_msix:
+ adf_disable_msix(&accel_dev->accel_pci_dev);
+
+err_free_msix_table:
+ adf_isr_free_msix_entry_table(accel_dev);
+
err_out:
- adf_isr_resource_free(accel_dev);
- return -EFAULT;
+ return ret;
}
EXPORT_SYMBOL_GPL(adf_isr_resource_alloc);
diff --git a/drivers/crypto/qat/qat_common/adf_pf2vf_msg.c b/drivers/crypto/qat/qat_common/adf_pf2vf_msg.c
index 8b090b7ae8c6..a1b77bd7a894 100644
--- a/drivers/crypto/qat/qat_common/adf_pf2vf_msg.c
+++ b/drivers/crypto/qat/qat_common/adf_pf2vf_msg.c
@@ -169,7 +169,7 @@ out:
* @msg: Message to send
* @vf_nr: VF number to which the message will be sent
*
- * Function sends a messge from the PF to a VF
+ * Function sends a message from the PF to a VF
*
* Return: 0 on success, error code otherwise.
*/
diff --git a/drivers/crypto/qat/qat_common/adf_transport.c b/drivers/crypto/qat/qat_common/adf_transport.c
index 888c1e047295..8ba28409fb74 100644
--- a/drivers/crypto/qat/qat_common/adf_transport.c
+++ b/drivers/crypto/qat/qat_common/adf_transport.c
@@ -172,6 +172,7 @@ static int adf_init_ring(struct adf_etr_ring_data *ring)
dev_err(&GET_DEV(accel_dev), "Ring address not aligned\n");
dma_free_coherent(&GET_DEV(accel_dev), ring_size_bytes,
ring->base_addr, ring->dma_addr);
+ ring->base_addr = NULL;
return -EFAULT;
}
diff --git a/drivers/crypto/qat/qat_common/adf_vf2pf_msg.c b/drivers/crypto/qat/qat_common/adf_vf2pf_msg.c
index 2c98fb63f7b7..e85bd62d134a 100644
--- a/drivers/crypto/qat/qat_common/adf_vf2pf_msg.c
+++ b/drivers/crypto/qat/qat_common/adf_vf2pf_msg.c
@@ -8,7 +8,7 @@
* adf_vf2pf_init() - send init msg to PF
* @accel_dev: Pointer to acceleration VF device.
*
- * Function sends an init messge from the VF to a PF
+ * Function sends an init message from the VF to a PF
*
* Return: 0 on success, error code otherwise.
*/
@@ -31,7 +31,7 @@ EXPORT_SYMBOL_GPL(adf_vf2pf_init);
* adf_vf2pf_shutdown() - send shutdown msg to PF
* @accel_dev: Pointer to acceleration VF device.
*
- * Function sends a shutdown messge from the VF to a PF
+ * Function sends a shutdown message from the VF to a PF
*
* Return: void
*/
diff --git a/drivers/crypto/qat/qat_common/adf_vf_isr.c b/drivers/crypto/qat/qat_common/adf_vf_isr.c
index 38d316a42ba6..888388acb6bd 100644
--- a/drivers/crypto/qat/qat_common/adf_vf_isr.c
+++ b/drivers/crypto/qat/qat_common/adf_vf_isr.c
@@ -261,17 +261,26 @@ int adf_vf_isr_resource_alloc(struct adf_accel_dev *accel_dev)
goto err_out;
if (adf_setup_pf2vf_bh(accel_dev))
- goto err_out;
+ goto err_disable_msi;
if (adf_setup_bh(accel_dev))
- goto err_out;
+ goto err_cleanup_pf2vf_bh;
if (adf_request_msi_irq(accel_dev))
- goto err_out;
+ goto err_cleanup_bh;
return 0;
+
+err_cleanup_bh:
+ adf_cleanup_bh(accel_dev);
+
+err_cleanup_pf2vf_bh:
+ adf_cleanup_pf2vf_bh(accel_dev);
+
+err_disable_msi:
+ adf_disable_msi(accel_dev);
+
err_out:
- adf_vf_isr_resource_free(accel_dev);
return -EFAULT;
}
EXPORT_SYMBOL_GPL(adf_vf_isr_resource_alloc);
diff --git a/drivers/crypto/qat/qat_common/qat_algs.c b/drivers/crypto/qat/qat_common/qat_algs.c
index ff78c73c47e3..f998ed58457c 100644
--- a/drivers/crypto/qat/qat_common/qat_algs.c
+++ b/drivers/crypto/qat/qat_common/qat_algs.c
@@ -718,8 +718,8 @@ static int qat_alg_sgl_to_bufl(struct qat_crypto_instance *inst,
int n = sg_nents(sgl);
struct qat_alg_buf_list *bufl;
struct qat_alg_buf_list *buflout = NULL;
- dma_addr_t blp;
- dma_addr_t bloutp = 0;
+ dma_addr_t blp = DMA_MAPPING_ERROR;
+ dma_addr_t bloutp = DMA_MAPPING_ERROR;
struct scatterlist *sg;
size_t sz_out, sz = struct_size(bufl, bufers, n + 1);
@@ -731,9 +731,8 @@ static int qat_alg_sgl_to_bufl(struct qat_crypto_instance *inst,
if (unlikely(!bufl))
return -ENOMEM;
- blp = dma_map_single(dev, bufl, sz, DMA_TO_DEVICE);
- if (unlikely(dma_mapping_error(dev, blp)))
- goto err_in;
+ for_each_sg(sgl, sg, n, i)
+ bufl->bufers[i].addr = DMA_MAPPING_ERROR;
for_each_sg(sgl, sg, n, i) {
int y = sg_nctr;
@@ -750,6 +749,9 @@ static int qat_alg_sgl_to_bufl(struct qat_crypto_instance *inst,
sg_nctr++;
}
bufl->num_bufs = sg_nctr;
+ blp = dma_map_single(dev, bufl, sz, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(dev, blp)))
+ goto err_in;
qat_req->buf.bl = bufl;
qat_req->buf.blp = blp;
qat_req->buf.sz = sz;
@@ -764,10 +766,11 @@ static int qat_alg_sgl_to_bufl(struct qat_crypto_instance *inst,
dev_to_node(&GET_DEV(inst->accel_dev)));
if (unlikely(!buflout))
goto err_in;
- bloutp = dma_map_single(dev, buflout, sz_out, DMA_TO_DEVICE);
- if (unlikely(dma_mapping_error(dev, bloutp)))
- goto err_out;
+
bufers = buflout->bufers;
+ for_each_sg(sglout, sg, n, i)
+ bufers[i].addr = DMA_MAPPING_ERROR;
+
for_each_sg(sglout, sg, n, i) {
int y = sg_nctr;
@@ -784,6 +787,9 @@ static int qat_alg_sgl_to_bufl(struct qat_crypto_instance *inst,
}
buflout->num_bufs = sg_nctr;
buflout->num_mapped_bufs = sg_nctr;
+ bloutp = dma_map_single(dev, buflout, sz_out, DMA_TO_DEVICE);
+ if (unlikely(dma_mapping_error(dev, bloutp)))
+ goto err_out;
qat_req->buf.blout = buflout;
qat_req->buf.bloutp = bloutp;
qat_req->buf.sz_out = sz_out;
@@ -795,17 +801,21 @@ static int qat_alg_sgl_to_bufl(struct qat_crypto_instance *inst,
return 0;
err_out:
+ if (!dma_mapping_error(dev, bloutp))
+ dma_unmap_single(dev, bloutp, sz_out, DMA_TO_DEVICE);
+
n = sg_nents(sglout);
for (i = 0; i < n; i++)
if (!dma_mapping_error(dev, buflout->bufers[i].addr))
dma_unmap_single(dev, buflout->bufers[i].addr,
buflout->bufers[i].len,
DMA_BIDIRECTIONAL);
- if (!dma_mapping_error(dev, bloutp))
- dma_unmap_single(dev, bloutp, sz_out, DMA_TO_DEVICE);
kfree(buflout);
err_in:
+ if (!dma_mapping_error(dev, blp))
+ dma_unmap_single(dev, blp, sz, DMA_TO_DEVICE);
+
n = sg_nents(sgl);
for (i = 0; i < n; i++)
if (!dma_mapping_error(dev, bufl->bufers[i].addr))
@@ -813,8 +823,6 @@ err_in:
bufl->bufers[i].len,
DMA_BIDIRECTIONAL);
- if (!dma_mapping_error(dev, blp))
- dma_unmap_single(dev, blp, sz, DMA_TO_DEVICE);
kfree(bufl);
dev_err(dev, "Failed to map buf for dma\n");
diff --git a/drivers/crypto/qat/qat_dh895xccvf/adf_drv.c b/drivers/crypto/qat/qat_dh895xccvf/adf_drv.c
index c972554a755e..29999da716cc 100644
--- a/drivers/crypto/qat/qat_dh895xccvf/adf_drv.c
+++ b/drivers/crypto/qat/qat_dh895xccvf/adf_drv.c
@@ -184,12 +184,12 @@ static int adf_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
if (ret)
goto out_err_free_reg;
- set_bit(ADF_STATUS_PF_RUNNING, &accel_dev->status);
-
ret = adf_dev_init(accel_dev);
if (ret)
goto out_err_dev_shutdown;
+ set_bit(ADF_STATUS_PF_RUNNING, &accel_dev->status);
+
ret = adf_dev_start(accel_dev);
if (ret)
goto out_err_dev_stop;
diff --git a/drivers/crypto/qce/cipher.h b/drivers/crypto/qce/cipher.h
index cffa9fc628ff..850f257d00f3 100644
--- a/drivers/crypto/qce/cipher.h
+++ b/drivers/crypto/qce/cipher.h
@@ -40,7 +40,6 @@ struct qce_cipher_reqctx {
struct scatterlist result_sg;
struct sg_table dst_tbl;
struct scatterlist *dst_sg;
- struct sg_table src_tbl;
struct scatterlist *src_sg;
unsigned int cryptlen;
struct skcipher_request fallback_req; // keep at the end
diff --git a/drivers/crypto/qce/common.c b/drivers/crypto/qce/common.c
index a73db2a5637f..dceb9579d87a 100644
--- a/drivers/crypto/qce/common.c
+++ b/drivers/crypto/qce/common.c
@@ -140,8 +140,7 @@ static u32 qce_auth_cfg(unsigned long flags, u32 key_size)
return cfg;
}
-static int qce_setup_regs_ahash(struct crypto_async_request *async_req,
- u32 totallen, u32 offset)
+static int qce_setup_regs_ahash(struct crypto_async_request *async_req)
{
struct ahash_request *req = ahash_request_cast(async_req);
struct crypto_ahash *ahash = __crypto_ahash_cast(async_req->tfm);
@@ -295,19 +294,18 @@ static void qce_xtskey(struct qce_device *qce, const u8 *enckey,
{
u32 xtskey[QCE_MAX_CIPHER_KEY_SIZE / sizeof(u32)] = {0};
unsigned int xtsklen = enckeylen / (2 * sizeof(u32));
- unsigned int xtsdusize;
qce_cpu_to_be32p_array((__be32 *)xtskey, enckey + enckeylen / 2,
enckeylen / 2);
qce_write_array(qce, REG_ENCR_XTS_KEY0, xtskey, xtsklen);
- /* xts du size 512B */
- xtsdusize = min_t(u32, QCE_SECTOR_SIZE, cryptlen);
- qce_write(qce, REG_ENCR_XTS_DU_SIZE, xtsdusize);
+ /* Set data unit size to cryptlen. Anything else causes
+ * crypto engine to return back incorrect results.
+ */
+ qce_write(qce, REG_ENCR_XTS_DU_SIZE, cryptlen);
}
-static int qce_setup_regs_skcipher(struct crypto_async_request *async_req,
- u32 totallen, u32 offset)
+static int qce_setup_regs_skcipher(struct crypto_async_request *async_req)
{
struct skcipher_request *req = skcipher_request_cast(async_req);
struct qce_cipher_reqctx *rctx = skcipher_request_ctx(req);
@@ -367,7 +365,7 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req,
qce_write(qce, REG_ENCR_SEG_CFG, encr_cfg);
qce_write(qce, REG_ENCR_SEG_SIZE, rctx->cryptlen);
- qce_write(qce, REG_ENCR_SEG_START, offset & 0xffff);
+ qce_write(qce, REG_ENCR_SEG_START, 0);
if (IS_CTR(flags)) {
qce_write(qce, REG_CNTR_MASK, ~0);
@@ -376,7 +374,7 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req,
qce_write(qce, REG_CNTR_MASK2, ~0);
}
- qce_write(qce, REG_SEG_SIZE, totallen);
+ qce_write(qce, REG_SEG_SIZE, rctx->cryptlen);
/* get little endianness */
config = qce_config_reg(qce, 1);
@@ -388,17 +386,16 @@ static int qce_setup_regs_skcipher(struct crypto_async_request *async_req,
}
#endif
-int qce_start(struct crypto_async_request *async_req, u32 type, u32 totallen,
- u32 offset)
+int qce_start(struct crypto_async_request *async_req, u32 type)
{
switch (type) {
#ifdef CONFIG_CRYPTO_DEV_QCE_SKCIPHER
case CRYPTO_ALG_TYPE_SKCIPHER:
- return qce_setup_regs_skcipher(async_req, totallen, offset);
+ return qce_setup_regs_skcipher(async_req);
#endif
#ifdef CONFIG_CRYPTO_DEV_QCE_SHA
case CRYPTO_ALG_TYPE_AHASH:
- return qce_setup_regs_ahash(async_req, totallen, offset);
+ return qce_setup_regs_ahash(async_req);
#endif
default:
return -EINVAL;
diff --git a/drivers/crypto/qce/common.h b/drivers/crypto/qce/common.h
index 85ba16418a04..3bc244bcca2d 100644
--- a/drivers/crypto/qce/common.h
+++ b/drivers/crypto/qce/common.h
@@ -94,7 +94,6 @@ struct qce_alg_template {
void qce_cpu_to_be32p_array(__be32 *dst, const u8 *src, unsigned int len);
int qce_check_status(struct qce_device *qce, u32 *status);
void qce_get_version(struct qce_device *qce, u32 *major, u32 *minor, u32 *step);
-int qce_start(struct crypto_async_request *async_req, u32 type, u32 totallen,
- u32 offset);
+int qce_start(struct crypto_async_request *async_req, u32 type);
#endif /* _COMMON_H_ */
diff --git a/drivers/crypto/qce/sha.c b/drivers/crypto/qce/sha.c
index 61c418c12345..8e6fcf2c21cc 100644
--- a/drivers/crypto/qce/sha.c
+++ b/drivers/crypto/qce/sha.c
@@ -12,9 +12,15 @@
#include "core.h"
#include "sha.h"
-/* crypto hw padding constant for first operation */
-#define SHA_PADDING 64
-#define SHA_PADDING_MASK (SHA_PADDING - 1)
+struct qce_sha_saved_state {
+ u8 pending_buf[QCE_SHA_MAX_BLOCKSIZE];
+ u8 partial_digest[QCE_SHA_MAX_DIGESTSIZE];
+ __be32 byte_count[2];
+ unsigned int pending_buflen;
+ unsigned int flags;
+ u64 count;
+ bool first_blk;
+};
static LIST_HEAD(ahash_algs);
@@ -107,7 +113,7 @@ static int qce_ahash_async_req_handle(struct crypto_async_request *async_req)
qce_dma_issue_pending(&qce->dma);
- ret = qce_start(async_req, tmpl->crypto_alg_type, 0, 0);
+ ret = qce_start(async_req, tmpl->crypto_alg_type);
if (ret)
goto error_terminate;
@@ -139,97 +145,37 @@ static int qce_ahash_init(struct ahash_request *req)
static int qce_ahash_export(struct ahash_request *req, void *out)
{
- struct crypto_ahash *ahash = crypto_ahash_reqtfm(req);
struct qce_sha_reqctx *rctx = ahash_request_ctx(req);
- unsigned long flags = rctx->flags;
- unsigned int digestsize = crypto_ahash_digestsize(ahash);
- unsigned int blocksize =
- crypto_tfm_alg_blocksize(crypto_ahash_tfm(ahash));
-
- if (IS_SHA1(flags) || IS_SHA1_HMAC(flags)) {
- struct sha1_state *out_state = out;
-
- out_state->count = rctx->count;
- qce_cpu_to_be32p_array((__be32 *)out_state->state,
- rctx->digest, digestsize);
- memcpy(out_state->buffer, rctx->buf, blocksize);
- } else if (IS_SHA256(flags) || IS_SHA256_HMAC(flags)) {
- struct sha256_state *out_state = out;
-
- out_state->count = rctx->count;
- qce_cpu_to_be32p_array((__be32 *)out_state->state,
- rctx->digest, digestsize);
- memcpy(out_state->buf, rctx->buf, blocksize);
- } else {
- return -EINVAL;
- }
+ struct qce_sha_saved_state *export_state = out;
- return 0;
-}
-
-static int qce_import_common(struct ahash_request *req, u64 in_count,
- const u32 *state, const u8 *buffer, bool hmac)
-{
- struct crypto_ahash *ahash = crypto_ahash_reqtfm(req);
- struct qce_sha_reqctx *rctx = ahash_request_ctx(req);
- unsigned int digestsize = crypto_ahash_digestsize(ahash);
- unsigned int blocksize;
- u64 count = in_count;
-
- blocksize = crypto_tfm_alg_blocksize(crypto_ahash_tfm(ahash));
- rctx->count = in_count;
- memcpy(rctx->buf, buffer, blocksize);
-
- if (in_count <= blocksize) {
- rctx->first_blk = 1;
- } else {
- rctx->first_blk = 0;
- /*
- * For HMAC, there is a hardware padding done when first block
- * is set. Therefore the byte_count must be incremened by 64
- * after the first block operation.
- */
- if (hmac)
- count += SHA_PADDING;
- }
-
- rctx->byte_count[0] = (__force __be32)(count & ~SHA_PADDING_MASK);
- rctx->byte_count[1] = (__force __be32)(count >> 32);
- qce_cpu_to_be32p_array((__be32 *)rctx->digest, (const u8 *)state,
- digestsize);
- rctx->buflen = (unsigned int)(in_count & (blocksize - 1));
+ memcpy(export_state->pending_buf, rctx->buf, rctx->buflen);
+ memcpy(export_state->partial_digest, rctx->digest, sizeof(rctx->digest));
+ export_state->byte_count[0] = rctx->byte_count[0];
+ export_state->byte_count[1] = rctx->byte_count[1];
+ export_state->pending_buflen = rctx->buflen;
+ export_state->count = rctx->count;
+ export_state->first_blk = rctx->first_blk;
+ export_state->flags = rctx->flags;
return 0;
}
static int qce_ahash_import(struct ahash_request *req, const void *in)
{
- struct qce_sha_reqctx *rctx;
- unsigned long flags;
- bool hmac;
- int ret;
-
- ret = qce_ahash_init(req);
- if (ret)
- return ret;
-
- rctx = ahash_request_ctx(req);
- flags = rctx->flags;
- hmac = IS_SHA_HMAC(flags);
-
- if (IS_SHA1(flags) || IS_SHA1_HMAC(flags)) {
- const struct sha1_state *state = in;
-
- ret = qce_import_common(req, state->count, state->state,
- state->buffer, hmac);
- } else if (IS_SHA256(flags) || IS_SHA256_HMAC(flags)) {
- const struct sha256_state *state = in;
+ struct qce_sha_reqctx *rctx = ahash_request_ctx(req);
+ const struct qce_sha_saved_state *import_state = in;
- ret = qce_import_common(req, state->count, state->state,
- state->buf, hmac);
- }
+ memset(rctx, 0, sizeof(*rctx));
+ rctx->count = import_state->count;
+ rctx->buflen = import_state->pending_buflen;
+ rctx->first_blk = import_state->first_blk;
+ rctx->flags = import_state->flags;
+ rctx->byte_count[0] = import_state->byte_count[0];
+ rctx->byte_count[1] = import_state->byte_count[1];
+ memcpy(rctx->buf, import_state->pending_buf, rctx->buflen);
+ memcpy(rctx->digest, import_state->partial_digest, sizeof(rctx->digest));
- return ret;
+ return 0;
}
static int qce_ahash_update(struct ahash_request *req)
@@ -270,6 +216,25 @@ static int qce_ahash_update(struct ahash_request *req)
/* calculate how many bytes will be hashed later */
hash_later = total % blocksize;
+
+ /*
+ * At this point, there is more than one block size of data. If
+ * the available data to transfer is exactly a multiple of block
+ * size, save the last block to be transferred in qce_ahash_final
+ * (with the last block bit set) if this is indeed the end of data
+ * stream. If not this saved block will be transferred as part of
+ * next update. If this block is not held back and if this is
+ * indeed the end of data stream, the digest obtained will be wrong
+ * since qce_ahash_final will see that rctx->buflen is 0 and return
+ * doing nothing which in turn means that a digest will not be
+ * copied to the destination result buffer. qce_ahash_final cannot
+ * be made to alter this behavior and allowed to proceed if
+ * rctx->buflen is 0 because the crypto engine BAM does not allow
+ * for zero length transfers.
+ */
+ if (!hash_later)
+ hash_later = blocksize;
+
if (hash_later) {
unsigned int src_offset = req->nbytes - hash_later;
scatterwalk_map_and_copy(rctx->buf, req->src, src_offset,
@@ -450,7 +415,7 @@ static const struct qce_ahash_def ahash_def[] = {
.drv_name = "sha1-qce",
.digestsize = SHA1_DIGEST_SIZE,
.blocksize = SHA1_BLOCK_SIZE,
- .statesize = sizeof(struct sha1_state),
+ .statesize = sizeof(struct qce_sha_saved_state),
.std_iv = std_iv_sha1,
},
{
@@ -459,7 +424,7 @@ static const struct qce_ahash_def ahash_def[] = {
.drv_name = "sha256-qce",
.digestsize = SHA256_DIGEST_SIZE,
.blocksize = SHA256_BLOCK_SIZE,
- .statesize = sizeof(struct sha256_state),
+ .statesize = sizeof(struct qce_sha_saved_state),
.std_iv = std_iv_sha256,
},
{
@@ -468,7 +433,7 @@ static const struct qce_ahash_def ahash_def[] = {
.drv_name = "hmac-sha1-qce",
.digestsize = SHA1_DIGEST_SIZE,
.blocksize = SHA1_BLOCK_SIZE,
- .statesize = sizeof(struct sha1_state),
+ .statesize = sizeof(struct qce_sha_saved_state),
.std_iv = std_iv_sha1,
},
{
@@ -477,7 +442,7 @@ static const struct qce_ahash_def ahash_def[] = {
.drv_name = "hmac-sha256-qce",
.digestsize = SHA256_DIGEST_SIZE,
.blocksize = SHA256_BLOCK_SIZE,
- .statesize = sizeof(struct sha256_state),
+ .statesize = sizeof(struct qce_sha_saved_state),
.std_iv = std_iv_sha256,
},
};
diff --git a/drivers/crypto/qce/skcipher.c b/drivers/crypto/qce/skcipher.c
index a2d3da0ad95f..c0a0d8c4fce1 100644
--- a/drivers/crypto/qce/skcipher.c
+++ b/drivers/crypto/qce/skcipher.c
@@ -8,6 +8,7 @@
#include <linux/interrupt.h>
#include <linux/moduleparam.h>
#include <linux/types.h>
+#include <linux/errno.h>
#include <crypto/aes.h>
#include <crypto/internal/des.h>
#include <crypto/internal/skcipher.h>
@@ -143,7 +144,7 @@ qce_skcipher_async_req_handle(struct crypto_async_request *async_req)
qce_dma_issue_pending(&qce->dma);
- ret = qce_start(async_req, tmpl->crypto_alg_type, req->cryptlen, 0);
+ ret = qce_start(async_req, tmpl->crypto_alg_type);
if (ret)
goto error_terminate;
@@ -167,16 +168,33 @@ static int qce_skcipher_setkey(struct crypto_skcipher *ablk, const u8 *key,
struct crypto_tfm *tfm = crypto_skcipher_tfm(ablk);
struct qce_cipher_ctx *ctx = crypto_tfm_ctx(tfm);
unsigned long flags = to_cipher_tmpl(ablk)->alg_flags;
+ unsigned int __keylen;
int ret;
if (!key || !keylen)
return -EINVAL;
- switch (IS_XTS(flags) ? keylen >> 1 : keylen) {
+ /*
+ * AES XTS key1 = key2 not supported by crypto engine.
+ * Revisit to request a fallback cipher in this case.
+ */
+ if (IS_XTS(flags)) {
+ __keylen = keylen >> 1;
+ if (!memcmp(key, key + __keylen, __keylen))
+ return -ENOKEY;
+ } else {
+ __keylen = keylen;
+ }
+
+ switch (__keylen) {
case AES_KEYSIZE_128:
case AES_KEYSIZE_256:
memcpy(ctx->enc_key, key, keylen);
break;
+ case AES_KEYSIZE_192:
+ break;
+ default:
+ return -EINVAL;
}
ret = crypto_skcipher_setkey(ctx->fallback, key, keylen);
@@ -204,12 +222,27 @@ static int qce_des3_setkey(struct crypto_skcipher *ablk, const u8 *key,
unsigned int keylen)
{
struct qce_cipher_ctx *ctx = crypto_skcipher_ctx(ablk);
+ u32 _key[6];
int err;
err = verify_skcipher_des3_key(ablk, key);
if (err)
return err;
+ /*
+ * The crypto engine does not support any two keys
+ * being the same for triple des algorithms. The
+ * verify_skcipher_des3_key does not check for all the
+ * below conditions. Return -ENOKEY in case any two keys
+ * are the same. Revisit to see if a fallback cipher
+ * is needed to handle this condition.
+ */
+ memcpy(_key, key, DES3_EDE_KEY_SIZE);
+ if (!((_key[0] ^ _key[2]) | (_key[1] ^ _key[3])) ||
+ !((_key[2] ^ _key[4]) | (_key[3] ^ _key[5])) ||
+ !((_key[0] ^ _key[4]) | (_key[1] ^ _key[5])))
+ return -ENOKEY;
+
ctx->enc_keylen = keylen;
memcpy(ctx->enc_key, key, keylen);
return 0;
@@ -221,6 +254,7 @@ static int qce_skcipher_crypt(struct skcipher_request *req, int encrypt)
struct qce_cipher_ctx *ctx = crypto_skcipher_ctx(tfm);
struct qce_cipher_reqctx *rctx = skcipher_request_ctx(req);
struct qce_alg_template *tmpl = to_cipher_tmpl(tfm);
+ unsigned int blocksize = crypto_skcipher_blocksize(tfm);
int keylen;
int ret;
@@ -228,14 +262,31 @@ static int qce_skcipher_crypt(struct skcipher_request *req, int encrypt)
rctx->flags |= encrypt ? QCE_ENCRYPT : QCE_DECRYPT;
keylen = IS_XTS(rctx->flags) ? ctx->enc_keylen >> 1 : ctx->enc_keylen;
- /* qce is hanging when AES-XTS request len > QCE_SECTOR_SIZE and
- * is not a multiple of it; pass such requests to the fallback
+ /* CE does not handle 0 length messages */
+ if (!req->cryptlen)
+ return 0;
+
+ /*
+ * ECB and CBC algorithms require message lengths to be
+ * multiples of block size.
+ */
+ if (IS_ECB(rctx->flags) || IS_CBC(rctx->flags))
+ if (!IS_ALIGNED(req->cryptlen, blocksize))
+ return -EINVAL;
+
+ /*
+ * Conditions for requesting a fallback cipher
+ * AES-192 (not supported by crypto engine (CE))
+ * AES-XTS request with len <= 512 byte (not recommended to use CE)
+ * AES-XTS request with len > QCE_SECTOR_SIZE and
+ * is not a multiple of it.(Revisit this condition to check if it is
+ * needed in all versions of CE)
*/
if (IS_AES(rctx->flags) &&
- (((keylen != AES_KEYSIZE_128 && keylen != AES_KEYSIZE_256) ||
- req->cryptlen <= aes_sw_max_len) ||
- (IS_XTS(rctx->flags) && req->cryptlen > QCE_SECTOR_SIZE &&
- req->cryptlen % QCE_SECTOR_SIZE))) {
+ ((keylen != AES_KEYSIZE_128 && keylen != AES_KEYSIZE_256) ||
+ (IS_XTS(rctx->flags) && ((req->cryptlen <= aes_sw_max_len) ||
+ (req->cryptlen > QCE_SECTOR_SIZE &&
+ req->cryptlen % QCE_SECTOR_SIZE))))) {
skcipher_request_set_tfm(&rctx->fallback_req, ctx->fallback);
skcipher_request_set_callback(&rctx->fallback_req,
req->base.flags,
@@ -307,7 +358,7 @@ static const struct qce_skcipher_def skcipher_def[] = {
.name = "ecb(aes)",
.drv_name = "ecb-aes-qce",
.blocksize = AES_BLOCK_SIZE,
- .ivsize = AES_BLOCK_SIZE,
+ .ivsize = 0,
.min_keysize = AES_MIN_KEY_SIZE,
.max_keysize = AES_MAX_KEY_SIZE,
},
diff --git a/drivers/crypto/rockchip/rk3288_crypto_ahash.c b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
index 81befe7febaa..ed03058497bc 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_ahash.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
@@ -48,7 +48,7 @@ static void rk_ahash_reg_init(struct rk_crypto_info *dev)
{
struct ahash_request *req = ahash_request_cast(dev->async_req);
struct rk_ahash_rctx *rctx = ahash_request_ctx(req);
- int reg_status = 0;
+ int reg_status;
reg_status = CRYPTO_READ(dev, RK_CRYPTO_CTRL) |
RK_CRYPTO_HASH_FLUSH | _SBF(0xffff, 16);
diff --git a/drivers/crypto/s5p-sss.c b/drivers/crypto/s5p-sss.c
index 682c8a450a57..55aa3a71169b 100644
--- a/drivers/crypto/s5p-sss.c
+++ b/drivers/crypto/s5p-sss.c
@@ -20,6 +20,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/platform_device.h>
#include <linux/scatterlist.h>
@@ -401,7 +402,7 @@ static const struct samsung_aes_variant exynos_aes_data = {
static const struct samsung_aes_variant exynos5433_slim_aes_data = {
.aes_offset = 0x400,
.hash_offset = 0x800,
- .clk_names = { "pclk", "aclk", },
+ .clk_names = { "aclk", "pclk", },
};
static const struct of_device_id s5p_sss_dt_match[] = {
@@ -424,13 +425,9 @@ MODULE_DEVICE_TABLE(of, s5p_sss_dt_match);
static inline const struct samsung_aes_variant *find_s5p_sss_version
(const struct platform_device *pdev)
{
- if (IS_ENABLED(CONFIG_OF) && (pdev->dev.of_node)) {
- const struct of_device_id *match;
+ if (IS_ENABLED(CONFIG_OF) && (pdev->dev.of_node))
+ return of_device_get_match_data(&pdev->dev);
- match = of_match_node(s5p_sss_dt_match,
- pdev->dev.of_node);
- return (const struct samsung_aes_variant *)match->data;
- }
return (const struct samsung_aes_variant *)
platform_get_device_id(pdev)->driver_data;
}
@@ -2159,7 +2156,7 @@ static struct skcipher_alg algs[] = {
static int s5p_aes_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
- int i, j, err = -ENODEV;
+ int i, j, err;
const struct samsung_aes_variant *variant;
struct s5p_aes_dev *pdata;
struct resource *res;
@@ -2189,14 +2186,14 @@ static int s5p_aes_probe(struct platform_device *pdev)
}
pdata->res = res;
- pdata->ioaddr = devm_ioremap_resource(&pdev->dev, res);
+ pdata->ioaddr = devm_ioremap_resource(dev, res);
if (IS_ERR(pdata->ioaddr)) {
if (!pdata->use_hash)
return PTR_ERR(pdata->ioaddr);
/* try AES without HASH */
res->end -= 0x300;
pdata->use_hash = false;
- pdata->ioaddr = devm_ioremap_resource(&pdev->dev, res);
+ pdata->ioaddr = devm_ioremap_resource(dev, res);
if (IS_ERR(pdata->ioaddr))
return PTR_ERR(pdata->ioaddr);
}
diff --git a/drivers/crypto/sa2ul.c b/drivers/crypto/sa2ul.c
index f300b0a5958a..1c6929fb3a13 100644
--- a/drivers/crypto/sa2ul.c
+++ b/drivers/crypto/sa2ul.c
@@ -69,8 +69,24 @@
/* Max Authentication tag size */
#define SA_MAX_AUTH_TAG_SZ 64
-#define PRIV_ID 0x1
-#define PRIV 0x1
+enum sa_algo_id {
+ SA_ALG_CBC_AES = 0,
+ SA_ALG_EBC_AES,
+ SA_ALG_CBC_DES3,
+ SA_ALG_ECB_DES3,
+ SA_ALG_SHA1,
+ SA_ALG_SHA256,
+ SA_ALG_SHA512,
+ SA_ALG_AUTHENC_SHA1_AES,
+ SA_ALG_AUTHENC_SHA256_AES,
+};
+
+struct sa_match_data {
+ u8 priv;
+ u8 priv_id;
+ u32 supported_algos;
+ bool skip_engine_control;
+};
static struct device *sa_k3_dev;
@@ -696,8 +712,9 @@ static void sa_dump_sc(u8 *buf, dma_addr_t dma_addr)
}
static
-int sa_init_sc(struct sa_ctx_info *ctx, const u8 *enc_key,
- u16 enc_key_sz, const u8 *auth_key, u16 auth_key_sz,
+int sa_init_sc(struct sa_ctx_info *ctx, const struct sa_match_data *match_data,
+ const u8 *enc_key, u16 enc_key_sz,
+ const u8 *auth_key, u16 auth_key_sz,
struct algo_data *ad, u8 enc, u32 *swinfo)
{
int enc_sc_offset = 0;
@@ -732,8 +749,8 @@ int sa_init_sc(struct sa_ctx_info *ctx, const u8 *enc_key,
sc_buf[SA_CTX_SCCTL_OWNER_OFFSET] = 0;
memcpy(&sc_buf[2], &sc_id, 2);
sc_buf[4] = 0x0;
- sc_buf[5] = PRIV_ID;
- sc_buf[6] = PRIV;
+ sc_buf[5] = match_data->priv_id;
+ sc_buf[6] = match_data->priv;
sc_buf[7] = 0x0;
/* Prepare context for encryption engine */
@@ -892,8 +909,8 @@ static int sa_cipher_setkey(struct crypto_skcipher *tfm, const u8 *key,
return ret;
/* Setup Encryption Security Context & Command label template */
- if (sa_init_sc(&ctx->enc, key, keylen, NULL, 0, ad, 1,
- &ctx->enc.epib[1]))
+ if (sa_init_sc(&ctx->enc, ctx->dev_data->match_data, key, keylen, NULL, 0,
+ ad, 1, &ctx->enc.epib[1]))
goto badkey;
cmdl_len = sa_format_cmdl_gen(&cfg,
@@ -905,8 +922,8 @@ static int sa_cipher_setkey(struct crypto_skcipher *tfm, const u8 *key,
ctx->enc.cmdl_size = cmdl_len;
/* Setup Decryption Security Context & Command label template */
- if (sa_init_sc(&ctx->dec, key, keylen, NULL, 0, ad, 0,
- &ctx->dec.epib[1]))
+ if (sa_init_sc(&ctx->dec, ctx->dev_data->match_data, key, keylen, NULL, 0,
+ ad, 0, &ctx->dec.epib[1]))
goto badkey;
cfg.enc_eng_id = ad->enc_eng.eng_id;
@@ -1106,7 +1123,7 @@ static int sa_run(struct sa_req *req)
else
dma_rx = pdata->dma_rx1;
- ddev = dma_rx->device->dev;
+ ddev = dmaengine_get_dma_device(pdata->dma_tx);
rxd->ddev = ddev;
memcpy(cmdl, sa_ctx->cmdl, sa_ctx->cmdl_size);
@@ -1146,8 +1163,10 @@ static int sa_run(struct sa_req *req)
mapped_sg->sgt.sgl = src;
mapped_sg->sgt.orig_nents = src_nents;
ret = dma_map_sgtable(ddev, &mapped_sg->sgt, dir_src, 0);
- if (ret)
+ if (ret) {
+ kfree(rxd);
return ret;
+ }
mapped_sg->dir = dir_src;
mapped_sg->mapped = true;
@@ -1155,8 +1174,10 @@ static int sa_run(struct sa_req *req)
mapped_sg->sgt.sgl = req->src;
mapped_sg->sgt.orig_nents = sg_nents;
ret = dma_map_sgtable(ddev, &mapped_sg->sgt, dir_src, 0);
- if (ret)
+ if (ret) {
+ kfree(rxd);
return ret;
+ }
mapped_sg->dir = dir_src;
mapped_sg->mapped = true;
@@ -1446,9 +1467,10 @@ static int sa_sha_setup(struct sa_tfm_ctx *ctx, struct algo_data *ad)
cfg.akey = NULL;
cfg.akey_len = 0;
+ ctx->dev_data = dev_get_drvdata(sa_k3_dev);
/* Setup Encryption Security Context & Command label template */
- if (sa_init_sc(&ctx->enc, NULL, 0, NULL, 0, ad, 0,
- &ctx->enc.epib[1]))
+ if (sa_init_sc(&ctx->enc, ctx->dev_data->match_data, NULL, 0, NULL, 0,
+ ad, 0, &ctx->enc.epib[1]))
goto badkey;
cmdl_len = sa_format_cmdl_gen(&cfg,
@@ -1716,6 +1738,7 @@ static int sa_cra_init_aead(struct crypto_aead *tfm, const char *hash,
int ret;
memzero_explicit(ctx, sizeof(*ctx));
+ ctx->dev_data = data;
ctx->shash = crypto_alloc_shash(hash, 0, CRYPTO_ALG_NEED_FALLBACK);
if (IS_ERR(ctx->shash)) {
@@ -1817,8 +1840,8 @@ static int sa_aead_setkey(struct crypto_aead *authenc,
cfg.akey_len = keys.authkeylen;
/* Setup Encryption Security Context & Command label template */
- if (sa_init_sc(&ctx->enc, keys.enckey, keys.enckeylen,
- keys.authkey, keys.authkeylen,
+ if (sa_init_sc(&ctx->enc, ctx->dev_data->match_data, keys.enckey,
+ keys.enckeylen, keys.authkey, keys.authkeylen,
ad, 1, &ctx->enc.epib[1]))
return -EINVAL;
@@ -1831,8 +1854,8 @@ static int sa_aead_setkey(struct crypto_aead *authenc,
ctx->enc.cmdl_size = cmdl_len;
/* Setup Decryption Security Context & Command label template */
- if (sa_init_sc(&ctx->dec, keys.enckey, keys.enckeylen,
- keys.authkey, keys.authkeylen,
+ if (sa_init_sc(&ctx->dec, ctx->dev_data->match_data, keys.enckey,
+ keys.enckeylen, keys.authkey, keys.authkeylen,
ad, 0, &ctx->dec.epib[1]))
return -EINVAL;
@@ -1950,7 +1973,7 @@ static int sa_aead_decrypt(struct aead_request *req)
}
static struct sa_alg_tmpl sa_algs[] = {
- {
+ [SA_ALG_CBC_AES] = {
.type = CRYPTO_ALG_TYPE_SKCIPHER,
.alg.skcipher = {
.base.cra_name = "cbc(aes)",
@@ -1973,7 +1996,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.decrypt = sa_decrypt,
}
},
- {
+ [SA_ALG_EBC_AES] = {
.type = CRYPTO_ALG_TYPE_SKCIPHER,
.alg.skcipher = {
.base.cra_name = "ecb(aes)",
@@ -1995,7 +2018,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.decrypt = sa_decrypt,
}
},
- {
+ [SA_ALG_CBC_DES3] = {
.type = CRYPTO_ALG_TYPE_SKCIPHER,
.alg.skcipher = {
.base.cra_name = "cbc(des3_ede)",
@@ -2018,7 +2041,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.decrypt = sa_decrypt,
}
},
- {
+ [SA_ALG_ECB_DES3] = {
.type = CRYPTO_ALG_TYPE_SKCIPHER,
.alg.skcipher = {
.base.cra_name = "ecb(des3_ede)",
@@ -2040,7 +2063,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.decrypt = sa_decrypt,
}
},
- {
+ [SA_ALG_SHA1] = {
.type = CRYPTO_ALG_TYPE_AHASH,
.alg.ahash = {
.halg.base = {
@@ -2069,7 +2092,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.import = sa_sha_import,
},
},
- {
+ [SA_ALG_SHA256] = {
.type = CRYPTO_ALG_TYPE_AHASH,
.alg.ahash = {
.halg.base = {
@@ -2098,7 +2121,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.import = sa_sha_import,
},
},
- {
+ [SA_ALG_SHA512] = {
.type = CRYPTO_ALG_TYPE_AHASH,
.alg.ahash = {
.halg.base = {
@@ -2127,7 +2150,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.import = sa_sha_import,
},
},
- {
+ [SA_ALG_AUTHENC_SHA1_AES] = {
.type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
@@ -2154,7 +2177,7 @@ static struct sa_alg_tmpl sa_algs[] = {
.decrypt = sa_aead_decrypt,
},
},
- {
+ [SA_ALG_AUTHENC_SHA256_AES] = {
.type = CRYPTO_ALG_TYPE_AEAD,
.alg.aead = {
.base = {
@@ -2185,13 +2208,19 @@ static struct sa_alg_tmpl sa_algs[] = {
};
/* Register the algorithms in crypto framework */
-static void sa_register_algos(const struct device *dev)
+static void sa_register_algos(struct sa_crypto_data *dev_data)
{
+ const struct sa_match_data *match_data = dev_data->match_data;
+ struct device *dev = dev_data->dev;
char *alg_name;
u32 type;
int i, err;
for (i = 0; i < ARRAY_SIZE(sa_algs); i++) {
+ /* Skip unsupported algos */
+ if (!(match_data->supported_algos & BIT(i)))
+ continue;
+
type = sa_algs[i].type;
if (type == CRYPTO_ALG_TYPE_SKCIPHER) {
alg_name = sa_algs[i].alg.skcipher.base.cra_name;
@@ -2329,14 +2358,39 @@ static int sa_link_child(struct device *dev, void *data)
return 0;
}
+static struct sa_match_data am654_match_data = {
+ .priv = 1,
+ .priv_id = 1,
+ .supported_algos = GENMASK(SA_ALG_AUTHENC_SHA256_AES, 0),
+};
+
+static struct sa_match_data am64_match_data = {
+ .priv = 0,
+ .priv_id = 0,
+ .supported_algos = BIT(SA_ALG_CBC_AES) |
+ BIT(SA_ALG_EBC_AES) |
+ BIT(SA_ALG_SHA256) |
+ BIT(SA_ALG_SHA512) |
+ BIT(SA_ALG_AUTHENC_SHA256_AES),
+ .skip_engine_control = true,
+};
+
+static const struct of_device_id of_match[] = {
+ { .compatible = "ti,j721e-sa2ul", .data = &am654_match_data, },
+ { .compatible = "ti,am654-sa2ul", .data = &am654_match_data, },
+ { .compatible = "ti,am64-sa2ul", .data = &am64_match_data, },
+ {},
+};
+MODULE_DEVICE_TABLE(of, of_match);
+
static int sa_ul_probe(struct platform_device *pdev)
{
+ const struct of_device_id *match;
struct device *dev = &pdev->dev;
struct device_node *node = dev->of_node;
struct resource *res;
static void __iomem *saul_base;
struct sa_crypto_data *dev_data;
- u32 val;
int ret;
dev_data = devm_kzalloc(dev, sizeof(*dev_data), GFP_KERNEL);
@@ -2350,7 +2404,7 @@ static int sa_ul_probe(struct platform_device *pdev)
dev_set_drvdata(sa_k3_dev, dev_data);
pm_runtime_enable(dev);
- ret = pm_runtime_get_sync(dev);
+ ret = pm_runtime_resume_and_get(dev);
if (ret < 0) {
dev_err(&pdev->dev, "%s: failed to get sync: %d\n", __func__,
ret);
@@ -2362,18 +2416,28 @@ static int sa_ul_probe(struct platform_device *pdev)
if (ret)
goto disable_pm_runtime;
+ match = of_match_node(of_match, dev->of_node);
+ if (!match) {
+ dev_err(dev, "No compatible match found\n");
+ return -ENODEV;
+ }
+ dev_data->match_data = match->data;
+
spin_lock_init(&dev_data->scid_lock);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
saul_base = devm_ioremap_resource(dev, res);
dev_data->base = saul_base;
- val = SA_EEC_ENCSS_EN | SA_EEC_AUTHSS_EN | SA_EEC_CTXCACH_EN |
- SA_EEC_CPPI_PORT_IN_EN | SA_EEC_CPPI_PORT_OUT_EN |
- SA_EEC_TRNG_EN;
- writel_relaxed(val, saul_base + SA_ENGINE_ENABLE_CONTROL);
+ if (!dev_data->match_data->skip_engine_control) {
+ u32 val = SA_EEC_ENCSS_EN | SA_EEC_AUTHSS_EN | SA_EEC_CTXCACH_EN |
+ SA_EEC_CPPI_PORT_IN_EN | SA_EEC_CPPI_PORT_OUT_EN |
+ SA_EEC_TRNG_EN;
- sa_register_algos(dev);
+ writel_relaxed(val, saul_base + SA_ENGINE_ENABLE_CONTROL);
+ }
+
+ sa_register_algos(dev_data);
ret = of_platform_populate(node, NULL, NULL, &pdev->dev);
if (ret)
@@ -2419,13 +2483,6 @@ static int sa_ul_remove(struct platform_device *pdev)
return 0;
}
-static const struct of_device_id of_match[] = {
- {.compatible = "ti,j721e-sa2ul",},
- {.compatible = "ti,am654-sa2ul",},
- {},
-};
-MODULE_DEVICE_TABLE(of, of_match);
-
static struct platform_driver sa_ul_driver = {
.probe = sa_ul_probe,
.remove = sa_ul_remove,
diff --git a/drivers/crypto/sa2ul.h b/drivers/crypto/sa2ul.h
index f597ddecde34..ed66d1f111db 100644
--- a/drivers/crypto/sa2ul.h
+++ b/drivers/crypto/sa2ul.h
@@ -171,9 +171,12 @@ struct sa_tfm_ctx;
#define SA_UNSAFE_DATA_SZ_MIN 240
#define SA_UNSAFE_DATA_SZ_MAX 256
+struct sa_match_data;
+
/**
* struct sa_crypto_data - Crypto driver instance data
* @base: Base address of the register space
+ * @soc_data: Pointer to SoC specific data
* @pdev: Platform device pointer
* @sc_pool: security context pool
* @dev: Device pointer
@@ -189,6 +192,7 @@ struct sa_tfm_ctx;
*/
struct sa_crypto_data {
void __iomem *base;
+ const struct sa_match_data *match_data;
struct platform_device *pdev;
struct dma_pool *sc_pool;
struct device *dev;
diff --git a/drivers/crypto/stm32/stm32-cryp.c b/drivers/crypto/stm32/stm32-cryp.c
index 2a4793176c71..7389a0536ff0 100644
--- a/drivers/crypto/stm32/stm32-cryp.c
+++ b/drivers/crypto/stm32/stm32-cryp.c
@@ -542,7 +542,7 @@ static int stm32_cryp_hw_init(struct stm32_cryp *cryp)
int ret;
u32 cfg, hw_mode;
- pm_runtime_get_sync(cryp->dev);
+ pm_runtime_resume_and_get(cryp->dev);
/* Disable interrupt */
stm32_cryp_write(cryp, CRYP_IMSCR, 0);
@@ -2043,7 +2043,7 @@ static int stm32_cryp_remove(struct platform_device *pdev)
if (!cryp)
return -ENODEV;
- ret = pm_runtime_get_sync(cryp->dev);
+ ret = pm_runtime_resume_and_get(cryp->dev);
if (ret < 0)
return ret;
diff --git a/drivers/crypto/stm32/stm32-hash.c b/drivers/crypto/stm32/stm32-hash.c
index 7ac0573ef663..389de9e3302d 100644
--- a/drivers/crypto/stm32/stm32-hash.c
+++ b/drivers/crypto/stm32/stm32-hash.c
@@ -813,7 +813,7 @@ static void stm32_hash_finish_req(struct ahash_request *req, int err)
static int stm32_hash_hw_init(struct stm32_hash_dev *hdev,
struct stm32_hash_request_ctx *rctx)
{
- pm_runtime_get_sync(hdev->dev);
+ pm_runtime_resume_and_get(hdev->dev);
if (!(HASH_FLAGS_INIT & hdev->flags)) {
stm32_hash_write(hdev, HASH_CR, HASH_CR_INIT);
@@ -962,7 +962,7 @@ static int stm32_hash_export(struct ahash_request *req, void *out)
u32 *preg;
unsigned int i;
- pm_runtime_get_sync(hdev->dev);
+ pm_runtime_resume_and_get(hdev->dev);
while ((stm32_hash_read(hdev, HASH_SR) & HASH_SR_BUSY))
cpu_relax();
@@ -1000,7 +1000,7 @@ static int stm32_hash_import(struct ahash_request *req, const void *in)
preg = rctx->hw_context;
- pm_runtime_get_sync(hdev->dev);
+ pm_runtime_resume_and_get(hdev->dev);
stm32_hash_write(hdev, HASH_IMR, *preg++);
stm32_hash_write(hdev, HASH_STR, *preg++);
@@ -1566,7 +1566,7 @@ static int stm32_hash_remove(struct platform_device *pdev)
if (!hdev)
return -ENODEV;
- ret = pm_runtime_get_sync(hdev->dev);
+ ret = pm_runtime_resume_and_get(hdev->dev);
if (ret < 0)
return ret;
diff --git a/drivers/crypto/ux500/cryp/cryp.c b/drivers/crypto/ux500/cryp/cryp.c
index 9866c2a5e9a7..759d0d9786fd 100644
--- a/drivers/crypto/ux500/cryp/cryp.c
+++ b/drivers/crypto/ux500/cryp/cryp.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* Copyright (C) ST-Ericsson SA 2010
* Author: Shujuan Chen <shujuan.chen@stericsson.com> for ST-Ericsson.
* Author: Jonas Linde <jonas.linde@stericsson.com> for ST-Ericsson.
@@ -15,7 +15,7 @@
#include "cryp_p.h"
#include "cryp.h"
-/**
+/*
* cryp_wait_until_done - wait until the device logic is not busy
*/
void cryp_wait_until_done(struct cryp_device_data *device_data)
@@ -285,6 +285,7 @@ int cryp_configure_init_vector(struct cryp_device_data *device_data,
* other device context parameter
* @device_data: Pointer to the device data struct for base address.
* @ctx: Crypto device context
+ * @cryp_mode: Mode: Polling, Interrupt or DMA
*/
void cryp_save_device_context(struct cryp_device_data *device_data,
struct cryp_device_context *ctx,
diff --git a/drivers/crypto/ux500/cryp/cryp.h b/drivers/crypto/ux500/cryp/cryp.h
index 8da7f87b339b..db5713d7c940 100644
--- a/drivers/crypto/ux500/cryp/cryp.h
+++ b/drivers/crypto/ux500/cryp/cryp.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-only */
-/**
+/*
* Copyright (C) ST-Ericsson SA 2010
* Author: Shujuan Chen <shujuan.chen@stericsson.com> for ST-Ericsson.
* Author: Jonas Linde <jonas.linde@stericsson.com> for ST-Ericsson.
diff --git a/drivers/crypto/ux500/cryp/cryp_core.c b/drivers/crypto/ux500/cryp/cryp_core.c
index c3adeb2e5823..30cdd5253929 100644
--- a/drivers/crypto/ux500/cryp/cryp_core.c
+++ b/drivers/crypto/ux500/cryp/cryp_core.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* Copyright (C) ST-Ericsson SA 2010
* Author: Shujuan Chen <shujuan.chen@stericsson.com> for ST-Ericsson.
* Author: Joakim Bech <joakim.xx.bech@stericsson.com> for ST-Ericsson.
@@ -62,7 +62,7 @@ struct cryp_driver_data {
/**
* struct cryp_ctx - Crypto context
* @config: Crypto mode.
- * @key[CRYP_MAX_KEY_SIZE]: Key.
+ * @key: Key array.
* @keylen: Length of key.
* @iv: Pointer to initialization vector.
* @indata: Pointer to indata.
@@ -73,6 +73,7 @@ struct cryp_driver_data {
* @updated: Updated flag.
* @dev_ctx: Device dependent context.
* @device: Pointer to the device.
+ * @session_id: Atomic session ID.
*/
struct cryp_ctx {
struct cryp_config config;
@@ -608,12 +609,12 @@ static void cryp_dma_done(struct cryp_ctx *ctx)
chan = ctx->device->dma.chan_mem2cryp;
dmaengine_terminate_all(chan);
dma_unmap_sg(chan->device->dev, ctx->device->dma.sg_src,
- ctx->device->dma.sg_src_len, DMA_TO_DEVICE);
+ ctx->device->dma.nents_src, DMA_TO_DEVICE);
chan = ctx->device->dma.chan_cryp2mem;
dmaengine_terminate_all(chan);
dma_unmap_sg(chan->device->dev, ctx->device->dma.sg_dst,
- ctx->device->dma.sg_dst_len, DMA_FROM_DEVICE);
+ ctx->device->dma.nents_dst, DMA_FROM_DEVICE);
}
static int cryp_dma_write(struct cryp_ctx *ctx, struct scatterlist *sg,
@@ -1290,7 +1291,6 @@ static int ux500_cryp_probe(struct platform_device *pdev)
device_data->phybase = res->start;
device_data->base = devm_ioremap_resource(dev, res);
if (IS_ERR(device_data->base)) {
- dev_err(dev, "[%s]: ioremap failed!", __func__);
ret = PTR_ERR(device_data->base);
goto out;
}
diff --git a/drivers/crypto/ux500/cryp/cryp_irq.c b/drivers/crypto/ux500/cryp/cryp_irq.c
index 7ebde69e8c76..6d2f07bec98a 100644
--- a/drivers/crypto/ux500/cryp/cryp_irq.c
+++ b/drivers/crypto/ux500/cryp/cryp_irq.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* Copyright (C) ST-Ericsson SA 2010
* Author: Shujuan Chen <shujuan.chen@stericsson.com> for ST-Ericsson.
* Author: Jonas Linde <jonas.linde@stericsson.com> for ST-Ericsson.
diff --git a/drivers/crypto/ux500/cryp/cryp_irq.h b/drivers/crypto/ux500/cryp/cryp_irq.h
index 1984f30100ff..da90029ea141 100644
--- a/drivers/crypto/ux500/cryp/cryp_irq.h
+++ b/drivers/crypto/ux500/cryp/cryp_irq.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-only */
-/**
+/*
* Copyright (C) ST-Ericsson SA 2010
* Author: Shujuan Chen <shujuan.chen@stericsson.com> for ST-Ericsson.
* Author: Jonas Linde <jonas.linde@stericsson.com> for ST-Ericsson.
@@ -19,7 +19,7 @@ enum cryp_irq_src_id {
CRYP_IRQ_SRC_ALL = 0x3
};
-/**
+/*
* M0 Funtions
*/
void cryp_enable_irq_src(struct cryp_device_data *device_data, u32 irq_src);
diff --git a/drivers/crypto/ux500/cryp/cryp_irqp.h b/drivers/crypto/ux500/cryp/cryp_irqp.h
index 879ed68a12d7..4981a3f461e5 100644
--- a/drivers/crypto/ux500/cryp/cryp_irqp.h
+++ b/drivers/crypto/ux500/cryp/cryp_irqp.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-only */
-/**
+/*
* Copyright (C) ST-Ericsson SA 2010
* Author: Shujuan Chen <shujuan.chen@stericsson.com> for ST-Ericsson.
* Author: Jonas Linde <jonas.linde@stericsson.com> for ST-Ericsson.
@@ -13,7 +13,7 @@
#include "cryp_irq.h"
-/**
+/*
*
* CRYP Registers - Offset mapping
* +-----------------+
diff --git a/drivers/crypto/ux500/cryp/cryp_p.h b/drivers/crypto/ux500/cryp/cryp_p.h
index 0df84eaa8531..60b47fe4de35 100644
--- a/drivers/crypto/ux500/cryp/cryp_p.h
+++ b/drivers/crypto/ux500/cryp/cryp_p.h
@@ -1,5 +1,5 @@
/* SPDX-License-Identifier: GPL-2.0-only */
-/**
+/*
* Copyright (C) ST-Ericsson SA 2010
* Author: Shujuan Chen <shujuan.chen@stericsson.com> for ST-Ericsson.
* Author: Jonas Linde <jonas.linde@stericsson.com> for ST-Ericsson.
@@ -17,7 +17,7 @@
#include "cryp.h"
#include "cryp_irqp.h"
-/**
+/*
* Generic Macros
*/
#define CRYP_SET_BITS(reg_name, mask) \
@@ -34,7 +34,7 @@
writel_relaxed(((readl_relaxed(reg) & ~(mask)) | \
(((u32)val << shift) & (mask))), reg)
-/**
+/*
* CRYP specific Macros
*/
#define CRYP_PERIPHERAL_ID0 0xE3
@@ -48,7 +48,7 @@
#define CRYP_PCELL_ID2 0x05
#define CRYP_PCELL_ID3 0xB1
-/**
+/*
* CRYP register default values
*/
#define MAX_DEVICE_SUPPORT 2
@@ -62,7 +62,7 @@
#define CRYP_KEY_DEFAULT 0x0
#define CRYP_INIT_VECT_DEFAULT 0x0
-/**
+/*
* CRYP Control register specific mask
*/
#define CRYP_CR_SECURE_MASK BIT(0)
@@ -81,7 +81,6 @@
CRYP_CR_PRLG_MASK |\
CRYP_CR_ALGODIR_MASK |\
CRYP_CR_ALGOMODE_MASK |\
- CRYP_CR_DATATYPE_MASK |\
CRYP_CR_KEYSIZE_MASK |\
CRYP_CR_KEYRDEN_MASK |\
CRYP_CR_DATATYPE_MASK)
@@ -91,7 +90,7 @@
#define CRYP_SR_IFEM_MASK BIT(0)
#define CRYP_SR_BUSY_MASK BIT(4)
-/**
+/*
* Bit position used while setting bits in register
*/
#define CRYP_CR_PRLG_POS 1
@@ -107,7 +106,7 @@
#define CRYP_SR_BUSY_POS 4
-/**
+/*
* CRYP PCRs------PC_NAND control register
* BIT_MASK
*/
diff --git a/drivers/crypto/ux500/hash/hash_core.c b/drivers/crypto/ux500/hash/hash_core.c
index da284b0ea1b2..ecb7412e84e3 100644
--- a/drivers/crypto/ux500/hash/hash_core.c
+++ b/drivers/crypto/ux500/hash/hash_core.c
@@ -190,7 +190,7 @@ static void hash_dma_done(struct hash_ctx *ctx)
chan = ctx->device->dma.chan_mem2hash;
dmaengine_terminate_all(chan);
dma_unmap_sg(chan->device->dev, ctx->device->dma.sg,
- ctx->device->dma.sg_len, DMA_TO_DEVICE);
+ ctx->device->dma.nents, DMA_TO_DEVICE);
}
static int hash_dma_write(struct hash_ctx *ctx,
@@ -356,7 +356,7 @@ out:
/**
* hash_get_device_data - Checks for an available hash device and return it.
- * @hash_ctx: Structure for the hash context.
+ * @ctx: Structure for the hash context.
* @device_data: Structure for the hash device.
*
* This function check for an available hash device and return it to
@@ -542,7 +542,7 @@ static bool hash_dma_valid_data(struct scatterlist *sg, int datasize)
}
/**
- * hash_init - Common hash init function for SHA1/SHA2 (SHA256).
+ * ux500_hash_init - Common hash init function for SHA1/SHA2 (SHA256).
* @req: The hash request for the job.
*
* Initialize structures.
@@ -585,6 +585,7 @@ static int ux500_hash_init(struct ahash_request *req)
* @device_data: Structure for the hash device.
* @message: Block (512 bits) of message to be written to
* the HASH hardware.
+ * @length: Message length
*
*/
static void hash_processblock(struct hash_device_data *device_data,
@@ -1295,7 +1296,7 @@ void hash_get_digest(struct hash_device_data *device_data,
}
/**
- * hash_update - The hash update function for SHA1/SHA2 (SHA256).
+ * ahash_update - The hash update function for SHA1/SHA2 (SHA256).
* @req: The hash request for the job.
*/
static int ahash_update(struct ahash_request *req)
@@ -1315,7 +1316,7 @@ static int ahash_update(struct ahash_request *req)
}
/**
- * hash_final - The hash final function for SHA1/SHA2 (SHA256).
+ * ahash_final - The hash final function for SHA1/SHA2 (SHA256).
* @req: The hash request for the job.
*/
static int ahash_final(struct ahash_request *req)
@@ -1615,9 +1616,6 @@ static struct hash_algo_template hash_algs[] = {
}
};
-/**
- * hash_algs_register_all -
- */
static int ahash_algs_register_all(struct hash_device_data *device_data)
{
int ret;
@@ -1640,9 +1638,6 @@ unreg:
return ret;
}
-/**
- * hash_algs_unregister_all -
- */
static void ahash_algs_unregister_all(struct hash_device_data *device_data)
{
int i;
@@ -1681,7 +1676,6 @@ static int ux500_hash_probe(struct platform_device *pdev)
device_data->phybase = res->start;
device_data->base = devm_ioremap_resource(dev, res);
if (IS_ERR(device_data->base)) {
- dev_err(dev, "%s: ioremap() failed!\n", __func__);
ret = PTR_ERR(device_data->base);
goto out;
}
diff --git a/drivers/crypto/vmx/aes.c b/drivers/crypto/vmx/aes.c
index d05c02baebcf..ec06189fbf99 100644
--- a/drivers/crypto/vmx/aes.c
+++ b/drivers/crypto/vmx/aes.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES routines supporting VMX instructions on the Power 8
*
* Copyright (C) 2015 International Business Machines Inc.
diff --git a/drivers/crypto/vmx/aes_cbc.c b/drivers/crypto/vmx/aes_cbc.c
index d88084447f1c..ed0debc7acb5 100644
--- a/drivers/crypto/vmx/aes_cbc.c
+++ b/drivers/crypto/vmx/aes_cbc.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES CBC routines supporting VMX instructions on the Power 8
*
* Copyright (C) 2015 International Business Machines Inc.
diff --git a/drivers/crypto/vmx/aes_ctr.c b/drivers/crypto/vmx/aes_ctr.c
index 79ba062ee1c1..9a3da8cd62f3 100644
--- a/drivers/crypto/vmx/aes_ctr.c
+++ b/drivers/crypto/vmx/aes_ctr.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES CTR routines supporting VMX instructions on the Power 8
*
* Copyright (C) 2015 International Business Machines Inc.
diff --git a/drivers/crypto/vmx/aes_xts.c b/drivers/crypto/vmx/aes_xts.c
index 9fee1b1532a4..dabbccb41550 100644
--- a/drivers/crypto/vmx/aes_xts.c
+++ b/drivers/crypto/vmx/aes_xts.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* AES XTS routines supporting VMX In-core instructions on Power 8
*
* Copyright (C) 2015 International Business Machines Inc.
diff --git a/drivers/crypto/vmx/ghash.c b/drivers/crypto/vmx/ghash.c
index 14807ac2e3b9..5bc5710a6de0 100644
--- a/drivers/crypto/vmx/ghash.c
+++ b/drivers/crypto/vmx/ghash.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0
-/**
+/*
* GHASH routines supporting VMX instructions on the Power 8
*
* Copyright (C) 2015, 2019 International Business Machines Inc.
diff --git a/drivers/crypto/vmx/vmx.c b/drivers/crypto/vmx/vmx.c
index a40d08e75fc0..7eb713cc87c8 100644
--- a/drivers/crypto/vmx/vmx.c
+++ b/drivers/crypto/vmx/vmx.c
@@ -1,5 +1,5 @@
// SPDX-License-Identifier: GPL-2.0-only
-/**
+/*
* Routines supporting VMX instructions on the Power 8
*
* Copyright (C) 2015 International Business Machines Inc.
diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c
index 244cb7d89678..2acc6173da36 100644
--- a/drivers/cxl/mem.c
+++ b/drivers/cxl/mem.c
@@ -4,6 +4,7 @@
#include <linux/security.h>
#include <linux/debugfs.h>
#include <linux/module.h>
+#include <linux/sizes.h>
#include <linux/mutex.h>
#include <linux/cdev.h>
#include <linux/idr.h>
@@ -96,21 +97,18 @@ struct mbox_cmd {
* @dev: driver core device object
* @cdev: char dev core object for ioctl operations
* @cxlm: pointer to the parent device driver data
- * @ops_active: active user of @cxlm in ops handlers
- * @ops_dead: completion when all @cxlm ops users have exited
* @id: id number of this memdev instance.
*/
struct cxl_memdev {
struct device dev;
struct cdev cdev;
struct cxl_mem *cxlm;
- struct percpu_ref ops_active;
- struct completion ops_dead;
int id;
};
static int cxl_mem_major;
static DEFINE_IDA(cxl_memdev_ida);
+static DECLARE_RWSEM(cxl_memdev_rwsem);
static struct dentry *cxl_debugfs;
static bool cxl_raw_allow_all;
@@ -169,7 +167,7 @@ struct cxl_mem_command {
* table will be validated against the user's input. For example, if size_in is
* 0, and the user passed in 1, it is an error.
*/
-static struct cxl_mem_command mem_commands[] = {
+static struct cxl_mem_command mem_commands[CXL_MEM_COMMAND_ID_MAX] = {
CXL_CMD(IDENTIFY, 0, 0x43, CXL_CMD_FLAG_FORCE_ENABLE),
#ifdef CONFIG_CXL_MEM_RAW_COMMANDS
CXL_CMD(RAW, ~0, ~0, 0),
@@ -776,26 +774,43 @@ static long __cxl_memdev_ioctl(struct cxl_memdev *cxlmd, unsigned int cmd,
static long cxl_memdev_ioctl(struct file *file, unsigned int cmd,
unsigned long arg)
{
- struct cxl_memdev *cxlmd;
- struct inode *inode;
- int rc = -ENOTTY;
+ struct cxl_memdev *cxlmd = file->private_data;
+ int rc = -ENXIO;
- inode = file_inode(file);
- cxlmd = container_of(inode->i_cdev, typeof(*cxlmd), cdev);
+ down_read(&cxl_memdev_rwsem);
+ if (cxlmd->cxlm)
+ rc = __cxl_memdev_ioctl(cxlmd, cmd, arg);
+ up_read(&cxl_memdev_rwsem);
- if (!percpu_ref_tryget_live(&cxlmd->ops_active))
- return -ENXIO;
+ return rc;
+}
- rc = __cxl_memdev_ioctl(cxlmd, cmd, arg);
+static int cxl_memdev_open(struct inode *inode, struct file *file)
+{
+ struct cxl_memdev *cxlmd =
+ container_of(inode->i_cdev, typeof(*cxlmd), cdev);
- percpu_ref_put(&cxlmd->ops_active);
+ get_device(&cxlmd->dev);
+ file->private_data = cxlmd;
- return rc;
+ return 0;
+}
+
+static int cxl_memdev_release_file(struct inode *inode, struct file *file)
+{
+ struct cxl_memdev *cxlmd =
+ container_of(inode->i_cdev, typeof(*cxlmd), cdev);
+
+ put_device(&cxlmd->dev);
+
+ return 0;
}
static const struct file_operations cxl_memdev_fops = {
.owner = THIS_MODULE,
.unlocked_ioctl = cxl_memdev_ioctl,
+ .open = cxl_memdev_open,
+ .release = cxl_memdev_release_file,
.compat_ioctl = compat_ptr_ioctl,
.llseek = noop_llseek,
};
@@ -984,7 +999,7 @@ static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo,
return NULL;
}
- offset = ((u64)reg_hi << 32) | FIELD_GET(CXL_REGLOC_ADDR_MASK, reg_lo);
+ offset = ((u64)reg_hi << 32) | (reg_lo & CXL_REGLOC_ADDR_MASK);
bar = FIELD_GET(CXL_REGLOC_BIR_MASK, reg_lo);
/* Basic sanity check that BAR is big enough */
@@ -1049,7 +1064,6 @@ static void cxl_memdev_release(struct device *dev)
{
struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
- percpu_ref_exit(&cxlmd->ops_active);
ida_free(&cxl_memdev_ida, cxlmd->id);
kfree(cxlmd);
}
@@ -1066,7 +1080,7 @@ static ssize_t firmware_version_show(struct device *dev,
struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
struct cxl_mem *cxlm = cxlmd->cxlm;
- return sprintf(buf, "%.16s\n", cxlm->firmware_version);
+ return sysfs_emit(buf, "%.16s\n", cxlm->firmware_version);
}
static DEVICE_ATTR_RO(firmware_version);
@@ -1076,7 +1090,7 @@ static ssize_t payload_max_show(struct device *dev,
struct cxl_memdev *cxlmd = to_cxl_memdev(dev);
struct cxl_mem *cxlm = cxlmd->cxlm;
- return sprintf(buf, "%zu\n", cxlm->payload_size);
+ return sysfs_emit(buf, "%zu\n", cxlm->payload_size);
}
static DEVICE_ATTR_RO(payload_max);
@@ -1087,7 +1101,7 @@ static ssize_t ram_size_show(struct device *dev, struct device_attribute *attr,
struct cxl_mem *cxlm = cxlmd->cxlm;
unsigned long long len = range_len(&cxlm->ram_range);
- return sprintf(buf, "%#llx\n", len);
+ return sysfs_emit(buf, "%#llx\n", len);
}
static struct device_attribute dev_attr_ram_size =
@@ -1100,7 +1114,7 @@ static ssize_t pmem_size_show(struct device *dev, struct device_attribute *attr,
struct cxl_mem *cxlm = cxlmd->cxlm;
unsigned long long len = range_len(&cxlm->pmem_range);
- return sprintf(buf, "%#llx\n", len);
+ return sysfs_emit(buf, "%#llx\n", len);
}
static struct device_attribute dev_attr_pmem_size =
@@ -1150,27 +1164,24 @@ static const struct device_type cxl_memdev_type = {
.groups = cxl_memdev_attribute_groups,
};
-static void cxlmdev_unregister(void *_cxlmd)
+static void cxl_memdev_shutdown(struct cxl_memdev *cxlmd)
{
- struct cxl_memdev *cxlmd = _cxlmd;
- struct device *dev = &cxlmd->dev;
-
- percpu_ref_kill(&cxlmd->ops_active);
- cdev_device_del(&cxlmd->cdev, dev);
- wait_for_completion(&cxlmd->ops_dead);
+ down_write(&cxl_memdev_rwsem);
cxlmd->cxlm = NULL;
- put_device(dev);
+ up_write(&cxl_memdev_rwsem);
}
-static void cxlmdev_ops_active_release(struct percpu_ref *ref)
+static void cxl_memdev_unregister(void *_cxlmd)
{
- struct cxl_memdev *cxlmd =
- container_of(ref, typeof(*cxlmd), ops_active);
+ struct cxl_memdev *cxlmd = _cxlmd;
+ struct device *dev = &cxlmd->dev;
- complete(&cxlmd->ops_dead);
+ cdev_device_del(&cxlmd->cdev, dev);
+ cxl_memdev_shutdown(cxlmd);
+ put_device(dev);
}
-static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
+static struct cxl_memdev *cxl_memdev_alloc(struct cxl_mem *cxlm)
{
struct pci_dev *pdev = cxlm->pdev;
struct cxl_memdev *cxlmd;
@@ -1180,22 +1191,11 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
cxlmd = kzalloc(sizeof(*cxlmd), GFP_KERNEL);
if (!cxlmd)
- return -ENOMEM;
- init_completion(&cxlmd->ops_dead);
-
- /*
- * @cxlm is deallocated when the driver unbinds so operations
- * that are using it need to hold a live reference.
- */
- cxlmd->cxlm = cxlm;
- rc = percpu_ref_init(&cxlmd->ops_active, cxlmdev_ops_active_release, 0,
- GFP_KERNEL);
- if (rc)
- goto err_ref;
+ return ERR_PTR(-ENOMEM);
rc = ida_alloc_range(&cxl_memdev_ida, 0, CXL_MEM_MAX_DEVS, GFP_KERNEL);
if (rc < 0)
- goto err_id;
+ goto err;
cxlmd->id = rc;
dev = &cxlmd->dev;
@@ -1204,30 +1204,54 @@ static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
dev->bus = &cxl_bus_type;
dev->devt = MKDEV(cxl_mem_major, cxlmd->id);
dev->type = &cxl_memdev_type;
- dev_set_name(dev, "mem%d", cxlmd->id);
+ device_set_pm_not_required(dev);
cdev = &cxlmd->cdev;
cdev_init(cdev, &cxl_memdev_fops);
+ return cxlmd;
+
+err:
+ kfree(cxlmd);
+ return ERR_PTR(rc);
+}
+
+static int cxl_mem_add_memdev(struct cxl_mem *cxlm)
+{
+ struct cxl_memdev *cxlmd;
+ struct device *dev;
+ struct cdev *cdev;
+ int rc;
+
+ cxlmd = cxl_memdev_alloc(cxlm);
+ if (IS_ERR(cxlmd))
+ return PTR_ERR(cxlmd);
+
+ dev = &cxlmd->dev;
+ rc = dev_set_name(dev, "mem%d", cxlmd->id);
+ if (rc)
+ goto err;
+
+ /*
+ * Activate ioctl operations, no cxl_memdev_rwsem manipulation
+ * needed as this is ordered with cdev_add() publishing the device.
+ */
+ cxlmd->cxlm = cxlm;
+ cdev = &cxlmd->cdev;
rc = cdev_device_add(cdev, dev);
if (rc)
- goto err_add;
+ goto err;
- return devm_add_action_or_reset(dev->parent, cxlmdev_unregister, cxlmd);
+ return devm_add_action_or_reset(dev->parent, cxl_memdev_unregister,
+ cxlmd);
-err_add:
- ida_free(&cxl_memdev_ida, cxlmd->id);
-err_id:
+err:
/*
- * Theoretically userspace could have already entered the fops,
- * so flush ops_active.
+ * The cdev was briefly live, shutdown any ioctl operations that
+ * saw that state.
*/
- percpu_ref_kill(&cxlmd->ops_active);
- wait_for_completion(&cxlmd->ops_dead);
- percpu_ref_exit(&cxlmd->ops_active);
-err_ref:
- kfree(cxlmd);
-
+ cxl_memdev_shutdown(cxlmd);
+ put_device(dev);
return rc;
}
@@ -1396,6 +1420,7 @@ out:
*/
static int cxl_mem_identify(struct cxl_mem *cxlm)
{
+ /* See CXL 2.0 Table 175 Identify Memory Device Output Payload */
struct cxl_mbox_identify {
char fw_revision[0x10];
__le64 total_capacity;
@@ -1424,10 +1449,11 @@ static int cxl_mem_identify(struct cxl_mem *cxlm)
* For now, only the capacity is exported in sysfs
*/
cxlm->ram_range.start = 0;
- cxlm->ram_range.end = le64_to_cpu(id.volatile_capacity) - 1;
+ cxlm->ram_range.end = le64_to_cpu(id.volatile_capacity) * SZ_256M - 1;
cxlm->pmem_range.start = 0;
- cxlm->pmem_range.end = le64_to_cpu(id.persistent_capacity) - 1;
+ cxlm->pmem_range.end =
+ le64_to_cpu(id.persistent_capacity) * SZ_256M - 1;
memcpy(cxlm->firmware_version, id.fw_revision, sizeof(id.fw_revision));
diff --git a/drivers/dax/bus.c b/drivers/dax/bus.c
index 452e85ae87a8..5aee26e1bbd6 100644
--- a/drivers/dax/bus.c
+++ b/drivers/dax/bus.c
@@ -90,13 +90,11 @@ static ssize_t do_id_store(struct device_driver *drv, const char *buf,
list_add(&dax_id->list, &dax_drv->ids);
} else
rc = -ENOMEM;
- } else
- /* nothing to remove */;
+ }
} else if (action == ID_REMOVE) {
list_del(&dax_id->list);
kfree(dax_id);
- } else
- /* dax_id already added */;
+ }
mutex_unlock(&dax_bus_lock);
if (rc < 0)
diff --git a/drivers/devfreq/Kconfig b/drivers/devfreq/Kconfig
index 00704efe6398..20373a893b44 100644
--- a/drivers/devfreq/Kconfig
+++ b/drivers/devfreq/Kconfig
@@ -62,7 +62,7 @@ config DEVFREQ_GOV_USERSPACE
help
Sets the frequency at the user specified one.
This governor returns the user configured frequency if there
- has been an input to /sys/devices/.../power/devfreq_set_freq.
+ has been an input to /sys/devices/.../userspace/set_freq.
Otherwise, the governor does not change the frequency
given at the initialization.
diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
index bf3047896e41..fe08c46642f7 100644
--- a/drivers/devfreq/devfreq.c
+++ b/drivers/devfreq/devfreq.c
@@ -11,6 +11,7 @@
#include <linux/kmod.h>
#include <linux/sched.h>
#include <linux/debugfs.h>
+#include <linux/devfreq_cooling.h>
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/init.h>
@@ -387,7 +388,7 @@ static int devfreq_set_target(struct devfreq *devfreq, unsigned long new_freq,
devfreq->previous_freq = new_freq;
if (devfreq->suspend_freq)
- devfreq->resume_freq = cur_freq;
+ devfreq->resume_freq = new_freq;
return err;
}
@@ -821,7 +822,8 @@ struct devfreq *devfreq_add_device(struct device *dev,
if (devfreq->profile->timer < 0
|| devfreq->profile->timer >= DEVFREQ_TIMER_NUM) {
- goto err_out;
+ mutex_unlock(&devfreq->lock);
+ goto err_dev;
}
if (!devfreq->profile->max_state && !devfreq->profile->freq_table) {
@@ -935,6 +937,12 @@ struct devfreq *devfreq_add_device(struct device *dev,
mutex_unlock(&devfreq_list_lock);
+ if (devfreq->profile->is_cooling_device) {
+ devfreq->cdev = devfreq_cooling_em_register(devfreq, NULL);
+ if (IS_ERR(devfreq->cdev))
+ devfreq->cdev = NULL;
+ }
+
return devfreq;
err_init:
@@ -960,6 +968,8 @@ int devfreq_remove_device(struct devfreq *devfreq)
if (!devfreq)
return -EINVAL;
+ devfreq_cooling_unregister(devfreq->cdev);
+
if (devfreq->governor) {
devfreq->governor->event_handler(devfreq,
DEVFREQ_GOV_STOP, NULL);
diff --git a/drivers/devfreq/governor.h b/drivers/devfreq/governor.h
index 70f44b3ca42e..2d69a0ce6291 100644
--- a/drivers/devfreq/governor.h
+++ b/drivers/devfreq/governor.h
@@ -57,8 +57,6 @@
* Basically, get_target_freq will run
* devfreq_dev_profile.get_dev_status() to get the
* status of the device (load = busy_time / total_time).
- * If no_central_polling is set, this callback is called
- * only with update_devfreq() notified by OPP.
* @event_handler: Callback for devfreq core framework to notify events
* to governors. Events include per device governor
* init and exit, opp changes out of devfreq, suspend
@@ -91,6 +89,9 @@ int devfreq_update_target(struct devfreq *devfreq, unsigned long freq);
static inline int devfreq_update_stats(struct devfreq *df)
{
+ if (!df->profile->get_dev_status)
+ return -EINVAL;
+
return df->profile->get_dev_status(df->dev.parent, &df->last_status);
}
#endif /* _GOVERNOR_H */
diff --git a/drivers/devfreq/imx-bus.c b/drivers/devfreq/imx-bus.c
index 4f38455ad742..3fc3fd77492d 100644
--- a/drivers/devfreq/imx-bus.c
+++ b/drivers/devfreq/imx-bus.c
@@ -169,7 +169,7 @@ static struct platform_driver imx_bus_platdrv = {
.probe = imx_bus_probe,
.driver = {
.name = "imx-bus-devfreq",
- .of_match_table = of_match_ptr(imx_bus_of_match),
+ .of_match_table = imx_bus_of_match,
},
};
module_platform_driver(imx_bus_platdrv);
diff --git a/drivers/devfreq/imx8m-ddrc.c b/drivers/devfreq/imx8m-ddrc.c
index bc82d3653bff..16636973eb10 100644
--- a/drivers/devfreq/imx8m-ddrc.c
+++ b/drivers/devfreq/imx8m-ddrc.c
@@ -280,18 +280,6 @@ static int imx8m_ddrc_get_cur_freq(struct device *dev, unsigned long *freq)
return 0;
}
-static int imx8m_ddrc_get_dev_status(struct device *dev,
- struct devfreq_dev_status *stat)
-{
- struct imx8m_ddrc *priv = dev_get_drvdata(dev);
-
- stat->busy_time = 0;
- stat->total_time = 0;
- stat->current_frequency = clk_get_rate(priv->dram_core);
-
- return 0;
-}
-
static int imx8m_ddrc_init_freq_info(struct device *dev)
{
struct imx8m_ddrc *priv = dev_get_drvdata(dev);
@@ -429,9 +417,7 @@ static int imx8m_ddrc_probe(struct platform_device *pdev)
if (ret < 0)
goto err;
- priv->profile.polling_ms = 1000;
priv->profile.target = imx8m_ddrc_target;
- priv->profile.get_dev_status = imx8m_ddrc_get_dev_status;
priv->profile.exit = imx8m_ddrc_exit;
priv->profile.get_cur_freq = imx8m_ddrc_get_cur_freq;
priv->profile.initial_freq = clk_get_rate(priv->dram_core);
@@ -461,7 +447,7 @@ static struct platform_driver imx8m_ddrc_platdrv = {
.probe = imx8m_ddrc_probe,
.driver = {
.name = "imx8m-ddrc-devfreq",
- .of_match_table = of_match_ptr(imx8m_ddrc_of_match),
+ .of_match_table = imx8m_ddrc_of_match,
},
};
module_platform_driver(imx8m_ddrc_platdrv);
diff --git a/drivers/devfreq/rk3399_dmc.c b/drivers/devfreq/rk3399_dmc.c
index 9e9d3b4c6d48..293857ebfd75 100644
--- a/drivers/devfreq/rk3399_dmc.c
+++ b/drivers/devfreq/rk3399_dmc.c
@@ -324,22 +324,14 @@ static int rk3399_dmcfreq_probe(struct platform_device *pdev)
mutex_init(&data->lock);
data->vdd_center = devm_regulator_get(dev, "center");
- if (IS_ERR(data->vdd_center)) {
- if (PTR_ERR(data->vdd_center) == -EPROBE_DEFER)
- return -EPROBE_DEFER;
-
- dev_err(dev, "Cannot get the regulator \"center\"\n");
- return PTR_ERR(data->vdd_center);
- }
+ if (IS_ERR(data->vdd_center))
+ return dev_err_probe(dev, PTR_ERR(data->vdd_center),
+ "Cannot get the regulator \"center\"\n");
data->dmc_clk = devm_clk_get(dev, "dmc_clk");
- if (IS_ERR(data->dmc_clk)) {
- if (PTR_ERR(data->dmc_clk) == -EPROBE_DEFER)
- return -EPROBE_DEFER;
-
- dev_err(dev, "Cannot get the clk dmc_clk\n");
- return PTR_ERR(data->dmc_clk);
- }
+ if (IS_ERR(data->dmc_clk))
+ return dev_err_probe(dev, PTR_ERR(data->dmc_clk),
+ "Cannot get the clk dmc_clk\n");
data->edev = devfreq_event_get_edev_by_phandle(dev, "devfreq-events", 0);
if (IS_ERR(data->edev))
diff --git a/drivers/dma-buf/dma-fence.c b/drivers/dma-buf/dma-fence.c
index d64fc03929be..ce0f5eff575d 100644
--- a/drivers/dma-buf/dma-fence.c
+++ b/drivers/dma-buf/dma-fence.c
@@ -123,7 +123,9 @@ static const struct dma_fence_ops dma_fence_stub_ops = {
/**
* dma_fence_get_stub - return a signaled fence
*
- * Return a stub fence which is already signaled.
+ * Return a stub fence which is already signaled. The fence's
+ * timestamp corresponds to the first time after boot this
+ * function is called.
*/
struct dma_fence *dma_fence_get_stub(void)
{
@@ -142,6 +144,29 @@ struct dma_fence *dma_fence_get_stub(void)
EXPORT_SYMBOL(dma_fence_get_stub);
/**
+ * dma_fence_allocate_private_stub - return a private, signaled fence
+ *
+ * Return a newly allocated and signaled stub fence.
+ */
+struct dma_fence *dma_fence_allocate_private_stub(void)
+{
+ struct dma_fence *fence;
+
+ fence = kzalloc(sizeof(*fence), GFP_KERNEL);
+ if (fence == NULL)
+ return ERR_PTR(-ENOMEM);
+
+ dma_fence_init(fence,
+ &dma_fence_stub_ops,
+ &dma_fence_stub_lock,
+ 0, 0);
+ dma_fence_signal(fence);
+
+ return fence;
+}
+EXPORT_SYMBOL(dma_fence_allocate_private_stub);
+
+/**
* dma_fence_context_alloc - allocate an array of fence contexts
* @num: amount of contexts to allocate
*
diff --git a/drivers/dma-buf/dma-heap.c b/drivers/dma-buf/dma-heap.c
index 6b5db954569f..56bf5ad01ad5 100644
--- a/drivers/dma-buf/dma-heap.c
+++ b/drivers/dma-buf/dma-heap.c
@@ -202,6 +202,18 @@ void *dma_heap_get_drvdata(struct dma_heap *heap)
return heap->priv;
}
+/**
+ * dma_heap_get_name() - get heap name
+ * @heap: DMA-Heap to retrieve private data for
+ *
+ * Returns:
+ * The char* for the heap name.
+ */
+const char *dma_heap_get_name(struct dma_heap *heap)
+{
+ return heap->name;
+}
+
struct dma_heap *dma_heap_add(const struct dma_heap_export_info *exp_info)
{
struct dma_heap *heap, *h, *err_ret;
diff --git a/drivers/dma-buf/heaps/cma_heap.c b/drivers/dma-buf/heaps/cma_heap.c
index 5d64eccd21d6..0c05b79870f9 100644
--- a/drivers/dma-buf/heaps/cma_heap.c
+++ b/drivers/dma-buf/heaps/cma_heap.c
@@ -339,6 +339,7 @@ static struct dma_buf *cma_heap_allocate(struct dma_heap *heap,
buffer->pagecount = pagecount;
/* create the dmabuf */
+ exp_info.exp_name = dma_heap_get_name(heap);
exp_info.ops = &cma_heap_buf_ops;
exp_info.size = buffer->len;
exp_info.flags = fd_flags;
diff --git a/drivers/dma-buf/heaps/system_heap.c b/drivers/dma-buf/heaps/system_heap.c
index 29e49ac17251..23a7e74ef966 100644
--- a/drivers/dma-buf/heaps/system_heap.c
+++ b/drivers/dma-buf/heaps/system_heap.c
@@ -390,6 +390,7 @@ static struct dma_buf *system_heap_allocate(struct dma_heap *heap,
}
/* create the dmabuf */
+ exp_info.exp_name = dma_heap_get_name(heap);
exp_info.ops = &system_heap_buf_ops;
exp_info.size = buffer->len;
exp_info.flags = fd_flags;
diff --git a/drivers/dma/Kconfig b/drivers/dma/Kconfig
index 0c2827fd8c19..6ab9d9a488a6 100644
--- a/drivers/dma/Kconfig
+++ b/drivers/dma/Kconfig
@@ -100,7 +100,7 @@ config AT_XDMAC
config AXI_DMAC
tristate "Analog Devices AXI-DMAC DMA support"
- depends on MICROBLAZE || NIOS2 || ARCH_ZYNQ || ARCH_ZYNQMP || ARCH_SOCFPGA || COMPILE_TEST
+ depends on MICROBLAZE || NIOS2 || ARCH_ZYNQ || ARCH_ZYNQMP || ARCH_INTEL_SOCFPGA || COMPILE_TEST
select DMA_ENGINE
select DMA_VIRTUAL_CHANNELS
select REGMAP_MMIO
@@ -300,6 +300,18 @@ config INTEL_IDXD_SVM
depends on PCI_PASID
depends on PCI_IOV
+config INTEL_IDXD_PERFMON
+ bool "Intel Data Accelerators performance monitor support"
+ depends on INTEL_IDXD
+ help
+ Enable performance monitor (pmu) support for the Intel(R)
+ data accelerators present in Intel Xeon CPU. With this
+ enabled, perf can be used to monitor the DSA (Intel Data
+ Streaming Accelerator) events described in the Intel DSA
+ spec.
+
+ If unsure, say N.
+
config INTEL_IOATDMA
tristate "Intel I/OAT DMA support"
depends on PCI && X86_64
diff --git a/drivers/dma/at_xdmac.c b/drivers/dma/at_xdmac.c
index fe45ad5d06c4..64a52bf4d737 100644
--- a/drivers/dma/at_xdmac.c
+++ b/drivers/dma/at_xdmac.c
@@ -344,17 +344,6 @@ static inline int at_xdmac_chan_is_paused(struct at_xdmac_chan *atchan)
return test_bit(AT_XDMAC_CHAN_IS_PAUSED, &atchan->status);
}
-static inline int at_xdmac_csize(u32 maxburst)
-{
- int csize;
-
- csize = ffs(maxburst) - 1;
- if (csize > 4)
- csize = -EINVAL;
-
- return csize;
-};
-
static inline bool at_xdmac_chan_is_peripheral_xfer(u32 cfg)
{
return cfg & AT_XDMAC_CC_TYPE_PER_TRAN;
diff --git a/drivers/dma/dmaengine.c b/drivers/dma/dmaengine.c
index fe6a460c4373..af3ee288bc11 100644
--- a/drivers/dma/dmaengine.c
+++ b/drivers/dma/dmaengine.c
@@ -1086,6 +1086,7 @@ static int __dma_async_device_channel_register(struct dma_device *device,
kfree(chan->dev);
err_free_local:
free_percpu(chan->local);
+ chan->local = NULL;
return rc;
}
diff --git a/drivers/dma/dw-edma/dw-edma-core.c b/drivers/dma/dw-edma/dw-edma-core.c
index 08d71dafa001..53289927dd0d 100644
--- a/drivers/dma/dw-edma/dw-edma-core.c
+++ b/drivers/dma/dw-edma/dw-edma-core.c
@@ -81,8 +81,13 @@ static struct dw_edma_chunk *dw_edma_alloc_chunk(struct dw_edma_desc *desc)
* - Even chunks originate CB equal to 1
*/
chunk->cb = !(desc->chunks_alloc % 2);
- chunk->ll_region.paddr = dw->ll_region.paddr + chan->ll_off;
- chunk->ll_region.vaddr = dw->ll_region.vaddr + chan->ll_off;
+ if (chan->dir == EDMA_DIR_WRITE) {
+ chunk->ll_region.paddr = dw->ll_region_wr[chan->id].paddr;
+ chunk->ll_region.vaddr = dw->ll_region_wr[chan->id].vaddr;
+ } else {
+ chunk->ll_region.paddr = dw->ll_region_rd[chan->id].paddr;
+ chunk->ll_region.vaddr = dw->ll_region_rd[chan->id].vaddr;
+ }
if (desc->chunk) {
/* Create and add new element into the linked list */
@@ -329,22 +334,22 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer)
struct dw_edma_chunk *chunk;
struct dw_edma_burst *burst;
struct dw_edma_desc *desc;
- u32 cnt;
+ u32 cnt = 0;
int i;
if (!chan->configured)
return NULL;
switch (chan->config.direction) {
- case DMA_DEV_TO_MEM: /* local dma */
+ case DMA_DEV_TO_MEM: /* local DMA */
if (dir == DMA_DEV_TO_MEM && chan->dir == EDMA_DIR_READ)
break;
return NULL;
- case DMA_MEM_TO_DEV: /* local dma */
+ case DMA_MEM_TO_DEV: /* local DMA */
if (dir == DMA_MEM_TO_DEV && chan->dir == EDMA_DIR_WRITE)
break;
return NULL;
- default: /* remote dma */
+ default: /* remote DMA */
if (dir == DMA_MEM_TO_DEV && chan->dir == EDMA_DIR_READ)
break;
if (dir == DMA_DEV_TO_MEM && chan->dir == EDMA_DIR_WRITE)
@@ -352,12 +357,19 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer)
return NULL;
}
- if (xfer->cyclic) {
+ if (xfer->type == EDMA_XFER_CYCLIC) {
if (!xfer->xfer.cyclic.len || !xfer->xfer.cyclic.cnt)
return NULL;
- } else {
+ } else if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
if (xfer->xfer.sg.len < 1)
return NULL;
+ } else if (xfer->type == EDMA_XFER_INTERLEAVED) {
+ if (!xfer->xfer.il->numf)
+ return NULL;
+ if (xfer->xfer.il->numf > 0 && xfer->xfer.il->frame_size > 0)
+ return NULL;
+ } else {
+ return NULL;
}
desc = dw_edma_alloc_desc(chan);
@@ -368,18 +380,28 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer)
if (unlikely(!chunk))
goto err_alloc;
- src_addr = chan->config.src_addr;
- dst_addr = chan->config.dst_addr;
+ if (xfer->type == EDMA_XFER_INTERLEAVED) {
+ src_addr = xfer->xfer.il->src_start;
+ dst_addr = xfer->xfer.il->dst_start;
+ } else {
+ src_addr = chan->config.src_addr;
+ dst_addr = chan->config.dst_addr;
+ }
- if (xfer->cyclic) {
+ if (xfer->type == EDMA_XFER_CYCLIC) {
cnt = xfer->xfer.cyclic.cnt;
- } else {
+ } else if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
cnt = xfer->xfer.sg.len;
sg = xfer->xfer.sg.sgl;
+ } else if (xfer->type == EDMA_XFER_INTERLEAVED) {
+ if (xfer->xfer.il->numf > 0)
+ cnt = xfer->xfer.il->numf;
+ else
+ cnt = xfer->xfer.il->frame_size;
}
for (i = 0; i < cnt; i++) {
- if (!xfer->cyclic && !sg)
+ if (xfer->type == EDMA_XFER_SCATTER_GATHER && !sg)
break;
if (chunk->bursts_alloc == chan->ll_max) {
@@ -392,20 +414,23 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer)
if (unlikely(!burst))
goto err_alloc;
- if (xfer->cyclic)
+ if (xfer->type == EDMA_XFER_CYCLIC)
burst->sz = xfer->xfer.cyclic.len;
- else
+ else if (xfer->type == EDMA_XFER_SCATTER_GATHER)
burst->sz = sg_dma_len(sg);
+ else if (xfer->type == EDMA_XFER_INTERLEAVED)
+ burst->sz = xfer->xfer.il->sgl[i].size;
chunk->ll_region.sz += burst->sz;
desc->alloc_sz += burst->sz;
if (chan->dir == EDMA_DIR_WRITE) {
burst->sar = src_addr;
- if (xfer->cyclic) {
+ if (xfer->type == EDMA_XFER_CYCLIC) {
burst->dar = xfer->xfer.cyclic.paddr;
- } else {
- burst->dar = dst_addr;
+ } else if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
+ src_addr += sg_dma_len(sg);
+ burst->dar = sg_dma_address(sg);
/* Unlike the typical assumption by other
* drivers/IPs the peripheral memory isn't
* a FIFO memory, in this case, it's a
@@ -416,10 +441,11 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer)
}
} else {
burst->dar = dst_addr;
- if (xfer->cyclic) {
+ if (xfer->type == EDMA_XFER_CYCLIC) {
burst->sar = xfer->xfer.cyclic.paddr;
- } else {
- burst->sar = src_addr;
+ } else if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
+ dst_addr += sg_dma_len(sg);
+ burst->sar = sg_dma_address(sg);
/* Unlike the typical assumption by other
* drivers/IPs the peripheral memory isn't
* a FIFO memory, in this case, it's a
@@ -430,10 +456,22 @@ dw_edma_device_transfer(struct dw_edma_transfer *xfer)
}
}
- if (!xfer->cyclic) {
- src_addr += sg_dma_len(sg);
- dst_addr += sg_dma_len(sg);
+ if (xfer->type == EDMA_XFER_SCATTER_GATHER) {
sg = sg_next(sg);
+ } else if (xfer->type == EDMA_XFER_INTERLEAVED &&
+ xfer->xfer.il->frame_size > 0) {
+ struct dma_interleaved_template *il = xfer->xfer.il;
+ struct data_chunk *dc = &il->sgl[i];
+
+ if (il->src_sgl) {
+ src_addr += burst->sz;
+ src_addr += dmaengine_get_src_icg(il, dc);
+ }
+
+ if (il->dst_sgl) {
+ dst_addr += burst->sz;
+ dst_addr += dmaengine_get_dst_icg(il, dc);
+ }
}
}
@@ -459,7 +497,7 @@ dw_edma_device_prep_slave_sg(struct dma_chan *dchan, struct scatterlist *sgl,
xfer.xfer.sg.sgl = sgl;
xfer.xfer.sg.len = len;
xfer.flags = flags;
- xfer.cyclic = false;
+ xfer.type = EDMA_XFER_SCATTER_GATHER;
return dw_edma_device_transfer(&xfer);
}
@@ -478,7 +516,23 @@ dw_edma_device_prep_dma_cyclic(struct dma_chan *dchan, dma_addr_t paddr,
xfer.xfer.cyclic.len = len;
xfer.xfer.cyclic.cnt = count;
xfer.flags = flags;
- xfer.cyclic = true;
+ xfer.type = EDMA_XFER_CYCLIC;
+
+ return dw_edma_device_transfer(&xfer);
+}
+
+static struct dma_async_tx_descriptor *
+dw_edma_device_prep_interleaved_dma(struct dma_chan *dchan,
+ struct dma_interleaved_template *ilt,
+ unsigned long flags)
+{
+ struct dw_edma_transfer xfer;
+
+ xfer.dchan = dchan;
+ xfer.direction = ilt->dir;
+ xfer.xfer.il = ilt;
+ xfer.flags = flags;
+ xfer.type = EDMA_XFER_INTERLEAVED;
return dw_edma_device_transfer(&xfer);
}
@@ -642,24 +696,13 @@ static int dw_edma_channel_setup(struct dw_edma_chip *chip, bool write,
struct device *dev = chip->dev;
struct dw_edma *dw = chip->dw;
struct dw_edma_chan *chan;
- size_t ll_chunk, dt_chunk;
struct dw_edma_irq *irq;
struct dma_device *dma;
- u32 i, j, cnt, ch_cnt;
u32 alloc, off_alloc;
+ u32 i, j, cnt;
int err = 0;
u32 pos;
- ch_cnt = dw->wr_ch_cnt + dw->rd_ch_cnt;
- ll_chunk = dw->ll_region.sz;
- dt_chunk = dw->dt_region.sz;
-
- /* Calculate linked list chunk for each channel */
- ll_chunk /= roundup_pow_of_two(ch_cnt);
-
- /* Calculate linked list chunk for each channel */
- dt_chunk /= roundup_pow_of_two(ch_cnt);
-
if (write) {
i = 0;
cnt = dw->wr_ch_cnt;
@@ -691,14 +734,14 @@ static int dw_edma_channel_setup(struct dw_edma_chip *chip, bool write,
chan->request = EDMA_REQ_NONE;
chan->status = EDMA_ST_IDLE;
- chan->ll_off = (ll_chunk * i);
- chan->ll_max = (ll_chunk / EDMA_LL_SZ) - 1;
-
- chan->dt_off = (dt_chunk * i);
+ if (write)
+ chan->ll_max = (dw->ll_region_wr[j].sz / EDMA_LL_SZ);
+ else
+ chan->ll_max = (dw->ll_region_rd[j].sz / EDMA_LL_SZ);
+ chan->ll_max -= 1;
- dev_vdbg(dev, "L. List:\tChannel %s[%u] off=0x%.8lx, max_cnt=%u\n",
- write ? "write" : "read", j,
- chan->ll_off, chan->ll_max);
+ dev_vdbg(dev, "L. List:\tChannel %s[%u] max_cnt=%u\n",
+ write ? "write" : "read", j, chan->ll_max);
if (dw->nr_irqs == 1)
pos = 0;
@@ -723,12 +766,15 @@ static int dw_edma_channel_setup(struct dw_edma_chip *chip, bool write,
chan->vc.desc_free = vchan_free_desc;
vchan_init(&chan->vc, dma);
- dt_region->paddr = dw->dt_region.paddr + chan->dt_off;
- dt_region->vaddr = dw->dt_region.vaddr + chan->dt_off;
- dt_region->sz = dt_chunk;
-
- dev_vdbg(dev, "Data:\tChannel %s[%u] off=0x%.8lx\n",
- write ? "write" : "read", j, chan->dt_off);
+ if (write) {
+ dt_region->paddr = dw->dt_region_wr[j].paddr;
+ dt_region->vaddr = dw->dt_region_wr[j].vaddr;
+ dt_region->sz = dw->dt_region_wr[j].sz;
+ } else {
+ dt_region->paddr = dw->dt_region_rd[j].paddr;
+ dt_region->vaddr = dw->dt_region_rd[j].vaddr;
+ dt_region->sz = dw->dt_region_rd[j].sz;
+ }
dw_edma_v0_core_device_config(chan);
}
@@ -738,6 +784,7 @@ static int dw_edma_channel_setup(struct dw_edma_chip *chip, bool write,
dma_cap_set(DMA_SLAVE, dma->cap_mask);
dma_cap_set(DMA_CYCLIC, dma->cap_mask);
dma_cap_set(DMA_PRIVATE, dma->cap_mask);
+ dma_cap_set(DMA_INTERLEAVE, dma->cap_mask);
dma->directions = BIT(write ? DMA_DEV_TO_MEM : DMA_MEM_TO_DEV);
dma->src_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
dma->dst_addr_widths = BIT(DMA_SLAVE_BUSWIDTH_4_BYTES);
@@ -756,6 +803,7 @@ static int dw_edma_channel_setup(struct dw_edma_chip *chip, bool write,
dma->device_tx_status = dw_edma_device_tx_status;
dma->device_prep_slave_sg = dw_edma_device_prep_slave_sg;
dma->device_prep_dma_cyclic = dw_edma_device_prep_dma_cyclic;
+ dma->device_prep_interleaved_dma = dw_edma_device_prep_interleaved_dma;
dma_set_max_seg_size(dma->dev, U32_MAX);
@@ -863,14 +911,15 @@ int dw_edma_probe(struct dw_edma_chip *chip)
raw_spin_lock_init(&dw->lock);
- /* Find out how many write channels are supported by hardware */
- dw->wr_ch_cnt = dw_edma_v0_core_ch_count(dw, EDMA_DIR_WRITE);
- if (!dw->wr_ch_cnt)
- return -EINVAL;
+ dw->wr_ch_cnt = min_t(u16, dw->wr_ch_cnt,
+ dw_edma_v0_core_ch_count(dw, EDMA_DIR_WRITE));
+ dw->wr_ch_cnt = min_t(u16, dw->wr_ch_cnt, EDMA_MAX_WR_CH);
- /* Find out how many read channels are supported by hardware */
- dw->rd_ch_cnt = dw_edma_v0_core_ch_count(dw, EDMA_DIR_READ);
- if (!dw->rd_ch_cnt)
+ dw->rd_ch_cnt = min_t(u16, dw->rd_ch_cnt,
+ dw_edma_v0_core_ch_count(dw, EDMA_DIR_READ));
+ dw->rd_ch_cnt = min_t(u16, dw->rd_ch_cnt, EDMA_MAX_RD_CH);
+
+ if (!dw->wr_ch_cnt && !dw->rd_ch_cnt)
return -EINVAL;
dev_vdbg(dev, "Channels:\twrite=%d, read=%d\n",
@@ -937,24 +986,23 @@ int dw_edma_remove(struct dw_edma_chip *chip)
/* Power management */
pm_runtime_disable(dev);
+ /* Deregister eDMA device */
+ dma_async_device_unregister(&dw->wr_edma);
list_for_each_entry_safe(chan, _chan, &dw->wr_edma.channels,
vc.chan.device_node) {
- list_del(&chan->vc.chan.device_node);
tasklet_kill(&chan->vc.task);
+ list_del(&chan->vc.chan.device_node);
}
+ dma_async_device_unregister(&dw->rd_edma);
list_for_each_entry_safe(chan, _chan, &dw->rd_edma.channels,
vc.chan.device_node) {
- list_del(&chan->vc.chan.device_node);
tasklet_kill(&chan->vc.task);
+ list_del(&chan->vc.chan.device_node);
}
- /* Deregister eDMA device */
- dma_async_device_unregister(&dw->wr_edma);
- dma_async_device_unregister(&dw->rd_edma);
-
/* Turn debugfs off */
- dw_edma_v0_core_debugfs_off();
+ dw_edma_v0_core_debugfs_off(chip);
return 0;
}
diff --git a/drivers/dma/dw-edma/dw-edma-core.h b/drivers/dma/dw-edma/dw-edma-core.h
index 31fc50d31792..60316d408c3e 100644
--- a/drivers/dma/dw-edma/dw-edma-core.h
+++ b/drivers/dma/dw-edma/dw-edma-core.h
@@ -15,15 +15,18 @@
#include "../virt-dma.h"
#define EDMA_LL_SZ 24
+#define EDMA_MAX_WR_CH 8
+#define EDMA_MAX_RD_CH 8
enum dw_edma_dir {
EDMA_DIR_WRITE = 0,
EDMA_DIR_READ
};
-enum dw_edma_mode {
- EDMA_MODE_LEGACY = 0,
- EDMA_MODE_UNROLL
+enum dw_edma_map_format {
+ EDMA_MF_EDMA_LEGACY = 0x0,
+ EDMA_MF_EDMA_UNROLL = 0x1,
+ EDMA_MF_HDMA_COMPAT = 0x5
};
enum dw_edma_request {
@@ -38,6 +41,12 @@ enum dw_edma_status {
EDMA_ST_BUSY
};
+enum dw_edma_xfer_type {
+ EDMA_XFER_SCATTER_GATHER = 0,
+ EDMA_XFER_CYCLIC,
+ EDMA_XFER_INTERLEAVED
+};
+
struct dw_edma_chan;
struct dw_edma_chunk;
@@ -82,11 +91,8 @@ struct dw_edma_chan {
int id;
enum dw_edma_dir dir;
- off_t ll_off;
u32 ll_max;
- off_t dt_off;
-
struct msi_msg msi;
enum dw_edma_request request;
@@ -117,19 +123,23 @@ struct dw_edma {
u16 rd_ch_cnt;
struct dw_edma_region rg_region; /* Registers */
- struct dw_edma_region ll_region; /* Linked list */
- struct dw_edma_region dt_region; /* Data */
+ struct dw_edma_region ll_region_wr[EDMA_MAX_WR_CH];
+ struct dw_edma_region ll_region_rd[EDMA_MAX_RD_CH];
+ struct dw_edma_region dt_region_wr[EDMA_MAX_WR_CH];
+ struct dw_edma_region dt_region_rd[EDMA_MAX_RD_CH];
struct dw_edma_irq *irq;
int nr_irqs;
- u32 version;
- enum dw_edma_mode mode;
+ enum dw_edma_map_format mf;
struct dw_edma_chan *chan;
const struct dw_edma_core_ops *ops;
raw_spinlock_t lock; /* Only for legacy */
+#ifdef CONFIG_DEBUG_FS
+ struct dentry *debugfs;
+#endif /* CONFIG_DEBUG_FS */
};
struct dw_edma_sg {
@@ -146,12 +156,13 @@ struct dw_edma_cyclic {
struct dw_edma_transfer {
struct dma_chan *dchan;
union dw_edma_xfer {
- struct dw_edma_sg sg;
- struct dw_edma_cyclic cyclic;
+ struct dw_edma_sg sg;
+ struct dw_edma_cyclic cyclic;
+ struct dma_interleaved_template *il;
} xfer;
enum dma_transfer_direction direction;
unsigned long flags;
- bool cyclic;
+ enum dw_edma_xfer_type type;
};
static inline
diff --git a/drivers/dma/dw-edma/dw-edma-pcie.c b/drivers/dma/dw-edma/dw-edma-pcie.c
index 1eafc602e17e..44f6e09bdb53 100644
--- a/drivers/dma/dw-edma/dw-edma-pcie.c
+++ b/drivers/dma/dw-edma/dw-edma-pcie.c
@@ -13,45 +13,81 @@
#include <linux/dma/edma.h>
#include <linux/pci-epf.h>
#include <linux/msi.h>
+#include <linux/bitfield.h>
#include "dw-edma-core.h"
+#define DW_PCIE_VSEC_DMA_ID 0x6
+#define DW_PCIE_VSEC_DMA_BAR GENMASK(10, 8)
+#define DW_PCIE_VSEC_DMA_MAP GENMASK(2, 0)
+#define DW_PCIE_VSEC_DMA_WR_CH GENMASK(9, 0)
+#define DW_PCIE_VSEC_DMA_RD_CH GENMASK(25, 16)
+
+#define DW_BLOCK(a, b, c) \
+ { \
+ .bar = a, \
+ .off = b, \
+ .sz = c, \
+ },
+
+struct dw_edma_block {
+ enum pci_barno bar;
+ off_t off;
+ size_t sz;
+};
+
struct dw_edma_pcie_data {
/* eDMA registers location */
- enum pci_barno rg_bar;
- off_t rg_off;
- size_t rg_sz;
+ struct dw_edma_block rg;
/* eDMA memory linked list location */
- enum pci_barno ll_bar;
- off_t ll_off;
- size_t ll_sz;
+ struct dw_edma_block ll_wr[EDMA_MAX_WR_CH];
+ struct dw_edma_block ll_rd[EDMA_MAX_RD_CH];
/* eDMA memory data location */
- enum pci_barno dt_bar;
- off_t dt_off;
- size_t dt_sz;
+ struct dw_edma_block dt_wr[EDMA_MAX_WR_CH];
+ struct dw_edma_block dt_rd[EDMA_MAX_RD_CH];
/* Other */
- u32 version;
- enum dw_edma_mode mode;
+ enum dw_edma_map_format mf;
u8 irqs;
+ u16 wr_ch_cnt;
+ u16 rd_ch_cnt;
};
static const struct dw_edma_pcie_data snps_edda_data = {
/* eDMA registers location */
- .rg_bar = BAR_0,
- .rg_off = 0x00001000, /* 4 Kbytes */
- .rg_sz = 0x00002000, /* 8 Kbytes */
+ .rg.bar = BAR_0,
+ .rg.off = 0x00001000, /* 4 Kbytes */
+ .rg.sz = 0x00002000, /* 8 Kbytes */
/* eDMA memory linked list location */
- .ll_bar = BAR_2,
- .ll_off = 0x00000000, /* 0 Kbytes */
- .ll_sz = 0x00800000, /* 8 Mbytes */
+ .ll_wr = {
+ /* Channel 0 - BAR 2, offset 0 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00000000, 0x00000800)
+ /* Channel 1 - BAR 2, offset 2 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00200000, 0x00000800)
+ },
+ .ll_rd = {
+ /* Channel 0 - BAR 2, offset 4 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00400000, 0x00000800)
+ /* Channel 1 - BAR 2, offset 6 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00600000, 0x00000800)
+ },
/* eDMA memory data location */
- .dt_bar = BAR_2,
- .dt_off = 0x00800000, /* 8 Mbytes */
- .dt_sz = 0x03800000, /* 56 Mbytes */
+ .dt_wr = {
+ /* Channel 0 - BAR 2, offset 8 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00800000, 0x00000800)
+ /* Channel 1 - BAR 2, offset 9 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00900000, 0x00000800)
+ },
+ .dt_rd = {
+ /* Channel 0 - BAR 2, offset 10 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00a00000, 0x00000800)
+ /* Channel 1 - BAR 2, offset 11 Mbytes, size 2 Kbytes */
+ DW_BLOCK(BAR_2, 0x00b00000, 0x00000800)
+ },
/* Other */
- .version = 0,
- .mode = EDMA_MODE_UNROLL,
+ .mf = EDMA_MF_EDMA_UNROLL,
.irqs = 1,
+ .wr_ch_cnt = 2,
+ .rd_ch_cnt = 2,
};
static int dw_edma_pcie_irq_vector(struct device *dev, unsigned int nr)
@@ -63,14 +99,58 @@ static const struct dw_edma_core_ops dw_edma_pcie_core_ops = {
.irq_vector = dw_edma_pcie_irq_vector,
};
+static void dw_edma_pcie_get_vsec_dma_data(struct pci_dev *pdev,
+ struct dw_edma_pcie_data *pdata)
+{
+ u32 val, map;
+ u16 vsec;
+ u64 off;
+
+ vsec = pci_find_vsec_capability(pdev, PCI_VENDOR_ID_SYNOPSYS,
+ DW_PCIE_VSEC_DMA_ID);
+ if (!vsec)
+ return;
+
+ pci_read_config_dword(pdev, vsec + PCI_VNDR_HEADER, &val);
+ if (PCI_VNDR_HEADER_REV(val) != 0x00 ||
+ PCI_VNDR_HEADER_LEN(val) != 0x18)
+ return;
+
+ pci_dbg(pdev, "Detected PCIe Vendor-Specific Extended Capability DMA\n");
+ pci_read_config_dword(pdev, vsec + 0x8, &val);
+ map = FIELD_GET(DW_PCIE_VSEC_DMA_MAP, val);
+ if (map != EDMA_MF_EDMA_LEGACY &&
+ map != EDMA_MF_EDMA_UNROLL &&
+ map != EDMA_MF_HDMA_COMPAT)
+ return;
+
+ pdata->mf = map;
+ pdata->rg.bar = FIELD_GET(DW_PCIE_VSEC_DMA_BAR, val);
+
+ pci_read_config_dword(pdev, vsec + 0xc, &val);
+ pdata->wr_ch_cnt = min_t(u16, pdata->wr_ch_cnt,
+ FIELD_GET(DW_PCIE_VSEC_DMA_WR_CH, val));
+ pdata->rd_ch_cnt = min_t(u16, pdata->rd_ch_cnt,
+ FIELD_GET(DW_PCIE_VSEC_DMA_RD_CH, val));
+
+ pci_read_config_dword(pdev, vsec + 0x14, &val);
+ off = val;
+ pci_read_config_dword(pdev, vsec + 0x10, &val);
+ off <<= 32;
+ off |= val;
+ pdata->rg.off = off;
+}
+
static int dw_edma_pcie_probe(struct pci_dev *pdev,
const struct pci_device_id *pid)
{
- const struct dw_edma_pcie_data *pdata = (void *)pid->driver_data;
+ struct dw_edma_pcie_data *pdata = (void *)pid->driver_data;
+ struct dw_edma_pcie_data vsec_data;
struct device *dev = &pdev->dev;
struct dw_edma_chip *chip;
- int err, nr_irqs;
struct dw_edma *dw;
+ int err, nr_irqs;
+ int i, mask;
/* Enable PCI device */
err = pcim_enable_device(pdev);
@@ -79,11 +159,25 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev,
return err;
}
+ memcpy(&vsec_data, pdata, sizeof(struct dw_edma_pcie_data));
+
+ /*
+ * Tries to find if exists a PCIe Vendor-Specific Extended Capability
+ * for the DMA, if one exists, then reconfigures it.
+ */
+ dw_edma_pcie_get_vsec_dma_data(pdev, &vsec_data);
+
/* Mapping PCI BAR regions */
- err = pcim_iomap_regions(pdev, BIT(pdata->rg_bar) |
- BIT(pdata->ll_bar) |
- BIT(pdata->dt_bar),
- pci_name(pdev));
+ mask = BIT(vsec_data.rg.bar);
+ for (i = 0; i < vsec_data.wr_ch_cnt; i++) {
+ mask |= BIT(vsec_data.ll_wr[i].bar);
+ mask |= BIT(vsec_data.dt_wr[i].bar);
+ }
+ for (i = 0; i < vsec_data.rd_ch_cnt; i++) {
+ mask |= BIT(vsec_data.ll_rd[i].bar);
+ mask |= BIT(vsec_data.dt_rd[i].bar);
+ }
+ err = pcim_iomap_regions(pdev, mask, pci_name(pdev));
if (err) {
pci_err(pdev, "eDMA BAR I/O remapping failed\n");
return err;
@@ -125,7 +219,7 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev,
return -ENOMEM;
/* IRQs allocation */
- nr_irqs = pci_alloc_irq_vectors(pdev, 1, pdata->irqs,
+ nr_irqs = pci_alloc_irq_vectors(pdev, 1, vsec_data.irqs,
PCI_IRQ_MSI | PCI_IRQ_MSIX);
if (nr_irqs < 1) {
pci_err(pdev, "fail to alloc IRQ vector (number of IRQs=%u)\n",
@@ -139,46 +233,109 @@ static int dw_edma_pcie_probe(struct pci_dev *pdev,
chip->id = pdev->devfn;
chip->irq = pdev->irq;
- dw->rg_region.vaddr = pcim_iomap_table(pdev)[pdata->rg_bar];
- dw->rg_region.vaddr += pdata->rg_off;
- dw->rg_region.paddr = pdev->resource[pdata->rg_bar].start;
- dw->rg_region.paddr += pdata->rg_off;
- dw->rg_region.sz = pdata->rg_sz;
-
- dw->ll_region.vaddr = pcim_iomap_table(pdev)[pdata->ll_bar];
- dw->ll_region.vaddr += pdata->ll_off;
- dw->ll_region.paddr = pdev->resource[pdata->ll_bar].start;
- dw->ll_region.paddr += pdata->ll_off;
- dw->ll_region.sz = pdata->ll_sz;
-
- dw->dt_region.vaddr = pcim_iomap_table(pdev)[pdata->dt_bar];
- dw->dt_region.vaddr += pdata->dt_off;
- dw->dt_region.paddr = pdev->resource[pdata->dt_bar].start;
- dw->dt_region.paddr += pdata->dt_off;
- dw->dt_region.sz = pdata->dt_sz;
-
- dw->version = pdata->version;
- dw->mode = pdata->mode;
+ dw->mf = vsec_data.mf;
dw->nr_irqs = nr_irqs;
dw->ops = &dw_edma_pcie_core_ops;
+ dw->wr_ch_cnt = vsec_data.wr_ch_cnt;
+ dw->rd_ch_cnt = vsec_data.rd_ch_cnt;
- /* Debug info */
- pci_dbg(pdev, "Version:\t%u\n", dw->version);
+ dw->rg_region.vaddr = pcim_iomap_table(pdev)[vsec_data.rg.bar];
+ if (!dw->rg_region.vaddr)
+ return -ENOMEM;
+
+ dw->rg_region.vaddr += vsec_data.rg.off;
+ dw->rg_region.paddr = pdev->resource[vsec_data.rg.bar].start;
+ dw->rg_region.paddr += vsec_data.rg.off;
+ dw->rg_region.sz = vsec_data.rg.sz;
+
+ for (i = 0; i < dw->wr_ch_cnt; i++) {
+ struct dw_edma_region *ll_region = &dw->ll_region_wr[i];
+ struct dw_edma_region *dt_region = &dw->dt_region_wr[i];
+ struct dw_edma_block *ll_block = &vsec_data.ll_wr[i];
+ struct dw_edma_block *dt_block = &vsec_data.dt_wr[i];
+
+ ll_region->vaddr = pcim_iomap_table(pdev)[ll_block->bar];
+ if (!ll_region->vaddr)
+ return -ENOMEM;
+
+ ll_region->vaddr += ll_block->off;
+ ll_region->paddr = pdev->resource[ll_block->bar].start;
+ ll_region->paddr += ll_block->off;
+ ll_region->sz = ll_block->sz;
+
+ dt_region->vaddr = pcim_iomap_table(pdev)[dt_block->bar];
+ if (!dt_region->vaddr)
+ return -ENOMEM;
+
+ dt_region->vaddr += dt_block->off;
+ dt_region->paddr = pdev->resource[dt_block->bar].start;
+ dt_region->paddr += dt_block->off;
+ dt_region->sz = dt_block->sz;
+ }
- pci_dbg(pdev, "Mode:\t%s\n",
- dw->mode == EDMA_MODE_LEGACY ? "Legacy" : "Unroll");
+ for (i = 0; i < dw->rd_ch_cnt; i++) {
+ struct dw_edma_region *ll_region = &dw->ll_region_rd[i];
+ struct dw_edma_region *dt_region = &dw->dt_region_rd[i];
+ struct dw_edma_block *ll_block = &vsec_data.ll_rd[i];
+ struct dw_edma_block *dt_block = &vsec_data.dt_rd[i];
+
+ ll_region->vaddr = pcim_iomap_table(pdev)[ll_block->bar];
+ if (!ll_region->vaddr)
+ return -ENOMEM;
+
+ ll_region->vaddr += ll_block->off;
+ ll_region->paddr = pdev->resource[ll_block->bar].start;
+ ll_region->paddr += ll_block->off;
+ ll_region->sz = ll_block->sz;
+
+ dt_region->vaddr = pcim_iomap_table(pdev)[dt_block->bar];
+ if (!dt_region->vaddr)
+ return -ENOMEM;
+
+ dt_region->vaddr += dt_block->off;
+ dt_region->paddr = pdev->resource[dt_block->bar].start;
+ dt_region->paddr += dt_block->off;
+ dt_region->sz = dt_block->sz;
+ }
+
+ /* Debug info */
+ if (dw->mf == EDMA_MF_EDMA_LEGACY)
+ pci_dbg(pdev, "Version:\teDMA Port Logic (0x%x)\n", dw->mf);
+ else if (dw->mf == EDMA_MF_EDMA_UNROLL)
+ pci_dbg(pdev, "Version:\teDMA Unroll (0x%x)\n", dw->mf);
+ else if (dw->mf == EDMA_MF_HDMA_COMPAT)
+ pci_dbg(pdev, "Version:\tHDMA Compatible (0x%x)\n", dw->mf);
+ else
+ pci_dbg(pdev, "Version:\tUnknown (0x%x)\n", dw->mf);
pci_dbg(pdev, "Registers:\tBAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%p, p=%pa)\n",
- pdata->rg_bar, pdata->rg_off, pdata->rg_sz,
+ vsec_data.rg.bar, vsec_data.rg.off, vsec_data.rg.sz,
dw->rg_region.vaddr, &dw->rg_region.paddr);
- pci_dbg(pdev, "L. List:\tBAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%p, p=%pa)\n",
- pdata->ll_bar, pdata->ll_off, pdata->ll_sz,
- dw->ll_region.vaddr, &dw->ll_region.paddr);
- pci_dbg(pdev, "Data:\tBAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%p, p=%pa)\n",
- pdata->dt_bar, pdata->dt_off, pdata->dt_sz,
- dw->dt_region.vaddr, &dw->dt_region.paddr);
+ for (i = 0; i < dw->wr_ch_cnt; i++) {
+ pci_dbg(pdev, "L. List:\tWRITE CH%.2u, BAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%p, p=%pa)\n",
+ i, vsec_data.ll_wr[i].bar,
+ vsec_data.ll_wr[i].off, dw->ll_region_wr[i].sz,
+ dw->ll_region_wr[i].vaddr, &dw->ll_region_wr[i].paddr);
+
+ pci_dbg(pdev, "Data:\tWRITE CH%.2u, BAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%p, p=%pa)\n",
+ i, vsec_data.dt_wr[i].bar,
+ vsec_data.dt_wr[i].off, dw->dt_region_wr[i].sz,
+ dw->dt_region_wr[i].vaddr, &dw->dt_region_wr[i].paddr);
+ }
+
+ for (i = 0; i < dw->rd_ch_cnt; i++) {
+ pci_dbg(pdev, "L. List:\tREAD CH%.2u, BAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%p, p=%pa)\n",
+ i, vsec_data.ll_rd[i].bar,
+ vsec_data.ll_rd[i].off, dw->ll_region_rd[i].sz,
+ dw->ll_region_rd[i].vaddr, &dw->ll_region_rd[i].paddr);
+
+ pci_dbg(pdev, "Data:\tREAD CH%.2u, BAR=%u, off=0x%.8lx, sz=0x%zx bytes, addr(v=%p, p=%pa)\n",
+ i, vsec_data.dt_rd[i].bar,
+ vsec_data.dt_rd[i].off, dw->dt_region_rd[i].sz,
+ dw->dt_region_rd[i].vaddr, &dw->dt_region_rd[i].paddr);
+ }
pci_dbg(pdev, "Nr. IRQs:\t%u\n", dw->nr_irqs);
diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.c b/drivers/dma/dw-edma/dw-edma-v0-core.c
index 692de47b1670..329fc2e57b70 100644
--- a/drivers/dma/dw-edma/dw-edma-v0-core.c
+++ b/drivers/dma/dw-edma/dw-edma-v0-core.c
@@ -28,35 +28,75 @@ static inline struct dw_edma_v0_regs __iomem *__dw_regs(struct dw_edma *dw)
return dw->rg_region.vaddr;
}
-#define SET(dw, name, value) \
+#define SET_32(dw, name, value) \
writel(value, &(__dw_regs(dw)->name))
-#define GET(dw, name) \
+#define GET_32(dw, name) \
readl(&(__dw_regs(dw)->name))
-#define SET_RW(dw, dir, name, value) \
+#define SET_RW_32(dw, dir, name, value) \
do { \
if ((dir) == EDMA_DIR_WRITE) \
- SET(dw, wr_##name, value); \
+ SET_32(dw, wr_##name, value); \
else \
- SET(dw, rd_##name, value); \
+ SET_32(dw, rd_##name, value); \
} while (0)
-#define GET_RW(dw, dir, name) \
+#define GET_RW_32(dw, dir, name) \
((dir) == EDMA_DIR_WRITE \
- ? GET(dw, wr_##name) \
- : GET(dw, rd_##name))
+ ? GET_32(dw, wr_##name) \
+ : GET_32(dw, rd_##name))
-#define SET_BOTH(dw, name, value) \
+#define SET_BOTH_32(dw, name, value) \
do { \
- SET(dw, wr_##name, value); \
- SET(dw, rd_##name, value); \
+ SET_32(dw, wr_##name, value); \
+ SET_32(dw, rd_##name, value); \
+ } while (0)
+
+#ifdef CONFIG_64BIT
+
+#define SET_64(dw, name, value) \
+ writeq(value, &(__dw_regs(dw)->name))
+
+#define GET_64(dw, name) \
+ readq(&(__dw_regs(dw)->name))
+
+#define SET_RW_64(dw, dir, name, value) \
+ do { \
+ if ((dir) == EDMA_DIR_WRITE) \
+ SET_64(dw, wr_##name, value); \
+ else \
+ SET_64(dw, rd_##name, value); \
+ } while (0)
+
+#define GET_RW_64(dw, dir, name) \
+ ((dir) == EDMA_DIR_WRITE \
+ ? GET_64(dw, wr_##name) \
+ : GET_64(dw, rd_##name))
+
+#define SET_BOTH_64(dw, name, value) \
+ do { \
+ SET_64(dw, wr_##name, value); \
+ SET_64(dw, rd_##name, value); \
+ } while (0)
+
+#endif /* CONFIG_64BIT */
+
+#define SET_COMPAT(dw, name, value) \
+ writel(value, &(__dw_regs(dw)->type.unroll.name))
+
+#define SET_RW_COMPAT(dw, dir, name, value) \
+ do { \
+ if ((dir) == EDMA_DIR_WRITE) \
+ SET_COMPAT(dw, wr_##name, value); \
+ else \
+ SET_COMPAT(dw, rd_##name, value); \
} while (0)
static inline struct dw_edma_v0_ch_regs __iomem *
__dw_ch_regs(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch)
{
- if (dw->mode == EDMA_MODE_LEGACY)
+ if (dw->mf == EDMA_MF_EDMA_LEGACY)
return &(__dw_regs(dw)->type.legacy.ch);
if (dir == EDMA_DIR_WRITE)
@@ -68,7 +108,7 @@ __dw_ch_regs(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch)
static inline void writel_ch(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch,
u32 value, void __iomem *addr)
{
- if (dw->mode == EDMA_MODE_LEGACY) {
+ if (dw->mf == EDMA_MF_EDMA_LEGACY) {
u32 viewport_sel;
unsigned long flags;
@@ -93,7 +133,7 @@ static inline u32 readl_ch(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch,
{
u32 value;
- if (dw->mode == EDMA_MODE_LEGACY) {
+ if (dw->mf == EDMA_MF_EDMA_LEGACY) {
u32 viewport_sel;
unsigned long flags;
@@ -115,21 +155,86 @@ static inline u32 readl_ch(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch,
return value;
}
-#define SET_CH(dw, dir, ch, name, value) \
+#define SET_CH_32(dw, dir, ch, name, value) \
writel_ch(dw, dir, ch, value, &(__dw_ch_regs(dw, dir, ch)->name))
-#define GET_CH(dw, dir, ch, name) \
+#define GET_CH_32(dw, dir, ch, name) \
readl_ch(dw, dir, ch, &(__dw_ch_regs(dw, dir, ch)->name))
-#define SET_LL(ll, value) \
+#define SET_LL_32(ll, value) \
writel(value, ll)
+#ifdef CONFIG_64BIT
+
+static inline void writeq_ch(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch,
+ u64 value, void __iomem *addr)
+{
+ if (dw->mf == EDMA_MF_EDMA_LEGACY) {
+ u32 viewport_sel;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&dw->lock, flags);
+
+ viewport_sel = FIELD_PREP(EDMA_V0_VIEWPORT_MASK, ch);
+ if (dir == EDMA_DIR_READ)
+ viewport_sel |= BIT(31);
+
+ writel(viewport_sel,
+ &(__dw_regs(dw)->type.legacy.viewport_sel));
+ writeq(value, addr);
+
+ raw_spin_unlock_irqrestore(&dw->lock, flags);
+ } else {
+ writeq(value, addr);
+ }
+}
+
+static inline u64 readq_ch(struct dw_edma *dw, enum dw_edma_dir dir, u16 ch,
+ const void __iomem *addr)
+{
+ u32 value;
+
+ if (dw->mf == EDMA_MF_EDMA_LEGACY) {
+ u32 viewport_sel;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&dw->lock, flags);
+
+ viewport_sel = FIELD_PREP(EDMA_V0_VIEWPORT_MASK, ch);
+ if (dir == EDMA_DIR_READ)
+ viewport_sel |= BIT(31);
+
+ writel(viewport_sel,
+ &(__dw_regs(dw)->type.legacy.viewport_sel));
+ value = readq(addr);
+
+ raw_spin_unlock_irqrestore(&dw->lock, flags);
+ } else {
+ value = readq(addr);
+ }
+
+ return value;
+}
+
+#define SET_CH_64(dw, dir, ch, name, value) \
+ writeq_ch(dw, dir, ch, value, &(__dw_ch_regs(dw, dir, ch)->name))
+
+#define GET_CH_64(dw, dir, ch, name) \
+ readq_ch(dw, dir, ch, &(__dw_ch_regs(dw, dir, ch)->name))
+
+#define SET_LL_64(ll, value) \
+ writeq(value, ll)
+
+#endif /* CONFIG_64BIT */
+
/* eDMA management callbacks */
void dw_edma_v0_core_off(struct dw_edma *dw)
{
- SET_BOTH(dw, int_mask, EDMA_V0_DONE_INT_MASK | EDMA_V0_ABORT_INT_MASK);
- SET_BOTH(dw, int_clear, EDMA_V0_DONE_INT_MASK | EDMA_V0_ABORT_INT_MASK);
- SET_BOTH(dw, engine_en, 0);
+ SET_BOTH_32(dw, int_mask,
+ EDMA_V0_DONE_INT_MASK | EDMA_V0_ABORT_INT_MASK);
+ SET_BOTH_32(dw, int_clear,
+ EDMA_V0_DONE_INT_MASK | EDMA_V0_ABORT_INT_MASK);
+ SET_BOTH_32(dw, engine_en, 0);
}
u16 dw_edma_v0_core_ch_count(struct dw_edma *dw, enum dw_edma_dir dir)
@@ -137,9 +242,11 @@ u16 dw_edma_v0_core_ch_count(struct dw_edma *dw, enum dw_edma_dir dir)
u32 num_ch;
if (dir == EDMA_DIR_WRITE)
- num_ch = FIELD_GET(EDMA_V0_WRITE_CH_COUNT_MASK, GET(dw, ctrl));
+ num_ch = FIELD_GET(EDMA_V0_WRITE_CH_COUNT_MASK,
+ GET_32(dw, ctrl));
else
- num_ch = FIELD_GET(EDMA_V0_READ_CH_COUNT_MASK, GET(dw, ctrl));
+ num_ch = FIELD_GET(EDMA_V0_READ_CH_COUNT_MASK,
+ GET_32(dw, ctrl));
if (num_ch > EDMA_V0_MAX_NR_CH)
num_ch = EDMA_V0_MAX_NR_CH;
@@ -153,7 +260,7 @@ enum dma_status dw_edma_v0_core_ch_status(struct dw_edma_chan *chan)
u32 tmp;
tmp = FIELD_GET(EDMA_V0_CH_STATUS_MASK,
- GET_CH(dw, chan->dir, chan->id, ch_control1));
+ GET_CH_32(dw, chan->dir, chan->id, ch_control1));
if (tmp == 1)
return DMA_IN_PROGRESS;
@@ -167,26 +274,28 @@ void dw_edma_v0_core_clear_done_int(struct dw_edma_chan *chan)
{
struct dw_edma *dw = chan->chip->dw;
- SET_RW(dw, chan->dir, int_clear,
- FIELD_PREP(EDMA_V0_DONE_INT_MASK, BIT(chan->id)));
+ SET_RW_32(dw, chan->dir, int_clear,
+ FIELD_PREP(EDMA_V0_DONE_INT_MASK, BIT(chan->id)));
}
void dw_edma_v0_core_clear_abort_int(struct dw_edma_chan *chan)
{
struct dw_edma *dw = chan->chip->dw;
- SET_RW(dw, chan->dir, int_clear,
- FIELD_PREP(EDMA_V0_ABORT_INT_MASK, BIT(chan->id)));
+ SET_RW_32(dw, chan->dir, int_clear,
+ FIELD_PREP(EDMA_V0_ABORT_INT_MASK, BIT(chan->id)));
}
u32 dw_edma_v0_core_status_done_int(struct dw_edma *dw, enum dw_edma_dir dir)
{
- return FIELD_GET(EDMA_V0_DONE_INT_MASK, GET_RW(dw, dir, int_status));
+ return FIELD_GET(EDMA_V0_DONE_INT_MASK,
+ GET_RW_32(dw, dir, int_status));
}
u32 dw_edma_v0_core_status_abort_int(struct dw_edma *dw, enum dw_edma_dir dir)
{
- return FIELD_GET(EDMA_V0_ABORT_INT_MASK, GET_RW(dw, dir, int_status));
+ return FIELD_GET(EDMA_V0_ABORT_INT_MASK,
+ GET_RW_32(dw, dir, int_status));
}
static void dw_edma_v0_core_write_chunk(struct dw_edma_chunk *chunk)
@@ -209,15 +318,23 @@ static void dw_edma_v0_core_write_chunk(struct dw_edma_chunk *chunk)
control |= (DW_EDMA_V0_LIE | DW_EDMA_V0_RIE);
/* Channel control */
- SET_LL(&lli[i].control, control);
+ SET_LL_32(&lli[i].control, control);
/* Transfer size */
- SET_LL(&lli[i].transfer_size, child->sz);
- /* SAR - low, high */
- SET_LL(&lli[i].sar_low, lower_32_bits(child->sar));
- SET_LL(&lli[i].sar_high, upper_32_bits(child->sar));
- /* DAR - low, high */
- SET_LL(&lli[i].dar_low, lower_32_bits(child->dar));
- SET_LL(&lli[i].dar_high, upper_32_bits(child->dar));
+ SET_LL_32(&lli[i].transfer_size, child->sz);
+ /* SAR */
+ #ifdef CONFIG_64BIT
+ SET_LL_64(&lli[i].sar.reg, child->sar);
+ #else /* CONFIG_64BIT */
+ SET_LL_32(&lli[i].sar.lsb, lower_32_bits(child->sar));
+ SET_LL_32(&lli[i].sar.msb, upper_32_bits(child->sar));
+ #endif /* CONFIG_64BIT */
+ /* DAR */
+ #ifdef CONFIG_64BIT
+ SET_LL_64(&lli[i].dar.reg, child->dar);
+ #else /* CONFIG_64BIT */
+ SET_LL_32(&lli[i].dar.lsb, lower_32_bits(child->dar));
+ SET_LL_32(&lli[i].dar.msb, upper_32_bits(child->dar));
+ #endif /* CONFIG_64BIT */
i++;
}
@@ -227,10 +344,14 @@ static void dw_edma_v0_core_write_chunk(struct dw_edma_chunk *chunk)
control |= DW_EDMA_V0_CB;
/* Channel control */
- SET_LL(&llp->control, control);
- /* Linked list - low, high */
- SET_LL(&llp->llp_low, lower_32_bits(chunk->ll_region.paddr));
- SET_LL(&llp->llp_high, upper_32_bits(chunk->ll_region.paddr));
+ SET_LL_32(&llp->control, control);
+ /* Linked list */
+ #ifdef CONFIG_64BIT
+ SET_LL_64(&llp->llp.reg, chunk->ll_region.paddr);
+ #else /* CONFIG_64BIT */
+ SET_LL_32(&llp->llp.lsb, lower_32_bits(chunk->ll_region.paddr));
+ SET_LL_32(&llp->llp.msb, upper_32_bits(chunk->ll_region.paddr));
+ #endif /* CONFIG_64BIT */
}
void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
@@ -243,28 +364,69 @@ void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first)
if (first) {
/* Enable engine */
- SET_RW(dw, chan->dir, engine_en, BIT(0));
+ SET_RW_32(dw, chan->dir, engine_en, BIT(0));
+ if (dw->mf == EDMA_MF_HDMA_COMPAT) {
+ switch (chan->id) {
+ case 0:
+ SET_RW_COMPAT(dw, chan->dir, ch0_pwr_en,
+ BIT(0));
+ break;
+ case 1:
+ SET_RW_COMPAT(dw, chan->dir, ch1_pwr_en,
+ BIT(0));
+ break;
+ case 2:
+ SET_RW_COMPAT(dw, chan->dir, ch2_pwr_en,
+ BIT(0));
+ break;
+ case 3:
+ SET_RW_COMPAT(dw, chan->dir, ch3_pwr_en,
+ BIT(0));
+ break;
+ case 4:
+ SET_RW_COMPAT(dw, chan->dir, ch4_pwr_en,
+ BIT(0));
+ break;
+ case 5:
+ SET_RW_COMPAT(dw, chan->dir, ch5_pwr_en,
+ BIT(0));
+ break;
+ case 6:
+ SET_RW_COMPAT(dw, chan->dir, ch6_pwr_en,
+ BIT(0));
+ break;
+ case 7:
+ SET_RW_COMPAT(dw, chan->dir, ch7_pwr_en,
+ BIT(0));
+ break;
+ }
+ }
/* Interrupt unmask - done, abort */
- tmp = GET_RW(dw, chan->dir, int_mask);
+ tmp = GET_RW_32(dw, chan->dir, int_mask);
tmp &= ~FIELD_PREP(EDMA_V0_DONE_INT_MASK, BIT(chan->id));
tmp &= ~FIELD_PREP(EDMA_V0_ABORT_INT_MASK, BIT(chan->id));
- SET_RW(dw, chan->dir, int_mask, tmp);
+ SET_RW_32(dw, chan->dir, int_mask, tmp);
/* Linked list error */
- tmp = GET_RW(dw, chan->dir, linked_list_err_en);
+ tmp = GET_RW_32(dw, chan->dir, linked_list_err_en);
tmp |= FIELD_PREP(EDMA_V0_LINKED_LIST_ERR_MASK, BIT(chan->id));
- SET_RW(dw, chan->dir, linked_list_err_en, tmp);
+ SET_RW_32(dw, chan->dir, linked_list_err_en, tmp);
/* Channel control */
- SET_CH(dw, chan->dir, chan->id, ch_control1,
- (DW_EDMA_V0_CCS | DW_EDMA_V0_LLE));
- /* Linked list - low, high */
- SET_CH(dw, chan->dir, chan->id, llp_low,
- lower_32_bits(chunk->ll_region.paddr));
- SET_CH(dw, chan->dir, chan->id, llp_high,
- upper_32_bits(chunk->ll_region.paddr));
+ SET_CH_32(dw, chan->dir, chan->id, ch_control1,
+ (DW_EDMA_V0_CCS | DW_EDMA_V0_LLE));
+ /* Linked list */
+ #ifdef CONFIG_64BIT
+ SET_CH_64(dw, chan->dir, chan->id, llp.reg,
+ chunk->ll_region.paddr);
+ #else /* CONFIG_64BIT */
+ SET_CH_32(dw, chan->dir, chan->id, llp.lsb,
+ lower_32_bits(chunk->ll_region.paddr));
+ SET_CH_32(dw, chan->dir, chan->id, llp.msb,
+ upper_32_bits(chunk->ll_region.paddr));
+ #endif /* CONFIG_64BIT */
}
/* Doorbell */
- SET_RW(dw, chan->dir, doorbell,
- FIELD_PREP(EDMA_V0_DOORBELL_CH_MASK, chan->id));
+ SET_RW_32(dw, chan->dir, doorbell,
+ FIELD_PREP(EDMA_V0_DOORBELL_CH_MASK, chan->id));
}
int dw_edma_v0_core_device_config(struct dw_edma_chan *chan)
@@ -273,31 +435,31 @@ int dw_edma_v0_core_device_config(struct dw_edma_chan *chan)
u32 tmp = 0;
/* MSI done addr - low, high */
- SET_RW(dw, chan->dir, done_imwr_low, chan->msi.address_lo);
- SET_RW(dw, chan->dir, done_imwr_high, chan->msi.address_hi);
+ SET_RW_32(dw, chan->dir, done_imwr.lsb, chan->msi.address_lo);
+ SET_RW_32(dw, chan->dir, done_imwr.msb, chan->msi.address_hi);
/* MSI abort addr - low, high */
- SET_RW(dw, chan->dir, abort_imwr_low, chan->msi.address_lo);
- SET_RW(dw, chan->dir, abort_imwr_high, chan->msi.address_hi);
+ SET_RW_32(dw, chan->dir, abort_imwr.lsb, chan->msi.address_lo);
+ SET_RW_32(dw, chan->dir, abort_imwr.msb, chan->msi.address_hi);
/* MSI data - low, high */
switch (chan->id) {
case 0:
case 1:
- tmp = GET_RW(dw, chan->dir, ch01_imwr_data);
+ tmp = GET_RW_32(dw, chan->dir, ch01_imwr_data);
break;
case 2:
case 3:
- tmp = GET_RW(dw, chan->dir, ch23_imwr_data);
+ tmp = GET_RW_32(dw, chan->dir, ch23_imwr_data);
break;
case 4:
case 5:
- tmp = GET_RW(dw, chan->dir, ch45_imwr_data);
+ tmp = GET_RW_32(dw, chan->dir, ch45_imwr_data);
break;
case 6:
case 7:
- tmp = GET_RW(dw, chan->dir, ch67_imwr_data);
+ tmp = GET_RW_32(dw, chan->dir, ch67_imwr_data);
break;
}
@@ -316,22 +478,22 @@ int dw_edma_v0_core_device_config(struct dw_edma_chan *chan)
switch (chan->id) {
case 0:
case 1:
- SET_RW(dw, chan->dir, ch01_imwr_data, tmp);
+ SET_RW_32(dw, chan->dir, ch01_imwr_data, tmp);
break;
case 2:
case 3:
- SET_RW(dw, chan->dir, ch23_imwr_data, tmp);
+ SET_RW_32(dw, chan->dir, ch23_imwr_data, tmp);
break;
case 4:
case 5:
- SET_RW(dw, chan->dir, ch45_imwr_data, tmp);
+ SET_RW_32(dw, chan->dir, ch45_imwr_data, tmp);
break;
case 6:
case 7:
- SET_RW(dw, chan->dir, ch67_imwr_data, tmp);
+ SET_RW_32(dw, chan->dir, ch67_imwr_data, tmp);
break;
}
@@ -344,7 +506,7 @@ void dw_edma_v0_core_debugfs_on(struct dw_edma_chip *chip)
dw_edma_v0_debugfs_on(chip);
}
-void dw_edma_v0_core_debugfs_off(void)
+void dw_edma_v0_core_debugfs_off(struct dw_edma_chip *chip)
{
- dw_edma_v0_debugfs_off();
+ dw_edma_v0_debugfs_off(chip);
}
diff --git a/drivers/dma/dw-edma/dw-edma-v0-core.h b/drivers/dma/dw-edma/dw-edma-v0-core.h
index abae1527f1f9..2afa626b8300 100644
--- a/drivers/dma/dw-edma/dw-edma-v0-core.h
+++ b/drivers/dma/dw-edma/dw-edma-v0-core.h
@@ -23,6 +23,6 @@ void dw_edma_v0_core_start(struct dw_edma_chunk *chunk, bool first);
int dw_edma_v0_core_device_config(struct dw_edma_chan *chan);
/* eDMA debug fs callbacks */
void dw_edma_v0_core_debugfs_on(struct dw_edma_chip *chip);
-void dw_edma_v0_core_debugfs_off(void);
+void dw_edma_v0_core_debugfs_off(struct dw_edma_chip *chip);
#endif /* _DW_EDMA_V0_CORE_H */
diff --git a/drivers/dma/dw-edma/dw-edma-v0-debugfs.c b/drivers/dma/dw-edma/dw-edma-v0-debugfs.c
index 6f62711a4c94..4b3bcffd15ef 100644
--- a/drivers/dma/dw-edma/dw-edma-v0-debugfs.c
+++ b/drivers/dma/dw-edma/dw-edma-v0-debugfs.c
@@ -38,7 +38,6 @@
#define CHANNEL_STR "channel"
#define REGISTERS_STR "registers"
-static struct dentry *base_dir;
static struct dw_edma *dw;
static struct dw_edma_v0_regs __iomem *regs;
@@ -55,7 +54,7 @@ struct debugfs_entries {
static int dw_edma_debugfs_u32_get(void *data, u64 *val)
{
void __iomem *reg = (void __force __iomem *)data;
- if (dw->mode == EDMA_MODE_LEGACY &&
+ if (dw->mf == EDMA_MF_EDMA_LEGACY &&
reg >= (void __iomem *)&regs->type.legacy.ch) {
void __iomem *ptr = &regs->type.legacy.ch;
u32 viewport_sel = 0;
@@ -114,12 +113,12 @@ static void dw_edma_debugfs_regs_ch(struct dw_edma_v0_ch_regs __iomem *regs,
REGISTER(ch_control1),
REGISTER(ch_control2),
REGISTER(transfer_size),
- REGISTER(sar_low),
- REGISTER(sar_high),
- REGISTER(dar_low),
- REGISTER(dar_high),
- REGISTER(llp_low),
- REGISTER(llp_high),
+ REGISTER(sar.lsb),
+ REGISTER(sar.msb),
+ REGISTER(dar.lsb),
+ REGISTER(dar.msb),
+ REGISTER(llp.lsb),
+ REGISTER(llp.msb),
};
nr_entries = ARRAY_SIZE(debugfs_regs);
@@ -132,17 +131,17 @@ static void dw_edma_debugfs_regs_wr(struct dentry *dir)
/* eDMA global registers */
WR_REGISTER(engine_en),
WR_REGISTER(doorbell),
- WR_REGISTER(ch_arb_weight_low),
- WR_REGISTER(ch_arb_weight_high),
+ WR_REGISTER(ch_arb_weight.lsb),
+ WR_REGISTER(ch_arb_weight.msb),
/* eDMA interrupts registers */
WR_REGISTER(int_status),
WR_REGISTER(int_mask),
WR_REGISTER(int_clear),
WR_REGISTER(err_status),
- WR_REGISTER(done_imwr_low),
- WR_REGISTER(done_imwr_high),
- WR_REGISTER(abort_imwr_low),
- WR_REGISTER(abort_imwr_high),
+ WR_REGISTER(done_imwr.lsb),
+ WR_REGISTER(done_imwr.msb),
+ WR_REGISTER(abort_imwr.lsb),
+ WR_REGISTER(abort_imwr.msb),
WR_REGISTER(ch01_imwr_data),
WR_REGISTER(ch23_imwr_data),
WR_REGISTER(ch45_imwr_data),
@@ -152,8 +151,8 @@ static void dw_edma_debugfs_regs_wr(struct dentry *dir)
const struct debugfs_entries debugfs_unroll_regs[] = {
/* eDMA channel context grouping */
WR_REGISTER_UNROLL(engine_chgroup),
- WR_REGISTER_UNROLL(engine_hshake_cnt_low),
- WR_REGISTER_UNROLL(engine_hshake_cnt_high),
+ WR_REGISTER_UNROLL(engine_hshake_cnt.lsb),
+ WR_REGISTER_UNROLL(engine_hshake_cnt.msb),
WR_REGISTER_UNROLL(ch0_pwr_en),
WR_REGISTER_UNROLL(ch1_pwr_en),
WR_REGISTER_UNROLL(ch2_pwr_en),
@@ -174,7 +173,7 @@ static void dw_edma_debugfs_regs_wr(struct dentry *dir)
nr_entries = ARRAY_SIZE(debugfs_regs);
dw_edma_debugfs_create_x32(debugfs_regs, nr_entries, regs_dir);
- if (dw->mode == EDMA_MODE_UNROLL) {
+ if (dw->mf == EDMA_MF_HDMA_COMPAT) {
nr_entries = ARRAY_SIZE(debugfs_unroll_regs);
dw_edma_debugfs_create_x32(debugfs_unroll_regs, nr_entries,
regs_dir);
@@ -200,19 +199,19 @@ static void dw_edma_debugfs_regs_rd(struct dentry *dir)
/* eDMA global registers */
RD_REGISTER(engine_en),
RD_REGISTER(doorbell),
- RD_REGISTER(ch_arb_weight_low),
- RD_REGISTER(ch_arb_weight_high),
+ RD_REGISTER(ch_arb_weight.lsb),
+ RD_REGISTER(ch_arb_weight.msb),
/* eDMA interrupts registers */
RD_REGISTER(int_status),
RD_REGISTER(int_mask),
RD_REGISTER(int_clear),
- RD_REGISTER(err_status_low),
- RD_REGISTER(err_status_high),
+ RD_REGISTER(err_status.lsb),
+ RD_REGISTER(err_status.msb),
RD_REGISTER(linked_list_err_en),
- RD_REGISTER(done_imwr_low),
- RD_REGISTER(done_imwr_high),
- RD_REGISTER(abort_imwr_low),
- RD_REGISTER(abort_imwr_high),
+ RD_REGISTER(done_imwr.lsb),
+ RD_REGISTER(done_imwr.msb),
+ RD_REGISTER(abort_imwr.lsb),
+ RD_REGISTER(abort_imwr.msb),
RD_REGISTER(ch01_imwr_data),
RD_REGISTER(ch23_imwr_data),
RD_REGISTER(ch45_imwr_data),
@@ -221,8 +220,8 @@ static void dw_edma_debugfs_regs_rd(struct dentry *dir)
const struct debugfs_entries debugfs_unroll_regs[] = {
/* eDMA channel context grouping */
RD_REGISTER_UNROLL(engine_chgroup),
- RD_REGISTER_UNROLL(engine_hshake_cnt_low),
- RD_REGISTER_UNROLL(engine_hshake_cnt_high),
+ RD_REGISTER_UNROLL(engine_hshake_cnt.lsb),
+ RD_REGISTER_UNROLL(engine_hshake_cnt.msb),
RD_REGISTER_UNROLL(ch0_pwr_en),
RD_REGISTER_UNROLL(ch1_pwr_en),
RD_REGISTER_UNROLL(ch2_pwr_en),
@@ -243,7 +242,7 @@ static void dw_edma_debugfs_regs_rd(struct dentry *dir)
nr_entries = ARRAY_SIZE(debugfs_regs);
dw_edma_debugfs_create_x32(debugfs_regs, nr_entries, regs_dir);
- if (dw->mode == EDMA_MODE_UNROLL) {
+ if (dw->mf == EDMA_MF_HDMA_COMPAT) {
nr_entries = ARRAY_SIZE(debugfs_unroll_regs);
dw_edma_debugfs_create_x32(debugfs_unroll_regs, nr_entries,
regs_dir);
@@ -272,7 +271,7 @@ static void dw_edma_debugfs_regs(void)
struct dentry *regs_dir;
int nr_entries;
- regs_dir = debugfs_create_dir(REGISTERS_STR, base_dir);
+ regs_dir = debugfs_create_dir(REGISTERS_STR, dw->debugfs);
if (!regs_dir)
return;
@@ -293,19 +292,23 @@ void dw_edma_v0_debugfs_on(struct dw_edma_chip *chip)
if (!regs)
return;
- base_dir = debugfs_create_dir(dw->name, NULL);
- if (!base_dir)
+ dw->debugfs = debugfs_create_dir(dw->name, NULL);
+ if (!dw->debugfs)
return;
- debugfs_create_u32("version", 0444, base_dir, &dw->version);
- debugfs_create_u32("mode", 0444, base_dir, &dw->mode);
- debugfs_create_u16("wr_ch_cnt", 0444, base_dir, &dw->wr_ch_cnt);
- debugfs_create_u16("rd_ch_cnt", 0444, base_dir, &dw->rd_ch_cnt);
+ debugfs_create_u32("mf", 0444, dw->debugfs, &dw->mf);
+ debugfs_create_u16("wr_ch_cnt", 0444, dw->debugfs, &dw->wr_ch_cnt);
+ debugfs_create_u16("rd_ch_cnt", 0444, dw->debugfs, &dw->rd_ch_cnt);
dw_edma_debugfs_regs();
}
-void dw_edma_v0_debugfs_off(void)
+void dw_edma_v0_debugfs_off(struct dw_edma_chip *chip)
{
- debugfs_remove_recursive(base_dir);
+ dw = chip->dw;
+ if (!dw)
+ return;
+
+ debugfs_remove_recursive(dw->debugfs);
+ dw->debugfs = NULL;
}
diff --git a/drivers/dma/dw-edma/dw-edma-v0-debugfs.h b/drivers/dma/dw-edma/dw-edma-v0-debugfs.h
index 5450a0a94193..d0ff25a9ea5c 100644
--- a/drivers/dma/dw-edma/dw-edma-v0-debugfs.h
+++ b/drivers/dma/dw-edma/dw-edma-v0-debugfs.h
@@ -13,13 +13,13 @@
#ifdef CONFIG_DEBUG_FS
void dw_edma_v0_debugfs_on(struct dw_edma_chip *chip);
-void dw_edma_v0_debugfs_off(void);
+void dw_edma_v0_debugfs_off(struct dw_edma_chip *chip);
#else
static inline void dw_edma_v0_debugfs_on(struct dw_edma_chip *chip)
{
}
-static inline void dw_edma_v0_debugfs_off(void)
+static inline void dw_edma_v0_debugfs_off(struct dw_edma_chip *chip)
{
}
#endif /* CONFIG_DEBUG_FS */
diff --git a/drivers/dma/dw-edma/dw-edma-v0-regs.h b/drivers/dma/dw-edma/dw-edma-v0-regs.h
index dfd70e223c2f..e175f7b20480 100644
--- a/drivers/dma/dw-edma/dw-edma-v0-regs.h
+++ b/drivers/dma/dw-edma/dw-edma-v0-regs.h
@@ -25,134 +25,209 @@
#define EDMA_V0_CH_EVEN_MSI_DATA_MASK GENMASK(15, 0)
struct dw_edma_v0_ch_regs {
- u32 ch_control1; /* 0x000 */
- u32 ch_control2; /* 0x004 */
- u32 transfer_size; /* 0x008 */
- u32 sar_low; /* 0x00c */
- u32 sar_high; /* 0x010 */
- u32 dar_low; /* 0x014 */
- u32 dar_high; /* 0x018 */
- u32 llp_low; /* 0x01c */
- u32 llp_high; /* 0x020 */
-};
+ u32 ch_control1; /* 0x0000 */
+ u32 ch_control2; /* 0x0004 */
+ u32 transfer_size; /* 0x0008 */
+ union {
+ u64 reg; /* 0x000c..0x0010 */
+ struct {
+ u32 lsb; /* 0x000c */
+ u32 msb; /* 0x0010 */
+ };
+ } sar;
+ union {
+ u64 reg; /* 0x0014..0x0018 */
+ struct {
+ u32 lsb; /* 0x0014 */
+ u32 msb; /* 0x0018 */
+ };
+ } dar;
+ union {
+ u64 reg; /* 0x001c..0x0020 */
+ struct {
+ u32 lsb; /* 0x001c */
+ u32 msb; /* 0x0020 */
+ };
+ } llp;
+} __packed;
struct dw_edma_v0_ch {
- struct dw_edma_v0_ch_regs wr; /* 0x200 */
- u32 padding_1[55]; /* [0x224..0x2fc] */
- struct dw_edma_v0_ch_regs rd; /* 0x300 */
- u32 padding_2[55]; /* [0x324..0x3fc] */
-};
+ struct dw_edma_v0_ch_regs wr; /* 0x0200 */
+ u32 padding_1[55]; /* 0x0224..0x02fc */
+ struct dw_edma_v0_ch_regs rd; /* 0x0300 */
+ u32 padding_2[55]; /* 0x0324..0x03fc */
+} __packed;
struct dw_edma_v0_unroll {
- u32 padding_1; /* 0x0f8 */
- u32 wr_engine_chgroup; /* 0x100 */
- u32 rd_engine_chgroup; /* 0x104 */
- u32 wr_engine_hshake_cnt_low; /* 0x108 */
- u32 wr_engine_hshake_cnt_high; /* 0x10c */
- u32 padding_2[2]; /* [0x110..0x114] */
- u32 rd_engine_hshake_cnt_low; /* 0x118 */
- u32 rd_engine_hshake_cnt_high; /* 0x11c */
- u32 padding_3[2]; /* [0x120..0x124] */
- u32 wr_ch0_pwr_en; /* 0x128 */
- u32 wr_ch1_pwr_en; /* 0x12c */
- u32 wr_ch2_pwr_en; /* 0x130 */
- u32 wr_ch3_pwr_en; /* 0x134 */
- u32 wr_ch4_pwr_en; /* 0x138 */
- u32 wr_ch5_pwr_en; /* 0x13c */
- u32 wr_ch6_pwr_en; /* 0x140 */
- u32 wr_ch7_pwr_en; /* 0x144 */
- u32 padding_4[8]; /* [0x148..0x164] */
- u32 rd_ch0_pwr_en; /* 0x168 */
- u32 rd_ch1_pwr_en; /* 0x16c */
- u32 rd_ch2_pwr_en; /* 0x170 */
- u32 rd_ch3_pwr_en; /* 0x174 */
- u32 rd_ch4_pwr_en; /* 0x178 */
- u32 rd_ch5_pwr_en; /* 0x18c */
- u32 rd_ch6_pwr_en; /* 0x180 */
- u32 rd_ch7_pwr_en; /* 0x184 */
- u32 padding_5[30]; /* [0x188..0x1fc] */
- struct dw_edma_v0_ch ch[EDMA_V0_MAX_NR_CH]; /* [0x200..0x1120] */
-};
+ u32 padding_1; /* 0x00f8 */
+ u32 wr_engine_chgroup; /* 0x0100 */
+ u32 rd_engine_chgroup; /* 0x0104 */
+ union {
+ u64 reg; /* 0x0108..0x010c */
+ struct {
+ u32 lsb; /* 0x0108 */
+ u32 msb; /* 0x010c */
+ };
+ } wr_engine_hshake_cnt;
+ u32 padding_2[2]; /* 0x0110..0x0114 */
+ union {
+ u64 reg; /* 0x0120..0x0124 */
+ struct {
+ u32 lsb; /* 0x0120 */
+ u32 msb; /* 0x0124 */
+ };
+ } rd_engine_hshake_cnt;
+ u32 padding_3[2]; /* 0x0120..0x0124 */
+ u32 wr_ch0_pwr_en; /* 0x0128 */
+ u32 wr_ch1_pwr_en; /* 0x012c */
+ u32 wr_ch2_pwr_en; /* 0x0130 */
+ u32 wr_ch3_pwr_en; /* 0x0134 */
+ u32 wr_ch4_pwr_en; /* 0x0138 */
+ u32 wr_ch5_pwr_en; /* 0x013c */
+ u32 wr_ch6_pwr_en; /* 0x0140 */
+ u32 wr_ch7_pwr_en; /* 0x0144 */
+ u32 padding_4[8]; /* 0x0148..0x0164 */
+ u32 rd_ch0_pwr_en; /* 0x0168 */
+ u32 rd_ch1_pwr_en; /* 0x016c */
+ u32 rd_ch2_pwr_en; /* 0x0170 */
+ u32 rd_ch3_pwr_en; /* 0x0174 */
+ u32 rd_ch4_pwr_en; /* 0x0178 */
+ u32 rd_ch5_pwr_en; /* 0x018c */
+ u32 rd_ch6_pwr_en; /* 0x0180 */
+ u32 rd_ch7_pwr_en; /* 0x0184 */
+ u32 padding_5[30]; /* 0x0188..0x01fc */
+ struct dw_edma_v0_ch ch[EDMA_V0_MAX_NR_CH]; /* 0x0200..0x1120 */
+} __packed;
struct dw_edma_v0_legacy {
- u32 viewport_sel; /* 0x0f8 */
- struct dw_edma_v0_ch_regs ch; /* [0x100..0x120] */
-};
+ u32 viewport_sel; /* 0x00f8 */
+ struct dw_edma_v0_ch_regs ch; /* 0x0100..0x0120 */
+} __packed;
struct dw_edma_v0_regs {
/* eDMA global registers */
- u32 ctrl_data_arb_prior; /* 0x000 */
- u32 padding_1; /* 0x004 */
- u32 ctrl; /* 0x008 */
- u32 wr_engine_en; /* 0x00c */
- u32 wr_doorbell; /* 0x010 */
- u32 padding_2; /* 0x014 */
- u32 wr_ch_arb_weight_low; /* 0x018 */
- u32 wr_ch_arb_weight_high; /* 0x01c */
- u32 padding_3[3]; /* [0x020..0x028] */
- u32 rd_engine_en; /* 0x02c */
- u32 rd_doorbell; /* 0x030 */
- u32 padding_4; /* 0x034 */
- u32 rd_ch_arb_weight_low; /* 0x038 */
- u32 rd_ch_arb_weight_high; /* 0x03c */
- u32 padding_5[3]; /* [0x040..0x048] */
+ u32 ctrl_data_arb_prior; /* 0x0000 */
+ u32 padding_1; /* 0x0004 */
+ u32 ctrl; /* 0x0008 */
+ u32 wr_engine_en; /* 0x000c */
+ u32 wr_doorbell; /* 0x0010 */
+ u32 padding_2; /* 0x0014 */
+ union {
+ u64 reg; /* 0x0018..0x001c */
+ struct {
+ u32 lsb; /* 0x0018 */
+ u32 msb; /* 0x001c */
+ };
+ } wr_ch_arb_weight;
+ u32 padding_3[3]; /* 0x0020..0x0028 */
+ u32 rd_engine_en; /* 0x002c */
+ u32 rd_doorbell; /* 0x0030 */
+ u32 padding_4; /* 0x0034 */
+ union {
+ u64 reg; /* 0x0038..0x003c */
+ struct {
+ u32 lsb; /* 0x0038 */
+ u32 msb; /* 0x003c */
+ };
+ } rd_ch_arb_weight;
+ u32 padding_5[3]; /* 0x0040..0x0048 */
/* eDMA interrupts registers */
- u32 wr_int_status; /* 0x04c */
- u32 padding_6; /* 0x050 */
- u32 wr_int_mask; /* 0x054 */
- u32 wr_int_clear; /* 0x058 */
- u32 wr_err_status; /* 0x05c */
- u32 wr_done_imwr_low; /* 0x060 */
- u32 wr_done_imwr_high; /* 0x064 */
- u32 wr_abort_imwr_low; /* 0x068 */
- u32 wr_abort_imwr_high; /* 0x06c */
- u32 wr_ch01_imwr_data; /* 0x070 */
- u32 wr_ch23_imwr_data; /* 0x074 */
- u32 wr_ch45_imwr_data; /* 0x078 */
- u32 wr_ch67_imwr_data; /* 0x07c */
- u32 padding_7[4]; /* [0x080..0x08c] */
- u32 wr_linked_list_err_en; /* 0x090 */
- u32 padding_8[3]; /* [0x094..0x09c] */
- u32 rd_int_status; /* 0x0a0 */
- u32 padding_9; /* 0x0a4 */
- u32 rd_int_mask; /* 0x0a8 */
- u32 rd_int_clear; /* 0x0ac */
- u32 padding_10; /* 0x0b0 */
- u32 rd_err_status_low; /* 0x0b4 */
- u32 rd_err_status_high; /* 0x0b8 */
- u32 padding_11[2]; /* [0x0bc..0x0c0] */
- u32 rd_linked_list_err_en; /* 0x0c4 */
- u32 padding_12; /* 0x0c8 */
- u32 rd_done_imwr_low; /* 0x0cc */
- u32 rd_done_imwr_high; /* 0x0d0 */
- u32 rd_abort_imwr_low; /* 0x0d4 */
- u32 rd_abort_imwr_high; /* 0x0d8 */
- u32 rd_ch01_imwr_data; /* 0x0dc */
- u32 rd_ch23_imwr_data; /* 0x0e0 */
- u32 rd_ch45_imwr_data; /* 0x0e4 */
- u32 rd_ch67_imwr_data; /* 0x0e8 */
- u32 padding_13[4]; /* [0x0ec..0x0f8] */
+ u32 wr_int_status; /* 0x004c */
+ u32 padding_6; /* 0x0050 */
+ u32 wr_int_mask; /* 0x0054 */
+ u32 wr_int_clear; /* 0x0058 */
+ u32 wr_err_status; /* 0x005c */
+ union {
+ u64 reg; /* 0x0060..0x0064 */
+ struct {
+ u32 lsb; /* 0x0060 */
+ u32 msb; /* 0x0064 */
+ };
+ } wr_done_imwr;
+ union {
+ u64 reg; /* 0x0068..0x006c */
+ struct {
+ u32 lsb; /* 0x0068 */
+ u32 msb; /* 0x006c */
+ };
+ } wr_abort_imwr;
+ u32 wr_ch01_imwr_data; /* 0x0070 */
+ u32 wr_ch23_imwr_data; /* 0x0074 */
+ u32 wr_ch45_imwr_data; /* 0x0078 */
+ u32 wr_ch67_imwr_data; /* 0x007c */
+ u32 padding_7[4]; /* 0x0080..0x008c */
+ u32 wr_linked_list_err_en; /* 0x0090 */
+ u32 padding_8[3]; /* 0x0094..0x009c */
+ u32 rd_int_status; /* 0x00a0 */
+ u32 padding_9; /* 0x00a4 */
+ u32 rd_int_mask; /* 0x00a8 */
+ u32 rd_int_clear; /* 0x00ac */
+ u32 padding_10; /* 0x00b0 */
+ union {
+ u64 reg; /* 0x00b4..0x00b8 */
+ struct {
+ u32 lsb; /* 0x00b4 */
+ u32 msb; /* 0x00b8 */
+ };
+ } rd_err_status;
+ u32 padding_11[2]; /* 0x00bc..0x00c0 */
+ u32 rd_linked_list_err_en; /* 0x00c4 */
+ u32 padding_12; /* 0x00c8 */
+ union {
+ u64 reg; /* 0x00cc..0x00d0 */
+ struct {
+ u32 lsb; /* 0x00cc */
+ u32 msb; /* 0x00d0 */
+ };
+ } rd_done_imwr;
+ union {
+ u64 reg; /* 0x00d4..0x00d8 */
+ struct {
+ u32 lsb; /* 0x00d4 */
+ u32 msb; /* 0x00d8 */
+ };
+ } rd_abort_imwr;
+ u32 rd_ch01_imwr_data; /* 0x00dc */
+ u32 rd_ch23_imwr_data; /* 0x00e0 */
+ u32 rd_ch45_imwr_data; /* 0x00e4 */
+ u32 rd_ch67_imwr_data; /* 0x00e8 */
+ u32 padding_13[4]; /* 0x00ec..0x00f8 */
/* eDMA channel context grouping */
union dw_edma_v0_type {
- struct dw_edma_v0_legacy legacy; /* [0x0f8..0x120] */
- struct dw_edma_v0_unroll unroll; /* [0x0f8..0x1120] */
+ struct dw_edma_v0_legacy legacy; /* 0x00f8..0x0120 */
+ struct dw_edma_v0_unroll unroll; /* 0x00f8..0x1120 */
} type;
-};
+} __packed;
struct dw_edma_v0_lli {
u32 control;
u32 transfer_size;
- u32 sar_low;
- u32 sar_high;
- u32 dar_low;
- u32 dar_high;
-};
+ union {
+ u64 reg;
+ struct {
+ u32 lsb;
+ u32 msb;
+ };
+ } sar;
+ union {
+ u64 reg;
+ struct {
+ u32 lsb;
+ u32 msb;
+ };
+ } dar;
+} __packed;
struct dw_edma_v0_llp {
u32 control;
u32 reserved;
- u32 llp_low;
- u32 llp_high;
-};
+ union {
+ u64 reg;
+ struct {
+ u32 lsb;
+ u32 msb;
+ };
+ } llp;
+} __packed;
#endif /* _DW_EDMA_V0_REGS_H */
diff --git a/drivers/dma/dw/Kconfig b/drivers/dma/dw/Kconfig
index e5162690de8f..db25f9b7778c 100644
--- a/drivers/dma/dw/Kconfig
+++ b/drivers/dma/dw/Kconfig
@@ -10,6 +10,7 @@ config DW_DMAC_CORE
config DW_DMAC
tristate "Synopsys DesignWare AHB DMA platform driver"
+ depends on HAS_IOMEM
select DW_DMAC_CORE
help
Support the Synopsys DesignWare AHB DMA controller. This
@@ -18,6 +19,7 @@ config DW_DMAC
config DW_DMAC_PCI
tristate "Synopsys DesignWare AHB DMA PCI driver"
depends on PCI
+ depends on HAS_IOMEM
select DW_DMAC_CORE
help
Support the Synopsys DesignWare AHB DMA controller on the
diff --git a/drivers/dma/idxd/Makefile b/drivers/dma/idxd/Makefile
index 8978b898d777..6d11558756f8 100644
--- a/drivers/dma/idxd/Makefile
+++ b/drivers/dma/idxd/Makefile
@@ -1,2 +1,4 @@
obj-$(CONFIG_INTEL_IDXD) += idxd.o
idxd-y := init.o irq.o device.o sysfs.o submit.o dma.o cdev.o
+
+idxd-$(CONFIG_INTEL_IDXD_PERFMON) += perfmon.o
diff --git a/drivers/dma/idxd/cdev.c b/drivers/dma/idxd/cdev.c
index 0db9b82ed8cf..302cba5ff779 100644
--- a/drivers/dma/idxd/cdev.c
+++ b/drivers/dma/idxd/cdev.c
@@ -39,15 +39,15 @@ struct idxd_user_context {
struct iommu_sva *sva;
};
-enum idxd_cdev_cleanup {
- CDEV_NORMAL = 0,
- CDEV_FAILED,
-};
-
static void idxd_cdev_dev_release(struct device *dev)
{
- dev_dbg(dev, "releasing cdev device\n");
- kfree(dev);
+ struct idxd_cdev *idxd_cdev = container_of(dev, struct idxd_cdev, dev);
+ struct idxd_cdev_context *cdev_ctx;
+ struct idxd_wq *wq = idxd_cdev->wq;
+
+ cdev_ctx = &ictx[wq->idxd->data->type];
+ ida_simple_remove(&cdev_ctx->minor_ida, idxd_cdev->minor);
+ kfree(idxd_cdev);
}
static struct device_type idxd_cdev_device_type = {
@@ -62,14 +62,11 @@ static inline struct idxd_cdev *inode_idxd_cdev(struct inode *inode)
return container_of(cdev, struct idxd_cdev, cdev);
}
-static inline struct idxd_wq *idxd_cdev_wq(struct idxd_cdev *idxd_cdev)
-{
- return container_of(idxd_cdev, struct idxd_wq, idxd_cdev);
-}
-
static inline struct idxd_wq *inode_wq(struct inode *inode)
{
- return idxd_cdev_wq(inode_idxd_cdev(inode));
+ struct idxd_cdev *idxd_cdev = inode_idxd_cdev(inode);
+
+ return idxd_cdev->wq;
}
static int idxd_cdev_open(struct inode *inode, struct file *filp)
@@ -220,11 +217,10 @@ static __poll_t idxd_cdev_poll(struct file *filp,
struct idxd_user_context *ctx = filp->private_data;
struct idxd_wq *wq = ctx->wq;
struct idxd_device *idxd = wq->idxd;
- struct idxd_cdev *idxd_cdev = &wq->idxd_cdev;
unsigned long flags;
__poll_t out = 0;
- poll_wait(filp, &idxd_cdev->err_queue, wait);
+ poll_wait(filp, &wq->err_queue, wait);
spin_lock_irqsave(&idxd->dev_lock, flags);
if (idxd->sw_err.valid)
out = EPOLLIN | EPOLLRDNORM;
@@ -243,101 +239,69 @@ static const struct file_operations idxd_cdev_fops = {
int idxd_cdev_get_major(struct idxd_device *idxd)
{
- return MAJOR(ictx[idxd->type].devt);
+ return MAJOR(ictx[idxd->data->type].devt);
}
-static int idxd_wq_cdev_dev_setup(struct idxd_wq *wq)
+int idxd_wq_add_cdev(struct idxd_wq *wq)
{
struct idxd_device *idxd = wq->idxd;
- struct idxd_cdev *idxd_cdev = &wq->idxd_cdev;
- struct idxd_cdev_context *cdev_ctx;
+ struct idxd_cdev *idxd_cdev;
+ struct cdev *cdev;
struct device *dev;
- int minor, rc;
+ struct idxd_cdev_context *cdev_ctx;
+ int rc, minor;
- idxd_cdev->dev = kzalloc(sizeof(*idxd_cdev->dev), GFP_KERNEL);
- if (!idxd_cdev->dev)
+ idxd_cdev = kzalloc(sizeof(*idxd_cdev), GFP_KERNEL);
+ if (!idxd_cdev)
return -ENOMEM;
- dev = idxd_cdev->dev;
- dev->parent = &idxd->pdev->dev;
- dev_set_name(dev, "%s/wq%u.%u", idxd_get_dev_name(idxd),
- idxd->id, wq->id);
- dev->bus = idxd_get_bus_type(idxd);
-
- cdev_ctx = &ictx[wq->idxd->type];
+ idxd_cdev->wq = wq;
+ cdev = &idxd_cdev->cdev;
+ dev = &idxd_cdev->dev;
+ cdev_ctx = &ictx[wq->idxd->data->type];
minor = ida_simple_get(&cdev_ctx->minor_ida, 0, MINORMASK, GFP_KERNEL);
if (minor < 0) {
- rc = minor;
- kfree(dev);
- goto ida_err;
- }
-
- dev->devt = MKDEV(MAJOR(cdev_ctx->devt), minor);
- dev->type = &idxd_cdev_device_type;
- rc = device_register(dev);
- if (rc < 0) {
- dev_err(&idxd->pdev->dev, "device register failed\n");
- goto dev_reg_err;
+ kfree(idxd_cdev);
+ return minor;
}
idxd_cdev->minor = minor;
- return 0;
-
- dev_reg_err:
- ida_simple_remove(&cdev_ctx->minor_ida, MINOR(dev->devt));
- put_device(dev);
- ida_err:
- idxd_cdev->dev = NULL;
- return rc;
-}
-
-static void idxd_wq_cdev_cleanup(struct idxd_wq *wq,
- enum idxd_cdev_cleanup cdev_state)
-{
- struct idxd_cdev *idxd_cdev = &wq->idxd_cdev;
- struct idxd_cdev_context *cdev_ctx;
-
- cdev_ctx = &ictx[wq->idxd->type];
- if (cdev_state == CDEV_NORMAL)
- cdev_del(&idxd_cdev->cdev);
- device_unregister(idxd_cdev->dev);
- /*
- * The device_type->release() will be called on the device and free
- * the allocated struct device. We can just forget it.
- */
- ida_simple_remove(&cdev_ctx->minor_ida, idxd_cdev->minor);
- idxd_cdev->dev = NULL;
- idxd_cdev->minor = -1;
-}
-
-int idxd_wq_add_cdev(struct idxd_wq *wq)
-{
- struct idxd_cdev *idxd_cdev = &wq->idxd_cdev;
- struct cdev *cdev = &idxd_cdev->cdev;
- struct device *dev;
- int rc;
+ device_initialize(dev);
+ dev->parent = &wq->conf_dev;
+ dev->bus = &dsa_bus_type;
+ dev->type = &idxd_cdev_device_type;
+ dev->devt = MKDEV(MAJOR(cdev_ctx->devt), minor);
- rc = idxd_wq_cdev_dev_setup(wq);
+ rc = dev_set_name(dev, "%s/wq%u.%u", idxd->data->name_prefix, idxd->id, wq->id);
if (rc < 0)
- return rc;
+ goto err;
- dev = idxd_cdev->dev;
+ wq->idxd_cdev = idxd_cdev;
cdev_init(cdev, &idxd_cdev_fops);
- cdev_set_parent(cdev, &dev->kobj);
- rc = cdev_add(cdev, dev->devt, 1);
+ rc = cdev_device_add(cdev, dev);
if (rc) {
dev_dbg(&wq->idxd->pdev->dev, "cdev_add failed: %d\n", rc);
- idxd_wq_cdev_cleanup(wq, CDEV_FAILED);
- return rc;
+ goto err;
}
- init_waitqueue_head(&idxd_cdev->err_queue);
return 0;
+
+ err:
+ put_device(dev);
+ wq->idxd_cdev = NULL;
+ return rc;
}
void idxd_wq_del_cdev(struct idxd_wq *wq)
{
- idxd_wq_cdev_cleanup(wq, CDEV_NORMAL);
+ struct idxd_cdev *idxd_cdev;
+ struct idxd_cdev_context *cdev_ctx;
+
+ cdev_ctx = &ictx[wq->idxd->data->type];
+ idxd_cdev = wq->idxd_cdev;
+ wq->idxd_cdev = NULL;
+ cdev_device_del(&idxd_cdev->cdev, &idxd_cdev->dev);
+ put_device(&idxd_cdev->dev);
}
int idxd_cdev_register(void)
diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c
index 84a6ea60ecf0..420b93fe5feb 100644
--- a/drivers/dma/idxd/device.c
+++ b/drivers/dma/idxd/device.c
@@ -19,7 +19,7 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand,
/* Interrupt control bits */
void idxd_mask_msix_vector(struct idxd_device *idxd, int vec_id)
{
- struct irq_data *data = irq_get_irq_data(idxd->msix_entries[vec_id].vector);
+ struct irq_data *data = irq_get_irq_data(idxd->irq_entries[vec_id].vector);
pci_msi_mask_irq(data);
}
@@ -36,7 +36,7 @@ void idxd_mask_msix_vectors(struct idxd_device *idxd)
void idxd_unmask_msix_vector(struct idxd_device *idxd, int vec_id)
{
- struct irq_data *data = irq_get_irq_data(idxd->msix_entries[vec_id].vector);
+ struct irq_data *data = irq_get_irq_data(idxd->irq_entries[vec_id].vector);
pci_msi_unmask_irq(data);
}
@@ -47,6 +47,7 @@ void idxd_unmask_error_interrupts(struct idxd_device *idxd)
genctrl.bits = ioread32(idxd->reg_base + IDXD_GENCTRL_OFFSET);
genctrl.softerr_int_en = 1;
+ genctrl.halt_int_en = 1;
iowrite32(genctrl.bits, idxd->reg_base + IDXD_GENCTRL_OFFSET);
}
@@ -56,6 +57,7 @@ void idxd_mask_error_interrupts(struct idxd_device *idxd)
genctrl.bits = ioread32(idxd->reg_base + IDXD_GENCTRL_OFFSET);
genctrl.softerr_int_en = 0;
+ genctrl.halt_int_en = 0;
iowrite32(genctrl.bits, idxd->reg_base + IDXD_GENCTRL_OFFSET);
}
@@ -144,14 +146,8 @@ int idxd_wq_alloc_resources(struct idxd_wq *wq)
if (rc < 0)
return rc;
- if (idxd->type == IDXD_TYPE_DSA)
- align = 32;
- else if (idxd->type == IDXD_TYPE_IAX)
- align = 64;
- else
- return -ENODEV;
-
- wq->compls_size = num_descs * idxd->compl_size + align;
+ align = idxd->data->align;
+ wq->compls_size = num_descs * idxd->data->compl_size + align;
wq->compls_raw = dma_alloc_coherent(dev, wq->compls_size,
&wq->compls_addr_raw, GFP_KERNEL);
if (!wq->compls_raw) {
@@ -178,16 +174,14 @@ int idxd_wq_alloc_resources(struct idxd_wq *wq)
struct idxd_desc *desc = wq->descs[i];
desc->hw = wq->hw_descs[i];
- if (idxd->type == IDXD_TYPE_DSA)
+ if (idxd->data->type == IDXD_TYPE_DSA)
desc->completion = &wq->compls[i];
- else if (idxd->type == IDXD_TYPE_IAX)
+ else if (idxd->data->type == IDXD_TYPE_IAX)
desc->iax_completion = &wq->iax_compls[i];
- desc->compl_dma = wq->compls_addr + idxd->compl_size * i;
+ desc->compl_dma = wq->compls_addr + idxd->data->compl_size * i;
desc->id = i;
desc->wq = wq;
desc->cpu = -1;
- dma_async_tx_descriptor_init(&desc->txd, &wq->dma_chan);
- desc->txd.tx_submit = idxd_dma_tx_submit;
}
return 0;
@@ -282,6 +276,22 @@ void idxd_wq_drain(struct idxd_wq *wq)
idxd_cmd_exec(idxd, IDXD_CMD_DRAIN_WQ, operand, NULL);
}
+void idxd_wq_reset(struct idxd_wq *wq)
+{
+ struct idxd_device *idxd = wq->idxd;
+ struct device *dev = &idxd->pdev->dev;
+ u32 operand;
+
+ if (wq->state != IDXD_WQ_ENABLED) {
+ dev_dbg(dev, "WQ %d in wrong state: %d\n", wq->id, wq->state);
+ return;
+ }
+
+ operand = BIT(wq->id % 16) | ((wq->id / 16) << 16);
+ idxd_cmd_exec(idxd, IDXD_CMD_RESET_WQ, operand, NULL);
+ wq->state = IDXD_WQ_DISABLED;
+}
+
int idxd_wq_map_portal(struct idxd_wq *wq)
{
struct idxd_device *idxd = wq->idxd;
@@ -304,6 +314,19 @@ void idxd_wq_unmap_portal(struct idxd_wq *wq)
struct device *dev = &wq->idxd->pdev->dev;
devm_iounmap(dev, wq->portal);
+ wq->portal = NULL;
+}
+
+void idxd_wqs_unmap_portal(struct idxd_device *idxd)
+{
+ int i;
+
+ for (i = 0; i < idxd->max_wqs; i++) {
+ struct idxd_wq *wq = idxd->wqs[i];
+
+ if (wq->portal)
+ idxd_wq_unmap_portal(wq);
+ }
}
int idxd_wq_set_pasid(struct idxd_wq *wq, int pasid)
@@ -363,8 +386,6 @@ int idxd_wq_disable_pasid(struct idxd_wq *wq)
void idxd_wq_disable_cleanup(struct idxd_wq *wq)
{
struct idxd_device *idxd = wq->idxd;
- struct device *dev = &idxd->pdev->dev;
- int i, wq_offset;
lockdep_assert_held(&idxd->dev_lock);
memset(wq->wqcfg, 0, idxd->wqcfg_size);
@@ -376,14 +397,32 @@ void idxd_wq_disable_cleanup(struct idxd_wq *wq)
wq->ats_dis = 0;
clear_bit(WQ_FLAG_DEDICATED, &wq->flags);
memset(wq->name, 0, WQ_NAME_SIZE);
+}
- for (i = 0; i < WQCFG_STRIDES(idxd); i++) {
- wq_offset = WQCFG_OFFSET(idxd, wq->id, i);
- iowrite32(0, idxd->reg_base + wq_offset);
- dev_dbg(dev, "WQ[%d][%d][%#x]: %#x\n",
- wq->id, i, wq_offset,
- ioread32(idxd->reg_base + wq_offset));
- }
+static void idxd_wq_ref_release(struct percpu_ref *ref)
+{
+ struct idxd_wq *wq = container_of(ref, struct idxd_wq, wq_active);
+
+ complete(&wq->wq_dead);
+}
+
+int idxd_wq_init_percpu_ref(struct idxd_wq *wq)
+{
+ int rc;
+
+ memset(&wq->wq_active, 0, sizeof(wq->wq_active));
+ rc = percpu_ref_init(&wq->wq_active, idxd_wq_ref_release, 0, GFP_KERNEL);
+ if (rc < 0)
+ return rc;
+ reinit_completion(&wq->wq_dead);
+ return 0;
+}
+
+void idxd_wq_quiesce(struct idxd_wq *wq)
+{
+ percpu_ref_kill(&wq->wq_active);
+ wait_for_completion(&wq->wq_dead);
+ percpu_ref_exit(&wq->wq_active);
}
/* Device control bits */
@@ -426,13 +465,13 @@ int idxd_device_init_reset(struct idxd_device *idxd)
memset(&cmd, 0, sizeof(cmd));
cmd.cmd = IDXD_CMD_RESET_DEVICE;
dev_dbg(dev, "%s: sending reset for init.\n", __func__);
- spin_lock_irqsave(&idxd->dev_lock, flags);
+ spin_lock_irqsave(&idxd->cmd_lock, flags);
iowrite32(cmd.bits, idxd->reg_base + IDXD_CMD_OFFSET);
while (ioread32(idxd->reg_base + IDXD_CMDSTS_OFFSET) &
IDXD_CMDSTS_ACTIVE)
cpu_relax();
- spin_unlock_irqrestore(&idxd->dev_lock, flags);
+ spin_unlock_irqrestore(&idxd->cmd_lock, flags);
return 0;
}
@@ -445,7 +484,8 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand,
if (idxd_device_is_halted(idxd)) {
dev_warn(&idxd->pdev->dev, "Device is HALTED!\n");
- *status = IDXD_CMDSTS_HW_ERR;
+ if (status)
+ *status = IDXD_CMDSTS_HW_ERR;
return;
}
@@ -454,10 +494,10 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand,
cmd.operand = operand;
cmd.int_req = 1;
- spin_lock_irqsave(&idxd->dev_lock, flags);
+ spin_lock_irqsave(&idxd->cmd_lock, flags);
wait_event_lock_irq(idxd->cmd_waitq,
!test_bit(IDXD_FLAG_CMD_RUNNING, &idxd->flags),
- idxd->dev_lock);
+ idxd->cmd_lock);
dev_dbg(&idxd->pdev->dev, "%s: sending cmd: %#x op: %#x\n",
__func__, cmd_code, operand);
@@ -471,9 +511,9 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand,
* After command submitted, release lock and go to sleep until
* the command completes via interrupt.
*/
- spin_unlock_irqrestore(&idxd->dev_lock, flags);
+ spin_unlock_irqrestore(&idxd->cmd_lock, flags);
wait_for_completion(&done);
- spin_lock_irqsave(&idxd->dev_lock, flags);
+ spin_lock_irqsave(&idxd->cmd_lock, flags);
if (status) {
*status = ioread32(idxd->reg_base + IDXD_CMDSTS_OFFSET);
idxd->cmd_status = *status & GENMASK(7, 0);
@@ -482,7 +522,7 @@ static void idxd_cmd_exec(struct idxd_device *idxd, int cmd_code, u32 operand,
__clear_bit(IDXD_FLAG_CMD_RUNNING, &idxd->flags);
/* Wake up other pending commands */
wake_up(&idxd->cmd_waitq);
- spin_unlock_irqrestore(&idxd->dev_lock, flags);
+ spin_unlock_irqrestore(&idxd->cmd_lock, flags);
}
int idxd_device_enable(struct idxd_device *idxd)
@@ -515,7 +555,7 @@ void idxd_device_wqs_clear_state(struct idxd_device *idxd)
lockdep_assert_held(&idxd->dev_lock);
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
if (wq->state == IDXD_WQ_ENABLED) {
idxd_wq_disable_cleanup(wq);
@@ -573,7 +613,108 @@ void idxd_device_drain_pasid(struct idxd_device *idxd, int pasid)
dev_dbg(dev, "pasid %d drained\n", pasid);
}
+int idxd_device_request_int_handle(struct idxd_device *idxd, int idx, int *handle,
+ enum idxd_interrupt_type irq_type)
+{
+ struct device *dev = &idxd->pdev->dev;
+ u32 operand, status;
+
+ if (!(idxd->hw.cmd_cap & BIT(IDXD_CMD_REQUEST_INT_HANDLE)))
+ return -EOPNOTSUPP;
+
+ dev_dbg(dev, "get int handle, idx %d\n", idx);
+
+ operand = idx & GENMASK(15, 0);
+ if (irq_type == IDXD_IRQ_IMS)
+ operand |= CMD_INT_HANDLE_IMS;
+
+ dev_dbg(dev, "cmd: %u operand: %#x\n", IDXD_CMD_REQUEST_INT_HANDLE, operand);
+
+ idxd_cmd_exec(idxd, IDXD_CMD_REQUEST_INT_HANDLE, operand, &status);
+
+ if ((status & IDXD_CMDSTS_ERR_MASK) != IDXD_CMDSTS_SUCCESS) {
+ dev_dbg(dev, "request int handle failed: %#x\n", status);
+ return -ENXIO;
+ }
+
+ *handle = (status >> IDXD_CMDSTS_RES_SHIFT) & GENMASK(15, 0);
+
+ dev_dbg(dev, "int handle acquired: %u\n", *handle);
+ return 0;
+}
+
+int idxd_device_release_int_handle(struct idxd_device *idxd, int handle,
+ enum idxd_interrupt_type irq_type)
+{
+ struct device *dev = &idxd->pdev->dev;
+ u32 operand, status;
+ union idxd_command_reg cmd;
+ unsigned long flags;
+
+ if (!(idxd->hw.cmd_cap & BIT(IDXD_CMD_RELEASE_INT_HANDLE)))
+ return -EOPNOTSUPP;
+
+ dev_dbg(dev, "release int handle, handle %d\n", handle);
+
+ memset(&cmd, 0, sizeof(cmd));
+ operand = handle & GENMASK(15, 0);
+
+ if (irq_type == IDXD_IRQ_IMS)
+ operand |= CMD_INT_HANDLE_IMS;
+
+ cmd.cmd = IDXD_CMD_RELEASE_INT_HANDLE;
+ cmd.operand = operand;
+
+ dev_dbg(dev, "cmd: %u operand: %#x\n", IDXD_CMD_RELEASE_INT_HANDLE, operand);
+
+ spin_lock_irqsave(&idxd->cmd_lock, flags);
+ iowrite32(cmd.bits, idxd->reg_base + IDXD_CMD_OFFSET);
+
+ while (ioread32(idxd->reg_base + IDXD_CMDSTS_OFFSET) & IDXD_CMDSTS_ACTIVE)
+ cpu_relax();
+ status = ioread32(idxd->reg_base + IDXD_CMDSTS_OFFSET);
+ spin_unlock_irqrestore(&idxd->cmd_lock, flags);
+
+ if ((status & IDXD_CMDSTS_ERR_MASK) != IDXD_CMDSTS_SUCCESS) {
+ dev_dbg(dev, "release int handle failed: %#x\n", status);
+ return -ENXIO;
+ }
+
+ dev_dbg(dev, "int handle released.\n");
+ return 0;
+}
+
/* Device configuration bits */
+void idxd_msix_perm_setup(struct idxd_device *idxd)
+{
+ union msix_perm mperm;
+ int i, msixcnt;
+
+ msixcnt = pci_msix_vec_count(idxd->pdev);
+ if (msixcnt < 0)
+ return;
+
+ mperm.bits = 0;
+ mperm.pasid = idxd->pasid;
+ mperm.pasid_en = device_pasid_enabled(idxd);
+ for (i = 1; i < msixcnt; i++)
+ iowrite32(mperm.bits, idxd->reg_base + idxd->msix_perm_offset + i * 8);
+}
+
+void idxd_msix_perm_clear(struct idxd_device *idxd)
+{
+ union msix_perm mperm;
+ int i, msixcnt;
+
+ msixcnt = pci_msix_vec_count(idxd->pdev);
+ if (msixcnt < 0)
+ return;
+
+ mperm.bits = 0;
+ for (i = 1; i < msixcnt; i++)
+ iowrite32(mperm.bits, idxd->reg_base + idxd->msix_perm_offset + i * 8);
+}
+
static void idxd_group_config_write(struct idxd_group *group)
{
struct idxd_device *idxd = group->idxd;
@@ -624,7 +765,7 @@ static int idxd_groups_config_write(struct idxd_device *idxd)
ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET));
for (i = 0; i < idxd->max_groups; i++) {
- struct idxd_group *group = &idxd->groups[i];
+ struct idxd_group *group = idxd->groups[i];
idxd_group_config_write(group);
}
@@ -642,7 +783,14 @@ static int idxd_wq_config_write(struct idxd_wq *wq)
if (!wq->group)
return 0;
- memset(wq->wqcfg, 0, idxd->wqcfg_size);
+ /*
+ * Instead of memset the entire shadow copy of WQCFG, copy from the hardware after
+ * wq reset. This will copy back the sticky values that are present on some devices.
+ */
+ for (i = 0; i < WQCFG_STRIDES(idxd); i++) {
+ wq_offset = WQCFG_OFFSET(idxd, wq->id, i);
+ wq->wqcfg->bits[i] = ioread32(idxd->reg_base + wq_offset);
+ }
/* byte 0-3 */
wq->wqcfg->wq_size = wq->size;
@@ -696,7 +844,7 @@ static int idxd_wqs_config_write(struct idxd_device *idxd)
int i, rc;
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
rc = idxd_wq_config_write(wq);
if (rc < 0)
@@ -712,7 +860,7 @@ static void idxd_group_flags_setup(struct idxd_device *idxd)
/* TC-A 0 and TC-B 1 should be defaults */
for (i = 0; i < idxd->max_groups; i++) {
- struct idxd_group *group = &idxd->groups[i];
+ struct idxd_group *group = idxd->groups[i];
if (group->tc_a == -1)
group->tc_a = group->grpcfg.flags.tc_a = 0;
@@ -739,12 +887,12 @@ static int idxd_engines_setup(struct idxd_device *idxd)
struct idxd_group *group;
for (i = 0; i < idxd->max_groups; i++) {
- group = &idxd->groups[i];
+ group = idxd->groups[i];
group->grpcfg.engines = 0;
}
for (i = 0; i < idxd->max_engines; i++) {
- eng = &idxd->engines[i];
+ eng = idxd->engines[i];
group = eng->group;
if (!group)
@@ -768,13 +916,13 @@ static int idxd_wqs_setup(struct idxd_device *idxd)
struct device *dev = &idxd->pdev->dev;
for (i = 0; i < idxd->max_groups; i++) {
- group = &idxd->groups[i];
+ group = idxd->groups[i];
for (j = 0; j < 4; j++)
group->grpcfg.wqs[j] = 0;
}
for (i = 0; i < idxd->max_wqs; i++) {
- wq = &idxd->wqs[i];
+ wq = idxd->wqs[i];
group = wq->group;
if (!wq->group)
@@ -822,3 +970,119 @@ int idxd_device_config(struct idxd_device *idxd)
return 0;
}
+
+static int idxd_wq_load_config(struct idxd_wq *wq)
+{
+ struct idxd_device *idxd = wq->idxd;
+ struct device *dev = &idxd->pdev->dev;
+ int wqcfg_offset;
+ int i;
+
+ wqcfg_offset = WQCFG_OFFSET(idxd, wq->id, 0);
+ memcpy_fromio(wq->wqcfg, idxd->reg_base + wqcfg_offset, idxd->wqcfg_size);
+
+ wq->size = wq->wqcfg->wq_size;
+ wq->threshold = wq->wqcfg->wq_thresh;
+ if (wq->wqcfg->priv)
+ wq->type = IDXD_WQT_KERNEL;
+
+ /* The driver does not support shared WQ mode in read-only config yet */
+ if (wq->wqcfg->mode == 0 || wq->wqcfg->pasid_en)
+ return -EOPNOTSUPP;
+
+ set_bit(WQ_FLAG_DEDICATED, &wq->flags);
+
+ wq->priority = wq->wqcfg->priority;
+
+ for (i = 0; i < WQCFG_STRIDES(idxd); i++) {
+ wqcfg_offset = WQCFG_OFFSET(idxd, wq->id, i);
+ dev_dbg(dev, "WQ[%d][%d][%#x]: %#x\n", wq->id, i, wqcfg_offset, wq->wqcfg->bits[i]);
+ }
+
+ return 0;
+}
+
+static void idxd_group_load_config(struct idxd_group *group)
+{
+ struct idxd_device *idxd = group->idxd;
+ struct device *dev = &idxd->pdev->dev;
+ int i, j, grpcfg_offset;
+
+ /*
+ * Load WQS bit fields
+ * Iterate through all 256 bits 64 bits at a time
+ */
+ for (i = 0; i < GRPWQCFG_STRIDES; i++) {
+ struct idxd_wq *wq;
+
+ grpcfg_offset = GRPWQCFG_OFFSET(idxd, group->id, i);
+ group->grpcfg.wqs[i] = ioread64(idxd->reg_base + grpcfg_offset);
+ dev_dbg(dev, "GRPCFG wq[%d:%d: %#x]: %#llx\n",
+ group->id, i, grpcfg_offset, group->grpcfg.wqs[i]);
+
+ if (i * 64 >= idxd->max_wqs)
+ break;
+
+ /* Iterate through all 64 bits and check for wq set */
+ for (j = 0; j < 64; j++) {
+ int id = i * 64 + j;
+
+ /* No need to check beyond max wqs */
+ if (id >= idxd->max_wqs)
+ break;
+
+ /* Set group assignment for wq if wq bit is set */
+ if (group->grpcfg.wqs[i] & BIT(j)) {
+ wq = idxd->wqs[id];
+ wq->group = group;
+ }
+ }
+ }
+
+ grpcfg_offset = GRPENGCFG_OFFSET(idxd, group->id);
+ group->grpcfg.engines = ioread64(idxd->reg_base + grpcfg_offset);
+ dev_dbg(dev, "GRPCFG engs[%d: %#x]: %#llx\n", group->id,
+ grpcfg_offset, group->grpcfg.engines);
+
+ /* Iterate through all 64 bits to check engines set */
+ for (i = 0; i < 64; i++) {
+ if (i >= idxd->max_engines)
+ break;
+
+ if (group->grpcfg.engines & BIT(i)) {
+ struct idxd_engine *engine = idxd->engines[i];
+
+ engine->group = group;
+ }
+ }
+
+ grpcfg_offset = GRPFLGCFG_OFFSET(idxd, group->id);
+ group->grpcfg.flags.bits = ioread32(idxd->reg_base + grpcfg_offset);
+ dev_dbg(dev, "GRPFLAGS flags[%d: %#x]: %#x\n",
+ group->id, grpcfg_offset, group->grpcfg.flags.bits);
+}
+
+int idxd_device_load_config(struct idxd_device *idxd)
+{
+ union gencfg_reg reg;
+ int i, rc;
+
+ reg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET);
+ idxd->token_limit = reg.token_limit;
+
+ for (i = 0; i < idxd->max_groups; i++) {
+ struct idxd_group *group = idxd->groups[i];
+
+ idxd_group_load_config(group);
+ }
+
+ for (i = 0; i < idxd->max_wqs; i++) {
+ struct idxd_wq *wq = idxd->wqs[i];
+
+ rc = idxd_wq_load_config(wq);
+ if (rc < 0)
+ return rc;
+ }
+
+ return 0;
+}
diff --git a/drivers/dma/idxd/dma.c b/drivers/dma/idxd/dma.c
index a15e50126434..77439b645044 100644
--- a/drivers/dma/idxd/dma.c
+++ b/drivers/dma/idxd/dma.c
@@ -14,7 +14,10 @@
static inline struct idxd_wq *to_idxd_wq(struct dma_chan *c)
{
- return container_of(c, struct idxd_wq, dma_chan);
+ struct idxd_dma_chan *idxd_chan;
+
+ idxd_chan = container_of(c, struct idxd_dma_chan, chan);
+ return idxd_chan->wq;
}
void idxd_dma_complete_txd(struct idxd_desc *desc,
@@ -135,7 +138,7 @@ static void idxd_dma_issue_pending(struct dma_chan *dma_chan)
{
}
-dma_cookie_t idxd_dma_tx_submit(struct dma_async_tx_descriptor *tx)
+static dma_cookie_t idxd_dma_tx_submit(struct dma_async_tx_descriptor *tx)
{
struct dma_chan *c = tx->chan;
struct idxd_wq *wq = to_idxd_wq(c);
@@ -156,14 +159,25 @@ dma_cookie_t idxd_dma_tx_submit(struct dma_async_tx_descriptor *tx)
static void idxd_dma_release(struct dma_device *device)
{
+ struct idxd_dma_dev *idxd_dma = container_of(device, struct idxd_dma_dev, dma);
+
+ kfree(idxd_dma);
}
int idxd_register_dma_device(struct idxd_device *idxd)
{
- struct dma_device *dma = &idxd->dma_dev;
+ struct idxd_dma_dev *idxd_dma;
+ struct dma_device *dma;
+ struct device *dev = &idxd->pdev->dev;
+ int rc;
+ idxd_dma = kzalloc_node(sizeof(*idxd_dma), GFP_KERNEL, dev_to_node(dev));
+ if (!idxd_dma)
+ return -ENOMEM;
+
+ dma = &idxd_dma->dma;
INIT_LIST_HEAD(&dma->channels);
- dma->dev = &idxd->pdev->dev;
+ dma->dev = dev;
dma_cap_set(DMA_PRIVATE, dma->cap_mask);
dma_cap_set(DMA_COMPLETION_NO_ORDER, dma->cap_mask);
@@ -179,35 +193,72 @@ int idxd_register_dma_device(struct idxd_device *idxd)
dma->device_alloc_chan_resources = idxd_dma_alloc_chan_resources;
dma->device_free_chan_resources = idxd_dma_free_chan_resources;
- return dma_async_device_register(&idxd->dma_dev);
+ rc = dma_async_device_register(dma);
+ if (rc < 0) {
+ kfree(idxd_dma);
+ return rc;
+ }
+
+ idxd_dma->idxd = idxd;
+ /*
+ * This pointer is protected by the refs taken by the dma_chan. It will remain valid
+ * as long as there are outstanding channels.
+ */
+ idxd->idxd_dma = idxd_dma;
+ return 0;
}
void idxd_unregister_dma_device(struct idxd_device *idxd)
{
- dma_async_device_unregister(&idxd->dma_dev);
+ dma_async_device_unregister(&idxd->idxd_dma->dma);
}
int idxd_register_dma_channel(struct idxd_wq *wq)
{
struct idxd_device *idxd = wq->idxd;
- struct dma_device *dma = &idxd->dma_dev;
- struct dma_chan *chan = &wq->dma_chan;
- int rc;
+ struct dma_device *dma = &idxd->idxd_dma->dma;
+ struct device *dev = &idxd->pdev->dev;
+ struct idxd_dma_chan *idxd_chan;
+ struct dma_chan *chan;
+ int rc, i;
+
+ idxd_chan = kzalloc_node(sizeof(*idxd_chan), GFP_KERNEL, dev_to_node(dev));
+ if (!idxd_chan)
+ return -ENOMEM;
- memset(&wq->dma_chan, 0, sizeof(struct dma_chan));
+ chan = &idxd_chan->chan;
chan->device = dma;
list_add_tail(&chan->device_node, &dma->channels);
+
+ for (i = 0; i < wq->num_descs; i++) {
+ struct idxd_desc *desc = wq->descs[i];
+
+ dma_async_tx_descriptor_init(&desc->txd, chan);
+ desc->txd.tx_submit = idxd_dma_tx_submit;
+ }
+
rc = dma_async_device_channel_register(dma, chan);
- if (rc < 0)
+ if (rc < 0) {
+ kfree(idxd_chan);
return rc;
+ }
+
+ wq->idxd_chan = idxd_chan;
+ idxd_chan->wq = wq;
+ get_device(&wq->conf_dev);
return 0;
}
void idxd_unregister_dma_channel(struct idxd_wq *wq)
{
- struct dma_chan *chan = &wq->dma_chan;
+ struct idxd_dma_chan *idxd_chan = wq->idxd_chan;
+ struct dma_chan *chan = &idxd_chan->chan;
+ struct idxd_dma_dev *idxd_dma = wq->idxd->idxd_dma;
- dma_async_device_channel_unregister(&wq->idxd->dma_dev, chan);
+ dma_async_device_channel_unregister(&idxd_dma->dma, chan);
list_del(&chan->device_node);
+ kfree(wq->idxd_chan);
+ wq->idxd_chan = NULL;
+ put_device(&wq->conf_dev);
}
diff --git a/drivers/dma/idxd/idxd.h b/drivers/dma/idxd/idxd.h
index 81a0e65fd316..26482c7d4c3a 100644
--- a/drivers/dma/idxd/idxd.h
+++ b/drivers/dma/idxd/idxd.h
@@ -8,12 +8,18 @@
#include <linux/percpu-rwsem.h>
#include <linux/wait.h>
#include <linux/cdev.h>
+#include <linux/idr.h>
+#include <linux/pci.h>
+#include <linux/perf_event.h>
#include "registers.h"
#define IDXD_DRIVER_VERSION "1.00"
extern struct kmem_cache *idxd_desc_pool;
+struct idxd_device;
+struct idxd_wq;
+
#define IDXD_REG_TIMEOUT 50
#define IDXD_DRAIN_TIMEOUT 5000
@@ -25,6 +31,7 @@ enum idxd_type {
};
#define IDXD_NAME_SIZE 128
+#define IDXD_PMU_EVENT_MAX 64
struct idxd_device_driver {
struct device_driver drv;
@@ -33,6 +40,7 @@ struct idxd_device_driver {
struct idxd_irq_entry {
struct idxd_device *idxd;
int id;
+ int vector;
struct llist_head pending_llist;
struct list_head work_list;
/*
@@ -56,6 +64,31 @@ struct idxd_group {
int tc_b;
};
+struct idxd_pmu {
+ struct idxd_device *idxd;
+
+ struct perf_event *event_list[IDXD_PMU_EVENT_MAX];
+ int n_events;
+
+ DECLARE_BITMAP(used_mask, IDXD_PMU_EVENT_MAX);
+
+ struct pmu pmu;
+ char name[IDXD_NAME_SIZE];
+ int cpu;
+
+ int n_counters;
+ int counter_width;
+ int n_event_categories;
+
+ bool per_counter_caps_supported;
+ unsigned long supported_event_categories;
+
+ unsigned long supported_filters;
+ int n_filters;
+
+ struct hlist_node cpuhp_node;
+};
+
#define IDXD_MAX_PRIORITY 0xf
enum idxd_wq_state {
@@ -75,10 +108,10 @@ enum idxd_wq_type {
};
struct idxd_cdev {
+ struct idxd_wq *wq;
struct cdev cdev;
- struct device *dev;
+ struct device dev;
int minor;
- struct wait_queue_head err_queue;
};
#define IDXD_ALLOCATED_BATCH_SIZE 128U
@@ -96,10 +129,18 @@ enum idxd_complete_type {
IDXD_COMPLETE_DEV_FAIL,
};
+struct idxd_dma_chan {
+ struct dma_chan chan;
+ struct idxd_wq *wq;
+};
+
struct idxd_wq {
void __iomem *portal;
+ struct percpu_ref wq_active;
+ struct completion wq_dead;
struct device conf_dev;
- struct idxd_cdev idxd_cdev;
+ struct idxd_cdev *idxd_cdev;
+ struct wait_queue_head err_queue;
struct idxd_device *idxd;
int id;
enum idxd_wq_type type;
@@ -125,7 +166,7 @@ struct idxd_wq {
int compls_size;
struct idxd_desc **descs;
struct sbitmap_queue sbq;
- struct dma_chan dma_chan;
+ struct idxd_dma_chan *idxd_chan;
char name[WQ_NAME_SIZE + 1];
u64 max_xfer_bytes;
u32 max_batch_size;
@@ -147,6 +188,7 @@ struct idxd_hw {
union group_cap_reg group_cap;
union engine_cap_reg engine_cap;
struct opcap opcap;
+ u32 cmd_cap;
};
enum idxd_device_state {
@@ -162,9 +204,22 @@ enum idxd_device_flag {
IDXD_FLAG_PASID_ENABLED,
};
-struct idxd_device {
+struct idxd_dma_dev {
+ struct idxd_device *idxd;
+ struct dma_device dma;
+};
+
+struct idxd_driver_data {
+ const char *name_prefix;
enum idxd_type type;
+ struct device_type *dev_type;
+ int compl_size;
+ int align;
+};
+
+struct idxd_device {
struct device conf_dev;
+ struct idxd_driver_data *data;
struct list_head list;
struct idxd_hw hw;
enum idxd_device_state state;
@@ -177,10 +232,11 @@ struct idxd_device {
void __iomem *reg_base;
spinlock_t dev_lock; /* spinlock for device */
+ spinlock_t cmd_lock; /* spinlock for device commands */
struct completion *cmd_done;
- struct idxd_group *groups;
- struct idxd_wq *wqs;
- struct idxd_engine *engines;
+ struct idxd_group **groups;
+ struct idxd_wq **wqs;
+ struct idxd_engine **engines;
struct iommu_sva *sva;
unsigned int pasid;
@@ -202,17 +258,19 @@ struct idxd_device {
int token_limit;
int nr_tokens; /* non-reserved tokens */
unsigned int wqcfg_size;
- int compl_size;
union sw_err_reg sw_err;
wait_queue_head_t cmd_waitq;
- struct msix_entry *msix_entries;
int num_wq_irqs;
struct idxd_irq_entry *irq_entries;
- struct dma_device dma_dev;
+ struct idxd_dma_dev *idxd_dma;
struct workqueue_struct *wq;
struct work_struct work;
+
+ int *int_handles;
+
+ struct idxd_pmu *idxd_pmu;
};
/* IDXD software descriptor */
@@ -232,6 +290,7 @@ struct idxd_desc {
struct list_head list;
int id;
int cpu;
+ unsigned int vector;
struct idxd_wq *wq;
};
@@ -242,6 +301,44 @@ extern struct bus_type dsa_bus_type;
extern struct bus_type iax_bus_type;
extern bool support_enqcmd;
+extern struct ida idxd_ida;
+extern struct device_type dsa_device_type;
+extern struct device_type iax_device_type;
+extern struct device_type idxd_wq_device_type;
+extern struct device_type idxd_engine_device_type;
+extern struct device_type idxd_group_device_type;
+
+static inline bool is_dsa_dev(struct device *dev)
+{
+ return dev->type == &dsa_device_type;
+}
+
+static inline bool is_iax_dev(struct device *dev)
+{
+ return dev->type == &iax_device_type;
+}
+
+static inline bool is_idxd_dev(struct device *dev)
+{
+ return is_dsa_dev(dev) || is_iax_dev(dev);
+}
+
+static inline bool is_idxd_wq_dev(struct device *dev)
+{
+ return dev->type == &idxd_wq_device_type;
+}
+
+static inline bool is_idxd_wq_dmaengine(struct idxd_wq *wq)
+{
+ if (wq->type == IDXD_WQT_KERNEL && strcmp(wq->name, "dmaengine") == 0)
+ return true;
+ return false;
+}
+
+static inline bool is_idxd_wq_cdev(struct idxd_wq *wq)
+{
+ return wq->type == IDXD_WQT_USER;
+}
static inline bool wq_dedicated(struct idxd_wq *wq)
{
@@ -268,6 +365,11 @@ enum idxd_portal_prot {
IDXD_PORTAL_LIMITED,
};
+enum idxd_interrupt_type {
+ IDXD_IRQ_MSIX = 0,
+ IDXD_IRQ_IMS,
+};
+
static inline int idxd_get_wq_portal_offset(enum idxd_portal_prot prot)
{
return prot * 0x1000;
@@ -279,18 +381,6 @@ static inline int idxd_get_wq_portal_full_offset(int wq_id,
return ((wq_id * 4) << PAGE_SHIFT) + idxd_get_wq_portal_offset(prot);
}
-static inline void idxd_set_type(struct idxd_device *idxd)
-{
- struct pci_dev *pdev = idxd->pdev;
-
- if (pdev->device == PCI_DEVICE_ID_INTEL_DSA_SPR0)
- idxd->type = IDXD_TYPE_DSA;
- else if (pdev->device == PCI_DEVICE_ID_INTEL_IAX_SPR0)
- idxd->type = IDXD_TYPE_IAX;
- else
- idxd->type = IDXD_TYPE_UNKNOWN;
-}
-
static inline void idxd_wq_get(struct idxd_wq *wq)
{
wq->client_count++;
@@ -306,17 +396,17 @@ static inline int idxd_wq_refcount(struct idxd_wq *wq)
return wq->client_count;
};
-const char *idxd_get_dev_name(struct idxd_device *idxd);
int idxd_register_bus_type(void);
void idxd_unregister_bus_type(void);
-int idxd_setup_sysfs(struct idxd_device *idxd);
-void idxd_cleanup_sysfs(struct idxd_device *idxd);
+int idxd_register_devices(struct idxd_device *idxd);
+void idxd_unregister_devices(struct idxd_device *idxd);
int idxd_register_driver(void);
void idxd_unregister_driver(void);
-struct bus_type *idxd_get_bus_type(struct idxd_device *idxd);
+void idxd_wqs_quiesce(struct idxd_device *idxd);
/* device interrupt control */
-irqreturn_t idxd_irq_handler(int vec, void *data);
+void idxd_msix_perm_setup(struct idxd_device *idxd);
+void idxd_msix_perm_clear(struct idxd_device *idxd);
irqreturn_t idxd_misc_thread(int vec, void *data);
irqreturn_t idxd_wq_thread(int irq, void *data);
void idxd_mask_error_interrupts(struct idxd_device *idxd);
@@ -334,18 +424,27 @@ void idxd_device_cleanup(struct idxd_device *idxd);
int idxd_device_config(struct idxd_device *idxd);
void idxd_device_wqs_clear_state(struct idxd_device *idxd);
void idxd_device_drain_pasid(struct idxd_device *idxd, int pasid);
+int idxd_device_load_config(struct idxd_device *idxd);
+int idxd_device_request_int_handle(struct idxd_device *idxd, int idx, int *handle,
+ enum idxd_interrupt_type irq_type);
+int idxd_device_release_int_handle(struct idxd_device *idxd, int handle,
+ enum idxd_interrupt_type irq_type);
/* work queue control */
+void idxd_wqs_unmap_portal(struct idxd_device *idxd);
int idxd_wq_alloc_resources(struct idxd_wq *wq);
void idxd_wq_free_resources(struct idxd_wq *wq);
int idxd_wq_enable(struct idxd_wq *wq);
int idxd_wq_disable(struct idxd_wq *wq);
void idxd_wq_drain(struct idxd_wq *wq);
+void idxd_wq_reset(struct idxd_wq *wq);
int idxd_wq_map_portal(struct idxd_wq *wq);
void idxd_wq_unmap_portal(struct idxd_wq *wq);
void idxd_wq_disable_cleanup(struct idxd_wq *wq);
int idxd_wq_set_pasid(struct idxd_wq *wq, int pasid);
int idxd_wq_disable_pasid(struct idxd_wq *wq);
+void idxd_wq_quiesce(struct idxd_wq *wq);
+int idxd_wq_init_percpu_ref(struct idxd_wq *wq);
/* submission */
int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc);
@@ -360,7 +459,6 @@ void idxd_unregister_dma_channel(struct idxd_wq *wq);
void idxd_parse_completion_status(u8 status, enum dmaengine_tx_result *res);
void idxd_dma_complete_txd(struct idxd_desc *desc,
enum idxd_complete_type comp_type);
-dma_cookie_t idxd_dma_tx_submit(struct dma_async_tx_descriptor *tx);
/* cdev */
int idxd_cdev_register(void);
@@ -369,4 +467,19 @@ int idxd_cdev_get_major(struct idxd_device *idxd);
int idxd_wq_add_cdev(struct idxd_wq *wq);
void idxd_wq_del_cdev(struct idxd_wq *wq);
+/* perfmon */
+#if IS_ENABLED(CONFIG_INTEL_IDXD_PERFMON)
+int perfmon_pmu_init(struct idxd_device *idxd);
+void perfmon_pmu_remove(struct idxd_device *idxd);
+void perfmon_counter_overflow(struct idxd_device *idxd);
+void perfmon_init(void);
+void perfmon_exit(void);
+#else
+static inline int perfmon_pmu_init(struct idxd_device *idxd) { return 0; }
+static inline void perfmon_pmu_remove(struct idxd_device *idxd) {}
+static inline void perfmon_counter_overflow(struct idxd_device *idxd) {}
+static inline void perfmon_init(void) {}
+static inline void perfmon_exit(void) {}
+#endif
+
#endif
diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c
index 085a0c3b62c6..2a926bef87f2 100644
--- a/drivers/dma/idxd/init.c
+++ b/drivers/dma/idxd/init.c
@@ -21,6 +21,7 @@
#include "../dmaengine.h"
#include "registers.h"
#include "idxd.h"
+#include "perfmon.h"
MODULE_VERSION(IDXD_DRIVER_VERSION);
MODULE_LICENSE("GPL v2");
@@ -33,60 +34,53 @@ MODULE_PARM_DESC(sva, "Toggle SVA support on/off");
#define DRV_NAME "idxd"
bool support_enqcmd;
-
-static struct idr idxd_idrs[IDXD_TYPE_MAX];
-static DEFINE_MUTEX(idxd_idr_lock);
+DEFINE_IDA(idxd_ida);
+
+static struct idxd_driver_data idxd_driver_data[] = {
+ [IDXD_TYPE_DSA] = {
+ .name_prefix = "dsa",
+ .type = IDXD_TYPE_DSA,
+ .compl_size = sizeof(struct dsa_completion_record),
+ .align = 32,
+ .dev_type = &dsa_device_type,
+ },
+ [IDXD_TYPE_IAX] = {
+ .name_prefix = "iax",
+ .type = IDXD_TYPE_IAX,
+ .compl_size = sizeof(struct iax_completion_record),
+ .align = 64,
+ .dev_type = &iax_device_type,
+ },
+};
static struct pci_device_id idxd_pci_tbl[] = {
/* DSA ver 1.0 platforms */
- { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_DSA_SPR0) },
+ { PCI_DEVICE_DATA(INTEL, DSA_SPR0, &idxd_driver_data[IDXD_TYPE_DSA]) },
/* IAX ver 1.0 platforms */
- { PCI_VDEVICE(INTEL, PCI_DEVICE_ID_INTEL_IAX_SPR0) },
+ { PCI_DEVICE_DATA(INTEL, IAX_SPR0, &idxd_driver_data[IDXD_TYPE_IAX]) },
{ 0, }
};
MODULE_DEVICE_TABLE(pci, idxd_pci_tbl);
-static char *idxd_name[] = {
- "dsa",
- "iax"
-};
-
-const char *idxd_get_dev_name(struct idxd_device *idxd)
-{
- return idxd_name[idxd->type];
-}
-
static int idxd_setup_interrupts(struct idxd_device *idxd)
{
struct pci_dev *pdev = idxd->pdev;
struct device *dev = &pdev->dev;
- struct msix_entry *msix;
struct idxd_irq_entry *irq_entry;
int i, msixcnt;
int rc = 0;
- union msix_perm mperm;
msixcnt = pci_msix_vec_count(pdev);
if (msixcnt < 0) {
dev_err(dev, "Not MSI-X interrupt capable.\n");
- goto err_no_irq;
+ return -ENOSPC;
}
- idxd->msix_entries = devm_kzalloc(dev, sizeof(struct msix_entry) *
- msixcnt, GFP_KERNEL);
- if (!idxd->msix_entries) {
- rc = -ENOMEM;
- goto err_no_irq;
- }
-
- for (i = 0; i < msixcnt; i++)
- idxd->msix_entries[i].entry = i;
-
- rc = pci_enable_msix_exact(pdev, idxd->msix_entries, msixcnt);
- if (rc) {
- dev_err(dev, "Failed enabling %d MSIX entries.\n", msixcnt);
- goto err_no_irq;
+ rc = pci_alloc_irq_vectors(pdev, msixcnt, msixcnt, PCI_IRQ_MSIX);
+ if (rc != msixcnt) {
+ dev_err(dev, "Failed enabling %d MSIX entries: %d\n", msixcnt, rc);
+ return -ENOSPC;
}
dev_dbg(dev, "Enabled %d msix vectors\n", msixcnt);
@@ -94,126 +88,266 @@ static int idxd_setup_interrupts(struct idxd_device *idxd)
* We implement 1 completion list per MSI-X entry except for
* entry 0, which is for errors and others.
*/
- idxd->irq_entries = devm_kcalloc(dev, msixcnt,
- sizeof(struct idxd_irq_entry),
- GFP_KERNEL);
+ idxd->irq_entries = kcalloc_node(msixcnt, sizeof(struct idxd_irq_entry),
+ GFP_KERNEL, dev_to_node(dev));
if (!idxd->irq_entries) {
rc = -ENOMEM;
- goto err_no_irq;
+ goto err_irq_entries;
}
for (i = 0; i < msixcnt; i++) {
idxd->irq_entries[i].id = i;
idxd->irq_entries[i].idxd = idxd;
+ idxd->irq_entries[i].vector = pci_irq_vector(pdev, i);
spin_lock_init(&idxd->irq_entries[i].list_lock);
}
- msix = &idxd->msix_entries[0];
irq_entry = &idxd->irq_entries[0];
- rc = devm_request_threaded_irq(dev, msix->vector, idxd_irq_handler,
- idxd_misc_thread, 0, "idxd-misc",
- irq_entry);
+ rc = request_threaded_irq(irq_entry->vector, NULL, idxd_misc_thread,
+ 0, "idxd-misc", irq_entry);
if (rc < 0) {
dev_err(dev, "Failed to allocate misc interrupt.\n");
- goto err_no_irq;
+ goto err_misc_irq;
}
- dev_dbg(dev, "Allocated idxd-misc handler on msix vector %d\n",
- msix->vector);
+ dev_dbg(dev, "Allocated idxd-misc handler on msix vector %d\n", irq_entry->vector);
/* first MSI-X entry is not for wq interrupts */
idxd->num_wq_irqs = msixcnt - 1;
for (i = 1; i < msixcnt; i++) {
- msix = &idxd->msix_entries[i];
irq_entry = &idxd->irq_entries[i];
init_llist_head(&idxd->irq_entries[i].pending_llist);
INIT_LIST_HEAD(&idxd->irq_entries[i].work_list);
- rc = devm_request_threaded_irq(dev, msix->vector,
- idxd_irq_handler,
- idxd_wq_thread, 0,
- "idxd-portal", irq_entry);
+ rc = request_threaded_irq(irq_entry->vector, NULL,
+ idxd_wq_thread, 0, "idxd-portal", irq_entry);
if (rc < 0) {
- dev_err(dev, "Failed to allocate irq %d.\n",
- msix->vector);
- goto err_no_irq;
+ dev_err(dev, "Failed to allocate irq %d.\n", irq_entry->vector);
+ goto err_wq_irqs;
+ }
+
+ dev_dbg(dev, "Allocated idxd-msix %d for vector %d\n", i, irq_entry->vector);
+ if (idxd->hw.cmd_cap & BIT(IDXD_CMD_REQUEST_INT_HANDLE)) {
+ /*
+ * The MSIX vector enumeration starts at 1 with vector 0 being the
+ * misc interrupt that handles non I/O completion events. The
+ * interrupt handles are for IMS enumeration on guest. The misc
+ * interrupt vector does not require a handle and therefore we start
+ * the int_handles at index 0. Since 'i' starts at 1, the first
+ * int_handles index will be 0.
+ */
+ rc = idxd_device_request_int_handle(idxd, i, &idxd->int_handles[i - 1],
+ IDXD_IRQ_MSIX);
+ if (rc < 0) {
+ free_irq(irq_entry->vector, irq_entry);
+ goto err_wq_irqs;
+ }
+ dev_dbg(dev, "int handle requested: %u\n", idxd->int_handles[i - 1]);
}
- dev_dbg(dev, "Allocated idxd-msix %d for vector %d\n",
- i, msix->vector);
}
idxd_unmask_error_interrupts(idxd);
-
- /* Setup MSIX permission table */
- mperm.bits = 0;
- mperm.pasid = idxd->pasid;
- mperm.pasid_en = device_pasid_enabled(idxd);
- for (i = 1; i < msixcnt; i++)
- iowrite32(mperm.bits, idxd->reg_base + idxd->msix_perm_offset + i * 8);
-
+ idxd_msix_perm_setup(idxd);
return 0;
- err_no_irq:
+ err_wq_irqs:
+ while (--i >= 0) {
+ irq_entry = &idxd->irq_entries[i];
+ free_irq(irq_entry->vector, irq_entry);
+ if (i != 0)
+ idxd_device_release_int_handle(idxd,
+ idxd->int_handles[i], IDXD_IRQ_MSIX);
+ }
+ err_misc_irq:
/* Disable error interrupt generation */
idxd_mask_error_interrupts(idxd);
- pci_disable_msix(pdev);
+ err_irq_entries:
+ pci_free_irq_vectors(pdev);
dev_err(dev, "No usable interrupts\n");
return rc;
}
-static int idxd_setup_internals(struct idxd_device *idxd)
+static int idxd_setup_wqs(struct idxd_device *idxd)
{
struct device *dev = &idxd->pdev->dev;
- int i;
-
- init_waitqueue_head(&idxd->cmd_waitq);
- idxd->groups = devm_kcalloc(dev, idxd->max_groups,
- sizeof(struct idxd_group), GFP_KERNEL);
- if (!idxd->groups)
- return -ENOMEM;
+ struct idxd_wq *wq;
+ int i, rc;
- for (i = 0; i < idxd->max_groups; i++) {
- idxd->groups[i].idxd = idxd;
- idxd->groups[i].id = i;
- idxd->groups[i].tc_a = -1;
- idxd->groups[i].tc_b = -1;
- }
-
- idxd->wqs = devm_kcalloc(dev, idxd->max_wqs, sizeof(struct idxd_wq),
- GFP_KERNEL);
+ idxd->wqs = kcalloc_node(idxd->max_wqs, sizeof(struct idxd_wq *),
+ GFP_KERNEL, dev_to_node(dev));
if (!idxd->wqs)
return -ENOMEM;
- idxd->engines = devm_kcalloc(dev, idxd->max_engines,
- sizeof(struct idxd_engine), GFP_KERNEL);
- if (!idxd->engines)
- return -ENOMEM;
-
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ wq = kzalloc_node(sizeof(*wq), GFP_KERNEL, dev_to_node(dev));
+ if (!wq) {
+ rc = -ENOMEM;
+ goto err;
+ }
wq->id = i;
wq->idxd = idxd;
+ device_initialize(&wq->conf_dev);
+ wq->conf_dev.parent = &idxd->conf_dev;
+ wq->conf_dev.bus = &dsa_bus_type;
+ wq->conf_dev.type = &idxd_wq_device_type;
+ rc = dev_set_name(&wq->conf_dev, "wq%d.%d", idxd->id, wq->id);
+ if (rc < 0) {
+ put_device(&wq->conf_dev);
+ goto err;
+ }
+
mutex_init(&wq->wq_lock);
- wq->idxd_cdev.minor = -1;
+ init_waitqueue_head(&wq->err_queue);
+ init_completion(&wq->wq_dead);
wq->max_xfer_bytes = idxd->max_xfer_bytes;
wq->max_batch_size = idxd->max_batch_size;
- wq->wqcfg = devm_kzalloc(dev, idxd->wqcfg_size, GFP_KERNEL);
- if (!wq->wqcfg)
- return -ENOMEM;
+ wq->wqcfg = kzalloc_node(idxd->wqcfg_size, GFP_KERNEL, dev_to_node(dev));
+ if (!wq->wqcfg) {
+ put_device(&wq->conf_dev);
+ rc = -ENOMEM;
+ goto err;
+ }
+ idxd->wqs[i] = wq;
}
+ return 0;
+
+ err:
+ while (--i >= 0)
+ put_device(&idxd->wqs[i]->conf_dev);
+ return rc;
+}
+
+static int idxd_setup_engines(struct idxd_device *idxd)
+{
+ struct idxd_engine *engine;
+ struct device *dev = &idxd->pdev->dev;
+ int i, rc;
+
+ idxd->engines = kcalloc_node(idxd->max_engines, sizeof(struct idxd_engine *),
+ GFP_KERNEL, dev_to_node(dev));
+ if (!idxd->engines)
+ return -ENOMEM;
+
for (i = 0; i < idxd->max_engines; i++) {
- idxd->engines[i].idxd = idxd;
- idxd->engines[i].id = i;
+ engine = kzalloc_node(sizeof(*engine), GFP_KERNEL, dev_to_node(dev));
+ if (!engine) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ engine->id = i;
+ engine->idxd = idxd;
+ device_initialize(&engine->conf_dev);
+ engine->conf_dev.parent = &idxd->conf_dev;
+ engine->conf_dev.type = &idxd_engine_device_type;
+ rc = dev_set_name(&engine->conf_dev, "engine%d.%d", idxd->id, engine->id);
+ if (rc < 0) {
+ put_device(&engine->conf_dev);
+ goto err;
+ }
+
+ idxd->engines[i] = engine;
}
- idxd->wq = create_workqueue(dev_name(dev));
- if (!idxd->wq)
+ return 0;
+
+ err:
+ while (--i >= 0)
+ put_device(&idxd->engines[i]->conf_dev);
+ return rc;
+}
+
+static int idxd_setup_groups(struct idxd_device *idxd)
+{
+ struct device *dev = &idxd->pdev->dev;
+ struct idxd_group *group;
+ int i, rc;
+
+ idxd->groups = kcalloc_node(idxd->max_groups, sizeof(struct idxd_group *),
+ GFP_KERNEL, dev_to_node(dev));
+ if (!idxd->groups)
return -ENOMEM;
+ for (i = 0; i < idxd->max_groups; i++) {
+ group = kzalloc_node(sizeof(*group), GFP_KERNEL, dev_to_node(dev));
+ if (!group) {
+ rc = -ENOMEM;
+ goto err;
+ }
+
+ group->id = i;
+ group->idxd = idxd;
+ device_initialize(&group->conf_dev);
+ group->conf_dev.parent = &idxd->conf_dev;
+ group->conf_dev.bus = &dsa_bus_type;
+ group->conf_dev.type = &idxd_group_device_type;
+ rc = dev_set_name(&group->conf_dev, "group%d.%d", idxd->id, group->id);
+ if (rc < 0) {
+ put_device(&group->conf_dev);
+ goto err;
+ }
+
+ idxd->groups[i] = group;
+ group->tc_a = -1;
+ group->tc_b = -1;
+ }
+
+ return 0;
+
+ err:
+ while (--i >= 0)
+ put_device(&idxd->groups[i]->conf_dev);
+ return rc;
+}
+
+static int idxd_setup_internals(struct idxd_device *idxd)
+{
+ struct device *dev = &idxd->pdev->dev;
+ int rc, i;
+
+ init_waitqueue_head(&idxd->cmd_waitq);
+
+ if (idxd->hw.cmd_cap & BIT(IDXD_CMD_REQUEST_INT_HANDLE)) {
+ idxd->int_handles = devm_kcalloc(dev, idxd->max_wqs, sizeof(int), GFP_KERNEL);
+ if (!idxd->int_handles)
+ return -ENOMEM;
+ }
+
+ rc = idxd_setup_wqs(idxd);
+ if (rc < 0)
+ goto err_wqs;
+
+ rc = idxd_setup_engines(idxd);
+ if (rc < 0)
+ goto err_engine;
+
+ rc = idxd_setup_groups(idxd);
+ if (rc < 0)
+ goto err_group;
+
+ idxd->wq = create_workqueue(dev_name(dev));
+ if (!idxd->wq) {
+ rc = -ENOMEM;
+ goto err_wkq_create;
+ }
+
return 0;
+
+ err_wkq_create:
+ for (i = 0; i < idxd->max_groups; i++)
+ put_device(&idxd->groups[i]->conf_dev);
+ err_group:
+ for (i = 0; i < idxd->max_engines; i++)
+ put_device(&idxd->engines[i]->conf_dev);
+ err_engine:
+ for (i = 0; i < idxd->max_wqs; i++)
+ put_device(&idxd->wqs[i]->conf_dev);
+ err_wqs:
+ kfree(idxd->int_handles);
+ return rc;
}
static void idxd_read_table_offsets(struct idxd_device *idxd)
@@ -241,6 +375,12 @@ static void idxd_read_caps(struct idxd_device *idxd)
/* reading generic capabilities */
idxd->hw.gen_cap.bits = ioread64(idxd->reg_base + IDXD_GENCAP_OFFSET);
dev_dbg(dev, "gen_cap: %#llx\n", idxd->hw.gen_cap.bits);
+
+ if (idxd->hw.gen_cap.cmd_cap) {
+ idxd->hw.cmd_cap = ioread32(idxd->reg_base + IDXD_CMDCAP_OFFSET);
+ dev_dbg(dev, "cmd_cap: %#x\n", idxd->hw.cmd_cap);
+ }
+
idxd->max_xfer_bytes = 1ULL << idxd->hw.gen_cap.max_xfer_shift;
dev_dbg(dev, "max xfer size: %llu bytes\n", idxd->max_xfer_bytes);
idxd->max_batch_size = 1U << idxd->hw.gen_cap.max_batch_shift;
@@ -283,17 +423,34 @@ static void idxd_read_caps(struct idxd_device *idxd)
}
}
-static struct idxd_device *idxd_alloc(struct pci_dev *pdev)
+static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_data *data)
{
struct device *dev = &pdev->dev;
struct idxd_device *idxd;
+ int rc;
- idxd = devm_kzalloc(dev, sizeof(struct idxd_device), GFP_KERNEL);
+ idxd = kzalloc_node(sizeof(*idxd), GFP_KERNEL, dev_to_node(dev));
if (!idxd)
return NULL;
idxd->pdev = pdev;
+ idxd->data = data;
+ idxd->id = ida_alloc(&idxd_ida, GFP_KERNEL);
+ if (idxd->id < 0)
+ return NULL;
+
+ device_initialize(&idxd->conf_dev);
+ idxd->conf_dev.parent = dev;
+ idxd->conf_dev.bus = &dsa_bus_type;
+ idxd->conf_dev.type = idxd->data->dev_type;
+ rc = dev_set_name(&idxd->conf_dev, "%s%d", idxd->data->name_prefix, idxd->id);
+ if (rc < 0) {
+ put_device(&idxd->conf_dev);
+ return NULL;
+ }
+
spin_lock_init(&idxd->dev_lock);
+ spin_lock_init(&idxd->cmd_lock);
return idxd;
}
@@ -346,11 +503,18 @@ static int idxd_probe(struct idxd_device *idxd)
dev_dbg(dev, "IDXD reset complete\n");
if (IS_ENABLED(CONFIG_INTEL_IDXD_SVM) && sva) {
- rc = idxd_enable_system_pasid(idxd);
- if (rc < 0)
- dev_warn(dev, "Failed to enable PASID. No SVA support: %d\n", rc);
- else
- set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags);
+ rc = iommu_dev_enable_feature(dev, IOMMU_DEV_FEAT_SVA);
+ if (rc == 0) {
+ rc = idxd_enable_system_pasid(idxd);
+ if (rc < 0) {
+ iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA);
+ dev_warn(dev, "Failed to enable PASID. No SVA support: %d\n", rc);
+ } else {
+ set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags);
+ }
+ } else {
+ dev_warn(dev, "Unable to turn on SVA feature.\n");
+ }
} else if (!sva) {
dev_warn(dev, "User forced SVA off via module param.\n");
}
@@ -360,80 +524,75 @@ static int idxd_probe(struct idxd_device *idxd)
rc = idxd_setup_internals(idxd);
if (rc)
- goto err_setup;
+ goto err;
+
+ /* If the configs are readonly, then load them from device */
+ if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) {
+ dev_dbg(dev, "Loading RO device config\n");
+ rc = idxd_device_load_config(idxd);
+ if (rc < 0)
+ goto err;
+ }
rc = idxd_setup_interrupts(idxd);
if (rc)
- goto err_setup;
+ goto err;
dev_dbg(dev, "IDXD interrupt setup complete.\n");
- mutex_lock(&idxd_idr_lock);
- idxd->id = idr_alloc(&idxd_idrs[idxd->type], idxd, 0, 0, GFP_KERNEL);
- mutex_unlock(&idxd_idr_lock);
- if (idxd->id < 0) {
- rc = -ENOMEM;
- goto err_idr_fail;
- }
-
idxd->major = idxd_cdev_get_major(idxd);
+ rc = perfmon_pmu_init(idxd);
+ if (rc < 0)
+ dev_warn(dev, "Failed to initialize perfmon. No PMU support: %d\n", rc);
+
dev_dbg(dev, "IDXD device %d probed successfully\n", idxd->id);
return 0;
- err_idr_fail:
- idxd_mask_error_interrupts(idxd);
- idxd_mask_msix_vectors(idxd);
- err_setup:
+ err:
if (device_pasid_enabled(idxd))
idxd_disable_system_pasid(idxd);
+ iommu_dev_disable_feature(dev, IOMMU_DEV_FEAT_SVA);
return rc;
}
-static void idxd_type_init(struct idxd_device *idxd)
-{
- if (idxd->type == IDXD_TYPE_DSA)
- idxd->compl_size = sizeof(struct dsa_completion_record);
- else if (idxd->type == IDXD_TYPE_IAX)
- idxd->compl_size = sizeof(struct iax_completion_record);
-}
-
static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
struct device *dev = &pdev->dev;
struct idxd_device *idxd;
+ struct idxd_driver_data *data = (struct idxd_driver_data *)id->driver_data;
int rc;
- rc = pcim_enable_device(pdev);
+ rc = pci_enable_device(pdev);
if (rc)
return rc;
dev_dbg(dev, "Alloc IDXD context\n");
- idxd = idxd_alloc(pdev);
- if (!idxd)
- return -ENOMEM;
+ idxd = idxd_alloc(pdev, data);
+ if (!idxd) {
+ rc = -ENOMEM;
+ goto err_idxd_alloc;
+ }
dev_dbg(dev, "Mapping BARs\n");
- idxd->reg_base = pcim_iomap(pdev, IDXD_MMIO_BAR, 0);
- if (!idxd->reg_base)
- return -ENOMEM;
+ idxd->reg_base = pci_iomap(pdev, IDXD_MMIO_BAR, 0);
+ if (!idxd->reg_base) {
+ rc = -ENOMEM;
+ goto err_iomap;
+ }
dev_dbg(dev, "Set DMA masks\n");
rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
if (rc)
rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
if (rc)
- return rc;
+ goto err;
rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
if (rc)
rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
if (rc)
- return rc;
-
- idxd_set_type(idxd);
-
- idxd_type_init(idxd);
+ goto err;
dev_dbg(dev, "Set PCI master\n");
pci_set_master(pdev);
@@ -443,13 +602,13 @@ static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
rc = idxd_probe(idxd);
if (rc) {
dev_err(dev, "Intel(R) IDXD DMA Engine init failed\n");
- return -ENODEV;
+ goto err;
}
- rc = idxd_setup_sysfs(idxd);
+ rc = idxd_register_devices(idxd);
if (rc) {
dev_err(dev, "IDXD sysfs setup failed\n");
- return -ENODEV;
+ goto err;
}
idxd->state = IDXD_DEV_CONF_READY;
@@ -458,6 +617,14 @@ static int idxd_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
idxd->hw.version);
return 0;
+
+ err:
+ pci_iounmap(pdev, idxd->reg_base);
+ err_iomap:
+ put_device(&idxd->conf_dev);
+ err_idxd_alloc:
+ pci_disable_device(pdev);
+ return rc;
}
static void idxd_flush_pending_llist(struct idxd_irq_entry *ie)
@@ -486,6 +653,36 @@ static void idxd_flush_work_list(struct idxd_irq_entry *ie)
}
}
+void idxd_wqs_quiesce(struct idxd_device *idxd)
+{
+ struct idxd_wq *wq;
+ int i;
+
+ for (i = 0; i < idxd->max_wqs; i++) {
+ wq = idxd->wqs[i];
+ if (wq->state == IDXD_WQ_ENABLED && wq->type == IDXD_WQT_KERNEL)
+ idxd_wq_quiesce(wq);
+ }
+}
+
+static void idxd_release_int_handles(struct idxd_device *idxd)
+{
+ struct device *dev = &idxd->pdev->dev;
+ int i, rc;
+
+ for (i = 0; i < idxd->num_wq_irqs; i++) {
+ if (idxd->hw.cmd_cap & BIT(IDXD_CMD_RELEASE_INT_HANDLE)) {
+ rc = idxd_device_release_int_handle(idxd, idxd->int_handles[i],
+ IDXD_IRQ_MSIX);
+ if (rc < 0)
+ dev_warn(dev, "irq handle %d release failed\n",
+ idxd->int_handles[i]);
+ else
+ dev_dbg(dev, "int handle requested: %u\n", idxd->int_handles[i]);
+ }
+ }
+}
+
static void idxd_shutdown(struct pci_dev *pdev)
{
struct idxd_device *idxd = pci_get_drvdata(pdev);
@@ -503,13 +700,19 @@ static void idxd_shutdown(struct pci_dev *pdev)
for (i = 0; i < msixcnt; i++) {
irq_entry = &idxd->irq_entries[i];
- synchronize_irq(idxd->msix_entries[i].vector);
+ synchronize_irq(irq_entry->vector);
+ free_irq(irq_entry->vector, irq_entry);
if (i == 0)
continue;
idxd_flush_pending_llist(irq_entry);
idxd_flush_work_list(irq_entry);
}
+ idxd_msix_perm_clear(idxd);
+ idxd_release_int_handles(idxd);
+ pci_free_irq_vectors(pdev);
+ pci_iounmap(pdev, idxd->reg_base);
+ pci_disable_device(pdev);
destroy_workqueue(idxd->wq);
}
@@ -518,13 +721,12 @@ static void idxd_remove(struct pci_dev *pdev)
struct idxd_device *idxd = pci_get_drvdata(pdev);
dev_dbg(&pdev->dev, "%s called\n", __func__);
- idxd_cleanup_sysfs(idxd);
idxd_shutdown(pdev);
if (device_pasid_enabled(idxd))
idxd_disable_system_pasid(idxd);
- mutex_lock(&idxd_idr_lock);
- idr_remove(&idxd_idrs[idxd->type], idxd->id);
- mutex_unlock(&idxd_idr_lock);
+ idxd_unregister_devices(idxd);
+ perfmon_pmu_remove(idxd);
+ iommu_dev_disable_feature(&pdev->dev, IOMMU_DEV_FEAT_SVA);
}
static struct pci_driver idxd_pci_driver = {
@@ -537,7 +739,7 @@ static struct pci_driver idxd_pci_driver = {
static int __init idxd_init_module(void)
{
- int err, i;
+ int err;
/*
* If the CPU does not support MOVDIR64B or ENQCMDS, there's no point in
@@ -553,8 +755,7 @@ static int __init idxd_init_module(void)
else
support_enqcmd = true;
- for (i = 0; i < IDXD_TYPE_MAX; i++)
- idr_init(&idxd_idrs[i]);
+ perfmon_init();
err = idxd_register_bus_type();
if (err < 0)
@@ -589,5 +790,6 @@ static void __exit idxd_exit_module(void)
pci_unregister_driver(&idxd_pci_driver);
idxd_cdev_remove();
idxd_unregister_bus_type();
+ perfmon_exit();
}
module_exit(idxd_exit_module);
diff --git a/drivers/dma/idxd/irq.c b/drivers/dma/idxd/irq.c
index a60ca11a5784..ae68e1e5487a 100644
--- a/drivers/dma/idxd/irq.c
+++ b/drivers/dma/idxd/irq.c
@@ -45,7 +45,7 @@ static void idxd_device_reinit(struct work_struct *work)
goto out;
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
if (wq->state == IDXD_WQ_ENABLED) {
rc = idxd_wq_enable(wq);
@@ -102,15 +102,6 @@ static int idxd_device_schedule_fault_process(struct idxd_device *idxd,
return 0;
}
-irqreturn_t idxd_irq_handler(int vec, void *data)
-{
- struct idxd_irq_entry *irq_entry = data;
- struct idxd_device *idxd = irq_entry->idxd;
-
- idxd_mask_msix_vector(idxd, irq_entry->id);
- return IRQ_WAKE_THREAD;
-}
-
static int process_misc_interrupts(struct idxd_device *idxd, u32 cause)
{
struct device *dev = &idxd->pdev->dev;
@@ -124,22 +115,24 @@ static int process_misc_interrupts(struct idxd_device *idxd, u32 cause)
for (i = 0; i < 4; i++)
idxd->sw_err.bits[i] = ioread64(idxd->reg_base +
IDXD_SWERR_OFFSET + i * sizeof(u64));
- iowrite64(IDXD_SWERR_ACK, idxd->reg_base + IDXD_SWERR_OFFSET);
+
+ iowrite64(idxd->sw_err.bits[0] & IDXD_SWERR_ACK,
+ idxd->reg_base + IDXD_SWERR_OFFSET);
if (idxd->sw_err.valid && idxd->sw_err.wq_idx_valid) {
int id = idxd->sw_err.wq_idx;
- struct idxd_wq *wq = &idxd->wqs[id];
+ struct idxd_wq *wq = idxd->wqs[id];
if (wq->type == IDXD_WQT_USER)
- wake_up_interruptible(&wq->idxd_cdev.err_queue);
+ wake_up_interruptible(&wq->err_queue);
} else {
int i;
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
if (wq->type == IDXD_WQT_USER)
- wake_up_interruptible(&wq->idxd_cdev.err_queue);
+ wake_up_interruptible(&wq->err_queue);
}
}
@@ -163,11 +156,8 @@ static int process_misc_interrupts(struct idxd_device *idxd, u32 cause)
}
if (cause & IDXD_INTC_PERFMON_OVFL) {
- /*
- * Driver does not utilize perfmon counter overflow interrupt
- * yet.
- */
val |= IDXD_INTC_PERFMON_OVFL;
+ perfmon_counter_overflow(idxd);
}
val ^= cause;
@@ -200,6 +190,8 @@ static int process_misc_interrupts(struct idxd_device *idxd, u32 cause)
queue_work(idxd->wq, &idxd->work);
} else {
spin_lock_bh(&idxd->dev_lock);
+ idxd_wqs_quiesce(idxd);
+ idxd_wqs_unmap_portal(idxd);
idxd_device_wqs_clear_state(idxd);
dev_err(&idxd->pdev->dev,
"idxd halted, need %s.\n",
@@ -233,7 +225,6 @@ irqreturn_t idxd_misc_thread(int vec, void *data)
iowrite32(cause, idxd->reg_base + IDXD_INTCAUSE_OFFSET);
}
- idxd_unmask_msix_vector(idxd, irq_entry->id);
return IRQ_HANDLED;
}
@@ -390,8 +381,6 @@ irqreturn_t idxd_wq_thread(int irq, void *data)
int processed;
processed = idxd_desc_process(irq_entry);
- idxd_unmask_msix_vector(irq_entry->idxd, irq_entry->id);
-
if (processed == 0)
return IRQ_NONE;
diff --git a/drivers/dma/idxd/perfmon.c b/drivers/dma/idxd/perfmon.c
new file mode 100644
index 000000000000..d73004f47cf4
--- /dev/null
+++ b/drivers/dma/idxd/perfmon.c
@@ -0,0 +1,662 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright(c) 2020 Intel Corporation. All rights rsvd. */
+
+#include <linux/sched/task.h>
+#include <linux/io-64-nonatomic-lo-hi.h>
+#include "idxd.h"
+#include "perfmon.h"
+
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr,
+ char *buf);
+
+static cpumask_t perfmon_dsa_cpu_mask;
+static bool cpuhp_set_up;
+static enum cpuhp_state cpuhp_slot;
+
+/*
+ * perf userspace reads this attribute to determine which cpus to open
+ * counters on. It's connected to perfmon_dsa_cpu_mask, which is
+ * maintained by the cpu hotplug handlers.
+ */
+static DEVICE_ATTR_RO(cpumask);
+
+static struct attribute *perfmon_cpumask_attrs[] = {
+ &dev_attr_cpumask.attr,
+ NULL,
+};
+
+static struct attribute_group cpumask_attr_group = {
+ .attrs = perfmon_cpumask_attrs,
+};
+
+/*
+ * These attributes specify the bits in the config word that the perf
+ * syscall uses to pass the event ids and categories to perfmon.
+ */
+DEFINE_PERFMON_FORMAT_ATTR(event_category, "config:0-3");
+DEFINE_PERFMON_FORMAT_ATTR(event, "config:4-31");
+
+/*
+ * These attributes specify the bits in the config1 word that the perf
+ * syscall uses to pass filter data to perfmon.
+ */
+DEFINE_PERFMON_FORMAT_ATTR(filter_wq, "config1:0-31");
+DEFINE_PERFMON_FORMAT_ATTR(filter_tc, "config1:32-39");
+DEFINE_PERFMON_FORMAT_ATTR(filter_pgsz, "config1:40-43");
+DEFINE_PERFMON_FORMAT_ATTR(filter_sz, "config1:44-51");
+DEFINE_PERFMON_FORMAT_ATTR(filter_eng, "config1:52-59");
+
+#define PERFMON_FILTERS_START 2
+#define PERFMON_FILTERS_MAX 5
+
+static struct attribute *perfmon_format_attrs[] = {
+ &format_attr_idxd_event_category.attr,
+ &format_attr_idxd_event.attr,
+ &format_attr_idxd_filter_wq.attr,
+ &format_attr_idxd_filter_tc.attr,
+ &format_attr_idxd_filter_pgsz.attr,
+ &format_attr_idxd_filter_sz.attr,
+ &format_attr_idxd_filter_eng.attr,
+ NULL,
+};
+
+static struct attribute_group perfmon_format_attr_group = {
+ .name = "format",
+ .attrs = perfmon_format_attrs,
+};
+
+static const struct attribute_group *perfmon_attr_groups[] = {
+ &perfmon_format_attr_group,
+ &cpumask_attr_group,
+ NULL,
+};
+
+static ssize_t cpumask_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ return cpumap_print_to_pagebuf(true, buf, &perfmon_dsa_cpu_mask);
+}
+
+static bool is_idxd_event(struct idxd_pmu *idxd_pmu, struct perf_event *event)
+{
+ return &idxd_pmu->pmu == event->pmu;
+}
+
+static int perfmon_collect_events(struct idxd_pmu *idxd_pmu,
+ struct perf_event *leader,
+ bool do_grp)
+{
+ struct perf_event *event;
+ int n, max_count;
+
+ max_count = idxd_pmu->n_counters;
+ n = idxd_pmu->n_events;
+
+ if (n >= max_count)
+ return -EINVAL;
+
+ if (is_idxd_event(idxd_pmu, leader)) {
+ idxd_pmu->event_list[n] = leader;
+ idxd_pmu->event_list[n]->hw.idx = n;
+ n++;
+ }
+
+ if (!do_grp)
+ return n;
+
+ for_each_sibling_event(event, leader) {
+ if (!is_idxd_event(idxd_pmu, event) ||
+ event->state <= PERF_EVENT_STATE_OFF)
+ continue;
+
+ if (n >= max_count)
+ return -EINVAL;
+
+ idxd_pmu->event_list[n] = event;
+ idxd_pmu->event_list[n]->hw.idx = n;
+ n++;
+ }
+
+ return n;
+}
+
+static void perfmon_assign_hw_event(struct idxd_pmu *idxd_pmu,
+ struct perf_event *event, int idx)
+{
+ struct idxd_device *idxd = idxd_pmu->idxd;
+ struct hw_perf_event *hwc = &event->hw;
+
+ hwc->idx = idx;
+ hwc->config_base = ioread64(CNTRCFG_REG(idxd, idx));
+ hwc->event_base = ioread64(CNTRCFG_REG(idxd, idx));
+}
+
+static int perfmon_assign_event(struct idxd_pmu *idxd_pmu,
+ struct perf_event *event)
+{
+ int i;
+
+ for (i = 0; i < IDXD_PMU_EVENT_MAX; i++)
+ if (!test_and_set_bit(i, idxd_pmu->used_mask))
+ return i;
+
+ return -EINVAL;
+}
+
+/*
+ * Check whether there are enough counters to satisfy that all the
+ * events in the group can actually be scheduled at the same time.
+ *
+ * To do this, create a fake idxd_pmu object so the event collection
+ * and assignment functions can be used without affecting the internal
+ * state of the real idxd_pmu object.
+ */
+static int perfmon_validate_group(struct idxd_pmu *pmu,
+ struct perf_event *event)
+{
+ struct perf_event *leader = event->group_leader;
+ struct idxd_pmu *fake_pmu;
+ int i, ret = 0, n, idx;
+
+ fake_pmu = kzalloc(sizeof(*fake_pmu), GFP_KERNEL);
+ if (!fake_pmu)
+ return -ENOMEM;
+
+ fake_pmu->pmu.name = pmu->pmu.name;
+ fake_pmu->n_counters = pmu->n_counters;
+
+ n = perfmon_collect_events(fake_pmu, leader, true);
+ if (n < 0) {
+ ret = n;
+ goto out;
+ }
+
+ fake_pmu->n_events = n;
+ n = perfmon_collect_events(fake_pmu, event, false);
+ if (n < 0) {
+ ret = n;
+ goto out;
+ }
+
+ fake_pmu->n_events = n;
+
+ for (i = 0; i < n; i++) {
+ event = fake_pmu->event_list[i];
+
+ idx = perfmon_assign_event(fake_pmu, event);
+ if (idx < 0) {
+ ret = idx;
+ goto out;
+ }
+ }
+out:
+ kfree(fake_pmu);
+
+ return ret;
+}
+
+static int perfmon_pmu_event_init(struct perf_event *event)
+{
+ struct idxd_device *idxd;
+ int ret = 0;
+
+ idxd = event_to_idxd(event);
+ event->hw.idx = -1;
+
+ if (event->attr.type != event->pmu->type)
+ return -ENOENT;
+
+ /* sampling not supported */
+ if (event->attr.sample_period)
+ return -EINVAL;
+
+ if (event->cpu < 0)
+ return -EINVAL;
+
+ if (event->pmu != &idxd->idxd_pmu->pmu)
+ return -EINVAL;
+
+ event->hw.event_base = ioread64(PERFMON_TABLE_OFFSET(idxd));
+ event->cpu = idxd->idxd_pmu->cpu;
+ event->hw.config = event->attr.config;
+
+ if (event->group_leader != event)
+ /* non-group events have themselves as leader */
+ ret = perfmon_validate_group(idxd->idxd_pmu, event);
+
+ return ret;
+}
+
+static inline u64 perfmon_pmu_read_counter(struct perf_event *event)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct idxd_device *idxd;
+ int cntr = hwc->idx;
+
+ idxd = event_to_idxd(event);
+
+ return ioread64(CNTRDATA_REG(idxd, cntr));
+}
+
+static void perfmon_pmu_event_update(struct perf_event *event)
+{
+ struct idxd_device *idxd = event_to_idxd(event);
+ u64 prev_raw_count, new_raw_count, delta, p, n;
+ int shift = 64 - idxd->idxd_pmu->counter_width;
+ struct hw_perf_event *hwc = &event->hw;
+
+ do {
+ prev_raw_count = local64_read(&hwc->prev_count);
+ new_raw_count = perfmon_pmu_read_counter(event);
+ } while (local64_cmpxchg(&hwc->prev_count, prev_raw_count,
+ new_raw_count) != prev_raw_count);
+
+ n = (new_raw_count << shift);
+ p = (prev_raw_count << shift);
+
+ delta = ((n - p) >> shift);
+
+ local64_add(delta, &event->count);
+}
+
+void perfmon_counter_overflow(struct idxd_device *idxd)
+{
+ int i, n_counters, max_loop = OVERFLOW_SIZE;
+ struct perf_event *event;
+ unsigned long ovfstatus;
+
+ n_counters = min(idxd->idxd_pmu->n_counters, OVERFLOW_SIZE);
+
+ ovfstatus = ioread32(OVFSTATUS_REG(idxd));
+
+ /*
+ * While updating overflowed counters, other counters behind
+ * them could overflow and be missed in a given pass.
+ * Normally this could happen at most n_counters times, but in
+ * theory a tiny counter width could result in continual
+ * overflows and endless looping. max_loop provides a
+ * failsafe in that highly unlikely case.
+ */
+ while (ovfstatus && max_loop--) {
+ /* Figure out which counter(s) overflowed */
+ for_each_set_bit(i, &ovfstatus, n_counters) {
+ unsigned long ovfstatus_clear = 0;
+
+ /* Update event->count for overflowed counter */
+ event = idxd->idxd_pmu->event_list[i];
+ perfmon_pmu_event_update(event);
+ /* Writing 1 to OVFSTATUS bit clears it */
+ set_bit(i, &ovfstatus_clear);
+ iowrite32(ovfstatus_clear, OVFSTATUS_REG(idxd));
+ }
+
+ ovfstatus = ioread32(OVFSTATUS_REG(idxd));
+ }
+
+ /*
+ * Should never happen. If so, it means a counter(s) looped
+ * around twice while this handler was running.
+ */
+ WARN_ON_ONCE(ovfstatus);
+}
+
+static inline void perfmon_reset_config(struct idxd_device *idxd)
+{
+ iowrite32(CONFIG_RESET, PERFRST_REG(idxd));
+ iowrite32(0, OVFSTATUS_REG(idxd));
+ iowrite32(0, PERFFRZ_REG(idxd));
+}
+
+static inline void perfmon_reset_counters(struct idxd_device *idxd)
+{
+ iowrite32(CNTR_RESET, PERFRST_REG(idxd));
+}
+
+static inline void perfmon_reset(struct idxd_device *idxd)
+{
+ perfmon_reset_config(idxd);
+ perfmon_reset_counters(idxd);
+}
+
+static void perfmon_pmu_event_start(struct perf_event *event, int mode)
+{
+ u32 flt_wq, flt_tc, flt_pg_sz, flt_xfer_sz, flt_eng = 0;
+ u64 cntr_cfg, cntrdata, event_enc, event_cat = 0;
+ struct hw_perf_event *hwc = &event->hw;
+ union filter_cfg flt_cfg;
+ union event_cfg event_cfg;
+ struct idxd_device *idxd;
+ int cntr;
+
+ idxd = event_to_idxd(event);
+
+ event->hw.idx = hwc->idx;
+ cntr = hwc->idx;
+
+ /* Obtain event category and event value from user space */
+ event_cfg.val = event->attr.config;
+ flt_cfg.val = event->attr.config1;
+ event_cat = event_cfg.event_cat;
+ event_enc = event_cfg.event_enc;
+
+ /* Obtain filter configuration from user space */
+ flt_wq = flt_cfg.wq;
+ flt_tc = flt_cfg.tc;
+ flt_pg_sz = flt_cfg.pg_sz;
+ flt_xfer_sz = flt_cfg.xfer_sz;
+ flt_eng = flt_cfg.eng;
+
+ if (flt_wq && test_bit(FLT_WQ, &idxd->idxd_pmu->supported_filters))
+ iowrite32(flt_wq, FLTCFG_REG(idxd, cntr, FLT_WQ));
+ if (flt_tc && test_bit(FLT_TC, &idxd->idxd_pmu->supported_filters))
+ iowrite32(flt_tc, FLTCFG_REG(idxd, cntr, FLT_TC));
+ if (flt_pg_sz && test_bit(FLT_PG_SZ, &idxd->idxd_pmu->supported_filters))
+ iowrite32(flt_pg_sz, FLTCFG_REG(idxd, cntr, FLT_PG_SZ));
+ if (flt_xfer_sz && test_bit(FLT_XFER_SZ, &idxd->idxd_pmu->supported_filters))
+ iowrite32(flt_xfer_sz, FLTCFG_REG(idxd, cntr, FLT_XFER_SZ));
+ if (flt_eng && test_bit(FLT_ENG, &idxd->idxd_pmu->supported_filters))
+ iowrite32(flt_eng, FLTCFG_REG(idxd, cntr, FLT_ENG));
+
+ /* Read the start value */
+ cntrdata = ioread64(CNTRDATA_REG(idxd, cntr));
+ local64_set(&event->hw.prev_count, cntrdata);
+
+ /* Set counter to event/category */
+ cntr_cfg = event_cat << CNTRCFG_CATEGORY_SHIFT;
+ cntr_cfg |= event_enc << CNTRCFG_EVENT_SHIFT;
+ /* Set interrupt on overflow and counter enable bits */
+ cntr_cfg |= (CNTRCFG_IRQ_OVERFLOW | CNTRCFG_ENABLE);
+
+ iowrite64(cntr_cfg, CNTRCFG_REG(idxd, cntr));
+}
+
+static void perfmon_pmu_event_stop(struct perf_event *event, int mode)
+{
+ struct hw_perf_event *hwc = &event->hw;
+ struct idxd_device *idxd;
+ int i, cntr = hwc->idx;
+ u64 cntr_cfg;
+
+ idxd = event_to_idxd(event);
+
+ /* remove this event from event list */
+ for (i = 0; i < idxd->idxd_pmu->n_events; i++) {
+ if (event != idxd->idxd_pmu->event_list[i])
+ continue;
+
+ for (++i; i < idxd->idxd_pmu->n_events; i++)
+ idxd->idxd_pmu->event_list[i - 1] = idxd->idxd_pmu->event_list[i];
+ --idxd->idxd_pmu->n_events;
+ break;
+ }
+
+ cntr_cfg = ioread64(CNTRCFG_REG(idxd, cntr));
+ cntr_cfg &= ~CNTRCFG_ENABLE;
+ iowrite64(cntr_cfg, CNTRCFG_REG(idxd, cntr));
+
+ if (mode == PERF_EF_UPDATE)
+ perfmon_pmu_event_update(event);
+
+ event->hw.idx = -1;
+ clear_bit(cntr, idxd->idxd_pmu->used_mask);
+}
+
+static void perfmon_pmu_event_del(struct perf_event *event, int mode)
+{
+ perfmon_pmu_event_stop(event, PERF_EF_UPDATE);
+}
+
+static int perfmon_pmu_event_add(struct perf_event *event, int flags)
+{
+ struct idxd_device *idxd = event_to_idxd(event);
+ struct idxd_pmu *idxd_pmu = idxd->idxd_pmu;
+ struct hw_perf_event *hwc = &event->hw;
+ int idx, n;
+
+ n = perfmon_collect_events(idxd_pmu, event, false);
+ if (n < 0)
+ return n;
+
+ hwc->state = PERF_HES_UPTODATE | PERF_HES_STOPPED;
+ if (!(flags & PERF_EF_START))
+ hwc->state |= PERF_HES_ARCH;
+
+ idx = perfmon_assign_event(idxd_pmu, event);
+ if (idx < 0)
+ return idx;
+
+ perfmon_assign_hw_event(idxd_pmu, event, idx);
+
+ if (flags & PERF_EF_START)
+ perfmon_pmu_event_start(event, 0);
+
+ idxd_pmu->n_events = n;
+
+ return 0;
+}
+
+static void enable_perfmon_pmu(struct idxd_device *idxd)
+{
+ iowrite32(COUNTER_UNFREEZE, PERFFRZ_REG(idxd));
+}
+
+static void disable_perfmon_pmu(struct idxd_device *idxd)
+{
+ iowrite32(COUNTER_FREEZE, PERFFRZ_REG(idxd));
+}
+
+static void perfmon_pmu_enable(struct pmu *pmu)
+{
+ struct idxd_device *idxd = pmu_to_idxd(pmu);
+
+ enable_perfmon_pmu(idxd);
+}
+
+static void perfmon_pmu_disable(struct pmu *pmu)
+{
+ struct idxd_device *idxd = pmu_to_idxd(pmu);
+
+ disable_perfmon_pmu(idxd);
+}
+
+static void skip_filter(int i)
+{
+ int j;
+
+ for (j = i; j < PERFMON_FILTERS_MAX; j++)
+ perfmon_format_attrs[PERFMON_FILTERS_START + j] =
+ perfmon_format_attrs[PERFMON_FILTERS_START + j + 1];
+}
+
+static void idxd_pmu_init(struct idxd_pmu *idxd_pmu)
+{
+ int i;
+
+ for (i = 0 ; i < PERFMON_FILTERS_MAX; i++) {
+ if (!test_bit(i, &idxd_pmu->supported_filters))
+ skip_filter(i);
+ }
+
+ idxd_pmu->pmu.name = idxd_pmu->name;
+ idxd_pmu->pmu.attr_groups = perfmon_attr_groups;
+ idxd_pmu->pmu.task_ctx_nr = perf_invalid_context;
+ idxd_pmu->pmu.event_init = perfmon_pmu_event_init;
+ idxd_pmu->pmu.pmu_enable = perfmon_pmu_enable,
+ idxd_pmu->pmu.pmu_disable = perfmon_pmu_disable,
+ idxd_pmu->pmu.add = perfmon_pmu_event_add;
+ idxd_pmu->pmu.del = perfmon_pmu_event_del;
+ idxd_pmu->pmu.start = perfmon_pmu_event_start;
+ idxd_pmu->pmu.stop = perfmon_pmu_event_stop;
+ idxd_pmu->pmu.read = perfmon_pmu_event_update;
+ idxd_pmu->pmu.capabilities = PERF_PMU_CAP_NO_EXCLUDE;
+ idxd_pmu->pmu.module = THIS_MODULE;
+}
+
+void perfmon_pmu_remove(struct idxd_device *idxd)
+{
+ if (!idxd->idxd_pmu)
+ return;
+
+ cpuhp_state_remove_instance(cpuhp_slot, &idxd->idxd_pmu->cpuhp_node);
+ perf_pmu_unregister(&idxd->idxd_pmu->pmu);
+ kfree(idxd->idxd_pmu);
+ idxd->idxd_pmu = NULL;
+}
+
+static int perf_event_cpu_online(unsigned int cpu, struct hlist_node *node)
+{
+ struct idxd_pmu *idxd_pmu;
+
+ idxd_pmu = hlist_entry_safe(node, typeof(*idxd_pmu), cpuhp_node);
+
+ /* select the first online CPU as the designated reader */
+ if (cpumask_empty(&perfmon_dsa_cpu_mask)) {
+ cpumask_set_cpu(cpu, &perfmon_dsa_cpu_mask);
+ idxd_pmu->cpu = cpu;
+ }
+
+ return 0;
+}
+
+static int perf_event_cpu_offline(unsigned int cpu, struct hlist_node *node)
+{
+ struct idxd_pmu *idxd_pmu;
+ unsigned int target;
+
+ idxd_pmu = hlist_entry_safe(node, typeof(*idxd_pmu), cpuhp_node);
+
+ if (!cpumask_test_and_clear_cpu(cpu, &perfmon_dsa_cpu_mask))
+ return 0;
+
+ target = cpumask_any_but(cpu_online_mask, cpu);
+
+ /* migrate events if there is a valid target */
+ if (target < nr_cpu_ids)
+ cpumask_set_cpu(target, &perfmon_dsa_cpu_mask);
+ else
+ target = -1;
+
+ perf_pmu_migrate_context(&idxd_pmu->pmu, cpu, target);
+
+ return 0;
+}
+
+int perfmon_pmu_init(struct idxd_device *idxd)
+{
+ union idxd_perfcap perfcap;
+ struct idxd_pmu *idxd_pmu;
+ int rc = -ENODEV;
+
+ /*
+ * perfmon module initialization failed, nothing to do
+ */
+ if (!cpuhp_set_up)
+ return -ENODEV;
+
+ /*
+ * If perfmon_offset or num_counters is 0, it means perfmon is
+ * not supported on this hardware.
+ */
+ if (idxd->perfmon_offset == 0)
+ return -ENODEV;
+
+ idxd_pmu = kzalloc(sizeof(*idxd_pmu), GFP_KERNEL);
+ if (!idxd_pmu)
+ return -ENOMEM;
+
+ idxd_pmu->idxd = idxd;
+ idxd->idxd_pmu = idxd_pmu;
+
+ if (idxd->data->type == IDXD_TYPE_DSA) {
+ rc = sprintf(idxd_pmu->name, "dsa%d", idxd->id);
+ if (rc < 0)
+ goto free;
+ } else if (idxd->data->type == IDXD_TYPE_IAX) {
+ rc = sprintf(idxd_pmu->name, "iax%d", idxd->id);
+ if (rc < 0)
+ goto free;
+ } else {
+ goto free;
+ }
+
+ perfmon_reset(idxd);
+
+ perfcap.bits = ioread64(PERFCAP_REG(idxd));
+
+ /*
+ * If total perf counter is 0, stop further registration.
+ * This is necessary in order to support driver running on
+ * guest which does not have pmon support.
+ */
+ if (perfcap.num_perf_counter == 0)
+ goto free;
+
+ /* A counter width of 0 means it can't count */
+ if (perfcap.counter_width == 0)
+ goto free;
+
+ /* Overflow interrupt and counter freeze support must be available */
+ if (!perfcap.overflow_interrupt || !perfcap.counter_freeze)
+ goto free;
+
+ /* Number of event categories cannot be 0 */
+ if (perfcap.num_event_category == 0)
+ goto free;
+
+ /*
+ * We don't support per-counter capabilities for now.
+ */
+ if (perfcap.cap_per_counter)
+ goto free;
+
+ idxd_pmu->n_event_categories = perfcap.num_event_category;
+ idxd_pmu->supported_event_categories = perfcap.global_event_category;
+ idxd_pmu->per_counter_caps_supported = perfcap.cap_per_counter;
+
+ /* check filter capability. If 0, then filters are not supported */
+ idxd_pmu->supported_filters = perfcap.filter;
+ if (perfcap.filter)
+ idxd_pmu->n_filters = hweight8(perfcap.filter);
+
+ /* Store the total number of counters categories, and counter width */
+ idxd_pmu->n_counters = perfcap.num_perf_counter;
+ idxd_pmu->counter_width = perfcap.counter_width;
+
+ idxd_pmu_init(idxd_pmu);
+
+ rc = perf_pmu_register(&idxd_pmu->pmu, idxd_pmu->name, -1);
+ if (rc)
+ goto free;
+
+ rc = cpuhp_state_add_instance(cpuhp_slot, &idxd_pmu->cpuhp_node);
+ if (rc) {
+ perf_pmu_unregister(&idxd->idxd_pmu->pmu);
+ goto free;
+ }
+out:
+ return rc;
+free:
+ kfree(idxd_pmu);
+ idxd->idxd_pmu = NULL;
+
+ goto out;
+}
+
+void __init perfmon_init(void)
+{
+ int rc = cpuhp_setup_state_multi(CPUHP_AP_ONLINE_DYN,
+ "driver/dma/idxd/perf:online",
+ perf_event_cpu_online,
+ perf_event_cpu_offline);
+ if (WARN_ON(rc < 0))
+ return;
+
+ cpuhp_slot = rc;
+ cpuhp_set_up = true;
+}
+
+void __exit perfmon_exit(void)
+{
+ if (cpuhp_set_up)
+ cpuhp_remove_multi_state(cpuhp_slot);
+}
diff --git a/drivers/dma/idxd/perfmon.h b/drivers/dma/idxd/perfmon.h
new file mode 100644
index 000000000000..9a081a1bc605
--- /dev/null
+++ b/drivers/dma/idxd/perfmon.h
@@ -0,0 +1,119 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2020 Intel Corporation. All rights rsvd. */
+
+#ifndef _PERFMON_H_
+#define _PERFMON_H_
+
+#include <linux/slab.h>
+#include <linux/pci.h>
+#include <linux/sbitmap.h>
+#include <linux/dmaengine.h>
+#include <linux/percpu-rwsem.h>
+#include <linux/wait.h>
+#include <linux/cdev.h>
+#include <linux/uuid.h>
+#include <linux/idxd.h>
+#include <linux/perf_event.h>
+#include "registers.h"
+
+static inline struct idxd_pmu *event_to_pmu(struct perf_event *event)
+{
+ struct idxd_pmu *idxd_pmu;
+ struct pmu *pmu;
+
+ pmu = event->pmu;
+ idxd_pmu = container_of(pmu, struct idxd_pmu, pmu);
+
+ return idxd_pmu;
+}
+
+static inline struct idxd_device *event_to_idxd(struct perf_event *event)
+{
+ struct idxd_pmu *idxd_pmu;
+ struct pmu *pmu;
+
+ pmu = event->pmu;
+ idxd_pmu = container_of(pmu, struct idxd_pmu, pmu);
+
+ return idxd_pmu->idxd;
+}
+
+static inline struct idxd_device *pmu_to_idxd(struct pmu *pmu)
+{
+ struct idxd_pmu *idxd_pmu;
+
+ idxd_pmu = container_of(pmu, struct idxd_pmu, pmu);
+
+ return idxd_pmu->idxd;
+}
+
+enum dsa_perf_events {
+ DSA_PERF_EVENT_WQ = 0,
+ DSA_PERF_EVENT_ENGINE,
+ DSA_PERF_EVENT_ADDR_TRANS,
+ DSA_PERF_EVENT_OP,
+ DSA_PERF_EVENT_COMPL,
+ DSA_PERF_EVENT_MAX,
+};
+
+enum filter_enc {
+ FLT_WQ = 0,
+ FLT_TC,
+ FLT_PG_SZ,
+ FLT_XFER_SZ,
+ FLT_ENG,
+ FLT_MAX,
+};
+
+#define CONFIG_RESET 0x0000000000000001
+#define CNTR_RESET 0x0000000000000002
+#define CNTR_ENABLE 0x0000000000000001
+#define INTR_OVFL 0x0000000000000002
+
+#define COUNTER_FREEZE 0x00000000FFFFFFFF
+#define COUNTER_UNFREEZE 0x0000000000000000
+#define OVERFLOW_SIZE 32
+
+#define CNTRCFG_ENABLE BIT(0)
+#define CNTRCFG_IRQ_OVERFLOW BIT(1)
+#define CNTRCFG_CATEGORY_SHIFT 8
+#define CNTRCFG_EVENT_SHIFT 32
+
+#define PERFMON_TABLE_OFFSET(_idxd) \
+({ \
+ typeof(_idxd) __idxd = (_idxd); \
+ ((__idxd)->reg_base + (__idxd)->perfmon_offset); \
+})
+#define PERFMON_REG_OFFSET(idxd, offset) \
+ (PERFMON_TABLE_OFFSET(idxd) + (offset))
+
+#define PERFCAP_REG(idxd) (PERFMON_REG_OFFSET(idxd, IDXD_PERFCAP_OFFSET))
+#define PERFRST_REG(idxd) (PERFMON_REG_OFFSET(idxd, IDXD_PERFRST_OFFSET))
+#define OVFSTATUS_REG(idxd) (PERFMON_REG_OFFSET(idxd, IDXD_OVFSTATUS_OFFSET))
+#define PERFFRZ_REG(idxd) (PERFMON_REG_OFFSET(idxd, IDXD_PERFFRZ_OFFSET))
+
+#define FLTCFG_REG(idxd, cntr, flt) \
+ (PERFMON_REG_OFFSET(idxd, IDXD_FLTCFG_OFFSET) + ((cntr) * 32) + ((flt) * 4))
+
+#define CNTRCFG_REG(idxd, cntr) \
+ (PERFMON_REG_OFFSET(idxd, IDXD_CNTRCFG_OFFSET) + ((cntr) * 8))
+#define CNTRDATA_REG(idxd, cntr) \
+ (PERFMON_REG_OFFSET(idxd, IDXD_CNTRDATA_OFFSET) + ((cntr) * 8))
+#define CNTRCAP_REG(idxd, cntr) \
+ (PERFMON_REG_OFFSET(idxd, IDXD_CNTRCAP_OFFSET) + ((cntr) * 8))
+
+#define EVNTCAP_REG(idxd, category) \
+ (PERFMON_REG_OFFSET(idxd, IDXD_EVNTCAP_OFFSET) + ((category) * 8))
+
+#define DEFINE_PERFMON_FORMAT_ATTR(_name, _format) \
+static ssize_t __perfmon_idxd_##_name##_show(struct kobject *kobj, \
+ struct kobj_attribute *attr, \
+ char *page) \
+{ \
+ BUILD_BUG_ON(sizeof(_format) >= PAGE_SIZE); \
+ return sprintf(page, _format "\n"); \
+} \
+static struct kobj_attribute format_attr_idxd_##_name = \
+ __ATTR(_name, 0444, __perfmon_idxd_##_name##_show, NULL)
+
+#endif
diff --git a/drivers/dma/idxd/registers.h b/drivers/dma/idxd/registers.h
index 751ecb4f9f81..c970c3f025f0 100644
--- a/drivers/dma/idxd/registers.h
+++ b/drivers/dma/idxd/registers.h
@@ -24,8 +24,8 @@ union gen_cap_reg {
u64 overlap_copy:1;
u64 cache_control_mem:1;
u64 cache_control_cache:1;
+ u64 cmd_cap:1;
u64 rsvd:3;
- u64 int_handle_req:1;
u64 dest_readback:1;
u64 drain_readback:1;
u64 rsvd2:6;
@@ -120,7 +120,8 @@ union gencfg_reg {
union genctrl_reg {
struct {
u32 softerr_int_en:1;
- u32 rsvd:31;
+ u32 halt_int_en:1;
+ u32 rsvd:30;
};
u32 bits;
} __packed;
@@ -180,8 +181,11 @@ enum idxd_cmd {
IDXD_CMD_DRAIN_PASID,
IDXD_CMD_ABORT_PASID,
IDXD_CMD_REQUEST_INT_HANDLE,
+ IDXD_CMD_RELEASE_INT_HANDLE,
};
+#define CMD_INT_HANDLE_IMS 0x10000
+
#define IDXD_CMDSTS_OFFSET 0xa8
union cmdsts_reg {
struct {
@@ -193,6 +197,8 @@ union cmdsts_reg {
u32 bits;
} __packed;
#define IDXD_CMDSTS_ACTIVE 0x80000000
+#define IDXD_CMDSTS_ERR_MASK 0xff
+#define IDXD_CMDSTS_RES_SHIFT 8
enum idxd_cmdsts_err {
IDXD_CMDSTS_SUCCESS = 0,
@@ -228,6 +234,8 @@ enum idxd_cmdsts_err {
IDXD_CMDSTS_ERR_NO_HANDLE,
};
+#define IDXD_CMDCAP_OFFSET 0xb0
+
#define IDXD_SWERR_OFFSET 0xc0
#define IDXD_SWERR_VALID 0x00000001
#define IDXD_SWERR_OVERFLOW 0x00000002
@@ -378,4 +386,112 @@ union wqcfg {
#define GRPENGCFG_OFFSET(idxd_dev, n) ((idxd_dev)->grpcfg_offset + (n) * GRPCFG_SIZE + 32)
#define GRPFLGCFG_OFFSET(idxd_dev, n) ((idxd_dev)->grpcfg_offset + (n) * GRPCFG_SIZE + 40)
+/* Following is performance monitor registers */
+#define IDXD_PERFCAP_OFFSET 0x0
+union idxd_perfcap {
+ struct {
+ u64 num_perf_counter:6;
+ u64 rsvd1:2;
+ u64 counter_width:8;
+ u64 num_event_category:4;
+ u64 global_event_category:16;
+ u64 filter:8;
+ u64 rsvd2:8;
+ u64 cap_per_counter:1;
+ u64 writeable_counter:1;
+ u64 counter_freeze:1;
+ u64 overflow_interrupt:1;
+ u64 rsvd3:8;
+ };
+ u64 bits;
+} __packed;
+
+#define IDXD_EVNTCAP_OFFSET 0x80
+union idxd_evntcap {
+ struct {
+ u64 events:28;
+ u64 rsvd:36;
+ };
+ u64 bits;
+} __packed;
+
+struct idxd_event {
+ union {
+ struct {
+ u32 event_category:4;
+ u32 events:28;
+ };
+ u32 val;
+ };
+} __packed;
+
+#define IDXD_CNTRCAP_OFFSET 0x800
+struct idxd_cntrcap {
+ union {
+ struct {
+ u32 counter_width:8;
+ u32 rsvd:20;
+ u32 num_events:4;
+ };
+ u32 val;
+ };
+ struct idxd_event events[];
+} __packed;
+
+#define IDXD_PERFRST_OFFSET 0x10
+union idxd_perfrst {
+ struct {
+ u32 perfrst_config:1;
+ u32 perfrst_counter:1;
+ u32 rsvd:30;
+ };
+ u32 val;
+} __packed;
+
+#define IDXD_OVFSTATUS_OFFSET 0x30
+#define IDXD_PERFFRZ_OFFSET 0x20
+#define IDXD_CNTRCFG_OFFSET 0x100
+union idxd_cntrcfg {
+ struct {
+ u64 enable:1;
+ u64 interrupt_ovf:1;
+ u64 global_freeze_ovf:1;
+ u64 rsvd1:5;
+ u64 event_category:4;
+ u64 rsvd2:20;
+ u64 events:28;
+ u64 rsvd3:4;
+ };
+ u64 val;
+} __packed;
+
+#define IDXD_FLTCFG_OFFSET 0x300
+
+#define IDXD_CNTRDATA_OFFSET 0x200
+union idxd_cntrdata {
+ struct {
+ u64 event_count_value;
+ };
+ u64 val;
+} __packed;
+
+union event_cfg {
+ struct {
+ u64 event_cat:4;
+ u64 event_enc:28;
+ };
+ u64 val;
+} __packed;
+
+union filter_cfg {
+ struct {
+ u64 wq:32;
+ u64 tc:8;
+ u64 pg_sz:4;
+ u64 xfer_sz:8;
+ u64 eng:8;
+ };
+ u64 val;
+} __packed;
+
#endif
diff --git a/drivers/dma/idxd/submit.c b/drivers/dma/idxd/submit.c
index a7a61bcc17d5..19afb62abaff 100644
--- a/drivers/dma/idxd/submit.c
+++ b/drivers/dma/idxd/submit.c
@@ -15,18 +15,30 @@ static struct idxd_desc *__get_desc(struct idxd_wq *wq, int idx, int cpu)
desc = wq->descs[idx];
memset(desc->hw, 0, sizeof(struct dsa_hw_desc));
- memset(desc->completion, 0, idxd->compl_size);
+ memset(desc->completion, 0, idxd->data->compl_size);
desc->cpu = cpu;
if (device_pasid_enabled(idxd))
desc->hw->pasid = idxd->pasid;
/*
- * Descriptor completion vectors are 1-8 for MSIX. We will round
- * robin through the 8 vectors.
+ * Descriptor completion vectors are 1...N for MSIX. We will round
+ * robin through the N vectors.
*/
wq->vec_ptr = (wq->vec_ptr % idxd->num_wq_irqs) + 1;
- desc->hw->int_handle = wq->vec_ptr;
+ if (!idxd->int_handles) {
+ desc->hw->int_handle = wq->vec_ptr;
+ } else {
+ desc->vector = wq->vec_ptr;
+ /*
+ * int_handles are only for descriptor completion. However for device
+ * MSIX enumeration, vec 0 is used for misc interrupts. Therefore even
+ * though we are rotating through 1...N for descriptor interrupts, we
+ * need to acqurie the int_handles from 0..N-1.
+ */
+ desc->hw->int_handle = idxd->int_handles[desc->vector - 1];
+ }
+
return desc;
}
@@ -79,13 +91,15 @@ void idxd_free_desc(struct idxd_wq *wq, struct idxd_desc *desc)
int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc)
{
struct idxd_device *idxd = wq->idxd;
- int vec = desc->hw->int_handle;
void __iomem *portal;
int rc;
if (idxd->state != IDXD_DEV_ENABLED)
return -EIO;
+ if (!percpu_ref_tryget_live(&wq->wq_active))
+ return -ENXIO;
+
portal = wq->portal;
/*
@@ -108,13 +122,25 @@ int idxd_submit_desc(struct idxd_wq *wq, struct idxd_desc *desc)
return rc;
}
+ percpu_ref_put(&wq->wq_active);
+
/*
* Pending the descriptor to the lockless list for the irq_entry
* that we designated the descriptor to.
*/
- if (desc->hw->flags & IDXD_OP_FLAG_RCI)
- llist_add(&desc->llnode,
- &idxd->irq_entries[vec].pending_llist);
+ if (desc->hw->flags & IDXD_OP_FLAG_RCI) {
+ int vec;
+
+ /*
+ * If the driver is on host kernel, it would be the value
+ * assigned to interrupt handle, which is index for MSIX
+ * vector. If it's guest then can't use the int_handle since
+ * that is the index to IMS for the entire device. The guest
+ * device local index will be used.
+ */
+ vec = !idxd->int_handles ? desc->hw->int_handle : desc->vector;
+ llist_add(&desc->llnode, &idxd->irq_entries[vec].pending_llist);
+ }
return 0;
}
diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c
index 4dbb03c545e4..0460d58e3941 100644
--- a/drivers/dma/idxd/sysfs.c
+++ b/drivers/dma/idxd/sysfs.c
@@ -16,69 +16,6 @@ static char *idxd_wq_type_names[] = {
[IDXD_WQT_USER] = "user",
};
-static void idxd_conf_device_release(struct device *dev)
-{
- dev_dbg(dev, "%s for %s\n", __func__, dev_name(dev));
-}
-
-static struct device_type idxd_group_device_type = {
- .name = "group",
- .release = idxd_conf_device_release,
-};
-
-static struct device_type idxd_wq_device_type = {
- .name = "wq",
- .release = idxd_conf_device_release,
-};
-
-static struct device_type idxd_engine_device_type = {
- .name = "engine",
- .release = idxd_conf_device_release,
-};
-
-static struct device_type dsa_device_type = {
- .name = "dsa",
- .release = idxd_conf_device_release,
-};
-
-static struct device_type iax_device_type = {
- .name = "iax",
- .release = idxd_conf_device_release,
-};
-
-static inline bool is_dsa_dev(struct device *dev)
-{
- return dev ? dev->type == &dsa_device_type : false;
-}
-
-static inline bool is_iax_dev(struct device *dev)
-{
- return dev ? dev->type == &iax_device_type : false;
-}
-
-static inline bool is_idxd_dev(struct device *dev)
-{
- return is_dsa_dev(dev) || is_iax_dev(dev);
-}
-
-static inline bool is_idxd_wq_dev(struct device *dev)
-{
- return dev ? dev->type == &idxd_wq_device_type : false;
-}
-
-static inline bool is_idxd_wq_dmaengine(struct idxd_wq *wq)
-{
- if (wq->type == IDXD_WQT_KERNEL &&
- strcmp(wq->name, "dmaengine") == 0)
- return true;
- return false;
-}
-
-static inline bool is_idxd_wq_cdev(struct idxd_wq *wq)
-{
- return wq->type == IDXD_WQT_USER;
-}
-
static int idxd_config_bus_match(struct device *dev,
struct device_driver *drv)
{
@@ -110,9 +47,131 @@ static int idxd_config_bus_match(struct device *dev,
return matched;
}
-static int idxd_config_bus_probe(struct device *dev)
+static int enable_wq(struct idxd_wq *wq)
{
+ struct idxd_device *idxd = wq->idxd;
+ struct device *dev = &idxd->pdev->dev;
+ unsigned long flags;
int rc;
+
+ mutex_lock(&wq->wq_lock);
+
+ if (idxd->state != IDXD_DEV_ENABLED) {
+ mutex_unlock(&wq->wq_lock);
+ dev_warn(dev, "Enabling while device not enabled.\n");
+ return -EPERM;
+ }
+
+ if (wq->state != IDXD_WQ_DISABLED) {
+ mutex_unlock(&wq->wq_lock);
+ dev_warn(dev, "WQ %d already enabled.\n", wq->id);
+ return -EBUSY;
+ }
+
+ if (!wq->group) {
+ mutex_unlock(&wq->wq_lock);
+ dev_warn(dev, "WQ not attached to group.\n");
+ return -EINVAL;
+ }
+
+ if (strlen(wq->name) == 0) {
+ mutex_unlock(&wq->wq_lock);
+ dev_warn(dev, "WQ name not set.\n");
+ return -EINVAL;
+ }
+
+ /* Shared WQ checks */
+ if (wq_shared(wq)) {
+ if (!device_swq_supported(idxd)) {
+ dev_warn(dev, "PASID not enabled and shared WQ.\n");
+ mutex_unlock(&wq->wq_lock);
+ return -ENXIO;
+ }
+ /*
+ * Shared wq with the threshold set to 0 means the user
+ * did not set the threshold or transitioned from a
+ * dedicated wq but did not set threshold. A value
+ * of 0 would effectively disable the shared wq. The
+ * driver does not allow a value of 0 to be set for
+ * threshold via sysfs.
+ */
+ if (wq->threshold == 0) {
+ dev_warn(dev, "Shared WQ and threshold 0.\n");
+ mutex_unlock(&wq->wq_lock);
+ return -EINVAL;
+ }
+ }
+
+ rc = idxd_wq_alloc_resources(wq);
+ if (rc < 0) {
+ mutex_unlock(&wq->wq_lock);
+ dev_warn(dev, "WQ resource alloc failed\n");
+ return rc;
+ }
+
+ spin_lock_irqsave(&idxd->dev_lock, flags);
+ if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags))
+ rc = idxd_device_config(idxd);
+ spin_unlock_irqrestore(&idxd->dev_lock, flags);
+ if (rc < 0) {
+ mutex_unlock(&wq->wq_lock);
+ dev_warn(dev, "Writing WQ %d config failed: %d\n", wq->id, rc);
+ return rc;
+ }
+
+ rc = idxd_wq_enable(wq);
+ if (rc < 0) {
+ mutex_unlock(&wq->wq_lock);
+ dev_warn(dev, "WQ %d enabling failed: %d\n", wq->id, rc);
+ return rc;
+ }
+
+ rc = idxd_wq_map_portal(wq);
+ if (rc < 0) {
+ dev_warn(dev, "wq portal mapping failed: %d\n", rc);
+ rc = idxd_wq_disable(wq);
+ if (rc < 0)
+ dev_warn(dev, "IDXD wq disable failed\n");
+ mutex_unlock(&wq->wq_lock);
+ return rc;
+ }
+
+ wq->client_count = 0;
+
+ if (wq->type == IDXD_WQT_KERNEL) {
+ rc = idxd_wq_init_percpu_ref(wq);
+ if (rc < 0) {
+ dev_dbg(dev, "percpu_ref setup failed\n");
+ mutex_unlock(&wq->wq_lock);
+ return rc;
+ }
+ }
+
+ if (is_idxd_wq_dmaengine(wq)) {
+ rc = idxd_register_dma_channel(wq);
+ if (rc < 0) {
+ dev_dbg(dev, "DMA channel register failed\n");
+ mutex_unlock(&wq->wq_lock);
+ return rc;
+ }
+ } else if (is_idxd_wq_cdev(wq)) {
+ rc = idxd_wq_add_cdev(wq);
+ if (rc < 0) {
+ dev_dbg(dev, "Cdev creation failed\n");
+ mutex_unlock(&wq->wq_lock);
+ return rc;
+ }
+ }
+
+ mutex_unlock(&wq->wq_lock);
+ dev_info(dev, "wq %s enabled\n", dev_name(&wq->conf_dev));
+
+ return 0;
+}
+
+static int idxd_config_bus_probe(struct device *dev)
+{
+ int rc = 0;
unsigned long flags;
dev_dbg(dev, "%s called\n", __func__);
@@ -130,7 +189,8 @@ static int idxd_config_bus_probe(struct device *dev)
/* Perform IDXD configuration and enabling */
spin_lock_irqsave(&idxd->dev_lock, flags);
- rc = idxd_device_config(idxd);
+ if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags))
+ rc = idxd_device_config(idxd);
spin_unlock_irqrestore(&idxd->dev_lock, flags);
if (rc < 0) {
module_put(THIS_MODULE);
@@ -157,115 +217,8 @@ static int idxd_config_bus_probe(struct device *dev)
return 0;
} else if (is_idxd_wq_dev(dev)) {
struct idxd_wq *wq = confdev_to_wq(dev);
- struct idxd_device *idxd = wq->idxd;
-
- mutex_lock(&wq->wq_lock);
-
- if (idxd->state != IDXD_DEV_ENABLED) {
- mutex_unlock(&wq->wq_lock);
- dev_warn(dev, "Enabling while device not enabled.\n");
- return -EPERM;
- }
-
- if (wq->state != IDXD_WQ_DISABLED) {
- mutex_unlock(&wq->wq_lock);
- dev_warn(dev, "WQ %d already enabled.\n", wq->id);
- return -EBUSY;
- }
-
- if (!wq->group) {
- mutex_unlock(&wq->wq_lock);
- dev_warn(dev, "WQ not attached to group.\n");
- return -EINVAL;
- }
-
- if (strlen(wq->name) == 0) {
- mutex_unlock(&wq->wq_lock);
- dev_warn(dev, "WQ name not set.\n");
- return -EINVAL;
- }
-
- /* Shared WQ checks */
- if (wq_shared(wq)) {
- if (!device_swq_supported(idxd)) {
- dev_warn(dev,
- "PASID not enabled and shared WQ.\n");
- mutex_unlock(&wq->wq_lock);
- return -ENXIO;
- }
- /*
- * Shared wq with the threshold set to 0 means the user
- * did not set the threshold or transitioned from a
- * dedicated wq but did not set threshold. A value
- * of 0 would effectively disable the shared wq. The
- * driver does not allow a value of 0 to be set for
- * threshold via sysfs.
- */
- if (wq->threshold == 0) {
- dev_warn(dev,
- "Shared WQ and threshold 0.\n");
- mutex_unlock(&wq->wq_lock);
- return -EINVAL;
- }
- }
-
- rc = idxd_wq_alloc_resources(wq);
- if (rc < 0) {
- mutex_unlock(&wq->wq_lock);
- dev_warn(dev, "WQ resource alloc failed\n");
- return rc;
- }
-
- spin_lock_irqsave(&idxd->dev_lock, flags);
- rc = idxd_device_config(idxd);
- spin_unlock_irqrestore(&idxd->dev_lock, flags);
- if (rc < 0) {
- mutex_unlock(&wq->wq_lock);
- dev_warn(dev, "Writing WQ %d config failed: %d\n",
- wq->id, rc);
- return rc;
- }
-
- rc = idxd_wq_enable(wq);
- if (rc < 0) {
- mutex_unlock(&wq->wq_lock);
- dev_warn(dev, "WQ %d enabling failed: %d\n",
- wq->id, rc);
- return rc;
- }
-
- rc = idxd_wq_map_portal(wq);
- if (rc < 0) {
- dev_warn(dev, "wq portal mapping failed: %d\n", rc);
- rc = idxd_wq_disable(wq);
- if (rc < 0)
- dev_warn(dev, "IDXD wq disable failed\n");
- mutex_unlock(&wq->wq_lock);
- return rc;
- }
-
- wq->client_count = 0;
- dev_info(dev, "wq %s enabled\n", dev_name(&wq->conf_dev));
-
- if (is_idxd_wq_dmaengine(wq)) {
- rc = idxd_register_dma_channel(wq);
- if (rc < 0) {
- dev_dbg(dev, "DMA channel register failed\n");
- mutex_unlock(&wq->wq_lock);
- return rc;
- }
- } else if (is_idxd_wq_cdev(wq)) {
- rc = idxd_wq_add_cdev(wq);
- if (rc < 0) {
- dev_dbg(dev, "Cdev creation failed\n");
- mutex_unlock(&wq->wq_lock);
- return rc;
- }
- }
-
- mutex_unlock(&wq->wq_lock);
- return 0;
+ return enable_wq(wq);
}
return -ENODEV;
@@ -275,7 +228,6 @@ static void disable_wq(struct idxd_wq *wq)
{
struct idxd_device *idxd = wq->idxd;
struct device *dev = &idxd->pdev->dev;
- int rc;
mutex_lock(&wq->wq_lock);
dev_dbg(dev, "%s removing WQ %s\n", __func__, dev_name(&wq->conf_dev));
@@ -284,6 +236,9 @@ static void disable_wq(struct idxd_wq *wq)
return;
}
+ if (wq->type == IDXD_WQT_KERNEL)
+ idxd_wq_quiesce(wq);
+
if (is_idxd_wq_dmaengine(wq))
idxd_unregister_dma_channel(wq);
else if (is_idxd_wq_cdev(wq))
@@ -296,17 +251,13 @@ static void disable_wq(struct idxd_wq *wq)
idxd_wq_unmap_portal(wq);
idxd_wq_drain(wq);
- rc = idxd_wq_disable(wq);
+ idxd_wq_reset(wq);
idxd_wq_free_resources(wq);
wq->client_count = 0;
mutex_unlock(&wq->wq_lock);
- if (rc < 0)
- dev_warn(dev, "Failed to disable %s: %d\n",
- dev_name(&wq->conf_dev), rc);
- else
- dev_info(dev, "wq %s disabled\n", dev_name(&wq->conf_dev));
+ dev_info(dev, "wq %s disabled\n", dev_name(&wq->conf_dev));
}
static int idxd_config_bus_remove(struct device *dev)
@@ -327,7 +278,7 @@ static int idxd_config_bus_remove(struct device *dev)
dev_dbg(dev, "%s removing dev %s\n", __func__,
dev_name(&idxd->conf_dev));
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
if (wq->state == IDXD_WQ_DISABLED)
continue;
@@ -338,12 +289,14 @@ static int idxd_config_bus_remove(struct device *dev)
idxd_unregister_dma_device(idxd);
rc = idxd_device_disable(idxd);
- for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) {
+ for (i = 0; i < idxd->max_wqs; i++) {
+ struct idxd_wq *wq = idxd->wqs[i];
- mutex_lock(&wq->wq_lock);
- idxd_wq_disable_cleanup(wq);
- mutex_unlock(&wq->wq_lock);
+ mutex_lock(&wq->wq_lock);
+ idxd_wq_disable_cleanup(wq);
+ mutex_unlock(&wq->wq_lock);
+ }
}
module_put(THIS_MODULE);
if (rc < 0)
@@ -369,19 +322,6 @@ struct bus_type dsa_bus_type = {
.shutdown = idxd_config_bus_shutdown,
};
-struct bus_type iax_bus_type = {
- .name = "iax",
- .match = idxd_config_bus_match,
- .probe = idxd_config_bus_probe,
- .remove = idxd_config_bus_remove,
- .shutdown = idxd_config_bus_shutdown,
-};
-
-static struct bus_type *idxd_bus_types[] = {
- &dsa_bus_type,
- &iax_bus_type
-};
-
static struct idxd_device_driver dsa_drv = {
.drv = {
.name = "dsa",
@@ -391,60 +331,15 @@ static struct idxd_device_driver dsa_drv = {
},
};
-static struct idxd_device_driver iax_drv = {
- .drv = {
- .name = "iax",
- .bus = &iax_bus_type,
- .owner = THIS_MODULE,
- .mod_name = KBUILD_MODNAME,
- },
-};
-
-static struct idxd_device_driver *idxd_drvs[] = {
- &dsa_drv,
- &iax_drv
-};
-
-struct bus_type *idxd_get_bus_type(struct idxd_device *idxd)
-{
- return idxd_bus_types[idxd->type];
-}
-
-static struct device_type *idxd_get_device_type(struct idxd_device *idxd)
-{
- if (idxd->type == IDXD_TYPE_DSA)
- return &dsa_device_type;
- else if (idxd->type == IDXD_TYPE_IAX)
- return &iax_device_type;
- else
- return NULL;
-}
-
/* IDXD generic driver setup */
int idxd_register_driver(void)
{
- int i, rc;
-
- for (i = 0; i < IDXD_TYPE_MAX; i++) {
- rc = driver_register(&idxd_drvs[i]->drv);
- if (rc < 0)
- goto drv_fail;
- }
-
- return 0;
-
-drv_fail:
- while (--i >= 0)
- driver_unregister(&idxd_drvs[i]->drv);
- return rc;
+ return driver_register(&dsa_drv.drv);
}
void idxd_unregister_driver(void)
{
- int i;
-
- for (i = 0; i < IDXD_TYPE_MAX; i++)
- driver_unregister(&idxd_drvs[i]->drv);
+ driver_unregister(&dsa_drv.drv);
}
/* IDXD engine attributes */
@@ -455,9 +350,9 @@ static ssize_t engine_group_id_show(struct device *dev,
container_of(dev, struct idxd_engine, conf_dev);
if (engine->group)
- return sprintf(buf, "%d\n", engine->group->id);
+ return sysfs_emit(buf, "%d\n", engine->group->id);
else
- return sprintf(buf, "%d\n", -1);
+ return sysfs_emit(buf, "%d\n", -1);
}
static ssize_t engine_group_id_store(struct device *dev,
@@ -493,7 +388,7 @@ static ssize_t engine_group_id_store(struct device *dev,
if (prevg)
prevg->num_engines--;
- engine->group = &idxd->groups[id];
+ engine->group = idxd->groups[id];
engine->group->num_engines++;
return count;
@@ -517,6 +412,19 @@ static const struct attribute_group *idxd_engine_attribute_groups[] = {
NULL,
};
+static void idxd_conf_engine_release(struct device *dev)
+{
+ struct idxd_engine *engine = container_of(dev, struct idxd_engine, conf_dev);
+
+ kfree(engine);
+}
+
+struct device_type idxd_engine_device_type = {
+ .name = "engine",
+ .release = idxd_conf_engine_release,
+ .groups = idxd_engine_attribute_groups,
+};
+
/* Group attributes */
static void idxd_set_free_tokens(struct idxd_device *idxd)
@@ -524,7 +432,7 @@ static void idxd_set_free_tokens(struct idxd_device *idxd)
int i, tokens;
for (i = 0, tokens = 0; i < idxd->max_groups; i++) {
- struct idxd_group *g = &idxd->groups[i];
+ struct idxd_group *g = idxd->groups[i];
tokens += g->tokens_reserved;
}
@@ -539,7 +447,7 @@ static ssize_t group_tokens_reserved_show(struct device *dev,
struct idxd_group *group =
container_of(dev, struct idxd_group, conf_dev);
- return sprintf(buf, "%u\n", group->tokens_reserved);
+ return sysfs_emit(buf, "%u\n", group->tokens_reserved);
}
static ssize_t group_tokens_reserved_store(struct device *dev,
@@ -556,7 +464,7 @@ static ssize_t group_tokens_reserved_store(struct device *dev,
if (rc < 0)
return -EINVAL;
- if (idxd->type == IDXD_TYPE_IAX)
+ if (idxd->data->type == IDXD_TYPE_IAX)
return -EOPNOTSUPP;
if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags))
@@ -587,7 +495,7 @@ static ssize_t group_tokens_allowed_show(struct device *dev,
struct idxd_group *group =
container_of(dev, struct idxd_group, conf_dev);
- return sprintf(buf, "%u\n", group->tokens_allowed);
+ return sysfs_emit(buf, "%u\n", group->tokens_allowed);
}
static ssize_t group_tokens_allowed_store(struct device *dev,
@@ -604,7 +512,7 @@ static ssize_t group_tokens_allowed_store(struct device *dev,
if (rc < 0)
return -EINVAL;
- if (idxd->type == IDXD_TYPE_IAX)
+ if (idxd->data->type == IDXD_TYPE_IAX)
return -EOPNOTSUPP;
if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags))
@@ -632,7 +540,7 @@ static ssize_t group_use_token_limit_show(struct device *dev,
struct idxd_group *group =
container_of(dev, struct idxd_group, conf_dev);
- return sprintf(buf, "%u\n", group->use_token_limit);
+ return sysfs_emit(buf, "%u\n", group->use_token_limit);
}
static ssize_t group_use_token_limit_store(struct device *dev,
@@ -649,7 +557,7 @@ static ssize_t group_use_token_limit_store(struct device *dev,
if (rc < 0)
return -EINVAL;
- if (idxd->type == IDXD_TYPE_IAX)
+ if (idxd->data->type == IDXD_TYPE_IAX)
return -EOPNOTSUPP;
if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags))
@@ -675,22 +583,22 @@ static ssize_t group_engines_show(struct device *dev,
struct idxd_group *group =
container_of(dev, struct idxd_group, conf_dev);
int i, rc = 0;
- char *tmp = buf;
struct idxd_device *idxd = group->idxd;
for (i = 0; i < idxd->max_engines; i++) {
- struct idxd_engine *engine = &idxd->engines[i];
+ struct idxd_engine *engine = idxd->engines[i];
if (!engine->group)
continue;
if (engine->group->id == group->id)
- rc += sprintf(tmp + rc, "engine%d.%d ",
- idxd->id, engine->id);
+ rc += sysfs_emit_at(buf, rc, "engine%d.%d ", idxd->id, engine->id);
}
+ if (!rc)
+ return 0;
rc--;
- rc += sprintf(tmp + rc, "\n");
+ rc += sysfs_emit_at(buf, rc, "\n");
return rc;
}
@@ -704,22 +612,22 @@ static ssize_t group_work_queues_show(struct device *dev,
struct idxd_group *group =
container_of(dev, struct idxd_group, conf_dev);
int i, rc = 0;
- char *tmp = buf;
struct idxd_device *idxd = group->idxd;
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
if (!wq->group)
continue;
if (wq->group->id == group->id)
- rc += sprintf(tmp + rc, "wq%d.%d ",
- idxd->id, wq->id);
+ rc += sysfs_emit_at(buf, rc, "wq%d.%d ", idxd->id, wq->id);
}
+ if (!rc)
+ return 0;
rc--;
- rc += sprintf(tmp + rc, "\n");
+ rc += sysfs_emit_at(buf, rc, "\n");
return rc;
}
@@ -734,7 +642,7 @@ static ssize_t group_traffic_class_a_show(struct device *dev,
struct idxd_group *group =
container_of(dev, struct idxd_group, conf_dev);
- return sprintf(buf, "%d\n", group->tc_a);
+ return sysfs_emit(buf, "%d\n", group->tc_a);
}
static ssize_t group_traffic_class_a_store(struct device *dev,
@@ -775,7 +683,7 @@ static ssize_t group_traffic_class_b_show(struct device *dev,
struct idxd_group *group =
container_of(dev, struct idxd_group, conf_dev);
- return sprintf(buf, "%d\n", group->tc_b);
+ return sysfs_emit(buf, "%d\n", group->tc_b);
}
static ssize_t group_traffic_class_b_store(struct device *dev,
@@ -829,13 +737,26 @@ static const struct attribute_group *idxd_group_attribute_groups[] = {
NULL,
};
+static void idxd_conf_group_release(struct device *dev)
+{
+ struct idxd_group *group = container_of(dev, struct idxd_group, conf_dev);
+
+ kfree(group);
+}
+
+struct device_type idxd_group_device_type = {
+ .name = "group",
+ .release = idxd_conf_group_release,
+ .groups = idxd_group_attribute_groups,
+};
+
/* IDXD work queue attribs */
static ssize_t wq_clients_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%d\n", wq->client_count);
+ return sysfs_emit(buf, "%d\n", wq->client_count);
}
static struct device_attribute dev_attr_wq_clients =
@@ -848,12 +769,12 @@ static ssize_t wq_state_show(struct device *dev,
switch (wq->state) {
case IDXD_WQ_DISABLED:
- return sprintf(buf, "disabled\n");
+ return sysfs_emit(buf, "disabled\n");
case IDXD_WQ_ENABLED:
- return sprintf(buf, "enabled\n");
+ return sysfs_emit(buf, "enabled\n");
}
- return sprintf(buf, "unknown\n");
+ return sysfs_emit(buf, "unknown\n");
}
static struct device_attribute dev_attr_wq_state =
@@ -865,9 +786,9 @@ static ssize_t wq_group_id_show(struct device *dev,
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
if (wq->group)
- return sprintf(buf, "%u\n", wq->group->id);
+ return sysfs_emit(buf, "%u\n", wq->group->id);
else
- return sprintf(buf, "-1\n");
+ return sysfs_emit(buf, "-1\n");
}
static ssize_t wq_group_id_store(struct device *dev,
@@ -901,7 +822,7 @@ static ssize_t wq_group_id_store(struct device *dev,
return count;
}
- group = &idxd->groups[id];
+ group = idxd->groups[id];
prevg = wq->group;
if (prevg)
@@ -919,8 +840,7 @@ static ssize_t wq_mode_show(struct device *dev, struct device_attribute *attr,
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%s\n",
- wq_dedicated(wq) ? "dedicated" : "shared");
+ return sysfs_emit(buf, "%s\n", wq_dedicated(wq) ? "dedicated" : "shared");
}
static ssize_t wq_mode_store(struct device *dev,
@@ -956,7 +876,7 @@ static ssize_t wq_size_show(struct device *dev, struct device_attribute *attr,
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%u\n", wq->size);
+ return sysfs_emit(buf, "%u\n", wq->size);
}
static int total_claimed_wq_size(struct idxd_device *idxd)
@@ -965,7 +885,7 @@ static int total_claimed_wq_size(struct idxd_device *idxd)
int wq_size = 0;
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
wq_size += wq->size;
}
@@ -989,7 +909,7 @@ static ssize_t wq_size_store(struct device *dev,
if (!test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags))
return -EPERM;
- if (wq->state != IDXD_WQ_DISABLED)
+ if (idxd->state == IDXD_DEV_ENABLED)
return -EPERM;
if (size + total_claimed_wq_size(idxd) - wq->size > idxd->max_wq_size)
@@ -1007,7 +927,7 @@ static ssize_t wq_priority_show(struct device *dev,
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%u\n", wq->priority);
+ return sysfs_emit(buf, "%u\n", wq->priority);
}
static ssize_t wq_priority_store(struct device *dev,
@@ -1044,8 +964,7 @@ static ssize_t wq_block_on_fault_show(struct device *dev,
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%u\n",
- test_bit(WQ_FLAG_BLOCK_ON_FAULT, &wq->flags));
+ return sysfs_emit(buf, "%u\n", test_bit(WQ_FLAG_BLOCK_ON_FAULT, &wq->flags));
}
static ssize_t wq_block_on_fault_store(struct device *dev,
@@ -1084,7 +1003,7 @@ static ssize_t wq_threshold_show(struct device *dev,
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%u\n", wq->threshold);
+ return sysfs_emit(buf, "%u\n", wq->threshold);
}
static ssize_t wq_threshold_store(struct device *dev,
@@ -1127,15 +1046,12 @@ static ssize_t wq_type_show(struct device *dev,
switch (wq->type) {
case IDXD_WQT_KERNEL:
- return sprintf(buf, "%s\n",
- idxd_wq_type_names[IDXD_WQT_KERNEL]);
+ return sysfs_emit(buf, "%s\n", idxd_wq_type_names[IDXD_WQT_KERNEL]);
case IDXD_WQT_USER:
- return sprintf(buf, "%s\n",
- idxd_wq_type_names[IDXD_WQT_USER]);
+ return sysfs_emit(buf, "%s\n", idxd_wq_type_names[IDXD_WQT_USER]);
case IDXD_WQT_NONE:
default:
- return sprintf(buf, "%s\n",
- idxd_wq_type_names[IDXD_WQT_NONE]);
+ return sysfs_emit(buf, "%s\n", idxd_wq_type_names[IDXD_WQT_NONE]);
}
return -EINVAL;
@@ -1176,7 +1092,7 @@ static ssize_t wq_name_show(struct device *dev,
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%s\n", wq->name);
+ return sysfs_emit(buf, "%s\n", wq->name);
}
static ssize_t wq_name_store(struct device *dev,
@@ -1211,8 +1127,16 @@ static ssize_t wq_cdev_minor_show(struct device *dev,
struct device_attribute *attr, char *buf)
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
+ int minor = -1;
+
+ mutex_lock(&wq->wq_lock);
+ if (wq->idxd_cdev)
+ minor = wq->idxd_cdev->minor;
+ mutex_unlock(&wq->wq_lock);
- return sprintf(buf, "%d\n", wq->idxd_cdev.minor);
+ if (minor == -1)
+ return -ENXIO;
+ return sysfs_emit(buf, "%d\n", minor);
}
static struct device_attribute dev_attr_wq_cdev_minor =
@@ -1238,7 +1162,7 @@ static ssize_t wq_max_transfer_size_show(struct device *dev, struct device_attri
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%llu\n", wq->max_xfer_bytes);
+ return sysfs_emit(buf, "%llu\n", wq->max_xfer_bytes);
}
static ssize_t wq_max_transfer_size_store(struct device *dev, struct device_attribute *attr,
@@ -1272,7 +1196,7 @@ static ssize_t wq_max_batch_size_show(struct device *dev, struct device_attribut
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%u\n", wq->max_batch_size);
+ return sysfs_emit(buf, "%u\n", wq->max_batch_size);
}
static ssize_t wq_max_batch_size_store(struct device *dev, struct device_attribute *attr,
@@ -1305,7 +1229,7 @@ static ssize_t wq_ats_disable_show(struct device *dev, struct device_attribute *
{
struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
- return sprintf(buf, "%u\n", wq->ats_dis);
+ return sysfs_emit(buf, "%u\n", wq->ats_dis);
}
static ssize_t wq_ats_disable_store(struct device *dev, struct device_attribute *attr,
@@ -1361,6 +1285,20 @@ static const struct attribute_group *idxd_wq_attribute_groups[] = {
NULL,
};
+static void idxd_conf_wq_release(struct device *dev)
+{
+ struct idxd_wq *wq = container_of(dev, struct idxd_wq, conf_dev);
+
+ kfree(wq->wqcfg);
+ kfree(wq);
+}
+
+struct device_type idxd_wq_device_type = {
+ .name = "wq",
+ .release = idxd_conf_wq_release,
+ .groups = idxd_wq_attribute_groups,
+};
+
/* IDXD device attribs */
static ssize_t version_show(struct device *dev, struct device_attribute *attr,
char *buf)
@@ -1368,7 +1306,7 @@ static ssize_t version_show(struct device *dev, struct device_attribute *attr,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%#x\n", idxd->hw.version);
+ return sysfs_emit(buf, "%#x\n", idxd->hw.version);
}
static DEVICE_ATTR_RO(version);
@@ -1379,7 +1317,7 @@ static ssize_t max_work_queues_size_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->max_wq_size);
+ return sysfs_emit(buf, "%u\n", idxd->max_wq_size);
}
static DEVICE_ATTR_RO(max_work_queues_size);
@@ -1389,7 +1327,7 @@ static ssize_t max_groups_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->max_groups);
+ return sysfs_emit(buf, "%u\n", idxd->max_groups);
}
static DEVICE_ATTR_RO(max_groups);
@@ -1399,7 +1337,7 @@ static ssize_t max_work_queues_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->max_wqs);
+ return sysfs_emit(buf, "%u\n", idxd->max_wqs);
}
static DEVICE_ATTR_RO(max_work_queues);
@@ -1409,7 +1347,7 @@ static ssize_t max_engines_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->max_engines);
+ return sysfs_emit(buf, "%u\n", idxd->max_engines);
}
static DEVICE_ATTR_RO(max_engines);
@@ -1419,7 +1357,7 @@ static ssize_t numa_node_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%d\n", dev_to_node(&idxd->pdev->dev));
+ return sysfs_emit(buf, "%d\n", dev_to_node(&idxd->pdev->dev));
}
static DEVICE_ATTR_RO(numa_node);
@@ -1429,7 +1367,7 @@ static ssize_t max_batch_size_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->max_batch_size);
+ return sysfs_emit(buf, "%u\n", idxd->max_batch_size);
}
static DEVICE_ATTR_RO(max_batch_size);
@@ -1440,7 +1378,7 @@ static ssize_t max_transfer_size_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%llu\n", idxd->max_xfer_bytes);
+ return sysfs_emit(buf, "%llu\n", idxd->max_xfer_bytes);
}
static DEVICE_ATTR_RO(max_transfer_size);
@@ -1449,8 +1387,14 @@ static ssize_t op_cap_show(struct device *dev,
{
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
+ int i, rc = 0;
+
+ for (i = 0; i < 4; i++)
+ rc += sysfs_emit_at(buf, rc, "%#llx ", idxd->hw.opcap.bits[i]);
- return sprintf(buf, "%#llx\n", idxd->hw.opcap.bits[0]);
+ rc--;
+ rc += sysfs_emit_at(buf, rc, "\n");
+ return rc;
}
static DEVICE_ATTR_RO(op_cap);
@@ -1460,7 +1404,7 @@ static ssize_t gen_cap_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%#llx\n", idxd->hw.gen_cap.bits);
+ return sysfs_emit(buf, "%#llx\n", idxd->hw.gen_cap.bits);
}
static DEVICE_ATTR_RO(gen_cap);
@@ -1470,8 +1414,7 @@ static ssize_t configurable_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n",
- test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags));
+ return sysfs_emit(buf, "%u\n", test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags));
}
static DEVICE_ATTR_RO(configurable);
@@ -1485,13 +1428,13 @@ static ssize_t clients_show(struct device *dev,
spin_lock_irqsave(&idxd->dev_lock, flags);
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
count += wq->client_count;
}
spin_unlock_irqrestore(&idxd->dev_lock, flags);
- return sprintf(buf, "%d\n", count);
+ return sysfs_emit(buf, "%d\n", count);
}
static DEVICE_ATTR_RO(clients);
@@ -1501,7 +1444,7 @@ static ssize_t pasid_enabled_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", device_pasid_enabled(idxd));
+ return sysfs_emit(buf, "%u\n", device_pasid_enabled(idxd));
}
static DEVICE_ATTR_RO(pasid_enabled);
@@ -1514,14 +1457,14 @@ static ssize_t state_show(struct device *dev,
switch (idxd->state) {
case IDXD_DEV_DISABLED:
case IDXD_DEV_CONF_READY:
- return sprintf(buf, "disabled\n");
+ return sysfs_emit(buf, "disabled\n");
case IDXD_DEV_ENABLED:
- return sprintf(buf, "enabled\n");
+ return sysfs_emit(buf, "enabled\n");
case IDXD_DEV_HALTED:
- return sprintf(buf, "halted\n");
+ return sysfs_emit(buf, "halted\n");
}
- return sprintf(buf, "unknown\n");
+ return sysfs_emit(buf, "unknown\n");
}
static DEVICE_ATTR_RO(state);
@@ -1535,10 +1478,10 @@ static ssize_t errors_show(struct device *dev,
spin_lock_irqsave(&idxd->dev_lock, flags);
for (i = 0; i < 4; i++)
- out += sprintf(buf + out, "%#018llx ", idxd->sw_err.bits[i]);
+ out += sysfs_emit_at(buf, out, "%#018llx ", idxd->sw_err.bits[i]);
spin_unlock_irqrestore(&idxd->dev_lock, flags);
out--;
- out += sprintf(buf + out, "\n");
+ out += sysfs_emit_at(buf, out, "\n");
return out;
}
static DEVICE_ATTR_RO(errors);
@@ -1549,7 +1492,7 @@ static ssize_t max_tokens_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->max_tokens);
+ return sysfs_emit(buf, "%u\n", idxd->max_tokens);
}
static DEVICE_ATTR_RO(max_tokens);
@@ -1559,7 +1502,7 @@ static ssize_t token_limit_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->token_limit);
+ return sysfs_emit(buf, "%u\n", idxd->token_limit);
}
static ssize_t token_limit_store(struct device *dev,
@@ -1598,7 +1541,7 @@ static ssize_t cdev_major_show(struct device *dev,
struct idxd_device *idxd =
container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%u\n", idxd->major);
+ return sysfs_emit(buf, "%u\n", idxd->major);
}
static DEVICE_ATTR_RO(cdev_major);
@@ -1607,7 +1550,7 @@ static ssize_t cmd_status_show(struct device *dev,
{
struct idxd_device *idxd = container_of(dev, struct idxd_device, conf_dev);
- return sprintf(buf, "%#x\n", idxd->cmd_status);
+ return sysfs_emit(buf, "%#x\n", idxd->cmd_status);
}
static DEVICE_ATTR_RO(cmd_status);
@@ -1643,183 +1586,161 @@ static const struct attribute_group *idxd_attribute_groups[] = {
NULL,
};
-static int idxd_setup_engine_sysfs(struct idxd_device *idxd)
+static void idxd_conf_device_release(struct device *dev)
{
- struct device *dev = &idxd->pdev->dev;
- int i, rc;
+ struct idxd_device *idxd = container_of(dev, struct idxd_device, conf_dev);
+
+ kfree(idxd->groups);
+ kfree(idxd->wqs);
+ kfree(idxd->engines);
+ kfree(idxd->irq_entries);
+ kfree(idxd->int_handles);
+ ida_free(&idxd_ida, idxd->id);
+ kfree(idxd);
+}
+
+struct device_type dsa_device_type = {
+ .name = "dsa",
+ .release = idxd_conf_device_release,
+ .groups = idxd_attribute_groups,
+};
+
+struct device_type iax_device_type = {
+ .name = "iax",
+ .release = idxd_conf_device_release,
+ .groups = idxd_attribute_groups,
+};
+
+static int idxd_register_engine_devices(struct idxd_device *idxd)
+{
+ int i, j, rc;
for (i = 0; i < idxd->max_engines; i++) {
- struct idxd_engine *engine = &idxd->engines[i];
-
- engine->conf_dev.parent = &idxd->conf_dev;
- dev_set_name(&engine->conf_dev, "engine%d.%d",
- idxd->id, engine->id);
- engine->conf_dev.bus = idxd_get_bus_type(idxd);
- engine->conf_dev.groups = idxd_engine_attribute_groups;
- engine->conf_dev.type = &idxd_engine_device_type;
- dev_dbg(dev, "Engine device register: %s\n",
- dev_name(&engine->conf_dev));
- rc = device_register(&engine->conf_dev);
- if (rc < 0) {
- put_device(&engine->conf_dev);
+ struct idxd_engine *engine = idxd->engines[i];
+
+ rc = device_add(&engine->conf_dev);
+ if (rc < 0)
goto cleanup;
- }
}
return 0;
cleanup:
- while (i--) {
- struct idxd_engine *engine = &idxd->engines[i];
+ j = i - 1;
+ for (; i < idxd->max_engines; i++)
+ put_device(&idxd->engines[i]->conf_dev);
- device_unregister(&engine->conf_dev);
- }
+ while (j--)
+ device_unregister(&idxd->engines[j]->conf_dev);
return rc;
}
-static int idxd_setup_group_sysfs(struct idxd_device *idxd)
+static int idxd_register_group_devices(struct idxd_device *idxd)
{
- struct device *dev = &idxd->pdev->dev;
- int i, rc;
+ int i, j, rc;
for (i = 0; i < idxd->max_groups; i++) {
- struct idxd_group *group = &idxd->groups[i];
-
- group->conf_dev.parent = &idxd->conf_dev;
- dev_set_name(&group->conf_dev, "group%d.%d",
- idxd->id, group->id);
- group->conf_dev.bus = idxd_get_bus_type(idxd);
- group->conf_dev.groups = idxd_group_attribute_groups;
- group->conf_dev.type = &idxd_group_device_type;
- dev_dbg(dev, "Group device register: %s\n",
- dev_name(&group->conf_dev));
- rc = device_register(&group->conf_dev);
- if (rc < 0) {
- put_device(&group->conf_dev);
+ struct idxd_group *group = idxd->groups[i];
+
+ rc = device_add(&group->conf_dev);
+ if (rc < 0)
goto cleanup;
- }
}
return 0;
cleanup:
- while (i--) {
- struct idxd_group *group = &idxd->groups[i];
+ j = i - 1;
+ for (; i < idxd->max_groups; i++)
+ put_device(&idxd->groups[i]->conf_dev);
- device_unregister(&group->conf_dev);
- }
+ while (j--)
+ device_unregister(&idxd->groups[j]->conf_dev);
return rc;
}
-static int idxd_setup_wq_sysfs(struct idxd_device *idxd)
+static int idxd_register_wq_devices(struct idxd_device *idxd)
{
- struct device *dev = &idxd->pdev->dev;
- int i, rc;
+ int i, rc, j;
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
-
- wq->conf_dev.parent = &idxd->conf_dev;
- dev_set_name(&wq->conf_dev, "wq%d.%d", idxd->id, wq->id);
- wq->conf_dev.bus = idxd_get_bus_type(idxd);
- wq->conf_dev.groups = idxd_wq_attribute_groups;
- wq->conf_dev.type = &idxd_wq_device_type;
- dev_dbg(dev, "WQ device register: %s\n",
- dev_name(&wq->conf_dev));
- rc = device_register(&wq->conf_dev);
- if (rc < 0) {
- put_device(&wq->conf_dev);
+ struct idxd_wq *wq = idxd->wqs[i];
+
+ rc = device_add(&wq->conf_dev);
+ if (rc < 0)
goto cleanup;
- }
}
return 0;
cleanup:
- while (i--) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ j = i - 1;
+ for (; i < idxd->max_wqs; i++)
+ put_device(&idxd->wqs[i]->conf_dev);
- device_unregister(&wq->conf_dev);
- }
+ while (j--)
+ device_unregister(&idxd->wqs[j]->conf_dev);
return rc;
}
-static int idxd_setup_device_sysfs(struct idxd_device *idxd)
-{
- struct device *dev = &idxd->pdev->dev;
- int rc;
- char devname[IDXD_NAME_SIZE];
-
- sprintf(devname, "%s%d", idxd_get_dev_name(idxd), idxd->id);
- idxd->conf_dev.parent = dev;
- dev_set_name(&idxd->conf_dev, "%s", devname);
- idxd->conf_dev.bus = idxd_get_bus_type(idxd);
- idxd->conf_dev.groups = idxd_attribute_groups;
- idxd->conf_dev.type = idxd_get_device_type(idxd);
-
- dev_dbg(dev, "IDXD device register: %s\n", dev_name(&idxd->conf_dev));
- rc = device_register(&idxd->conf_dev);
- if (rc < 0) {
- put_device(&idxd->conf_dev);
- return rc;
- }
-
- return 0;
-}
-
-int idxd_setup_sysfs(struct idxd_device *idxd)
+int idxd_register_devices(struct idxd_device *idxd)
{
struct device *dev = &idxd->pdev->dev;
- int rc;
+ int rc, i;
- rc = idxd_setup_device_sysfs(idxd);
- if (rc < 0) {
- dev_dbg(dev, "Device sysfs registering failed: %d\n", rc);
+ rc = device_add(&idxd->conf_dev);
+ if (rc < 0)
return rc;
- }
- rc = idxd_setup_wq_sysfs(idxd);
+ rc = idxd_register_wq_devices(idxd);
if (rc < 0) {
- /* unregister conf dev */
- dev_dbg(dev, "Work Queue sysfs registering failed: %d\n", rc);
- return rc;
+ dev_dbg(dev, "WQ devices registering failed: %d\n", rc);
+ goto err_wq;
}
- rc = idxd_setup_group_sysfs(idxd);
+ rc = idxd_register_engine_devices(idxd);
if (rc < 0) {
- /* unregister conf dev */
- dev_dbg(dev, "Group sysfs registering failed: %d\n", rc);
- return rc;
+ dev_dbg(dev, "Engine devices registering failed: %d\n", rc);
+ goto err_engine;
}
- rc = idxd_setup_engine_sysfs(idxd);
+ rc = idxd_register_group_devices(idxd);
if (rc < 0) {
- /* unregister conf dev */
- dev_dbg(dev, "Engine sysfs registering failed: %d\n", rc);
- return rc;
+ dev_dbg(dev, "Group device registering failed: %d\n", rc);
+ goto err_group;
}
return 0;
+
+ err_group:
+ for (i = 0; i < idxd->max_engines; i++)
+ device_unregister(&idxd->engines[i]->conf_dev);
+ err_engine:
+ for (i = 0; i < idxd->max_wqs; i++)
+ device_unregister(&idxd->wqs[i]->conf_dev);
+ err_wq:
+ device_del(&idxd->conf_dev);
+ return rc;
}
-void idxd_cleanup_sysfs(struct idxd_device *idxd)
+void idxd_unregister_devices(struct idxd_device *idxd)
{
int i;
for (i = 0; i < idxd->max_wqs; i++) {
- struct idxd_wq *wq = &idxd->wqs[i];
+ struct idxd_wq *wq = idxd->wqs[i];
device_unregister(&wq->conf_dev);
}
for (i = 0; i < idxd->max_engines; i++) {
- struct idxd_engine *engine = &idxd->engines[i];
+ struct idxd_engine *engine = idxd->engines[i];
device_unregister(&engine->conf_dev);
}
for (i = 0; i < idxd->max_groups; i++) {
- struct idxd_group *group = &idxd->groups[i];
+ struct idxd_group *group = idxd->groups[i];
device_unregister(&group->conf_dev);
}
@@ -1829,26 +1750,10 @@ void idxd_cleanup_sysfs(struct idxd_device *idxd)
int idxd_register_bus_type(void)
{
- int i, rc;
-
- for (i = 0; i < IDXD_TYPE_MAX; i++) {
- rc = bus_register(idxd_bus_types[i]);
- if (rc < 0)
- goto bus_err;
- }
-
- return 0;
-
-bus_err:
- while (--i >= 0)
- bus_unregister(idxd_bus_types[i]);
- return rc;
+ return bus_register(&dsa_bus_type);
}
void idxd_unregister_bus_type(void)
{
- int i;
-
- for (i = 0; i < IDXD_TYPE_MAX; i++)
- bus_unregister(idxd_bus_types[i]);
+ bus_unregister(&dsa_bus_type);
}
diff --git a/drivers/dma/k3dma.c b/drivers/dma/k3dma.c
index d0b2e601e3e5..ecdaada95120 100644
--- a/drivers/dma/k3dma.c
+++ b/drivers/dma/k3dma.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (c) 2013 - 2015 Linaro Ltd.
- * Copyright (c) 2013 Hisilicon Limited.
+ * Copyright (c) 2013 HiSilicon Limited.
*/
#include <linux/sched.h>
#include <linux/device.h>
@@ -1039,6 +1039,6 @@ static struct platform_driver k3_pdma_driver = {
module_platform_driver(k3_pdma_driver);
-MODULE_DESCRIPTION("Hisilicon k3 DMA Driver");
+MODULE_DESCRIPTION("HiSilicon k3 DMA Driver");
MODULE_ALIAS("platform:k3dma");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/dma/plx_dma.c b/drivers/dma/plx_dma.c
index f387c5bbc170..166934544161 100644
--- a/drivers/dma/plx_dma.c
+++ b/drivers/dma/plx_dma.c
@@ -507,10 +507,8 @@ static int plx_dma_create(struct pci_dev *pdev)
rc = request_irq(pci_irq_vector(pdev, 0), plx_dma_isr, 0,
KBUILD_MODNAME, plxdev);
- if (rc) {
- kfree(plxdev);
- return rc;
- }
+ if (rc)
+ goto free_plx;
spin_lock_init(&plxdev->ring_lock);
tasklet_setup(&plxdev->desc_task, plx_dma_desc_task);
@@ -540,14 +538,20 @@ static int plx_dma_create(struct pci_dev *pdev)
rc = dma_async_device_register(dma);
if (rc) {
pci_err(pdev, "Failed to register dma device: %d\n", rc);
- free_irq(pci_irq_vector(pdev, 0), plxdev);
- kfree(plxdev);
- return rc;
+ goto put_device;
}
pci_set_drvdata(pdev, plxdev);
return 0;
+
+put_device:
+ put_device(&pdev->dev);
+ free_irq(pci_irq_vector(pdev, 0), plxdev);
+free_plx:
+ kfree(plxdev);
+
+ return rc;
}
static int plx_dma_probe(struct pci_dev *pdev,
diff --git a/drivers/dma/qcom/gpi.c b/drivers/dma/qcom/gpi.c
index 57f5ee4235c7..43ac3ab23d4c 100644
--- a/drivers/dma/qcom/gpi.c
+++ b/drivers/dma/qcom/gpi.c
@@ -2281,6 +2281,7 @@ static int gpi_probe(struct platform_device *pdev)
static const struct of_device_id gpi_of_match[] = {
{ .compatible = "qcom,sdm845-gpi-dma" },
+ { .compatible = "qcom,sm8150-gpi-dma" },
{ },
};
MODULE_DEVICE_TABLE(of, gpi_of_match);
diff --git a/drivers/dma/qcom/hidma.c b/drivers/dma/qcom/hidma.c
index 6c0f9eb8ecc6..23d64489d25f 100644
--- a/drivers/dma/qcom/hidma.c
+++ b/drivers/dma/qcom/hidma.c
@@ -90,12 +90,6 @@ static inline struct hidma_chan *to_hidma_chan(struct dma_chan *dmach)
return container_of(dmach, struct hidma_chan, chan);
}
-static inline
-struct hidma_desc *to_hidma_desc(struct dma_async_tx_descriptor *t)
-{
- return container_of(t, struct hidma_desc, desc);
-}
-
static void hidma_free(struct hidma_dev *dmadev)
{
INIT_LIST_HEAD(&dmadev->ddev.channels);
diff --git a/drivers/dma/tegra20-apb-dma.c b/drivers/dma/tegra20-apb-dma.c
index 71827d9b0aa1..b7260749e8ee 100644
--- a/drivers/dma/tegra20-apb-dma.c
+++ b/drivers/dma/tegra20-apb-dma.c
@@ -723,7 +723,7 @@ static void tegra_dma_issue_pending(struct dma_chan *dc)
goto end;
}
if (!tdc->busy) {
- err = pm_runtime_get_sync(tdc->tdma->dev);
+ err = pm_runtime_resume_and_get(tdc->tdma->dev);
if (err < 0) {
dev_err(tdc2dev(tdc), "Failed to enable DMA\n");
goto end;
@@ -818,7 +818,7 @@ static void tegra_dma_synchronize(struct dma_chan *dc)
struct tegra_dma_channel *tdc = to_tegra_dma_chan(dc);
int err;
- err = pm_runtime_get_sync(tdc->tdma->dev);
+ err = pm_runtime_resume_and_get(tdc->tdma->dev);
if (err < 0) {
dev_err(tdc2dev(tdc), "Failed to synchronize DMA: %d\n", err);
return;
diff --git a/drivers/dma/xilinx/xilinx_dma.c b/drivers/dma/xilinx/xilinx_dma.c
index 3aded7861fef..75c0b8e904e5 100644
--- a/drivers/dma/xilinx/xilinx_dma.c
+++ b/drivers/dma/xilinx/xilinx_dma.c
@@ -2453,6 +2453,13 @@ static int xilinx_dma_terminate_all(struct dma_chan *dchan)
return 0;
}
+static void xilinx_dma_synchronize(struct dma_chan *dchan)
+{
+ struct xilinx_dma_chan *chan = to_xilinx_chan(dchan);
+
+ tasklet_kill(&chan->tasklet);
+}
+
/**
* xilinx_dma_channel_set_config - Configure VDMA channel
* Run-time configuration for Axi VDMA, supports:
@@ -3074,6 +3081,7 @@ static int xilinx_dma_probe(struct platform_device *pdev)
xdev->common.device_free_chan_resources =
xilinx_dma_free_chan_resources;
xdev->common.device_terminate_all = xilinx_dma_terminate_all;
+ xdev->common.device_synchronize = xilinx_dma_synchronize;
xdev->common.device_tx_status = xilinx_dma_tx_status;
xdev->common.device_issue_pending = xilinx_dma_issue_pending;
if (xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA) {
diff --git a/drivers/dma/xilinx/xilinx_dpdma.c b/drivers/dma/xilinx/xilinx_dpdma.c
index 55df63dead8d..70b29bd079c9 100644
--- a/drivers/dma/xilinx/xilinx_dpdma.c
+++ b/drivers/dma/xilinx/xilinx_dpdma.c
@@ -839,6 +839,7 @@ static void xilinx_dpdma_chan_queue_transfer(struct xilinx_dpdma_chan *chan)
struct xilinx_dpdma_tx_desc *desc;
struct virt_dma_desc *vdesc;
u32 reg, channels;
+ bool first_frame;
lockdep_assert_held(&chan->lock);
@@ -852,14 +853,6 @@ static void xilinx_dpdma_chan_queue_transfer(struct xilinx_dpdma_chan *chan)
chan->running = true;
}
- if (chan->video_group)
- channels = xilinx_dpdma_chan_video_group_ready(chan);
- else
- channels = BIT(chan->id);
-
- if (!channels)
- return;
-
vdesc = vchan_next_desc(&chan->vchan);
if (!vdesc)
return;
@@ -884,13 +877,26 @@ static void xilinx_dpdma_chan_queue_transfer(struct xilinx_dpdma_chan *chan)
FIELD_PREP(XILINX_DPDMA_CH_DESC_START_ADDRE_MASK,
upper_32_bits(sw_desc->dma_addr)));
- if (chan->first_frame)
+ first_frame = chan->first_frame;
+ chan->first_frame = false;
+
+ if (chan->video_group) {
+ channels = xilinx_dpdma_chan_video_group_ready(chan);
+ /*
+ * Trigger the transfer only when all channels in the group are
+ * ready.
+ */
+ if (!channels)
+ return;
+ } else {
+ channels = BIT(chan->id);
+ }
+
+ if (first_frame)
reg = XILINX_DPDMA_GBL_TRIG_MASK(channels);
else
reg = XILINX_DPDMA_GBL_RETRIG_MASK(channels);
- chan->first_frame = false;
-
dpdma_write(xdev->reg, XILINX_DPDMA_GBL, reg);
}
@@ -1042,13 +1048,14 @@ static int xilinx_dpdma_chan_stop(struct xilinx_dpdma_chan *chan)
*/
static void xilinx_dpdma_chan_done_irq(struct xilinx_dpdma_chan *chan)
{
- struct xilinx_dpdma_tx_desc *active = chan->desc.active;
+ struct xilinx_dpdma_tx_desc *active;
unsigned long flags;
spin_lock_irqsave(&chan->lock, flags);
xilinx_dpdma_debugfs_desc_done_irq(chan);
+ active = chan->desc.active;
if (active)
vchan_cyclic_callback(&active->vdesc);
else
diff --git a/drivers/edac/Kconfig b/drivers/edac/Kconfig
index 27d0c4cdc58d..1e836e320edd 100644
--- a/drivers/edac/Kconfig
+++ b/drivers/edac/Kconfig
@@ -396,7 +396,7 @@ config EDAC_THUNDERX
config EDAC_ALTERA
bool "Altera SOCFPGA ECC"
- depends on EDAC=y && (ARCH_SOCFPGA || ARCH_STRATIX10)
+ depends on EDAC=y && ARCH_INTEL_SOCFPGA
help
Support for error detection and correction on the
Altera SOCs. This is the global enable for the
diff --git a/drivers/edac/altera_edac.c b/drivers/edac/altera_edac.c
index e91cf1147a4e..5f7fd79ec82f 100644
--- a/drivers/edac/altera_edac.c
+++ b/drivers/edac/altera_edac.c
@@ -1501,8 +1501,13 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
dci->mod_name = ecc_name;
dci->dev_name = ecc_name;
- /* Update the PortB IRQs - A10 has 4, S10 has 2, Index accordingly */
-#ifdef CONFIG_ARCH_STRATIX10
+ /*
+ * Update the PortB IRQs - A10 has 4, S10 has 2, Index accordingly
+ *
+ * FIXME: Instead of ifdefs with different architectures the driver
+ * should properly use compatibles.
+ */
+#ifdef CONFIG_64BIT
altdev->sb_irq = irq_of_parse_and_map(np, 1);
#else
altdev->sb_irq = irq_of_parse_and_map(np, 2);
@@ -1521,7 +1526,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
goto err_release_group_1;
}
-#ifdef CONFIG_ARCH_STRATIX10
+#ifdef CONFIG_64BIT
/* Use IRQ to determine SError origin instead of assigning IRQ */
rc = of_property_read_u32_index(np, "interrupts", 1, &altdev->db_irq);
if (rc) {
@@ -1931,7 +1936,7 @@ static int altr_edac_a10_device_add(struct altr_arria10_edac *edac,
goto err_release_group1;
}
-#ifdef CONFIG_ARCH_STRATIX10
+#ifdef CONFIG_64BIT
/* Use IRQ to determine SError origin instead of assigning IRQ */
rc = of_property_read_u32_index(np, "interrupts", 0, &altdev->db_irq);
if (rc) {
@@ -2016,7 +2021,7 @@ static const struct irq_domain_ops a10_eccmgr_ic_ops = {
/************** Stratix 10 EDAC Double Bit Error Handler ************/
#define to_a10edac(p, m) container_of(p, struct altr_arria10_edac, m)
-#ifdef CONFIG_ARCH_STRATIX10
+#ifdef CONFIG_64BIT
/* panic routine issues reboot on non-zero panic_timeout */
extern int panic_timeout;
@@ -2109,7 +2114,7 @@ static int altr_edac_a10_probe(struct platform_device *pdev)
altr_edac_a10_irq_handler,
edac);
-#ifdef CONFIG_ARCH_STRATIX10
+#ifdef CONFIG_64BIT
{
int dberror, err_addr;
diff --git a/drivers/edac/amd64_edac.c b/drivers/edac/amd64_edac.c
index 9fa4dfc6ebee..f0d8f60acee1 100644
--- a/drivers/edac/amd64_edac.c
+++ b/drivers/edac/amd64_edac.c
@@ -3083,7 +3083,7 @@ static void read_mc_regs(struct amd64_pvt *pvt)
edac_dbg(0, " TOP_MEM: 0x%016llx\n", pvt->top_mem);
/* Check first whether TOP_MEM2 is enabled: */
- rdmsrl(MSR_K8_SYSCFG, msr_val);
+ rdmsrl(MSR_AMD64_SYSCFG, msr_val);
if (msr_val & BIT(21)) {
rdmsrl(MSR_K8_TOP_MEM2, pvt->top_mem2);
edac_dbg(0, " TOP_MEM2: 0x%016llx\n", pvt->top_mem2);
diff --git a/drivers/extcon/Kconfig b/drivers/extcon/Kconfig
index af58ebca2bf6..e3db936becfd 100644
--- a/drivers/extcon/Kconfig
+++ b/drivers/extcon/Kconfig
@@ -21,14 +21,6 @@ config EXTCON_ADC_JACK
help
Say Y here to enable extcon device driver based on ADC values.
-config EXTCON_ARIZONA
- tristate "Wolfson Arizona EXTCON support"
- depends on MFD_ARIZONA && INPUT && SND_SOC
- help
- Say Y here to enable support for external accessory detection
- with Wolfson Arizona devices. These are audio CODECs with
- advanced audio accessory detection support.
-
config EXTCON_AXP288
tristate "X-Power AXP288 EXTCON support"
depends on MFD_AXP20X && USB_SUPPORT && X86 && ACPI
diff --git a/drivers/extcon/Makefile b/drivers/extcon/Makefile
index fe10a1b7d18b..1b390d934ca9 100644
--- a/drivers/extcon/Makefile
+++ b/drivers/extcon/Makefile
@@ -6,7 +6,6 @@
obj-$(CONFIG_EXTCON) += extcon-core.o
extcon-core-objs += extcon.o devres.o
obj-$(CONFIG_EXTCON_ADC_JACK) += extcon-adc-jack.o
-obj-$(CONFIG_EXTCON_ARIZONA) += extcon-arizona.o
obj-$(CONFIG_EXTCON_AXP288) += extcon-axp288.o
obj-$(CONFIG_EXTCON_FSA9480) += extcon-fsa9480.o
obj-$(CONFIG_EXTCON_GPIO) += extcon-gpio.o
diff --git a/drivers/extcon/extcon-arizona.c b/drivers/extcon/extcon-arizona.c
deleted file mode 100644
index aae82db542a5..000000000000
--- a/drivers/extcon/extcon-arizona.c
+++ /dev/null
@@ -1,1816 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * extcon-arizona.c - Extcon driver Wolfson Arizona devices
- *
- * Copyright (C) 2012-2014 Wolfson Microelectronics plc
- */
-
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/i2c.h>
-#include <linux/slab.h>
-#include <linux/interrupt.h>
-#include <linux/err.h>
-#include <linux/gpio/consumer.h>
-#include <linux/gpio.h>
-#include <linux/input.h>
-#include <linux/platform_device.h>
-#include <linux/pm_runtime.h>
-#include <linux/property.h>
-#include <linux/regulator/consumer.h>
-#include <linux/extcon-provider.h>
-
-#include <sound/soc.h>
-
-#include <linux/mfd/arizona/core.h>
-#include <linux/mfd/arizona/pdata.h>
-#include <linux/mfd/arizona/registers.h>
-#include <dt-bindings/mfd/arizona.h>
-
-#define ARIZONA_MAX_MICD_RANGE 8
-
-#define ARIZONA_MICD_CLAMP_MODE_JDL 0x4
-#define ARIZONA_MICD_CLAMP_MODE_JDH 0x5
-#define ARIZONA_MICD_CLAMP_MODE_JDL_GP5H 0x9
-#define ARIZONA_MICD_CLAMP_MODE_JDH_GP5H 0xb
-
-#define ARIZONA_TST_CAP_DEFAULT 0x3
-#define ARIZONA_TST_CAP_CLAMP 0x1
-
-#define ARIZONA_HPDET_MAX 10000
-
-#define HPDET_DEBOUNCE 500
-#define DEFAULT_MICD_TIMEOUT 2000
-
-#define ARIZONA_HPDET_WAIT_COUNT 15
-#define ARIZONA_HPDET_WAIT_DELAY_MS 20
-
-#define QUICK_HEADPHONE_MAX_OHM 3
-#define MICROPHONE_MIN_OHM 1257
-#define MICROPHONE_MAX_OHM 30000
-
-#define MICD_DBTIME_TWO_READINGS 2
-#define MICD_DBTIME_FOUR_READINGS 4
-
-#define MICD_LVL_1_TO_7 (ARIZONA_MICD_LVL_1 | ARIZONA_MICD_LVL_2 | \
- ARIZONA_MICD_LVL_3 | ARIZONA_MICD_LVL_4 | \
- ARIZONA_MICD_LVL_5 | ARIZONA_MICD_LVL_6 | \
- ARIZONA_MICD_LVL_7)
-
-#define MICD_LVL_0_TO_7 (ARIZONA_MICD_LVL_0 | MICD_LVL_1_TO_7)
-
-#define MICD_LVL_0_TO_8 (MICD_LVL_0_TO_7 | ARIZONA_MICD_LVL_8)
-
-struct arizona_extcon_info {
- struct device *dev;
- struct arizona *arizona;
- struct mutex lock;
- struct regulator *micvdd;
- struct input_dev *input;
-
- u16 last_jackdet;
-
- int micd_mode;
- const struct arizona_micd_config *micd_modes;
- int micd_num_modes;
-
- const struct arizona_micd_range *micd_ranges;
- int num_micd_ranges;
-
- bool micd_reva;
- bool micd_clamp;
-
- struct delayed_work hpdet_work;
- struct delayed_work micd_detect_work;
- struct delayed_work micd_timeout_work;
-
- bool hpdet_active;
- bool hpdet_done;
- bool hpdet_retried;
-
- int num_hpdet_res;
- unsigned int hpdet_res[3];
-
- bool mic;
- bool detecting;
- int jack_flips;
-
- int hpdet_ip_version;
-
- struct extcon_dev *edev;
-
- struct gpio_desc *micd_pol_gpio;
-};
-
-static const struct arizona_micd_config micd_default_modes[] = {
- { ARIZONA_ACCDET_SRC, 1, 0 },
- { 0, 2, 1 },
-};
-
-static const struct arizona_micd_range micd_default_ranges[] = {
- { .max = 11, .key = BTN_0 },
- { .max = 28, .key = BTN_1 },
- { .max = 54, .key = BTN_2 },
- { .max = 100, .key = BTN_3 },
- { .max = 186, .key = BTN_4 },
- { .max = 430, .key = BTN_5 },
-};
-
-/* The number of levels in arizona_micd_levels valid for button thresholds */
-#define ARIZONA_NUM_MICD_BUTTON_LEVELS 64
-
-static const int arizona_micd_levels[] = {
- 3, 6, 8, 11, 13, 16, 18, 21, 23, 26, 28, 31, 34, 36, 39, 41, 44, 46,
- 49, 52, 54, 57, 60, 62, 65, 67, 70, 73, 75, 78, 81, 83, 89, 94, 100,
- 105, 111, 116, 122, 127, 139, 150, 161, 173, 186, 196, 209, 220, 245,
- 270, 295, 321, 348, 375, 402, 430, 489, 550, 614, 681, 752, 903, 1071,
- 1257, 30000,
-};
-
-static const unsigned int arizona_cable[] = {
- EXTCON_MECHANICAL,
- EXTCON_JACK_MICROPHONE,
- EXTCON_JACK_HEADPHONE,
- EXTCON_JACK_LINE_OUT,
- EXTCON_NONE,
-};
-
-static void arizona_start_hpdet_acc_id(struct arizona_extcon_info *info);
-
-static void arizona_extcon_hp_clamp(struct arizona_extcon_info *info,
- bool clamp)
-{
- struct arizona *arizona = info->arizona;
- unsigned int mask = 0, val = 0;
- unsigned int cap_sel = 0;
- int ret;
-
- switch (arizona->type) {
- case WM8998:
- case WM1814:
- mask = 0;
- break;
- case WM5110:
- case WM8280:
- mask = ARIZONA_HP1L_SHRTO | ARIZONA_HP1L_FLWR |
- ARIZONA_HP1L_SHRTI;
- if (clamp) {
- val = ARIZONA_HP1L_SHRTO;
- cap_sel = ARIZONA_TST_CAP_CLAMP;
- } else {
- val = ARIZONA_HP1L_FLWR | ARIZONA_HP1L_SHRTI;
- cap_sel = ARIZONA_TST_CAP_DEFAULT;
- }
-
- ret = regmap_update_bits(arizona->regmap,
- ARIZONA_HP_TEST_CTRL_1,
- ARIZONA_HP1_TST_CAP_SEL_MASK,
- cap_sel);
- if (ret != 0)
- dev_warn(arizona->dev,
- "Failed to set TST_CAP_SEL: %d\n", ret);
- break;
- default:
- mask = ARIZONA_RMV_SHRT_HP1L;
- if (clamp)
- val = ARIZONA_RMV_SHRT_HP1L;
- break;
- }
-
- snd_soc_dapm_mutex_lock(arizona->dapm);
-
- arizona->hpdet_clamp = clamp;
-
- /* Keep the HP output stages disabled while doing the clamp */
- if (clamp) {
- ret = regmap_update_bits(arizona->regmap,
- ARIZONA_OUTPUT_ENABLES_1,
- ARIZONA_OUT1L_ENA |
- ARIZONA_OUT1R_ENA, 0);
- if (ret != 0)
- dev_warn(arizona->dev,
- "Failed to disable headphone outputs: %d\n",
- ret);
- }
-
- if (mask) {
- ret = regmap_update_bits(arizona->regmap, ARIZONA_HP_CTRL_1L,
- mask, val);
- if (ret != 0)
- dev_warn(arizona->dev, "Failed to do clamp: %d\n",
- ret);
-
- ret = regmap_update_bits(arizona->regmap, ARIZONA_HP_CTRL_1R,
- mask, val);
- if (ret != 0)
- dev_warn(arizona->dev, "Failed to do clamp: %d\n",
- ret);
- }
-
- /* Restore the desired state while not doing the clamp */
- if (!clamp) {
- ret = regmap_update_bits(arizona->regmap,
- ARIZONA_OUTPUT_ENABLES_1,
- ARIZONA_OUT1L_ENA |
- ARIZONA_OUT1R_ENA, arizona->hp_ena);
- if (ret != 0)
- dev_warn(arizona->dev,
- "Failed to restore headphone outputs: %d\n",
- ret);
- }
-
- snd_soc_dapm_mutex_unlock(arizona->dapm);
-}
-
-static void arizona_extcon_set_mode(struct arizona_extcon_info *info, int mode)
-{
- struct arizona *arizona = info->arizona;
-
- mode %= info->micd_num_modes;
-
- gpiod_set_value_cansleep(info->micd_pol_gpio,
- info->micd_modes[mode].gpio);
-
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_BIAS_SRC_MASK,
- info->micd_modes[mode].bias <<
- ARIZONA_MICD_BIAS_SRC_SHIFT);
- regmap_update_bits(arizona->regmap, ARIZONA_ACCESSORY_DETECT_MODE_1,
- ARIZONA_ACCDET_SRC, info->micd_modes[mode].src);
-
- info->micd_mode = mode;
-
- dev_dbg(arizona->dev, "Set jack polarity to %d\n", mode);
-}
-
-static const char *arizona_extcon_get_micbias(struct arizona_extcon_info *info)
-{
- switch (info->micd_modes[0].bias) {
- case 1:
- return "MICBIAS1";
- case 2:
- return "MICBIAS2";
- case 3:
- return "MICBIAS3";
- default:
- return "MICVDD";
- }
-}
-
-static void arizona_extcon_pulse_micbias(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- const char *widget = arizona_extcon_get_micbias(info);
- struct snd_soc_dapm_context *dapm = arizona->dapm;
- struct snd_soc_component *component = snd_soc_dapm_to_component(dapm);
- int ret;
-
- ret = snd_soc_component_force_enable_pin(component, widget);
- if (ret != 0)
- dev_warn(arizona->dev, "Failed to enable %s: %d\n",
- widget, ret);
-
- snd_soc_dapm_sync(dapm);
-
- if (!arizona->pdata.micd_force_micbias) {
- ret = snd_soc_component_disable_pin(component, widget);
- if (ret != 0)
- dev_warn(arizona->dev, "Failed to disable %s: %d\n",
- widget, ret);
-
- snd_soc_dapm_sync(dapm);
- }
-}
-
-static void arizona_start_mic(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- bool change;
- int ret;
- unsigned int mode;
-
- /* Microphone detection can't use idle mode */
- pm_runtime_get(info->dev);
-
- if (info->detecting) {
- ret = regulator_allow_bypass(info->micvdd, false);
- if (ret != 0) {
- dev_err(arizona->dev,
- "Failed to regulate MICVDD: %d\n",
- ret);
- }
- }
-
- ret = regulator_enable(info->micvdd);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to enable MICVDD: %d\n",
- ret);
- }
-
- if (info->micd_reva) {
- const struct reg_sequence reva[] = {
- { 0x80, 0x3 },
- { 0x294, 0x0 },
- { 0x80, 0x0 },
- };
-
- regmap_multi_reg_write(arizona->regmap, reva, ARRAY_SIZE(reva));
- }
-
- if (info->detecting && arizona->pdata.micd_software_compare)
- mode = ARIZONA_ACCDET_MODE_ADC;
- else
- mode = ARIZONA_ACCDET_MODE_MIC;
-
- regmap_update_bits(arizona->regmap,
- ARIZONA_ACCESSORY_DETECT_MODE_1,
- ARIZONA_ACCDET_MODE_MASK, mode);
-
- arizona_extcon_pulse_micbias(info);
-
- ret = regmap_update_bits_check(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_ENA, ARIZONA_MICD_ENA,
- &change);
- if (ret < 0) {
- dev_err(arizona->dev, "Failed to enable micd: %d\n", ret);
- } else if (!change) {
- regulator_disable(info->micvdd);
- pm_runtime_put_autosuspend(info->dev);
- }
-}
-
-static void arizona_stop_mic(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- const char *widget = arizona_extcon_get_micbias(info);
- struct snd_soc_dapm_context *dapm = arizona->dapm;
- struct snd_soc_component *component = snd_soc_dapm_to_component(dapm);
- bool change = false;
- int ret;
-
- ret = regmap_update_bits_check(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_ENA, 0,
- &change);
- if (ret < 0)
- dev_err(arizona->dev, "Failed to disable micd: %d\n", ret);
-
- ret = snd_soc_component_disable_pin(component, widget);
- if (ret != 0)
- dev_warn(arizona->dev,
- "Failed to disable %s: %d\n",
- widget, ret);
-
- snd_soc_dapm_sync(dapm);
-
- if (info->micd_reva) {
- const struct reg_sequence reva[] = {
- { 0x80, 0x3 },
- { 0x294, 0x2 },
- { 0x80, 0x0 },
- };
-
- regmap_multi_reg_write(arizona->regmap, reva, ARRAY_SIZE(reva));
- }
-
- ret = regulator_allow_bypass(info->micvdd, true);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to bypass MICVDD: %d\n",
- ret);
- }
-
- if (change) {
- regulator_disable(info->micvdd);
- pm_runtime_mark_last_busy(info->dev);
- pm_runtime_put_autosuspend(info->dev);
- }
-}
-
-static struct {
- unsigned int threshold;
- unsigned int factor_a;
- unsigned int factor_b;
-} arizona_hpdet_b_ranges[] = {
- { 100, 5528, 362464 },
- { 169, 11084, 6186851 },
- { 169, 11065, 65460395 },
-};
-
-#define ARIZONA_HPDET_B_RANGE_MAX 0x3fb
-
-static struct {
- int min;
- int max;
-} arizona_hpdet_c_ranges[] = {
- { 0, 30 },
- { 8, 100 },
- { 100, 1000 },
- { 1000, 10000 },
-};
-
-static int arizona_hpdet_read(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- unsigned int val, range;
- int ret;
-
- ret = regmap_read(arizona->regmap, ARIZONA_HEADPHONE_DETECT_2, &val);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to read HPDET status: %d\n",
- ret);
- return ret;
- }
-
- switch (info->hpdet_ip_version) {
- case 0:
- if (!(val & ARIZONA_HP_DONE)) {
- dev_err(arizona->dev, "HPDET did not complete: %x\n",
- val);
- return -EAGAIN;
- }
-
- val &= ARIZONA_HP_LVL_MASK;
- break;
-
- case 1:
- if (!(val & ARIZONA_HP_DONE_B)) {
- dev_err(arizona->dev, "HPDET did not complete: %x\n",
- val);
- return -EAGAIN;
- }
-
- ret = regmap_read(arizona->regmap, ARIZONA_HP_DACVAL, &val);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to read HP value: %d\n",
- ret);
- return -EAGAIN;
- }
-
- regmap_read(arizona->regmap, ARIZONA_HEADPHONE_DETECT_1,
- &range);
- range = (range & ARIZONA_HP_IMPEDANCE_RANGE_MASK)
- >> ARIZONA_HP_IMPEDANCE_RANGE_SHIFT;
-
- if (range < ARRAY_SIZE(arizona_hpdet_b_ranges) - 1 &&
- (val < arizona_hpdet_b_ranges[range].threshold ||
- val >= ARIZONA_HPDET_B_RANGE_MAX)) {
- range++;
- dev_dbg(arizona->dev, "Moving to HPDET range %d\n",
- range);
- regmap_update_bits(arizona->regmap,
- ARIZONA_HEADPHONE_DETECT_1,
- ARIZONA_HP_IMPEDANCE_RANGE_MASK,
- range <<
- ARIZONA_HP_IMPEDANCE_RANGE_SHIFT);
- return -EAGAIN;
- }
-
- /* If we go out of range report top of range */
- if (val < arizona_hpdet_b_ranges[range].threshold ||
- val >= ARIZONA_HPDET_B_RANGE_MAX) {
- dev_dbg(arizona->dev, "Measurement out of range\n");
- return ARIZONA_HPDET_MAX;
- }
-
- dev_dbg(arizona->dev, "HPDET read %d in range %d\n",
- val, range);
-
- val = arizona_hpdet_b_ranges[range].factor_b
- / ((val * 100) -
- arizona_hpdet_b_ranges[range].factor_a);
- break;
-
- case 2:
- if (!(val & ARIZONA_HP_DONE_B)) {
- dev_err(arizona->dev, "HPDET did not complete: %x\n",
- val);
- return -EAGAIN;
- }
-
- val &= ARIZONA_HP_LVL_B_MASK;
- /* Convert to ohms, the value is in 0.5 ohm increments */
- val /= 2;
-
- regmap_read(arizona->regmap, ARIZONA_HEADPHONE_DETECT_1,
- &range);
- range = (range & ARIZONA_HP_IMPEDANCE_RANGE_MASK)
- >> ARIZONA_HP_IMPEDANCE_RANGE_SHIFT;
-
- /* Skip up a range, or report? */
- if (range < ARRAY_SIZE(arizona_hpdet_c_ranges) - 1 &&
- (val >= arizona_hpdet_c_ranges[range].max)) {
- range++;
- dev_dbg(arizona->dev, "Moving to HPDET range %d-%d\n",
- arizona_hpdet_c_ranges[range].min,
- arizona_hpdet_c_ranges[range].max);
- regmap_update_bits(arizona->regmap,
- ARIZONA_HEADPHONE_DETECT_1,
- ARIZONA_HP_IMPEDANCE_RANGE_MASK,
- range <<
- ARIZONA_HP_IMPEDANCE_RANGE_SHIFT);
- return -EAGAIN;
- }
-
- if (range && (val < arizona_hpdet_c_ranges[range].min)) {
- dev_dbg(arizona->dev, "Reporting range boundary %d\n",
- arizona_hpdet_c_ranges[range].min);
- val = arizona_hpdet_c_ranges[range].min;
- }
- break;
-
- default:
- dev_warn(arizona->dev, "Unknown HPDET IP revision %d\n",
- info->hpdet_ip_version);
- return -EINVAL;
- }
-
- dev_dbg(arizona->dev, "HP impedance %d ohms\n", val);
- return val;
-}
-
-static int arizona_hpdet_do_id(struct arizona_extcon_info *info, int *reading,
- bool *mic)
-{
- struct arizona *arizona = info->arizona;
- int id_gpio = arizona->pdata.hpdet_id_gpio;
-
- if (!arizona->pdata.hpdet_acc_id)
- return 0;
-
- /*
- * If we're using HPDET for accessory identification we need
- * to take multiple measurements, step through them in sequence.
- */
- info->hpdet_res[info->num_hpdet_res++] = *reading;
-
- /* Only check the mic directly if we didn't already ID it */
- if (id_gpio && info->num_hpdet_res == 1) {
- dev_dbg(arizona->dev, "Measuring mic\n");
-
- regmap_update_bits(arizona->regmap,
- ARIZONA_ACCESSORY_DETECT_MODE_1,
- ARIZONA_ACCDET_MODE_MASK |
- ARIZONA_ACCDET_SRC,
- ARIZONA_ACCDET_MODE_HPR |
- info->micd_modes[0].src);
-
- gpio_set_value_cansleep(id_gpio, 1);
-
- regmap_update_bits(arizona->regmap, ARIZONA_HEADPHONE_DETECT_1,
- ARIZONA_HP_POLL, ARIZONA_HP_POLL);
- return -EAGAIN;
- }
-
- /* OK, got both. Now, compare... */
- dev_dbg(arizona->dev, "HPDET measured %d %d\n",
- info->hpdet_res[0], info->hpdet_res[1]);
-
- /* Take the headphone impedance for the main report */
- *reading = info->hpdet_res[0];
-
- /* Sometimes we get false readings due to slow insert */
- if (*reading >= ARIZONA_HPDET_MAX && !info->hpdet_retried) {
- dev_dbg(arizona->dev, "Retrying high impedance\n");
- info->num_hpdet_res = 0;
- info->hpdet_retried = true;
- arizona_start_hpdet_acc_id(info);
- pm_runtime_put(info->dev);
- return -EAGAIN;
- }
-
- /*
- * If we measure the mic as high impedance
- */
- if (!id_gpio || info->hpdet_res[1] > 50) {
- dev_dbg(arizona->dev, "Detected mic\n");
- *mic = true;
- info->detecting = true;
- } else {
- dev_dbg(arizona->dev, "Detected headphone\n");
- }
-
- /* Make sure everything is reset back to the real polarity */
- regmap_update_bits(arizona->regmap, ARIZONA_ACCESSORY_DETECT_MODE_1,
- ARIZONA_ACCDET_SRC, info->micd_modes[0].src);
-
- return 0;
-}
-
-static irqreturn_t arizona_hpdet_irq(int irq, void *data)
-{
- struct arizona_extcon_info *info = data;
- struct arizona *arizona = info->arizona;
- int id_gpio = arizona->pdata.hpdet_id_gpio;
- unsigned int report = EXTCON_JACK_HEADPHONE;
- int ret, reading;
- bool mic = false;
-
- mutex_lock(&info->lock);
-
- /* If we got a spurious IRQ for some reason then ignore it */
- if (!info->hpdet_active) {
- dev_warn(arizona->dev, "Spurious HPDET IRQ\n");
- mutex_unlock(&info->lock);
- return IRQ_NONE;
- }
-
- /* If the cable was removed while measuring ignore the result */
- ret = extcon_get_state(info->edev, EXTCON_MECHANICAL);
- if (ret < 0) {
- dev_err(arizona->dev, "Failed to check cable state: %d\n",
- ret);
- goto out;
- } else if (!ret) {
- dev_dbg(arizona->dev, "Ignoring HPDET for removed cable\n");
- goto done;
- }
-
- ret = arizona_hpdet_read(info);
- if (ret == -EAGAIN)
- goto out;
- else if (ret < 0)
- goto done;
- reading = ret;
-
- /* Reset back to starting range */
- regmap_update_bits(arizona->regmap,
- ARIZONA_HEADPHONE_DETECT_1,
- ARIZONA_HP_IMPEDANCE_RANGE_MASK | ARIZONA_HP_POLL,
- 0);
-
- ret = arizona_hpdet_do_id(info, &reading, &mic);
- if (ret == -EAGAIN)
- goto out;
- else if (ret < 0)
- goto done;
-
- /* Report high impedence cables as line outputs */
- if (reading >= 5000)
- report = EXTCON_JACK_LINE_OUT;
- else
- report = EXTCON_JACK_HEADPHONE;
-
- ret = extcon_set_state_sync(info->edev, report, true);
- if (ret != 0)
- dev_err(arizona->dev, "Failed to report HP/line: %d\n",
- ret);
-
-done:
- /* Reset back to starting range */
- regmap_update_bits(arizona->regmap,
- ARIZONA_HEADPHONE_DETECT_1,
- ARIZONA_HP_IMPEDANCE_RANGE_MASK | ARIZONA_HP_POLL,
- 0);
-
- arizona_extcon_hp_clamp(info, false);
-
- if (id_gpio)
- gpio_set_value_cansleep(id_gpio, 0);
-
- /* If we have a mic then reenable MICDET */
- if (mic || info->mic)
- arizona_start_mic(info);
-
- if (info->hpdet_active) {
- pm_runtime_put_autosuspend(info->dev);
- info->hpdet_active = false;
- }
-
- info->hpdet_done = true;
-
-out:
- mutex_unlock(&info->lock);
-
- return IRQ_HANDLED;
-}
-
-static void arizona_identify_headphone(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- int ret;
-
- if (info->hpdet_done)
- return;
-
- dev_dbg(arizona->dev, "Starting HPDET\n");
-
- /* Make sure we keep the device enabled during the measurement */
- pm_runtime_get(info->dev);
-
- info->hpdet_active = true;
-
- arizona_stop_mic(info);
-
- arizona_extcon_hp_clamp(info, true);
-
- ret = regmap_update_bits(arizona->regmap,
- ARIZONA_ACCESSORY_DETECT_MODE_1,
- ARIZONA_ACCDET_MODE_MASK,
- arizona->pdata.hpdet_channel);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to set HPDET mode: %d\n", ret);
- goto err;
- }
-
- ret = regmap_update_bits(arizona->regmap, ARIZONA_HEADPHONE_DETECT_1,
- ARIZONA_HP_POLL, ARIZONA_HP_POLL);
- if (ret != 0) {
- dev_err(arizona->dev, "Can't start HPDETL measurement: %d\n",
- ret);
- goto err;
- }
-
- return;
-
-err:
- arizona_extcon_hp_clamp(info, false);
- pm_runtime_put_autosuspend(info->dev);
-
- /* Just report headphone */
- ret = extcon_set_state_sync(info->edev, EXTCON_JACK_HEADPHONE, true);
- if (ret != 0)
- dev_err(arizona->dev, "Failed to report headphone: %d\n", ret);
-
- if (info->mic)
- arizona_start_mic(info);
-
- info->hpdet_active = false;
-}
-
-static void arizona_start_hpdet_acc_id(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- int hp_reading = 32;
- bool mic;
- int ret;
-
- dev_dbg(arizona->dev, "Starting identification via HPDET\n");
-
- /* Make sure we keep the device enabled during the measurement */
- pm_runtime_get_sync(info->dev);
-
- info->hpdet_active = true;
-
- arizona_extcon_hp_clamp(info, true);
-
- ret = regmap_update_bits(arizona->regmap,
- ARIZONA_ACCESSORY_DETECT_MODE_1,
- ARIZONA_ACCDET_SRC | ARIZONA_ACCDET_MODE_MASK,
- info->micd_modes[0].src |
- arizona->pdata.hpdet_channel);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to set HPDET mode: %d\n", ret);
- goto err;
- }
-
- if (arizona->pdata.hpdet_acc_id_line) {
- ret = regmap_update_bits(arizona->regmap,
- ARIZONA_HEADPHONE_DETECT_1,
- ARIZONA_HP_POLL, ARIZONA_HP_POLL);
- if (ret != 0) {
- dev_err(arizona->dev,
- "Can't start HPDETL measurement: %d\n",
- ret);
- goto err;
- }
- } else {
- arizona_hpdet_do_id(info, &hp_reading, &mic);
- }
-
- return;
-
-err:
- /* Just report headphone */
- ret = extcon_set_state_sync(info->edev, EXTCON_JACK_HEADPHONE, true);
- if (ret != 0)
- dev_err(arizona->dev, "Failed to report headphone: %d\n", ret);
-
- info->hpdet_active = false;
-}
-
-static void arizona_micd_timeout_work(struct work_struct *work)
-{
- struct arizona_extcon_info *info = container_of(work,
- struct arizona_extcon_info,
- micd_timeout_work.work);
-
- mutex_lock(&info->lock);
-
- dev_dbg(info->arizona->dev, "MICD timed out, reporting HP\n");
-
- info->detecting = false;
-
- arizona_identify_headphone(info);
-
- mutex_unlock(&info->lock);
-}
-
-static int arizona_micd_adc_read(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- unsigned int val;
- int ret;
-
- /* Must disable MICD before we read the ADCVAL */
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_ENA, 0);
-
- ret = regmap_read(arizona->regmap, ARIZONA_MIC_DETECT_4, &val);
- if (ret != 0) {
- dev_err(arizona->dev,
- "Failed to read MICDET_ADCVAL: %d\n", ret);
- return ret;
- }
-
- dev_dbg(arizona->dev, "MICDET_ADCVAL: %x\n", val);
-
- val &= ARIZONA_MICDET_ADCVAL_MASK;
- if (val < ARRAY_SIZE(arizona_micd_levels))
- val = arizona_micd_levels[val];
- else
- val = INT_MAX;
-
- if (val <= QUICK_HEADPHONE_MAX_OHM)
- val = ARIZONA_MICD_STS | ARIZONA_MICD_LVL_0;
- else if (val <= MICROPHONE_MIN_OHM)
- val = ARIZONA_MICD_STS | ARIZONA_MICD_LVL_1;
- else if (val <= MICROPHONE_MAX_OHM)
- val = ARIZONA_MICD_STS | ARIZONA_MICD_LVL_8;
- else
- val = ARIZONA_MICD_LVL_8;
-
- return val;
-}
-
-static int arizona_micd_read(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- unsigned int val = 0;
- int ret, i;
-
- for (i = 0; i < 10 && !(val & MICD_LVL_0_TO_8); i++) {
- ret = regmap_read(arizona->regmap, ARIZONA_MIC_DETECT_3, &val);
- if (ret != 0) {
- dev_err(arizona->dev,
- "Failed to read MICDET: %d\n", ret);
- return ret;
- }
-
- dev_dbg(arizona->dev, "MICDET: %x\n", val);
-
- if (!(val & ARIZONA_MICD_VALID)) {
- dev_warn(arizona->dev,
- "Microphone detection state invalid\n");
- return -EINVAL;
- }
- }
-
- if (i == 10 && !(val & MICD_LVL_0_TO_8)) {
- dev_err(arizona->dev, "Failed to get valid MICDET value\n");
- return -EINVAL;
- }
-
- return val;
-}
-
-static int arizona_micdet_reading(void *priv)
-{
- struct arizona_extcon_info *info = priv;
- struct arizona *arizona = info->arizona;
- int ret, val;
-
- if (info->detecting && arizona->pdata.micd_software_compare)
- ret = arizona_micd_adc_read(info);
- else
- ret = arizona_micd_read(info);
- if (ret < 0)
- return ret;
-
- val = ret;
-
- /* Due to jack detect this should never happen */
- if (!(val & ARIZONA_MICD_STS)) {
- dev_warn(arizona->dev, "Detected open circuit\n");
- info->mic = false;
- info->detecting = false;
- arizona_identify_headphone(info);
- return 0;
- }
-
- /* If we got a high impedence we should have a headset, report it. */
- if (val & ARIZONA_MICD_LVL_8) {
- info->mic = true;
- info->detecting = false;
-
- arizona_identify_headphone(info);
-
- ret = extcon_set_state_sync(info->edev,
- EXTCON_JACK_MICROPHONE, true);
- if (ret != 0)
- dev_err(arizona->dev, "Headset report failed: %d\n",
- ret);
-
- /* Don't need to regulate for button detection */
- ret = regulator_allow_bypass(info->micvdd, true);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to bypass MICVDD: %d\n",
- ret);
- }
-
- return 0;
- }
-
- /* If we detected a lower impedence during initial startup
- * then we probably have the wrong polarity, flip it. Don't
- * do this for the lowest impedences to speed up detection of
- * plain headphones. If both polarities report a low
- * impedence then give up and report headphones.
- */
- if (val & MICD_LVL_1_TO_7) {
- if (info->jack_flips >= info->micd_num_modes * 10) {
- dev_dbg(arizona->dev, "Detected HP/line\n");
-
- info->detecting = false;
-
- arizona_identify_headphone(info);
- } else {
- info->micd_mode++;
- if (info->micd_mode == info->micd_num_modes)
- info->micd_mode = 0;
- arizona_extcon_set_mode(info, info->micd_mode);
-
- info->jack_flips++;
-
- if (arizona->pdata.micd_software_compare)
- regmap_update_bits(arizona->regmap,
- ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_ENA,
- ARIZONA_MICD_ENA);
-
- queue_delayed_work(system_power_efficient_wq,
- &info->micd_timeout_work,
- msecs_to_jiffies(arizona->pdata.micd_timeout));
- }
-
- return 0;
- }
-
- /*
- * If we're still detecting and we detect a short then we've
- * got a headphone.
- */
- dev_dbg(arizona->dev, "Headphone detected\n");
- info->detecting = false;
-
- arizona_identify_headphone(info);
-
- return 0;
-}
-
-static int arizona_button_reading(void *priv)
-{
- struct arizona_extcon_info *info = priv;
- struct arizona *arizona = info->arizona;
- int val, key, lvl, i;
-
- val = arizona_micd_read(info);
- if (val < 0)
- return val;
-
- /*
- * If we're still detecting and we detect a short then we've
- * got a headphone. Otherwise it's a button press.
- */
- if (val & MICD_LVL_0_TO_7) {
- if (info->mic) {
- dev_dbg(arizona->dev, "Mic button detected\n");
-
- lvl = val & ARIZONA_MICD_LVL_MASK;
- lvl >>= ARIZONA_MICD_LVL_SHIFT;
-
- for (i = 0; i < info->num_micd_ranges; i++)
- input_report_key(info->input,
- info->micd_ranges[i].key, 0);
-
- if (lvl && ffs(lvl) - 1 < info->num_micd_ranges) {
- key = info->micd_ranges[ffs(lvl) - 1].key;
- input_report_key(info->input, key, 1);
- input_sync(info->input);
- } else {
- dev_err(arizona->dev, "Button out of range\n");
- }
- } else {
- dev_warn(arizona->dev, "Button with no mic: %x\n",
- val);
- }
- } else {
- dev_dbg(arizona->dev, "Mic button released\n");
- for (i = 0; i < info->num_micd_ranges; i++)
- input_report_key(info->input,
- info->micd_ranges[i].key, 0);
- input_sync(info->input);
- arizona_extcon_pulse_micbias(info);
- }
-
- return 0;
-}
-
-static void arizona_micd_detect(struct work_struct *work)
-{
- struct arizona_extcon_info *info = container_of(work,
- struct arizona_extcon_info,
- micd_detect_work.work);
- struct arizona *arizona = info->arizona;
- int ret;
-
- cancel_delayed_work_sync(&info->micd_timeout_work);
-
- mutex_lock(&info->lock);
-
- /* If the cable was removed while measuring ignore the result */
- ret = extcon_get_state(info->edev, EXTCON_MECHANICAL);
- if (ret < 0) {
- dev_err(arizona->dev, "Failed to check cable state: %d\n",
- ret);
- mutex_unlock(&info->lock);
- return;
- } else if (!ret) {
- dev_dbg(arizona->dev, "Ignoring MICDET for removed cable\n");
- mutex_unlock(&info->lock);
- return;
- }
-
- if (info->detecting)
- arizona_micdet_reading(info);
- else
- arizona_button_reading(info);
-
- pm_runtime_mark_last_busy(info->dev);
- mutex_unlock(&info->lock);
-}
-
-static irqreturn_t arizona_micdet(int irq, void *data)
-{
- struct arizona_extcon_info *info = data;
- struct arizona *arizona = info->arizona;
- int debounce = arizona->pdata.micd_detect_debounce;
-
- cancel_delayed_work_sync(&info->micd_detect_work);
- cancel_delayed_work_sync(&info->micd_timeout_work);
-
- mutex_lock(&info->lock);
- if (!info->detecting)
- debounce = 0;
- mutex_unlock(&info->lock);
-
- if (debounce)
- queue_delayed_work(system_power_efficient_wq,
- &info->micd_detect_work,
- msecs_to_jiffies(debounce));
- else
- arizona_micd_detect(&info->micd_detect_work.work);
-
- return IRQ_HANDLED;
-}
-
-static void arizona_hpdet_work(struct work_struct *work)
-{
- struct arizona_extcon_info *info = container_of(work,
- struct arizona_extcon_info,
- hpdet_work.work);
-
- mutex_lock(&info->lock);
- arizona_start_hpdet_acc_id(info);
- mutex_unlock(&info->lock);
-}
-
-static int arizona_hpdet_wait(struct arizona_extcon_info *info)
-{
- struct arizona *arizona = info->arizona;
- unsigned int val;
- int i, ret;
-
- for (i = 0; i < ARIZONA_HPDET_WAIT_COUNT; i++) {
- ret = regmap_read(arizona->regmap, ARIZONA_HEADPHONE_DETECT_2,
- &val);
- if (ret) {
- dev_err(arizona->dev,
- "Failed to read HPDET state: %d\n", ret);
- return ret;
- }
-
- switch (info->hpdet_ip_version) {
- case 0:
- if (val & ARIZONA_HP_DONE)
- return 0;
- break;
- default:
- if (val & ARIZONA_HP_DONE_B)
- return 0;
- break;
- }
-
- msleep(ARIZONA_HPDET_WAIT_DELAY_MS);
- }
-
- dev_warn(arizona->dev, "HPDET did not appear to complete\n");
-
- return -ETIMEDOUT;
-}
-
-static irqreturn_t arizona_jackdet(int irq, void *data)
-{
- struct arizona_extcon_info *info = data;
- struct arizona *arizona = info->arizona;
- unsigned int val, present, mask;
- bool cancelled_hp, cancelled_mic;
- int ret, i;
-
- cancelled_hp = cancel_delayed_work_sync(&info->hpdet_work);
- cancelled_mic = cancel_delayed_work_sync(&info->micd_timeout_work);
-
- pm_runtime_get_sync(info->dev);
-
- mutex_lock(&info->lock);
-
- if (info->micd_clamp) {
- mask = ARIZONA_MICD_CLAMP_STS;
- present = 0;
- } else {
- mask = ARIZONA_JD1_STS;
- if (arizona->pdata.jd_invert)
- present = 0;
- else
- present = ARIZONA_JD1_STS;
- }
-
- ret = regmap_read(arizona->regmap, ARIZONA_AOD_IRQ_RAW_STATUS, &val);
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to read jackdet status: %d\n",
- ret);
- mutex_unlock(&info->lock);
- pm_runtime_put_autosuspend(info->dev);
- return IRQ_NONE;
- }
-
- val &= mask;
- if (val == info->last_jackdet) {
- dev_dbg(arizona->dev, "Suppressing duplicate JACKDET\n");
- if (cancelled_hp)
- queue_delayed_work(system_power_efficient_wq,
- &info->hpdet_work,
- msecs_to_jiffies(HPDET_DEBOUNCE));
-
- if (cancelled_mic) {
- int micd_timeout = arizona->pdata.micd_timeout;
-
- queue_delayed_work(system_power_efficient_wq,
- &info->micd_timeout_work,
- msecs_to_jiffies(micd_timeout));
- }
-
- goto out;
- }
- info->last_jackdet = val;
-
- if (info->last_jackdet == present) {
- dev_dbg(arizona->dev, "Detected jack\n");
- ret = extcon_set_state_sync(info->edev,
- EXTCON_MECHANICAL, true);
-
- if (ret != 0)
- dev_err(arizona->dev, "Mechanical report failed: %d\n",
- ret);
-
- info->detecting = true;
- info->mic = false;
- info->jack_flips = 0;
-
- if (!arizona->pdata.hpdet_acc_id) {
- arizona_start_mic(info);
- } else {
- queue_delayed_work(system_power_efficient_wq,
- &info->hpdet_work,
- msecs_to_jiffies(HPDET_DEBOUNCE));
- }
-
- if (info->micd_clamp || !arizona->pdata.jd_invert)
- regmap_update_bits(arizona->regmap,
- ARIZONA_JACK_DETECT_DEBOUNCE,
- ARIZONA_MICD_CLAMP_DB |
- ARIZONA_JD1_DB, 0);
- } else {
- dev_dbg(arizona->dev, "Detected jack removal\n");
-
- arizona_stop_mic(info);
-
- info->num_hpdet_res = 0;
- for (i = 0; i < ARRAY_SIZE(info->hpdet_res); i++)
- info->hpdet_res[i] = 0;
- info->mic = false;
- info->hpdet_done = false;
- info->hpdet_retried = false;
-
- for (i = 0; i < info->num_micd_ranges; i++)
- input_report_key(info->input,
- info->micd_ranges[i].key, 0);
- input_sync(info->input);
-
- for (i = 0; i < ARRAY_SIZE(arizona_cable) - 1; i++) {
- ret = extcon_set_state_sync(info->edev,
- arizona_cable[i], false);
- if (ret != 0)
- dev_err(arizona->dev,
- "Removal report failed: %d\n", ret);
- }
-
- /*
- * If the jack was removed during a headphone detection we
- * need to wait for the headphone detection to finish, as
- * it can not be aborted. We don't want to be able to start
- * a new headphone detection from a fresh insert until this
- * one is finished.
- */
- arizona_hpdet_wait(info);
-
- regmap_update_bits(arizona->regmap,
- ARIZONA_JACK_DETECT_DEBOUNCE,
- ARIZONA_MICD_CLAMP_DB | ARIZONA_JD1_DB,
- ARIZONA_MICD_CLAMP_DB | ARIZONA_JD1_DB);
- }
-
-out:
- /* Clear trig_sts to make sure DCVDD is not forced up */
- regmap_write(arizona->regmap, ARIZONA_AOD_WKUP_AND_TRIG,
- ARIZONA_MICD_CLAMP_FALL_TRIG_STS |
- ARIZONA_MICD_CLAMP_RISE_TRIG_STS |
- ARIZONA_JD1_FALL_TRIG_STS |
- ARIZONA_JD1_RISE_TRIG_STS);
-
- mutex_unlock(&info->lock);
-
- pm_runtime_mark_last_busy(info->dev);
- pm_runtime_put_autosuspend(info->dev);
-
- return IRQ_HANDLED;
-}
-
-/* Map a level onto a slot in the register bank */
-static void arizona_micd_set_level(struct arizona *arizona, int index,
- unsigned int level)
-{
- int reg;
- unsigned int mask;
-
- reg = ARIZONA_MIC_DETECT_LEVEL_4 - (index / 2);
-
- if (!(index % 2)) {
- mask = 0x3f00;
- level <<= 8;
- } else {
- mask = 0x3f;
- }
-
- /* Program the level itself */
- regmap_update_bits(arizona->regmap, reg, mask, level);
-}
-
-static int arizona_extcon_get_micd_configs(struct device *dev,
- struct arizona *arizona)
-{
- const char * const prop = "wlf,micd-configs";
- const int entries_per_config = 3;
- struct arizona_micd_config *micd_configs;
- int nconfs, ret;
- int i, j;
- u32 *vals;
-
- nconfs = device_property_count_u32(arizona->dev, prop);
- if (nconfs <= 0)
- return 0;
-
- vals = kcalloc(nconfs, sizeof(u32), GFP_KERNEL);
- if (!vals)
- return -ENOMEM;
-
- ret = device_property_read_u32_array(arizona->dev, prop, vals, nconfs);
- if (ret < 0)
- goto out;
-
- nconfs /= entries_per_config;
- micd_configs = devm_kcalloc(dev, nconfs, sizeof(*micd_configs),
- GFP_KERNEL);
- if (!micd_configs) {
- ret = -ENOMEM;
- goto out;
- }
-
- for (i = 0, j = 0; i < nconfs; ++i) {
- micd_configs[i].src = vals[j++] ? ARIZONA_ACCDET_SRC : 0;
- micd_configs[i].bias = vals[j++];
- micd_configs[i].gpio = vals[j++];
- }
-
- arizona->pdata.micd_configs = micd_configs;
- arizona->pdata.num_micd_configs = nconfs;
-
-out:
- kfree(vals);
- return ret;
-}
-
-static int arizona_extcon_device_get_pdata(struct device *dev,
- struct arizona *arizona)
-{
- struct arizona_pdata *pdata = &arizona->pdata;
- unsigned int val = ARIZONA_ACCDET_MODE_HPL;
- int ret;
-
- device_property_read_u32(arizona->dev, "wlf,hpdet-channel", &val);
- switch (val) {
- case ARIZONA_ACCDET_MODE_HPL:
- case ARIZONA_ACCDET_MODE_HPR:
- pdata->hpdet_channel = val;
- break;
- default:
- dev_err(arizona->dev,
- "Wrong wlf,hpdet-channel DT value %d\n", val);
- pdata->hpdet_channel = ARIZONA_ACCDET_MODE_HPL;
- }
-
- device_property_read_u32(arizona->dev, "wlf,micd-detect-debounce",
- &pdata->micd_detect_debounce);
-
- device_property_read_u32(arizona->dev, "wlf,micd-bias-start-time",
- &pdata->micd_bias_start_time);
-
- device_property_read_u32(arizona->dev, "wlf,micd-rate",
- &pdata->micd_rate);
-
- device_property_read_u32(arizona->dev, "wlf,micd-dbtime",
- &pdata->micd_dbtime);
-
- device_property_read_u32(arizona->dev, "wlf,micd-timeout-ms",
- &pdata->micd_timeout);
-
- pdata->micd_force_micbias = device_property_read_bool(arizona->dev,
- "wlf,micd-force-micbias");
-
- pdata->micd_software_compare = device_property_read_bool(arizona->dev,
- "wlf,micd-software-compare");
-
- pdata->jd_invert = device_property_read_bool(arizona->dev,
- "wlf,jd-invert");
-
- device_property_read_u32(arizona->dev, "wlf,gpsw", &pdata->gpsw);
-
- pdata->jd_gpio5 = device_property_read_bool(arizona->dev,
- "wlf,use-jd2");
- pdata->jd_gpio5_nopull = device_property_read_bool(arizona->dev,
- "wlf,use-jd2-nopull");
-
- ret = arizona_extcon_get_micd_configs(dev, arizona);
- if (ret < 0)
- dev_err(arizona->dev, "Failed to read micd configs: %d\n", ret);
-
- return 0;
-}
-
-static int arizona_extcon_probe(struct platform_device *pdev)
-{
- struct arizona *arizona = dev_get_drvdata(pdev->dev.parent);
- struct arizona_pdata *pdata = &arizona->pdata;
- struct arizona_extcon_info *info;
- unsigned int val;
- unsigned int clamp_mode;
- int jack_irq_fall, jack_irq_rise;
- int ret, mode, i, j;
-
- if (!arizona->dapm || !arizona->dapm->card)
- return -EPROBE_DEFER;
-
- info = devm_kzalloc(&pdev->dev, sizeof(*info), GFP_KERNEL);
- if (!info)
- return -ENOMEM;
-
- if (!dev_get_platdata(arizona->dev))
- arizona_extcon_device_get_pdata(&pdev->dev, arizona);
-
- info->micvdd = devm_regulator_get(&pdev->dev, "MICVDD");
- if (IS_ERR(info->micvdd)) {
- ret = PTR_ERR(info->micvdd);
- dev_err(arizona->dev, "Failed to get MICVDD: %d\n", ret);
- return ret;
- }
-
- mutex_init(&info->lock);
- info->arizona = arizona;
- info->dev = &pdev->dev;
- info->last_jackdet = ~(ARIZONA_MICD_CLAMP_STS | ARIZONA_JD1_STS);
- INIT_DELAYED_WORK(&info->hpdet_work, arizona_hpdet_work);
- INIT_DELAYED_WORK(&info->micd_detect_work, arizona_micd_detect);
- INIT_DELAYED_WORK(&info->micd_timeout_work, arizona_micd_timeout_work);
- platform_set_drvdata(pdev, info);
-
- switch (arizona->type) {
- case WM5102:
- switch (arizona->rev) {
- case 0:
- info->micd_reva = true;
- break;
- default:
- info->micd_clamp = true;
- info->hpdet_ip_version = 1;
- break;
- }
- break;
- case WM5110:
- case WM8280:
- switch (arizona->rev) {
- case 0 ... 2:
- break;
- default:
- info->micd_clamp = true;
- info->hpdet_ip_version = 2;
- break;
- }
- break;
- case WM8998:
- case WM1814:
- info->micd_clamp = true;
- info->hpdet_ip_version = 2;
- break;
- default:
- break;
- }
-
- info->edev = devm_extcon_dev_allocate(&pdev->dev, arizona_cable);
- if (IS_ERR(info->edev)) {
- dev_err(&pdev->dev, "failed to allocate extcon device\n");
- return -ENOMEM;
- }
-
- ret = devm_extcon_dev_register(&pdev->dev, info->edev);
- if (ret < 0) {
- dev_err(arizona->dev, "extcon_dev_register() failed: %d\n",
- ret);
- return ret;
- }
-
- info->input = devm_input_allocate_device(&pdev->dev);
- if (!info->input) {
- dev_err(arizona->dev, "Can't allocate input dev\n");
- ret = -ENOMEM;
- return ret;
- }
-
- info->input->name = "Headset";
- info->input->phys = "arizona/extcon";
-
- if (!pdata->micd_timeout)
- pdata->micd_timeout = DEFAULT_MICD_TIMEOUT;
-
- if (pdata->num_micd_configs) {
- info->micd_modes = pdata->micd_configs;
- info->micd_num_modes = pdata->num_micd_configs;
- } else {
- info->micd_modes = micd_default_modes;
- info->micd_num_modes = ARRAY_SIZE(micd_default_modes);
- }
-
- if (arizona->pdata.gpsw > 0)
- regmap_update_bits(arizona->regmap, ARIZONA_GP_SWITCH_1,
- ARIZONA_SW1_MODE_MASK, arizona->pdata.gpsw);
-
- if (pdata->micd_pol_gpio > 0) {
- if (info->micd_modes[0].gpio)
- mode = GPIOF_OUT_INIT_HIGH;
- else
- mode = GPIOF_OUT_INIT_LOW;
-
- ret = devm_gpio_request_one(&pdev->dev, pdata->micd_pol_gpio,
- mode, "MICD polarity");
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to request GPIO%d: %d\n",
- pdata->micd_pol_gpio, ret);
- return ret;
- }
-
- info->micd_pol_gpio = gpio_to_desc(pdata->micd_pol_gpio);
- } else {
- if (info->micd_modes[0].gpio)
- mode = GPIOD_OUT_HIGH;
- else
- mode = GPIOD_OUT_LOW;
-
- /* We can't use devm here because we need to do the get
- * against the MFD device, as that is where the of_node
- * will reside, but if we devm against that the GPIO
- * will not be freed if the extcon driver is unloaded.
- */
- info->micd_pol_gpio = gpiod_get_optional(arizona->dev,
- "wlf,micd-pol",
- GPIOD_OUT_LOW);
- if (IS_ERR(info->micd_pol_gpio)) {
- ret = PTR_ERR(info->micd_pol_gpio);
- dev_err(arizona->dev,
- "Failed to get microphone polarity GPIO: %d\n",
- ret);
- return ret;
- }
- }
-
- if (arizona->pdata.hpdet_id_gpio > 0) {
- ret = devm_gpio_request_one(&pdev->dev,
- arizona->pdata.hpdet_id_gpio,
- GPIOF_OUT_INIT_LOW,
- "HPDET");
- if (ret != 0) {
- dev_err(arizona->dev, "Failed to request GPIO%d: %d\n",
- arizona->pdata.hpdet_id_gpio, ret);
- goto err_gpio;
- }
- }
-
- if (arizona->pdata.micd_bias_start_time)
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_BIAS_STARTTIME_MASK,
- arizona->pdata.micd_bias_start_time
- << ARIZONA_MICD_BIAS_STARTTIME_SHIFT);
-
- if (arizona->pdata.micd_rate)
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_RATE_MASK,
- arizona->pdata.micd_rate
- << ARIZONA_MICD_RATE_SHIFT);
-
- switch (arizona->pdata.micd_dbtime) {
- case MICD_DBTIME_FOUR_READINGS:
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_DBTIME_MASK,
- ARIZONA_MICD_DBTIME);
- break;
- case MICD_DBTIME_TWO_READINGS:
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_DBTIME_MASK, 0);
- break;
- default:
- break;
- }
-
- BUILD_BUG_ON(ARRAY_SIZE(arizona_micd_levels) <
- ARIZONA_NUM_MICD_BUTTON_LEVELS);
-
- if (arizona->pdata.num_micd_ranges) {
- info->micd_ranges = pdata->micd_ranges;
- info->num_micd_ranges = pdata->num_micd_ranges;
- } else {
- info->micd_ranges = micd_default_ranges;
- info->num_micd_ranges = ARRAY_SIZE(micd_default_ranges);
- }
-
- if (arizona->pdata.num_micd_ranges > ARIZONA_MAX_MICD_RANGE) {
- dev_err(arizona->dev, "Too many MICD ranges: %d\n",
- arizona->pdata.num_micd_ranges);
- }
-
- if (info->num_micd_ranges > 1) {
- for (i = 1; i < info->num_micd_ranges; i++) {
- if (info->micd_ranges[i - 1].max >
- info->micd_ranges[i].max) {
- dev_err(arizona->dev,
- "MICD ranges must be sorted\n");
- ret = -EINVAL;
- goto err_gpio;
- }
- }
- }
-
- /* Disable all buttons by default */
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_2,
- ARIZONA_MICD_LVL_SEL_MASK, 0x81);
-
- /* Set up all the buttons the user specified */
- for (i = 0; i < info->num_micd_ranges; i++) {
- for (j = 0; j < ARIZONA_NUM_MICD_BUTTON_LEVELS; j++)
- if (arizona_micd_levels[j] >= info->micd_ranges[i].max)
- break;
-
- if (j == ARIZONA_NUM_MICD_BUTTON_LEVELS) {
- dev_err(arizona->dev, "Unsupported MICD level %d\n",
- info->micd_ranges[i].max);
- ret = -EINVAL;
- goto err_gpio;
- }
-
- dev_dbg(arizona->dev, "%d ohms for MICD threshold %d\n",
- arizona_micd_levels[j], i);
-
- arizona_micd_set_level(arizona, i, j);
- input_set_capability(info->input, EV_KEY,
- info->micd_ranges[i].key);
-
- /* Enable reporting of that range */
- regmap_update_bits(arizona->regmap, ARIZONA_MIC_DETECT_2,
- 1 << i, 1 << i);
- }
-
- /* Set all the remaining keys to a maximum */
- for (; i < ARIZONA_MAX_MICD_RANGE; i++)
- arizona_micd_set_level(arizona, i, 0x3f);
-
- /*
- * If we have a clamp use it, activating in conjunction with
- * GPIO5 if that is connected for jack detect operation.
- */
- if (info->micd_clamp) {
- if (arizona->pdata.jd_gpio5) {
- /* Put the GPIO into input mode with optional pull */
- val = 0xc101;
- if (arizona->pdata.jd_gpio5_nopull)
- val &= ~ARIZONA_GPN_PU;
-
- regmap_write(arizona->regmap, ARIZONA_GPIO5_CTRL,
- val);
-
- if (arizona->pdata.jd_invert)
- clamp_mode = ARIZONA_MICD_CLAMP_MODE_JDH_GP5H;
- else
- clamp_mode = ARIZONA_MICD_CLAMP_MODE_JDL_GP5H;
- } else {
- if (arizona->pdata.jd_invert)
- clamp_mode = ARIZONA_MICD_CLAMP_MODE_JDH;
- else
- clamp_mode = ARIZONA_MICD_CLAMP_MODE_JDL;
- }
-
- regmap_update_bits(arizona->regmap,
- ARIZONA_MICD_CLAMP_CONTROL,
- ARIZONA_MICD_CLAMP_MODE_MASK, clamp_mode);
-
- regmap_update_bits(arizona->regmap,
- ARIZONA_JACK_DETECT_DEBOUNCE,
- ARIZONA_MICD_CLAMP_DB,
- ARIZONA_MICD_CLAMP_DB);
- }
-
- arizona_extcon_set_mode(info, 0);
-
- pm_runtime_enable(&pdev->dev);
- pm_runtime_idle(&pdev->dev);
- pm_runtime_get_sync(&pdev->dev);
-
- if (info->micd_clamp) {
- jack_irq_rise = ARIZONA_IRQ_MICD_CLAMP_RISE;
- jack_irq_fall = ARIZONA_IRQ_MICD_CLAMP_FALL;
- } else {
- jack_irq_rise = ARIZONA_IRQ_JD_RISE;
- jack_irq_fall = ARIZONA_IRQ_JD_FALL;
- }
-
- ret = arizona_request_irq(arizona, jack_irq_rise,
- "JACKDET rise", arizona_jackdet, info);
- if (ret != 0) {
- dev_err(&pdev->dev, "Failed to get JACKDET rise IRQ: %d\n",
- ret);
- goto err_pm;
- }
-
- ret = arizona_set_irq_wake(arizona, jack_irq_rise, 1);
- if (ret != 0) {
- dev_err(&pdev->dev, "Failed to set JD rise IRQ wake: %d\n",
- ret);
- goto err_rise;
- }
-
- ret = arizona_request_irq(arizona, jack_irq_fall,
- "JACKDET fall", arizona_jackdet, info);
- if (ret != 0) {
- dev_err(&pdev->dev, "Failed to get JD fall IRQ: %d\n", ret);
- goto err_rise_wake;
- }
-
- ret = arizona_set_irq_wake(arizona, jack_irq_fall, 1);
- if (ret != 0) {
- dev_err(&pdev->dev, "Failed to set JD fall IRQ wake: %d\n",
- ret);
- goto err_fall;
- }
-
- ret = arizona_request_irq(arizona, ARIZONA_IRQ_MICDET,
- "MICDET", arizona_micdet, info);
- if (ret != 0) {
- dev_err(&pdev->dev, "Failed to get MICDET IRQ: %d\n", ret);
- goto err_fall_wake;
- }
-
- ret = arizona_request_irq(arizona, ARIZONA_IRQ_HPDET,
- "HPDET", arizona_hpdet_irq, info);
- if (ret != 0) {
- dev_err(&pdev->dev, "Failed to get HPDET IRQ: %d\n", ret);
- goto err_micdet;
- }
-
- arizona_clk32k_enable(arizona);
- regmap_update_bits(arizona->regmap, ARIZONA_JACK_DETECT_DEBOUNCE,
- ARIZONA_JD1_DB, ARIZONA_JD1_DB);
- regmap_update_bits(arizona->regmap, ARIZONA_JACK_DETECT_ANALOGUE,
- ARIZONA_JD1_ENA, ARIZONA_JD1_ENA);
-
- ret = regulator_allow_bypass(info->micvdd, true);
- if (ret != 0)
- dev_warn(arizona->dev, "Failed to set MICVDD to bypass: %d\n",
- ret);
-
- ret = input_register_device(info->input);
- if (ret) {
- dev_err(&pdev->dev, "Can't register input device: %d\n", ret);
- goto err_hpdet;
- }
-
- pm_runtime_put(&pdev->dev);
-
- return 0;
-
-err_hpdet:
- arizona_free_irq(arizona, ARIZONA_IRQ_HPDET, info);
-err_micdet:
- arizona_free_irq(arizona, ARIZONA_IRQ_MICDET, info);
-err_fall_wake:
- arizona_set_irq_wake(arizona, jack_irq_fall, 0);
-err_fall:
- arizona_free_irq(arizona, jack_irq_fall, info);
-err_rise_wake:
- arizona_set_irq_wake(arizona, jack_irq_rise, 0);
-err_rise:
- arizona_free_irq(arizona, jack_irq_rise, info);
-err_pm:
- pm_runtime_put(&pdev->dev);
- pm_runtime_disable(&pdev->dev);
-err_gpio:
- gpiod_put(info->micd_pol_gpio);
- return ret;
-}
-
-static int arizona_extcon_remove(struct platform_device *pdev)
-{
- struct arizona_extcon_info *info = platform_get_drvdata(pdev);
- struct arizona *arizona = info->arizona;
- int jack_irq_rise, jack_irq_fall;
- bool change;
- int ret;
-
- ret = regmap_update_bits_check(arizona->regmap, ARIZONA_MIC_DETECT_1,
- ARIZONA_MICD_ENA, 0,
- &change);
- if (ret < 0) {
- dev_err(&pdev->dev, "Failed to disable micd on remove: %d\n",
- ret);
- } else if (change) {
- regulator_disable(info->micvdd);
- pm_runtime_put(info->dev);
- }
-
- gpiod_put(info->micd_pol_gpio);
-
- pm_runtime_disable(&pdev->dev);
-
- regmap_update_bits(arizona->regmap,
- ARIZONA_MICD_CLAMP_CONTROL,
- ARIZONA_MICD_CLAMP_MODE_MASK, 0);
-
- if (info->micd_clamp) {
- jack_irq_rise = ARIZONA_IRQ_MICD_CLAMP_RISE;
- jack_irq_fall = ARIZONA_IRQ_MICD_CLAMP_FALL;
- } else {
- jack_irq_rise = ARIZONA_IRQ_JD_RISE;
- jack_irq_fall = ARIZONA_IRQ_JD_FALL;
- }
-
- arizona_set_irq_wake(arizona, jack_irq_rise, 0);
- arizona_set_irq_wake(arizona, jack_irq_fall, 0);
- arizona_free_irq(arizona, ARIZONA_IRQ_HPDET, info);
- arizona_free_irq(arizona, ARIZONA_IRQ_MICDET, info);
- arizona_free_irq(arizona, jack_irq_rise, info);
- arizona_free_irq(arizona, jack_irq_fall, info);
- cancel_delayed_work_sync(&info->hpdet_work);
- regmap_update_bits(arizona->regmap, ARIZONA_JACK_DETECT_ANALOGUE,
- ARIZONA_JD1_ENA, 0);
- arizona_clk32k_disable(arizona);
-
- return 0;
-}
-
-static struct platform_driver arizona_extcon_driver = {
- .driver = {
- .name = "arizona-extcon",
- },
- .probe = arizona_extcon_probe,
- .remove = arizona_extcon_remove,
-};
-
-module_platform_driver(arizona_extcon_driver);
-
-MODULE_DESCRIPTION("Arizona Extcon driver");
-MODULE_AUTHOR("Mark Brown <broonie@opensource.wolfsonmicro.com>");
-MODULE_LICENSE("GPL");
-MODULE_ALIAS("platform:extcon-arizona");
diff --git a/drivers/extcon/extcon-gpio.c b/drivers/extcon/extcon-gpio.c
index c211222f5d0c..4105df74f2b0 100644
--- a/drivers/extcon/extcon-gpio.c
+++ b/drivers/extcon/extcon-gpio.c
@@ -9,6 +9,7 @@
* (originally switch class is supported)
*/
+#include <linux/devm-helpers.h>
#include <linux/extcon-provider.h>
#include <linux/gpio/consumer.h>
#include <linux/init.h>
@@ -112,7 +113,9 @@ static int gpio_extcon_probe(struct platform_device *pdev)
if (ret < 0)
return ret;
- INIT_DELAYED_WORK(&data->work, gpio_extcon_work);
+ ret = devm_delayed_work_autocancel(dev, &data->work, gpio_extcon_work);
+ if (ret)
+ return ret;
/*
* Request the interrupt of gpio to detect whether external connector
@@ -131,15 +134,6 @@ static int gpio_extcon_probe(struct platform_device *pdev)
return 0;
}
-static int gpio_extcon_remove(struct platform_device *pdev)
-{
- struct gpio_extcon_data *data = platform_get_drvdata(pdev);
-
- cancel_delayed_work_sync(&data->work);
-
- return 0;
-}
-
#ifdef CONFIG_PM_SLEEP
static int gpio_extcon_resume(struct device *dev)
{
@@ -158,7 +152,6 @@ static SIMPLE_DEV_PM_OPS(gpio_extcon_pm_ops, NULL, gpio_extcon_resume);
static struct platform_driver gpio_extcon_driver = {
.probe = gpio_extcon_probe,
- .remove = gpio_extcon_remove,
.driver = {
.name = "extcon-gpio",
.pm = &gpio_extcon_pm_ops,
diff --git a/drivers/extcon/extcon-intel-int3496.c b/drivers/extcon/extcon-intel-int3496.c
index 80c9abcc3f97..fb527c23639e 100644
--- a/drivers/extcon/extcon-intel-int3496.c
+++ b/drivers/extcon/extcon-intel-int3496.c
@@ -11,6 +11,7 @@
*/
#include <linux/acpi.h>
+#include <linux/devm-helpers.h>
#include <linux/extcon-provider.h>
#include <linux/gpio/consumer.h>
#include <linux/interrupt.h>
@@ -101,7 +102,9 @@ static int int3496_probe(struct platform_device *pdev)
return -ENOMEM;
data->dev = dev;
- INIT_DELAYED_WORK(&data->work, int3496_do_usb_id);
+ ret = devm_delayed_work_autocancel(dev, &data->work, int3496_do_usb_id);
+ if (ret)
+ return ret;
data->gpio_usb_id = devm_gpiod_get(dev, "id", GPIOD_IN);
if (IS_ERR(data->gpio_usb_id)) {
@@ -155,16 +158,6 @@ static int int3496_probe(struct platform_device *pdev)
return 0;
}
-static int int3496_remove(struct platform_device *pdev)
-{
- struct int3496_data *data = platform_get_drvdata(pdev);
-
- devm_free_irq(&pdev->dev, data->usb_id_irq, data);
- cancel_delayed_work_sync(&data->work);
-
- return 0;
-}
-
static const struct acpi_device_id int3496_acpi_match[] = {
{ "INT3496" },
{ }
@@ -177,7 +170,6 @@ static struct platform_driver int3496_driver = {
.acpi_match_table = int3496_acpi_match,
},
.probe = int3496_probe,
- .remove = int3496_remove,
};
module_platform_driver(int3496_driver);
diff --git a/drivers/extcon/extcon-max8997.c b/drivers/extcon/extcon-max8997.c
index 337b0eea4e62..e1408075ef7d 100644
--- a/drivers/extcon/extcon-max8997.c
+++ b/drivers/extcon/extcon-max8997.c
@@ -44,6 +44,8 @@ static struct max8997_muic_irq muic_irqs[] = {
{ MAX8997_MUICIRQ_ChgDetRun, "muic-CHGDETRUN" },
{ MAX8997_MUICIRQ_ChgTyp, "muic-CHGTYP" },
{ MAX8997_MUICIRQ_OVP, "muic-OVP" },
+ { MAX8997_PMICIRQ_CHGINS, "pmic-CHGINS" },
+ { MAX8997_PMICIRQ_CHGRM, "pmic-CHGRM" },
};
/* Define supported cable type */
@@ -538,6 +540,8 @@ static void max8997_muic_irq_work(struct work_struct *work)
case MAX8997_MUICIRQ_DCDTmr:
case MAX8997_MUICIRQ_ChgDetRun:
case MAX8997_MUICIRQ_ChgTyp:
+ case MAX8997_PMICIRQ_CHGINS:
+ case MAX8997_PMICIRQ_CHGRM:
/* Handle charger cable */
ret = max8997_muic_chg_handler(info);
break;
diff --git a/drivers/extcon/extcon-palmas.c b/drivers/extcon/extcon-palmas.c
index a2852bcc5f0d..d2c1a8b89c08 100644
--- a/drivers/extcon/extcon-palmas.c
+++ b/drivers/extcon/extcon-palmas.c
@@ -9,6 +9,7 @@
* Author: Hema HK <hemahk@ti.com>
*/
+#include <linux/devm-helpers.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#include <linux/platform_device.h>
@@ -237,7 +238,11 @@ static int palmas_usb_probe(struct platform_device *pdev)
palmas_usb->sw_debounce_jiffies = msecs_to_jiffies(debounce);
}
- INIT_DELAYED_WORK(&palmas_usb->wq_detectid, palmas_gpio_id_detect);
+ status = devm_delayed_work_autocancel(&pdev->dev,
+ &palmas_usb->wq_detectid,
+ palmas_gpio_id_detect);
+ if (status)
+ return status;
palmas->usb = palmas_usb;
palmas_usb->palmas = palmas;
@@ -359,15 +364,6 @@ static int palmas_usb_probe(struct platform_device *pdev)
return 0;
}
-static int palmas_usb_remove(struct platform_device *pdev)
-{
- struct palmas_usb *palmas_usb = platform_get_drvdata(pdev);
-
- cancel_delayed_work_sync(&palmas_usb->wq_detectid);
-
- return 0;
-}
-
#ifdef CONFIG_PM_SLEEP
static int palmas_usb_suspend(struct device *dev)
{
@@ -422,7 +418,6 @@ static const struct of_device_id of_palmas_match_tbl[] = {
static struct platform_driver palmas_usb_driver = {
.probe = palmas_usb_probe,
- .remove = palmas_usb_remove,
.driver = {
.name = "palmas-usb",
.of_match_table = of_palmas_match_tbl,
diff --git a/drivers/extcon/extcon-qcom-spmi-misc.c b/drivers/extcon/extcon-qcom-spmi-misc.c
index 6b836ae62176..eb02cb962b5e 100644
--- a/drivers/extcon/extcon-qcom-spmi-misc.c
+++ b/drivers/extcon/extcon-qcom-spmi-misc.c
@@ -1,12 +1,13 @@
// SPDX-License-Identifier: GPL-2.0-only
/**
* extcon-qcom-spmi-misc.c - Qualcomm USB extcon driver to support USB ID
- * detection based on extcon-usb-gpio.c.
+ * and VBUS detection based on extcon-usb-gpio.c.
*
* Copyright (C) 2016 Linaro, Ltd.
* Stephen Boyd <stephen.boyd@linaro.org>
*/
+#include <linux/devm-helpers.h>
#include <linux/extcon-provider.h>
#include <linux/init.h>
#include <linux/interrupt.h>
@@ -21,30 +22,56 @@
struct qcom_usb_extcon_info {
struct extcon_dev *edev;
- int irq;
+ int id_irq;
+ int vbus_irq;
struct delayed_work wq_detcable;
unsigned long debounce_jiffies;
};
static const unsigned int qcom_usb_extcon_cable[] = {
+ EXTCON_USB,
EXTCON_USB_HOST,
EXTCON_NONE,
};
static void qcom_usb_extcon_detect_cable(struct work_struct *work)
{
- bool id;
+ bool state = false;
int ret;
+ union extcon_property_value val;
struct qcom_usb_extcon_info *info = container_of(to_delayed_work(work),
struct qcom_usb_extcon_info,
wq_detcable);
- /* check ID and update cable state */
- ret = irq_get_irqchip_state(info->irq, IRQCHIP_STATE_LINE_LEVEL, &id);
- if (ret)
- return;
+ if (info->id_irq > 0) {
+ /* check ID and update cable state */
+ ret = irq_get_irqchip_state(info->id_irq,
+ IRQCHIP_STATE_LINE_LEVEL, &state);
+ if (ret)
+ return;
+
+ if (!state) {
+ val.intval = true;
+ extcon_set_property(info->edev, EXTCON_USB_HOST,
+ EXTCON_PROP_USB_SS, val);
+ }
+ extcon_set_state_sync(info->edev, EXTCON_USB_HOST, !state);
+ }
- extcon_set_state_sync(info->edev, EXTCON_USB_HOST, !id);
+ if (info->vbus_irq > 0) {
+ /* check VBUS and update cable state */
+ ret = irq_get_irqchip_state(info->vbus_irq,
+ IRQCHIP_STATE_LINE_LEVEL, &state);
+ if (ret)
+ return;
+
+ if (state) {
+ val.intval = true;
+ extcon_set_property(info->edev, EXTCON_USB,
+ EXTCON_PROP_USB_SS, val);
+ }
+ extcon_set_state_sync(info->edev, EXTCON_USB, state);
+ }
}
static irqreturn_t qcom_usb_irq_handler(int irq, void *dev_id)
@@ -79,21 +106,52 @@ static int qcom_usb_extcon_probe(struct platform_device *pdev)
return ret;
}
+ ret = extcon_set_property_capability(info->edev,
+ EXTCON_USB, EXTCON_PROP_USB_SS);
+ ret |= extcon_set_property_capability(info->edev,
+ EXTCON_USB_HOST, EXTCON_PROP_USB_SS);
+ if (ret) {
+ dev_err(dev, "failed to register extcon props rc=%d\n",
+ ret);
+ return ret;
+ }
+
info->debounce_jiffies = msecs_to_jiffies(USB_ID_DEBOUNCE_MS);
- INIT_DELAYED_WORK(&info->wq_detcable, qcom_usb_extcon_detect_cable);
- info->irq = platform_get_irq_byname(pdev, "usb_id");
- if (info->irq < 0)
- return info->irq;
+ ret = devm_delayed_work_autocancel(dev, &info->wq_detcable,
+ qcom_usb_extcon_detect_cable);
+ if (ret)
+ return ret;
- ret = devm_request_threaded_irq(dev, info->irq, NULL,
+ info->id_irq = platform_get_irq_byname(pdev, "usb_id");
+ if (info->id_irq > 0) {
+ ret = devm_request_threaded_irq(dev, info->id_irq, NULL,
qcom_usb_irq_handler,
IRQF_TRIGGER_RISING |
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
pdev->name, info);
- if (ret < 0) {
- dev_err(dev, "failed to request handler for ID IRQ\n");
- return ret;
+ if (ret < 0) {
+ dev_err(dev, "failed to request handler for ID IRQ\n");
+ return ret;
+ }
+ }
+
+ info->vbus_irq = platform_get_irq_byname(pdev, "usb_vbus");
+ if (info->vbus_irq > 0) {
+ ret = devm_request_threaded_irq(dev, info->vbus_irq, NULL,
+ qcom_usb_irq_handler,
+ IRQF_TRIGGER_RISING |
+ IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
+ pdev->name, info);
+ if (ret < 0) {
+ dev_err(dev, "failed to request handler for VBUS IRQ\n");
+ return ret;
+ }
+ }
+
+ if (info->id_irq < 0 && info->vbus_irq < 0) {
+ dev_err(dev, "ID and VBUS IRQ not found\n");
+ return -EINVAL;
}
platform_set_drvdata(pdev, info);
@@ -105,23 +163,18 @@ static int qcom_usb_extcon_probe(struct platform_device *pdev)
return 0;
}
-static int qcom_usb_extcon_remove(struct platform_device *pdev)
-{
- struct qcom_usb_extcon_info *info = platform_get_drvdata(pdev);
-
- cancel_delayed_work_sync(&info->wq_detcable);
-
- return 0;
-}
-
#ifdef CONFIG_PM_SLEEP
static int qcom_usb_extcon_suspend(struct device *dev)
{
struct qcom_usb_extcon_info *info = dev_get_drvdata(dev);
int ret = 0;
- if (device_may_wakeup(dev))
- ret = enable_irq_wake(info->irq);
+ if (device_may_wakeup(dev)) {
+ if (info->id_irq > 0)
+ ret = enable_irq_wake(info->id_irq);
+ if (info->vbus_irq > 0)
+ ret = enable_irq_wake(info->vbus_irq);
+ }
return ret;
}
@@ -131,8 +184,12 @@ static int qcom_usb_extcon_resume(struct device *dev)
struct qcom_usb_extcon_info *info = dev_get_drvdata(dev);
int ret = 0;
- if (device_may_wakeup(dev))
- ret = disable_irq_wake(info->irq);
+ if (device_may_wakeup(dev)) {
+ if (info->id_irq > 0)
+ ret = disable_irq_wake(info->id_irq);
+ if (info->vbus_irq > 0)
+ ret = disable_irq_wake(info->vbus_irq);
+ }
return ret;
}
@@ -149,7 +206,6 @@ MODULE_DEVICE_TABLE(of, qcom_usb_extcon_dt_match);
static struct platform_driver qcom_usb_extcon_driver = {
.probe = qcom_usb_extcon_probe,
- .remove = qcom_usb_extcon_remove,
.driver = {
.name = "extcon-pm8941-misc",
.pm = &qcom_usb_extcon_pm_ops,
diff --git a/drivers/extcon/extcon-sm5502.c b/drivers/extcon/extcon-sm5502.c
index 106d4da647bd..db41d1c58efd 100644
--- a/drivers/extcon/extcon-sm5502.c
+++ b/drivers/extcon/extcon-sm5502.c
@@ -144,6 +144,7 @@ enum sm5502_muic_acc_type {
SM5502_MUIC_ADC_AUDIO_TYPE1_FULL_REMOTE = 0x3e, /* | 001|11110| */
SM5502_MUIC_ADC_AUDIO_TYPE1_SEND_END = 0x5e, /* | 010|11110| */
/* |Dev Type1|--ADC| */
+ SM5502_MUIC_ADC_GROUND_USB_OTG = 0x80, /* | 100|00000| */
SM5502_MUIC_ADC_OPEN_USB = 0x5f, /* | 010|11111| */
SM5502_MUIC_ADC_OPEN_TA = 0xdf, /* | 110|11111| */
SM5502_MUIC_ADC_OPEN_USB_OTG = 0xff, /* | 111|11111| */
@@ -291,11 +292,27 @@ static unsigned int sm5502_muic_get_cable_type(struct sm5502_muic_info *info)
* connected with to MUIC device.
*/
cable_type = adc & SM5502_REG_ADC_MASK;
- if (cable_type == SM5502_MUIC_ADC_GROUND)
- return SM5502_MUIC_ADC_GROUND;
switch (cable_type) {
case SM5502_MUIC_ADC_GROUND:
+ ret = regmap_read(info->regmap, SM5502_REG_DEV_TYPE1,
+ &dev_type1);
+ if (ret) {
+ dev_err(info->dev, "failed to read DEV_TYPE1 reg\n");
+ return ret;
+ }
+
+ switch (dev_type1) {
+ case SM5502_REG_DEV_TYPE1_USB_OTG_MASK:
+ cable_type = SM5502_MUIC_ADC_GROUND_USB_OTG;
+ break;
+ default:
+ dev_dbg(info->dev,
+ "cannot identify the cable type: adc(0x%x), dev_type1(0x%x)\n",
+ adc, dev_type1);
+ return -EINVAL;
+ }
+ break;
case SM5502_MUIC_ADC_SEND_END_BUTTON:
case SM5502_MUIC_ADC_REMOTE_S1_BUTTON:
case SM5502_MUIC_ADC_REMOTE_S2_BUTTON:
@@ -396,6 +413,7 @@ static int sm5502_muic_cable_handler(struct sm5502_muic_info *info,
con_sw = DM_DP_SWITCH_OPEN;
vbus_sw = VBUSIN_SWITCH_VBUSOUT;
break;
+ case SM5502_MUIC_ADC_GROUND_USB_OTG:
case SM5502_MUIC_ADC_OPEN_USB_OTG:
id = EXTCON_USB_HOST;
con_sw = DM_DP_SWITCH_USB;
diff --git a/drivers/extcon/extcon.c b/drivers/extcon/extcon.c
index 0a6438cbb3f3..e7a9561a826d 100644
--- a/drivers/extcon/extcon.c
+++ b/drivers/extcon/extcon.c
@@ -1241,6 +1241,7 @@ int extcon_dev_register(struct extcon_dev *edev)
sizeof(*edev->nh), GFP_KERNEL);
if (!edev->nh) {
ret = -ENOMEM;
+ device_unregister(&edev->dev);
goto err_dev;
}
diff --git a/drivers/firewire/nosy.c b/drivers/firewire/nosy.c
index 5fd6a60b6741..88ed971e32c0 100644
--- a/drivers/firewire/nosy.c
+++ b/drivers/firewire/nosy.c
@@ -346,6 +346,7 @@ nosy_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
struct client *client = file->private_data;
spinlock_t *client_list_lock = &client->lynx->client_list_lock;
struct nosy_stats stats;
+ int ret;
switch (cmd) {
case NOSY_IOC_GET_STATS:
@@ -360,11 +361,15 @@ nosy_ioctl(struct file *file, unsigned int cmd, unsigned long arg)
return 0;
case NOSY_IOC_START:
+ ret = -EBUSY;
spin_lock_irq(client_list_lock);
- list_add_tail(&client->link, &client->lynx->client_list);
+ if (list_empty(&client->link)) {
+ list_add_tail(&client->link, &client->lynx->client_list);
+ ret = 0;
+ }
spin_unlock_irq(client_list_lock);
- return 0;
+ return ret;
case NOSY_IOC_STOP:
spin_lock_irq(client_list_lock);
diff --git a/drivers/firewire/ohci.c b/drivers/firewire/ohci.c
index 9811c40956e5..17c9d825188b 100644
--- a/drivers/firewire/ohci.c
+++ b/drivers/firewire/ohci.c
@@ -2545,7 +2545,7 @@ static int ohci_cancel_packet(struct fw_card *card, struct fw_packet *packet)
struct driver_data *driver_data = packet->driver_data;
int ret = -ENOENT;
- tasklet_disable(&ctx->tasklet);
+ tasklet_disable_in_atomic(&ctx->tasklet);
if (packet->ack != 0)
goto out;
@@ -3465,7 +3465,7 @@ static int ohci_flush_iso_completions(struct fw_iso_context *base)
struct iso_context *ctx = container_of(base, struct iso_context, base);
int ret = 0;
- tasklet_disable(&ctx->context.tasklet);
+ tasklet_disable_in_atomic(&ctx->context.tasklet);
if (!test_and_set_bit_lock(0, &ctx->flushing_completions)) {
context_tasklet((unsigned long)&ctx->context);
diff --git a/drivers/firmware/Kconfig b/drivers/firmware/Kconfig
index 3f14dffb9669..db0ea2d2d75a 100644
--- a/drivers/firmware/Kconfig
+++ b/drivers/firmware/Kconfig
@@ -206,7 +206,7 @@ config FW_CFG_SYSFS_CMDLINE
config INTEL_STRATIX10_SERVICE
tristate "Intel Stratix10 Service Layer"
- depends on (ARCH_STRATIX10 || ARCH_AGILEX) && HAVE_ARM_SMCCC
+ depends on ARCH_INTEL_SOCFPGA && ARM64 && HAVE_ARM_SMCCC
default n
help
Intel Stratix10 service layer runs at privileged exception level,
@@ -237,6 +237,7 @@ config INTEL_STRATIX10_RSU
config QCOM_SCM
bool
depends on ARM || ARM64
+ depends on HAVE_ARM_SMCCC
select RESET_CONTROLLER
config QCOM_SCM_DOWNLOAD_MODE_DEFAULT
diff --git a/drivers/firmware/arm_scmi/base.c b/drivers/firmware/arm_scmi/base.c
index 017e5d8bd869..de416f9e7921 100644
--- a/drivers/firmware/arm_scmi/base.c
+++ b/drivers/firmware/arm_scmi/base.c
@@ -2,11 +2,12 @@
/*
* System Control and Management Interface (SCMI) Base Protocol
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#define pr_fmt(fmt) "SCMI Notifications BASE - " fmt
+#include <linux/module.h>
#include <linux/scmi_protocol.h>
#include "common.h"
@@ -50,30 +51,30 @@ struct scmi_base_error_notify_payld {
* scmi_base_attributes_get() - gets the implementation details
* that are associated with the base protocol.
*
- * @handle: SCMI entity handle
+ * @ph: SCMI protocol handle
*
* Return: 0 on success, else appropriate SCMI error.
*/
-static int scmi_base_attributes_get(const struct scmi_handle *handle)
+static int scmi_base_attributes_get(const struct scmi_protocol_handle *ph)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_base_attributes *attr_info;
- struct scmi_revision_info *rev = handle->version;
+ struct scmi_revision_info *rev = ph->get_priv(ph);
- ret = scmi_xfer_get_init(handle, PROTOCOL_ATTRIBUTES,
- SCMI_PROTOCOL_BASE, 0, sizeof(*attr_info), &t);
+ ret = ph->xops->xfer_get_init(ph, PROTOCOL_ATTRIBUTES,
+ 0, sizeof(*attr_info), &t);
if (ret)
return ret;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
attr_info = t->rx.buf;
rev->num_protocols = attr_info->num_protocols;
rev->num_agents = attr_info->num_agents;
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -81,19 +82,20 @@ static int scmi_base_attributes_get(const struct scmi_handle *handle)
/**
* scmi_base_vendor_id_get() - gets vendor/subvendor identifier ASCII string.
*
- * @handle: SCMI entity handle
+ * @ph: SCMI protocol handle
* @sub_vendor: specify true if sub-vendor ID is needed
*
* Return: 0 on success, else appropriate SCMI error.
*/
static int
-scmi_base_vendor_id_get(const struct scmi_handle *handle, bool sub_vendor)
+scmi_base_vendor_id_get(const struct scmi_protocol_handle *ph, bool sub_vendor)
{
u8 cmd;
int ret, size;
char *vendor_id;
struct scmi_xfer *t;
- struct scmi_revision_info *rev = handle->version;
+ struct scmi_revision_info *rev = ph->get_priv(ph);
+
if (sub_vendor) {
cmd = BASE_DISCOVER_SUB_VENDOR;
@@ -105,15 +107,15 @@ scmi_base_vendor_id_get(const struct scmi_handle *handle, bool sub_vendor)
size = ARRAY_SIZE(rev->vendor_id);
}
- ret = scmi_xfer_get_init(handle, cmd, SCMI_PROTOCOL_BASE, 0, size, &t);
+ ret = ph->xops->xfer_get_init(ph, cmd, 0, size, &t);
if (ret)
return ret;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
memcpy(vendor_id, t->rx.buf, size);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -123,30 +125,30 @@ scmi_base_vendor_id_get(const struct scmi_handle *handle, bool sub_vendor)
* implementation 32-bit version. The format of the version number is
* vendor-specific
*
- * @handle: SCMI entity handle
+ * @ph: SCMI protocol handle
*
* Return: 0 on success, else appropriate SCMI error.
*/
static int
-scmi_base_implementation_version_get(const struct scmi_handle *handle)
+scmi_base_implementation_version_get(const struct scmi_protocol_handle *ph)
{
int ret;
__le32 *impl_ver;
struct scmi_xfer *t;
- struct scmi_revision_info *rev = handle->version;
+ struct scmi_revision_info *rev = ph->get_priv(ph);
- ret = scmi_xfer_get_init(handle, BASE_DISCOVER_IMPLEMENT_VERSION,
- SCMI_PROTOCOL_BASE, 0, sizeof(*impl_ver), &t);
+ ret = ph->xops->xfer_get_init(ph, BASE_DISCOVER_IMPLEMENT_VERSION,
+ 0, sizeof(*impl_ver), &t);
if (ret)
return ret;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
impl_ver = t->rx.buf;
rev->impl_ver = le32_to_cpu(*impl_ver);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -155,23 +157,24 @@ scmi_base_implementation_version_get(const struct scmi_handle *handle)
* scmi_base_implementation_list_get() - gets the list of protocols it is
* OSPM is allowed to access
*
- * @handle: SCMI entity handle
+ * @ph: SCMI protocol handle
* @protocols_imp: pointer to hold the list of protocol identifiers
*
* Return: 0 on success, else appropriate SCMI error.
*/
-static int scmi_base_implementation_list_get(const struct scmi_handle *handle,
- u8 *protocols_imp)
+static int
+scmi_base_implementation_list_get(const struct scmi_protocol_handle *ph,
+ u8 *protocols_imp)
{
u8 *list;
int ret, loop;
struct scmi_xfer *t;
__le32 *num_skip, *num_ret;
u32 tot_num_ret = 0, loop_num_ret;
- struct device *dev = handle->dev;
+ struct device *dev = ph->dev;
- ret = scmi_xfer_get_init(handle, BASE_DISCOVER_LIST_PROTOCOLS,
- SCMI_PROTOCOL_BASE, sizeof(*num_skip), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, BASE_DISCOVER_LIST_PROTOCOLS,
+ sizeof(*num_skip), 0, &t);
if (ret)
return ret;
@@ -183,7 +186,7 @@ static int scmi_base_implementation_list_get(const struct scmi_handle *handle,
/* Set the number of protocols to be skipped/already read */
*num_skip = cpu_to_le32(tot_num_ret);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (ret)
break;
@@ -198,10 +201,10 @@ static int scmi_base_implementation_list_get(const struct scmi_handle *handle,
tot_num_ret += loop_num_ret;
- scmi_reset_rx_to_maxsz(handle, t);
+ ph->xops->reset_rx_to_maxsz(ph, t);
} while (loop_num_ret);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -209,7 +212,7 @@ static int scmi_base_implementation_list_get(const struct scmi_handle *handle,
/**
* scmi_base_discover_agent_get() - discover the name of an agent
*
- * @handle: SCMI entity handle
+ * @ph: SCMI protocol handle
* @id: Agent identifier
* @name: Agent identifier ASCII string
*
@@ -218,63 +221,63 @@ static int scmi_base_implementation_list_get(const struct scmi_handle *handle,
*
* Return: 0 on success, else appropriate SCMI error.
*/
-static int scmi_base_discover_agent_get(const struct scmi_handle *handle,
+static int scmi_base_discover_agent_get(const struct scmi_protocol_handle *ph,
int id, char *name)
{
int ret;
struct scmi_xfer *t;
- ret = scmi_xfer_get_init(handle, BASE_DISCOVER_AGENT,
- SCMI_PROTOCOL_BASE, sizeof(__le32),
- SCMI_MAX_STR_SIZE, &t);
+ ret = ph->xops->xfer_get_init(ph, BASE_DISCOVER_AGENT,
+ sizeof(__le32), SCMI_MAX_STR_SIZE, &t);
if (ret)
return ret;
put_unaligned_le32(id, t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
strlcpy(name, t->rx.buf, SCMI_MAX_STR_SIZE);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_base_error_notify(const struct scmi_handle *handle, bool enable)
+static int scmi_base_error_notify(const struct scmi_protocol_handle *ph,
+ bool enable)
{
int ret;
u32 evt_cntl = enable ? BASE_TP_NOTIFY_ALL : 0;
struct scmi_xfer *t;
struct scmi_msg_base_error_notify *cfg;
- ret = scmi_xfer_get_init(handle, BASE_NOTIFY_ERRORS,
- SCMI_PROTOCOL_BASE, sizeof(*cfg), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, BASE_NOTIFY_ERRORS,
+ sizeof(*cfg), 0, &t);
if (ret)
return ret;
cfg = t->tx.buf;
cfg->event_control = cpu_to_le32(evt_cntl);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_base_set_notify_enabled(const struct scmi_handle *handle,
+static int scmi_base_set_notify_enabled(const struct scmi_protocol_handle *ph,
u8 evt_id, u32 src_id, bool enable)
{
int ret;
- ret = scmi_base_error_notify(handle, enable);
+ ret = scmi_base_error_notify(ph, enable);
if (ret)
pr_debug("FAIL_ENABLED - evt[%X] ret:%d\n", evt_id, ret);
return ret;
}
-static void *scmi_base_fill_custom_report(const struct scmi_handle *handle,
+static void *scmi_base_fill_custom_report(const struct scmi_protocol_handle *ph,
u8 evt_id, ktime_t timestamp,
const void *payld, size_t payld_sz,
void *report, u32 *src_id)
@@ -318,17 +321,24 @@ static const struct scmi_event_ops base_event_ops = {
.fill_custom_report = scmi_base_fill_custom_report,
};
-int scmi_base_protocol_init(struct scmi_handle *h)
+static const struct scmi_protocol_events base_protocol_events = {
+ .queue_sz = 4 * SCMI_PROTO_QUEUE_SZ,
+ .ops = &base_event_ops,
+ .evts = base_events,
+ .num_events = ARRAY_SIZE(base_events),
+ .num_sources = SCMI_BASE_NUM_SOURCES,
+};
+
+static int scmi_base_protocol_init(const struct scmi_protocol_handle *ph)
{
int id, ret;
u8 *prot_imp;
u32 version;
char name[SCMI_MAX_STR_SIZE];
- const struct scmi_handle *handle = h;
- struct device *dev = handle->dev;
- struct scmi_revision_info *rev = handle->version;
+ struct device *dev = ph->dev;
+ struct scmi_revision_info *rev = scmi_revision_area_get(ph);
- ret = scmi_version_get(handle, SCMI_PROTOCOL_BASE, &version);
+ ret = ph->xops->version_get(ph, &version);
if (ret)
return ret;
@@ -338,13 +348,15 @@ int scmi_base_protocol_init(struct scmi_handle *h)
rev->major_ver = PROTOCOL_REV_MAJOR(version),
rev->minor_ver = PROTOCOL_REV_MINOR(version);
+ ph->set_priv(ph, rev);
+
+ scmi_base_attributes_get(ph);
+ scmi_base_vendor_id_get(ph, false);
+ scmi_base_vendor_id_get(ph, true);
+ scmi_base_implementation_version_get(ph);
+ scmi_base_implementation_list_get(ph, prot_imp);
- scmi_base_attributes_get(handle);
- scmi_base_vendor_id_get(handle, false);
- scmi_base_vendor_id_get(handle, true);
- scmi_base_implementation_version_get(handle);
- scmi_base_implementation_list_get(handle, prot_imp);
- scmi_setup_protocol_implemented(handle, prot_imp);
+ scmi_setup_protocol_implemented(ph, prot_imp);
dev_info(dev, "SCMI Protocol v%d.%d '%s:%s' Firmware version 0x%x\n",
rev->major_ver, rev->minor_ver, rev->vendor_id,
@@ -352,16 +364,20 @@ int scmi_base_protocol_init(struct scmi_handle *h)
dev_dbg(dev, "Found %d protocol(s) %d agent(s)\n", rev->num_protocols,
rev->num_agents);
- scmi_register_protocol_events(handle, SCMI_PROTOCOL_BASE,
- (4 * SCMI_PROTO_QUEUE_SZ),
- &base_event_ops, base_events,
- ARRAY_SIZE(base_events),
- SCMI_BASE_NUM_SOURCES);
-
for (id = 0; id < rev->num_agents; id++) {
- scmi_base_discover_agent_get(handle, id, name);
+ scmi_base_discover_agent_get(ph, id, name);
dev_dbg(dev, "Agent %d: %s\n", id, name);
}
return 0;
}
+
+static const struct scmi_protocol scmi_base = {
+ .id = SCMI_PROTOCOL_BASE,
+ .owner = NULL,
+ .instance_init = &scmi_base_protocol_init,
+ .ops = NULL,
+ .events = &base_protocol_events,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(base, scmi_base)
diff --git a/drivers/firmware/arm_scmi/bus.c b/drivers/firmware/arm_scmi/bus.c
index 1377ec76a45d..784cf0027da3 100644
--- a/drivers/firmware/arm_scmi/bus.c
+++ b/drivers/firmware/arm_scmi/bus.c
@@ -2,7 +2,7 @@
/*
* System Control and Management Interface (SCMI) Message Protocol bus layer
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
@@ -51,18 +51,53 @@ static int scmi_dev_match(struct device *dev, struct device_driver *drv)
return 0;
}
-static int scmi_protocol_init(int protocol_id, struct scmi_handle *handle)
+static int scmi_match_by_id_table(struct device *dev, void *data)
{
- scmi_prot_init_fn_t fn = idr_find(&scmi_protocols, protocol_id);
+ struct scmi_device *sdev = to_scmi_dev(dev);
+ struct scmi_device_id *id_table = data;
- if (unlikely(!fn))
- return -EINVAL;
- return fn(handle);
+ return sdev->protocol_id == id_table->protocol_id &&
+ !strcmp(sdev->name, id_table->name);
}
-static int scmi_protocol_dummy_init(struct scmi_handle *handle)
+struct scmi_device *scmi_child_dev_find(struct device *parent,
+ int prot_id, const char *name)
{
- return 0;
+ struct scmi_device_id id_table;
+ struct device *dev;
+
+ id_table.protocol_id = prot_id;
+ id_table.name = name;
+
+ dev = device_find_child(parent, &id_table, scmi_match_by_id_table);
+ if (!dev)
+ return NULL;
+
+ return to_scmi_dev(dev);
+}
+
+const struct scmi_protocol *scmi_protocol_get(int protocol_id)
+{
+ const struct scmi_protocol *proto;
+
+ proto = idr_find(&scmi_protocols, protocol_id);
+ if (!proto || !try_module_get(proto->owner)) {
+ pr_warn("SCMI Protocol 0x%x not found!\n", protocol_id);
+ return NULL;
+ }
+
+ pr_debug("Found SCMI Protocol 0x%x\n", protocol_id);
+
+ return proto;
+}
+
+void scmi_protocol_put(int protocol_id)
+{
+ const struct scmi_protocol *proto;
+
+ proto = idr_find(&scmi_protocols, protocol_id);
+ if (proto)
+ module_put(proto->owner);
}
static int scmi_dev_probe(struct device *dev)
@@ -70,7 +105,6 @@ static int scmi_dev_probe(struct device *dev)
struct scmi_driver *scmi_drv = to_scmi_driver(dev->driver);
struct scmi_device *scmi_dev = to_scmi_dev(dev);
const struct scmi_device_id *id;
- int ret;
id = scmi_dev_match_id(scmi_dev, scmi_drv);
if (!id)
@@ -79,14 +113,6 @@ static int scmi_dev_probe(struct device *dev)
if (!scmi_dev->handle)
return -EPROBE_DEFER;
- ret = scmi_protocol_init(scmi_dev->protocol_id, scmi_dev->handle);
- if (ret)
- return ret;
-
- /* Skip protocol initialisation for additional devices */
- idr_replace(&scmi_protocols, &scmi_protocol_dummy_init,
- scmi_dev->protocol_id);
-
return scmi_drv->probe(scmi_dev);
}
@@ -113,6 +139,10 @@ int scmi_driver_register(struct scmi_driver *driver, struct module *owner,
{
int retval;
+ retval = scmi_protocol_device_request(driver->id_table);
+ if (retval)
+ return retval;
+
driver->driver.bus = &scmi_bus_type;
driver->driver.name = driver->name;
driver->driver.owner = owner;
@@ -129,6 +159,7 @@ EXPORT_SYMBOL_GPL(scmi_driver_register);
void scmi_driver_unregister(struct scmi_driver *driver)
{
driver_unregister(&driver->driver);
+ scmi_protocol_device_unrequest(driver->id_table);
}
EXPORT_SYMBOL_GPL(scmi_driver_unregister);
@@ -194,26 +225,45 @@ void scmi_set_handle(struct scmi_device *scmi_dev)
scmi_dev->handle = scmi_handle_get(&scmi_dev->dev);
}
-int scmi_protocol_register(int protocol_id, scmi_prot_init_fn_t fn)
+int scmi_protocol_register(const struct scmi_protocol *proto)
{
int ret;
+ if (!proto) {
+ pr_err("invalid protocol\n");
+ return -EINVAL;
+ }
+
+ if (!proto->instance_init) {
+ pr_err("missing init for protocol 0x%x\n", proto->id);
+ return -EINVAL;
+ }
+
spin_lock(&protocol_lock);
- ret = idr_alloc(&scmi_protocols, fn, protocol_id, protocol_id + 1,
- GFP_ATOMIC);
+ ret = idr_alloc(&scmi_protocols, (void *)proto,
+ proto->id, proto->id + 1, GFP_ATOMIC);
spin_unlock(&protocol_lock);
- if (ret != protocol_id)
- pr_err("unable to allocate SCMI idr slot, err %d\n", ret);
+ if (ret != proto->id) {
+ pr_err("unable to allocate SCMI idr slot for 0x%x - err %d\n",
+ proto->id, ret);
+ return ret;
+ }
- return ret;
+ pr_debug("Registered SCMI Protocol 0x%x\n", proto->id);
+
+ return 0;
}
EXPORT_SYMBOL_GPL(scmi_protocol_register);
-void scmi_protocol_unregister(int protocol_id)
+void scmi_protocol_unregister(const struct scmi_protocol *proto)
{
spin_lock(&protocol_lock);
- idr_remove(&scmi_protocols, protocol_id);
+ idr_remove(&scmi_protocols, proto->id);
spin_unlock(&protocol_lock);
+
+ pr_debug("Unregistered SCMI Protocol 0x%x\n", proto->id);
+
+ return;
}
EXPORT_SYMBOL_GPL(scmi_protocol_unregister);
diff --git a/drivers/firmware/arm_scmi/clock.c b/drivers/firmware/arm_scmi/clock.c
index 4645677d86f1..35b56c8ba0c0 100644
--- a/drivers/firmware/arm_scmi/clock.c
+++ b/drivers/firmware/arm_scmi/clock.c
@@ -2,9 +2,10 @@
/*
* System Control and Management Interface (SCMI) Clock Protocol
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
+#include <linux/module.h>
#include <linux/sort.h>
#include "common.h"
@@ -74,52 +75,53 @@ struct clock_info {
struct scmi_clock_info *clk;
};
-static int scmi_clock_protocol_attributes_get(const struct scmi_handle *handle,
- struct clock_info *ci)
+static int
+scmi_clock_protocol_attributes_get(const struct scmi_protocol_handle *ph,
+ struct clock_info *ci)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_clock_protocol_attributes *attr;
- ret = scmi_xfer_get_init(handle, PROTOCOL_ATTRIBUTES,
- SCMI_PROTOCOL_CLOCK, 0, sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, PROTOCOL_ATTRIBUTES,
+ 0, sizeof(*attr), &t);
if (ret)
return ret;
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
ci->num_clocks = le16_to_cpu(attr->num_clocks);
ci->max_async_req = attr->max_async_req;
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_clock_attributes_get(const struct scmi_handle *handle,
+static int scmi_clock_attributes_get(const struct scmi_protocol_handle *ph,
u32 clk_id, struct scmi_clock_info *clk)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_clock_attributes *attr;
- ret = scmi_xfer_get_init(handle, CLOCK_ATTRIBUTES, SCMI_PROTOCOL_CLOCK,
- sizeof(clk_id), sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, CLOCK_ATTRIBUTES,
+ sizeof(clk_id), sizeof(*attr), &t);
if (ret)
return ret;
put_unaligned_le32(clk_id, t->tx.buf);
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
strlcpy(clk->name, attr->name, SCMI_MAX_STR_SIZE);
else
clk->name[0] = '\0';
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -136,7 +138,7 @@ static int rate_cmp_func(const void *_r1, const void *_r2)
}
static int
-scmi_clock_describe_rates_get(const struct scmi_handle *handle, u32 clk_id,
+scmi_clock_describe_rates_get(const struct scmi_protocol_handle *ph, u32 clk_id,
struct scmi_clock_info *clk)
{
u64 *rate = NULL;
@@ -148,8 +150,8 @@ scmi_clock_describe_rates_get(const struct scmi_handle *handle, u32 clk_id,
struct scmi_msg_clock_describe_rates *clk_desc;
struct scmi_msg_resp_clock_describe_rates *rlist;
- ret = scmi_xfer_get_init(handle, CLOCK_DESCRIBE_RATES,
- SCMI_PROTOCOL_CLOCK, sizeof(*clk_desc), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, CLOCK_DESCRIBE_RATES,
+ sizeof(*clk_desc), 0, &t);
if (ret)
return ret;
@@ -161,7 +163,7 @@ scmi_clock_describe_rates_get(const struct scmi_handle *handle, u32 clk_id,
/* Set the number of rates to be skipped/already read */
clk_desc->rate_index = cpu_to_le32(tot_rate_cnt);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (ret)
goto err;
@@ -171,7 +173,7 @@ scmi_clock_describe_rates_get(const struct scmi_handle *handle, u32 clk_id,
num_returned = NUM_RETURNED(rates_flag);
if (tot_rate_cnt + num_returned > SCMI_MAX_NUM_RATES) {
- dev_err(handle->dev, "No. of rates > MAX_NUM_RATES");
+ dev_err(ph->dev, "No. of rates > MAX_NUM_RATES");
break;
}
@@ -179,7 +181,7 @@ scmi_clock_describe_rates_get(const struct scmi_handle *handle, u32 clk_id,
clk->range.min_rate = RATE_TO_U64(rlist->rate[0]);
clk->range.max_rate = RATE_TO_U64(rlist->rate[1]);
clk->range.step_size = RATE_TO_U64(rlist->rate[2]);
- dev_dbg(handle->dev, "Min %llu Max %llu Step %llu Hz\n",
+ dev_dbg(ph->dev, "Min %llu Max %llu Step %llu Hz\n",
clk->range.min_rate, clk->range.max_rate,
clk->range.step_size);
break;
@@ -188,12 +190,12 @@ scmi_clock_describe_rates_get(const struct scmi_handle *handle, u32 clk_id,
rate = &clk->list.rates[tot_rate_cnt];
for (cnt = 0; cnt < num_returned; cnt++, rate++) {
*rate = RATE_TO_U64(rlist->rate[cnt]);
- dev_dbg(handle->dev, "Rate %llu Hz\n", *rate);
+ dev_dbg(ph->dev, "Rate %llu Hz\n", *rate);
}
tot_rate_cnt += num_returned;
- scmi_reset_rx_to_maxsz(handle, t);
+ ph->xops->reset_rx_to_maxsz(ph, t);
/*
* check for both returned and remaining to avoid infinite
* loop due to buggy firmware
@@ -208,42 +210,42 @@ scmi_clock_describe_rates_get(const struct scmi_handle *handle, u32 clk_id,
clk->rate_discrete = rate_discrete;
err:
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
static int
-scmi_clock_rate_get(const struct scmi_handle *handle, u32 clk_id, u64 *value)
+scmi_clock_rate_get(const struct scmi_protocol_handle *ph,
+ u32 clk_id, u64 *value)
{
int ret;
struct scmi_xfer *t;
- ret = scmi_xfer_get_init(handle, CLOCK_RATE_GET, SCMI_PROTOCOL_CLOCK,
- sizeof(__le32), sizeof(u64), &t);
+ ret = ph->xops->xfer_get_init(ph, CLOCK_RATE_GET,
+ sizeof(__le32), sizeof(u64), &t);
if (ret)
return ret;
put_unaligned_le32(clk_id, t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
*value = get_unaligned_le64(t->rx.buf);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_clock_rate_set(const struct scmi_handle *handle, u32 clk_id,
- u64 rate)
+static int scmi_clock_rate_set(const struct scmi_protocol_handle *ph,
+ u32 clk_id, u64 rate)
{
int ret;
u32 flags = 0;
struct scmi_xfer *t;
struct scmi_clock_set_rate *cfg;
- struct clock_info *ci = handle->clk_priv;
+ struct clock_info *ci = ph->get_priv(ph);
- ret = scmi_xfer_get_init(handle, CLOCK_RATE_SET, SCMI_PROTOCOL_CLOCK,
- sizeof(*cfg), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, CLOCK_RATE_SET, sizeof(*cfg), 0, &t);
if (ret)
return ret;
@@ -258,26 +260,27 @@ static int scmi_clock_rate_set(const struct scmi_handle *handle, u32 clk_id,
cfg->value_high = cpu_to_le32(rate >> 32);
if (flags & CLOCK_SET_ASYNC)
- ret = scmi_do_xfer_with_response(handle, t);
+ ret = ph->xops->do_xfer_with_response(ph, t);
else
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (ci->max_async_req)
atomic_dec(&ci->cur_async_req);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
static int
-scmi_clock_config_set(const struct scmi_handle *handle, u32 clk_id, u32 config)
+scmi_clock_config_set(const struct scmi_protocol_handle *ph, u32 clk_id,
+ u32 config)
{
int ret;
struct scmi_xfer *t;
struct scmi_clock_set_config *cfg;
- ret = scmi_xfer_get_init(handle, CLOCK_CONFIG_SET, SCMI_PROTOCOL_CLOCK,
- sizeof(*cfg), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, CLOCK_CONFIG_SET,
+ sizeof(*cfg), 0, &t);
if (ret)
return ret;
@@ -285,33 +288,33 @@ scmi_clock_config_set(const struct scmi_handle *handle, u32 clk_id, u32 config)
cfg->id = cpu_to_le32(clk_id);
cfg->attributes = cpu_to_le32(config);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_clock_enable(const struct scmi_handle *handle, u32 clk_id)
+static int scmi_clock_enable(const struct scmi_protocol_handle *ph, u32 clk_id)
{
- return scmi_clock_config_set(handle, clk_id, CLOCK_ENABLE);
+ return scmi_clock_config_set(ph, clk_id, CLOCK_ENABLE);
}
-static int scmi_clock_disable(const struct scmi_handle *handle, u32 clk_id)
+static int scmi_clock_disable(const struct scmi_protocol_handle *ph, u32 clk_id)
{
- return scmi_clock_config_set(handle, clk_id, 0);
+ return scmi_clock_config_set(ph, clk_id, 0);
}
-static int scmi_clock_count_get(const struct scmi_handle *handle)
+static int scmi_clock_count_get(const struct scmi_protocol_handle *ph)
{
- struct clock_info *ci = handle->clk_priv;
+ struct clock_info *ci = ph->get_priv(ph);
return ci->num_clocks;
}
static const struct scmi_clock_info *
-scmi_clock_info_get(const struct scmi_handle *handle, u32 clk_id)
+scmi_clock_info_get(const struct scmi_protocol_handle *ph, u32 clk_id)
{
- struct clock_info *ci = handle->clk_priv;
+ struct clock_info *ci = ph->get_priv(ph);
struct scmi_clock_info *clk = ci->clk + clk_id;
if (!clk->name[0])
@@ -320,7 +323,7 @@ scmi_clock_info_get(const struct scmi_handle *handle, u32 clk_id)
return clk;
}
-static const struct scmi_clk_ops clk_ops = {
+static const struct scmi_clk_proto_ops clk_proto_ops = {
.count_get = scmi_clock_count_get,
.info_get = scmi_clock_info_get,
.rate_get = scmi_clock_rate_get,
@@ -329,24 +332,24 @@ static const struct scmi_clk_ops clk_ops = {
.disable = scmi_clock_disable,
};
-static int scmi_clock_protocol_init(struct scmi_handle *handle)
+static int scmi_clock_protocol_init(const struct scmi_protocol_handle *ph)
{
u32 version;
int clkid, ret;
struct clock_info *cinfo;
- scmi_version_get(handle, SCMI_PROTOCOL_CLOCK, &version);
+ ph->xops->version_get(ph, &version);
- dev_dbg(handle->dev, "Clock Version %d.%d\n",
+ dev_dbg(ph->dev, "Clock Version %d.%d\n",
PROTOCOL_REV_MAJOR(version), PROTOCOL_REV_MINOR(version));
- cinfo = devm_kzalloc(handle->dev, sizeof(*cinfo), GFP_KERNEL);
+ cinfo = devm_kzalloc(ph->dev, sizeof(*cinfo), GFP_KERNEL);
if (!cinfo)
return -ENOMEM;
- scmi_clock_protocol_attributes_get(handle, cinfo);
+ scmi_clock_protocol_attributes_get(ph, cinfo);
- cinfo->clk = devm_kcalloc(handle->dev, cinfo->num_clocks,
+ cinfo->clk = devm_kcalloc(ph->dev, cinfo->num_clocks,
sizeof(*cinfo->clk), GFP_KERNEL);
if (!cinfo->clk)
return -ENOMEM;
@@ -354,16 +357,20 @@ static int scmi_clock_protocol_init(struct scmi_handle *handle)
for (clkid = 0; clkid < cinfo->num_clocks; clkid++) {
struct scmi_clock_info *clk = cinfo->clk + clkid;
- ret = scmi_clock_attributes_get(handle, clkid, clk);
+ ret = scmi_clock_attributes_get(ph, clkid, clk);
if (!ret)
- scmi_clock_describe_rates_get(handle, clkid, clk);
+ scmi_clock_describe_rates_get(ph, clkid, clk);
}
cinfo->version = version;
- handle->clk_ops = &clk_ops;
- handle->clk_priv = cinfo;
-
- return 0;
+ return ph->set_priv(ph, cinfo);
}
-DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(SCMI_PROTOCOL_CLOCK, clock)
+static const struct scmi_protocol scmi_clock = {
+ .id = SCMI_PROTOCOL_CLOCK,
+ .owner = THIS_MODULE,
+ .instance_init = &scmi_clock_protocol_init,
+ .ops = &clk_proto_ops,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(clock, scmi_clock)
diff --git a/drivers/firmware/arm_scmi/common.h b/drivers/firmware/arm_scmi/common.h
index c0fb45e7c3e8..228bf4a71d23 100644
--- a/drivers/firmware/arm_scmi/common.h
+++ b/drivers/firmware/arm_scmi/common.h
@@ -4,7 +4,7 @@
* driver common header file containing some definitions, structures
* and function prototypes used in all the different SCMI protocols.
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#ifndef _SCMI_COMMON_H
#define _SCMI_COMMON_H
@@ -14,11 +14,14 @@
#include <linux/device.h>
#include <linux/errno.h>
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/scmi_protocol.h>
#include <linux/types.h>
#include <asm/unaligned.h>
+#include "notify.h"
+
#define PROTOCOL_REV_MINOR_MASK GENMASK(15, 0)
#define PROTOCOL_REV_MAJOR_MASK GENMASK(31, 16)
#define PROTOCOL_REV_MAJOR(x) (u16)(FIELD_GET(PROTOCOL_REV_MAJOR_MASK, (x)))
@@ -141,22 +144,92 @@ struct scmi_xfer {
struct completion *async_done;
};
-void scmi_xfer_put(const struct scmi_handle *h, struct scmi_xfer *xfer);
-int scmi_do_xfer(const struct scmi_handle *h, struct scmi_xfer *xfer);
-int scmi_do_xfer_with_response(const struct scmi_handle *h,
- struct scmi_xfer *xfer);
-int scmi_xfer_get_init(const struct scmi_handle *h, u8 msg_id, u8 prot_id,
- size_t tx_size, size_t rx_size, struct scmi_xfer **p);
-void scmi_reset_rx_to_maxsz(const struct scmi_handle *handle,
- struct scmi_xfer *xfer);
+struct scmi_xfer_ops;
+
+/**
+ * struct scmi_protocol_handle - Reference to an initialized protocol instance
+ *
+ * @dev: A reference to the associated SCMI instance device (handle->dev).
+ * @xops: A reference to a struct holding refs to the core xfer operations that
+ * can be used by the protocol implementation to generate SCMI messages.
+ * @set_priv: A method to set protocol private data for this instance.
+ * @get_priv: A method to get protocol private data previously set.
+ *
+ * This structure represents a protocol initialized against specific SCMI
+ * instance and it will be used as follows:
+ * - as a parameter fed from the core to the protocol initialization code so
+ * that it can access the core xfer operations to build and generate SCMI
+ * messages exclusively for the specific underlying protocol instance.
+ * - as an opaque handle fed by an SCMI driver user when it tries to access
+ * this protocol through its own protocol operations.
+ * In this case this handle will be returned as an opaque object together
+ * with the related protocol operations when the SCMI driver tries to access
+ * the protocol.
+ */
+struct scmi_protocol_handle {
+ struct device *dev;
+ const struct scmi_xfer_ops *xops;
+ int (*set_priv)(const struct scmi_protocol_handle *ph, void *priv);
+ void *(*get_priv)(const struct scmi_protocol_handle *ph);
+};
+
+/**
+ * struct scmi_xfer_ops - References to the core SCMI xfer operations.
+ * @version_get: Get this version protocol.
+ * @xfer_get_init: Initialize one struct xfer if any xfer slot is free.
+ * @reset_rx_to_maxsz: Reset rx size to max transport size.
+ * @do_xfer: Do the SCMI transfer.
+ * @do_xfer_with_response: Do the SCMI transfer waiting for a response.
+ * @xfer_put: Free the xfer slot.
+ *
+ * Note that all this operations expect a protocol handle as first parameter;
+ * they then internally use it to infer the underlying protocol number: this
+ * way is not possible for a protocol implementation to forge messages for
+ * another protocol.
+ */
+struct scmi_xfer_ops {
+ int (*version_get)(const struct scmi_protocol_handle *ph, u32 *version);
+ int (*xfer_get_init)(const struct scmi_protocol_handle *ph, u8 msg_id,
+ size_t tx_size, size_t rx_size,
+ struct scmi_xfer **p);
+ void (*reset_rx_to_maxsz)(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer);
+ int (*do_xfer)(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer);
+ int (*do_xfer_with_response)(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer);
+ void (*xfer_put)(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer);
+};
+
+struct scmi_revision_info *
+scmi_revision_area_get(const struct scmi_protocol_handle *ph);
int scmi_handle_put(const struct scmi_handle *handle);
struct scmi_handle *scmi_handle_get(struct device *dev);
void scmi_set_handle(struct scmi_device *scmi_dev);
-int scmi_version_get(const struct scmi_handle *h, u8 protocol, u32 *version);
-void scmi_setup_protocol_implemented(const struct scmi_handle *handle,
+void scmi_setup_protocol_implemented(const struct scmi_protocol_handle *ph,
u8 *prot_imp);
-int scmi_base_protocol_init(struct scmi_handle *h);
+typedef int (*scmi_prot_init_ph_fn_t)(const struct scmi_protocol_handle *);
+
+/**
+ * struct scmi_protocol - Protocol descriptor
+ * @id: Protocol ID.
+ * @owner: Module reference if any.
+ * @instance_init: Mandatory protocol initialization function.
+ * @instance_deinit: Optional protocol de-initialization function.
+ * @ops: Optional reference to the operations provided by the protocol and
+ * exposed in scmi_protocol.h.
+ * @events: An optional reference to the events supported by this protocol.
+ */
+struct scmi_protocol {
+ const u8 id;
+ struct module *owner;
+ const scmi_prot_init_ph_fn_t instance_init;
+ const scmi_prot_init_ph_fn_t instance_deinit;
+ const void *ops;
+ const struct scmi_protocol_events *events;
+};
int __init scmi_bus_init(void);
void __exit scmi_bus_exit(void);
@@ -164,6 +237,7 @@ void __exit scmi_bus_exit(void);
#define DECLARE_SCMI_REGISTER_UNREGISTER(func) \
int __init scmi_##func##_register(void); \
void __exit scmi_##func##_unregister(void)
+DECLARE_SCMI_REGISTER_UNREGISTER(base);
DECLARE_SCMI_REGISTER_UNREGISTER(clock);
DECLARE_SCMI_REGISTER_UNREGISTER(perf);
DECLARE_SCMI_REGISTER_UNREGISTER(power);
@@ -172,17 +246,25 @@ DECLARE_SCMI_REGISTER_UNREGISTER(sensors);
DECLARE_SCMI_REGISTER_UNREGISTER(voltage);
DECLARE_SCMI_REGISTER_UNREGISTER(system);
-#define DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(id, name) \
-int __init scmi_##name##_register(void) \
-{ \
- return scmi_protocol_register((id), &scmi_##name##_protocol_init); \
-} \
-\
-void __exit scmi_##name##_unregister(void) \
-{ \
- scmi_protocol_unregister((id)); \
+#define DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(name, proto) \
+static const struct scmi_protocol *__this_proto = &(proto); \
+ \
+int __init scmi_##name##_register(void) \
+{ \
+ return scmi_protocol_register(__this_proto); \
+} \
+ \
+void __exit scmi_##name##_unregister(void) \
+{ \
+ scmi_protocol_unregister(__this_proto); \
}
+const struct scmi_protocol *scmi_protocol_get(int protocol_id);
+void scmi_protocol_put(int protocol_id);
+
+int scmi_protocol_acquire(const struct scmi_handle *handle, u8 protocol_id);
+void scmi_protocol_release(const struct scmi_handle *handle, u8 protocol_id);
+
/* SCMI Transport */
/**
* struct scmi_chan_info - Structure representing a SCMI channel information
@@ -227,6 +309,11 @@ struct scmi_transport_ops {
bool (*poll_done)(struct scmi_chan_info *cinfo, struct scmi_xfer *xfer);
};
+int scmi_protocol_device_request(const struct scmi_device_id *id_table);
+void scmi_protocol_device_unrequest(const struct scmi_device_id *id_table);
+struct scmi_device *scmi_child_dev_find(struct device *parent,
+ int prot_id, const char *name);
+
/**
* struct scmi_desc - Description of SoC integration
*
@@ -265,4 +352,8 @@ void shmem_clear_channel(struct scmi_shared_mem __iomem *shmem);
bool shmem_poll_done(struct scmi_shared_mem __iomem *shmem,
struct scmi_xfer *xfer);
+void scmi_notification_instance_data_set(const struct scmi_handle *handle,
+ void *priv);
+void *scmi_notification_instance_data_get(const struct scmi_handle *handle);
+
#endif /* _SCMI_COMMON_H */
diff --git a/drivers/firmware/arm_scmi/driver.c b/drivers/firmware/arm_scmi/driver.c
index cacdf1589b10..66eb3f0e5daf 100644
--- a/drivers/firmware/arm_scmi/driver.c
+++ b/drivers/firmware/arm_scmi/driver.c
@@ -11,18 +11,22 @@
* various power domain DVFS including the core/cluster, certain system
* clocks configuration, thermal sensors and many others.
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#include <linux/bitmap.h>
+#include <linux/device.h>
#include <linux/export.h>
+#include <linux/idr.h>
#include <linux/io.h>
#include <linux/kernel.h>
#include <linux/ktime.h>
+#include <linux/list.h>
#include <linux/module.h>
#include <linux/of_address.h>
#include <linux/of_device.h>
#include <linux/processor.h>
+#include <linux/refcount.h>
#include <linux/slab.h>
#include "common.h"
@@ -53,6 +57,14 @@ static DEFINE_MUTEX(scmi_list_mutex);
/* Track the unique id for the transfers for debug & profiling purpose */
static atomic_t transfer_last_id;
+static DEFINE_IDR(scmi_requested_devices);
+static DEFINE_MUTEX(scmi_requested_devices_mtx);
+
+struct scmi_requested_dev {
+ const struct scmi_device_id *id_table;
+ struct list_head node;
+};
+
/**
* struct scmi_xfers_info - Structure to manage transfer information
*
@@ -69,6 +81,30 @@ struct scmi_xfers_info {
};
/**
+ * struct scmi_protocol_instance - Describe an initialized protocol instance.
+ * @handle: Reference to the SCMI handle associated to this protocol instance.
+ * @proto: A reference to the protocol descriptor.
+ * @gid: A reference for per-protocol devres management.
+ * @users: A refcount to track effective users of this protocol.
+ * @priv: Reference for optional protocol private data.
+ * @ph: An embedded protocol handle that will be passed down to protocol
+ * initialization code to identify this instance.
+ *
+ * Each protocol is initialized independently once for each SCMI platform in
+ * which is defined by DT and implemented by the SCMI server fw.
+ */
+struct scmi_protocol_instance {
+ const struct scmi_handle *handle;
+ const struct scmi_protocol *proto;
+ void *gid;
+ refcount_t users;
+ void *priv;
+ struct scmi_protocol_handle ph;
+};
+
+#define ph_to_pi(h) container_of(h, struct scmi_protocol_instance, ph)
+
+/**
* struct scmi_info - Structure representing a SCMI instance
*
* @dev: Device pointer
@@ -80,8 +116,15 @@ struct scmi_xfers_info {
* @rx_minfo: Universal Receive Message management info
* @tx_idr: IDR object to map protocol id to Tx channel info pointer
* @rx_idr: IDR object to map protocol id to Rx channel info pointer
+ * @protocols: IDR for protocols' instance descriptors initialized for
+ * this SCMI instance: populated on protocol's first attempted
+ * usage.
+ * @protocols_mtx: A mutex to protect protocols instances initialization.
* @protocols_imp: List of protocols implemented, currently maximum of
* MAX_PROTOCOLS_IMP elements allocated by the base protocol
+ * @active_protocols: IDR storing device_nodes for protocols actually defined
+ * in the DT and confirmed as implemented by fw.
+ * @notify_priv: Pointer to private data structure specific to notifications.
* @node: List head
* @users: Number of users of this instance
*/
@@ -94,7 +137,12 @@ struct scmi_info {
struct scmi_xfers_info rx_minfo;
struct idr tx_idr;
struct idr rx_idr;
+ struct idr protocols;
+ /* Ensure mutual exclusive access to protocols instance array */
+ struct mutex protocols_mtx;
u8 *protocols_imp;
+ struct idr active_protocols;
+ void *notify_priv;
struct list_head node;
int users;
};
@@ -136,6 +184,25 @@ static inline void scmi_dump_header_dbg(struct device *dev,
hdr->id, hdr->seq, hdr->protocol_id);
}
+void scmi_notification_instance_data_set(const struct scmi_handle *handle,
+ void *priv)
+{
+ struct scmi_info *info = handle_to_scmi_info(handle);
+
+ info->notify_priv = priv;
+ /* Ensure updated protocol private date are visible */
+ smp_wmb();
+}
+
+void *scmi_notification_instance_data_get(const struct scmi_handle *handle)
+{
+ struct scmi_info *info = handle_to_scmi_info(handle);
+
+ /* Ensure protocols_private_data has been updated */
+ smp_rmb();
+ return info->notify_priv;
+}
+
/**
* scmi_xfer_get() - Allocate one message
*
@@ -316,14 +383,16 @@ void scmi_rx_callback(struct scmi_chan_info *cinfo, u32 msg_hdr)
}
/**
- * scmi_xfer_put() - Release a transmit message
+ * xfer_put() - Release a transmit message
*
- * @handle: Pointer to SCMI entity handle
+ * @ph: Pointer to SCMI protocol handle
* @xfer: message that was reserved by scmi_xfer_get
*/
-void scmi_xfer_put(const struct scmi_handle *handle, struct scmi_xfer *xfer)
+static void xfer_put(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer)
{
- struct scmi_info *info = handle_to_scmi_info(handle);
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
+ struct scmi_info *info = handle_to_scmi_info(pi->handle);
__scmi_xfer_put(&info->tx_minfo, xfer);
}
@@ -340,23 +409,32 @@ static bool scmi_xfer_done_no_timeout(struct scmi_chan_info *cinfo,
}
/**
- * scmi_do_xfer() - Do one transfer
+ * do_xfer() - Do one transfer
*
- * @handle: Pointer to SCMI entity handle
+ * @ph: Pointer to SCMI protocol handle
* @xfer: Transfer to initiate and wait for response
*
* Return: -ETIMEDOUT in case of no response, if transmit error,
* return corresponding error, else if all goes well,
* return 0.
*/
-int scmi_do_xfer(const struct scmi_handle *handle, struct scmi_xfer *xfer)
+static int do_xfer(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer)
{
int ret;
int timeout;
- struct scmi_info *info = handle_to_scmi_info(handle);
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
+ struct scmi_info *info = handle_to_scmi_info(pi->handle);
struct device *dev = info->dev;
struct scmi_chan_info *cinfo;
+ /*
+ * Re-instate protocol id here from protocol handle so that cannot be
+ * overridden by mistake (or malice) by the protocol code mangling with
+ * the scmi_xfer structure.
+ */
+ xfer->hdr.protocol_id = pi->proto->id;
+
cinfo = idr_find(&info->tx_idr, xfer->hdr.protocol_id);
if (unlikely(!cinfo))
return -EINVAL;
@@ -402,10 +480,11 @@ int scmi_do_xfer(const struct scmi_handle *handle, struct scmi_xfer *xfer)
return ret;
}
-void scmi_reset_rx_to_maxsz(const struct scmi_handle *handle,
- struct scmi_xfer *xfer)
+static void reset_rx_to_maxsz(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer)
{
- struct scmi_info *info = handle_to_scmi_info(handle);
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
+ struct scmi_info *info = handle_to_scmi_info(pi->handle);
xfer->rx.len = info->desc->max_msg_size;
}
@@ -413,24 +492,27 @@ void scmi_reset_rx_to_maxsz(const struct scmi_handle *handle,
#define SCMI_MAX_RESPONSE_TIMEOUT (2 * MSEC_PER_SEC)
/**
- * scmi_do_xfer_with_response() - Do one transfer and wait until the delayed
+ * do_xfer_with_response() - Do one transfer and wait until the delayed
* response is received
*
- * @handle: Pointer to SCMI entity handle
+ * @ph: Pointer to SCMI protocol handle
* @xfer: Transfer to initiate and wait for response
*
* Return: -ETIMEDOUT in case of no delayed response, if transmit error,
* return corresponding error, else if all goes well, return 0.
*/
-int scmi_do_xfer_with_response(const struct scmi_handle *handle,
- struct scmi_xfer *xfer)
+static int do_xfer_with_response(const struct scmi_protocol_handle *ph,
+ struct scmi_xfer *xfer)
{
int ret, timeout = msecs_to_jiffies(SCMI_MAX_RESPONSE_TIMEOUT);
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
DECLARE_COMPLETION_ONSTACK(async_response);
+ xfer->hdr.protocol_id = pi->proto->id;
+
xfer->async_done = &async_response;
- ret = scmi_do_xfer(handle, xfer);
+ ret = do_xfer(ph, xfer);
if (!ret && !wait_for_completion_timeout(xfer->async_done, timeout))
ret = -ETIMEDOUT;
@@ -439,11 +521,10 @@ int scmi_do_xfer_with_response(const struct scmi_handle *handle,
}
/**
- * scmi_xfer_get_init() - Allocate and initialise one message for transmit
+ * xfer_get_init() - Allocate and initialise one message for transmit
*
- * @handle: Pointer to SCMI entity handle
+ * @ph: Pointer to SCMI protocol handle
* @msg_id: Message identifier
- * @prot_id: Protocol identifier for the message
* @tx_size: transmit message size
* @rx_size: receive message size
* @p: pointer to the allocated and initialised message
@@ -454,12 +535,14 @@ int scmi_do_xfer_with_response(const struct scmi_handle *handle,
* Return: 0 if all went fine with @p pointing to message, else
* corresponding error.
*/
-int scmi_xfer_get_init(const struct scmi_handle *handle, u8 msg_id, u8 prot_id,
- size_t tx_size, size_t rx_size, struct scmi_xfer **p)
+static int xfer_get_init(const struct scmi_protocol_handle *ph,
+ u8 msg_id, size_t tx_size, size_t rx_size,
+ struct scmi_xfer **p)
{
int ret;
struct scmi_xfer *xfer;
- struct scmi_info *info = handle_to_scmi_info(handle);
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
+ struct scmi_info *info = handle_to_scmi_info(pi->handle);
struct scmi_xfers_info *minfo = &info->tx_minfo;
struct device *dev = info->dev;
@@ -468,7 +551,7 @@ int scmi_xfer_get_init(const struct scmi_handle *handle, u8 msg_id, u8 prot_id,
tx_size > info->desc->max_msg_size)
return -ERANGE;
- xfer = scmi_xfer_get(handle, minfo);
+ xfer = scmi_xfer_get(pi->handle, minfo);
if (IS_ERR(xfer)) {
ret = PTR_ERR(xfer);
dev_err(dev, "failed to get free message slot(%d)\n", ret);
@@ -478,7 +561,7 @@ int scmi_xfer_get_init(const struct scmi_handle *handle, u8 msg_id, u8 prot_id,
xfer->tx.len = tx_size;
xfer->rx.len = rx_size ? : info->desc->max_msg_size;
xfer->hdr.id = msg_id;
- xfer->hdr.protocol_id = prot_id;
+ xfer->hdr.protocol_id = pi->proto->id;
xfer->hdr.poll_completion = false;
*p = xfer;
@@ -487,43 +570,276 @@ int scmi_xfer_get_init(const struct scmi_handle *handle, u8 msg_id, u8 prot_id,
}
/**
- * scmi_version_get() - command to get the revision of the SCMI entity
+ * version_get() - command to get the revision of the SCMI entity
*
- * @handle: Pointer to SCMI entity handle
- * @protocol: Protocol identifier for the message
+ * @ph: Pointer to SCMI protocol handle
* @version: Holds returned version of protocol.
*
* Updates the SCMI information in the internal data structure.
*
* Return: 0 if all went fine, else return appropriate error.
*/
-int scmi_version_get(const struct scmi_handle *handle, u8 protocol,
- u32 *version)
+static int version_get(const struct scmi_protocol_handle *ph, u32 *version)
{
int ret;
__le32 *rev_info;
struct scmi_xfer *t;
- ret = scmi_xfer_get_init(handle, PROTOCOL_VERSION, protocol, 0,
- sizeof(*version), &t);
+ ret = xfer_get_init(ph, PROTOCOL_VERSION, 0, sizeof(*version), &t);
if (ret)
return ret;
- ret = scmi_do_xfer(handle, t);
+ ret = do_xfer(ph, t);
if (!ret) {
rev_info = t->rx.buf;
*version = le32_to_cpu(*rev_info);
}
- scmi_xfer_put(handle, t);
+ xfer_put(ph, t);
return ret;
}
-void scmi_setup_protocol_implemented(const struct scmi_handle *handle,
- u8 *prot_imp)
+/**
+ * scmi_set_protocol_priv - Set protocol specific data at init time
+ *
+ * @ph: A reference to the protocol handle.
+ * @priv: The private data to set.
+ *
+ * Return: 0 on Success
+ */
+static int scmi_set_protocol_priv(const struct scmi_protocol_handle *ph,
+ void *priv)
{
+ struct scmi_protocol_instance *pi = ph_to_pi(ph);
+
+ pi->priv = priv;
+
+ return 0;
+}
+
+/**
+ * scmi_get_protocol_priv - Set protocol specific data at init time
+ *
+ * @ph: A reference to the protocol handle.
+ *
+ * Return: Protocol private data if any was set.
+ */
+static void *scmi_get_protocol_priv(const struct scmi_protocol_handle *ph)
+{
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
+
+ return pi->priv;
+}
+
+static const struct scmi_xfer_ops xfer_ops = {
+ .version_get = version_get,
+ .xfer_get_init = xfer_get_init,
+ .reset_rx_to_maxsz = reset_rx_to_maxsz,
+ .do_xfer = do_xfer,
+ .do_xfer_with_response = do_xfer_with_response,
+ .xfer_put = xfer_put,
+};
+
+/**
+ * scmi_revision_area_get - Retrieve version memory area.
+ *
+ * @ph: A reference to the protocol handle.
+ *
+ * A helper to grab the version memory area reference during SCMI Base protocol
+ * initialization.
+ *
+ * Return: A reference to the version memory area associated to the SCMI
+ * instance underlying this protocol handle.
+ */
+struct scmi_revision_info *
+scmi_revision_area_get(const struct scmi_protocol_handle *ph)
+{
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
+
+ return pi->handle->version;
+}
+
+/**
+ * scmi_alloc_init_protocol_instance - Allocate and initialize a protocol
+ * instance descriptor.
+ * @info: The reference to the related SCMI instance.
+ * @proto: The protocol descriptor.
+ *
+ * Allocate a new protocol instance descriptor, using the provided @proto
+ * description, against the specified SCMI instance @info, and initialize it;
+ * all resources management is handled via a dedicated per-protocol devres
+ * group.
+ *
+ * Context: Assumes to be called with @protocols_mtx already acquired.
+ * Return: A reference to a freshly allocated and initialized protocol instance
+ * or ERR_PTR on failure. On failure the @proto reference is at first
+ * put using @scmi_protocol_put() before releasing all the devres group.
+ */
+static struct scmi_protocol_instance *
+scmi_alloc_init_protocol_instance(struct scmi_info *info,
+ const struct scmi_protocol *proto)
+{
+ int ret = -ENOMEM;
+ void *gid;
+ struct scmi_protocol_instance *pi;
+ const struct scmi_handle *handle = &info->handle;
+
+ /* Protocol specific devres group */
+ gid = devres_open_group(handle->dev, NULL, GFP_KERNEL);
+ if (!gid) {
+ scmi_protocol_put(proto->id);
+ goto out;
+ }
+
+ pi = devm_kzalloc(handle->dev, sizeof(*pi), GFP_KERNEL);
+ if (!pi)
+ goto clean;
+
+ pi->gid = gid;
+ pi->proto = proto;
+ pi->handle = handle;
+ pi->ph.dev = handle->dev;
+ pi->ph.xops = &xfer_ops;
+ pi->ph.set_priv = scmi_set_protocol_priv;
+ pi->ph.get_priv = scmi_get_protocol_priv;
+ refcount_set(&pi->users, 1);
+ /* proto->init is assured NON NULL by scmi_protocol_register */
+ ret = pi->proto->instance_init(&pi->ph);
+ if (ret)
+ goto clean;
+
+ ret = idr_alloc(&info->protocols, pi, proto->id, proto->id + 1,
+ GFP_KERNEL);
+ if (ret != proto->id)
+ goto clean;
+
+ /*
+ * Warn but ignore events registration errors since we do not want
+ * to skip whole protocols if their notifications are messed up.
+ */
+ if (pi->proto->events) {
+ ret = scmi_register_protocol_events(handle, pi->proto->id,
+ &pi->ph,
+ pi->proto->events);
+ if (ret)
+ dev_warn(handle->dev,
+ "Protocol:%X - Events Registration Failed - err:%d\n",
+ pi->proto->id, ret);
+ }
+
+ devres_close_group(handle->dev, pi->gid);
+ dev_dbg(handle->dev, "Initialized protocol: 0x%X\n", pi->proto->id);
+
+ return pi;
+
+clean:
+ /* Take care to put the protocol module's owner before releasing all */
+ scmi_protocol_put(proto->id);
+ devres_release_group(handle->dev, gid);
+out:
+ return ERR_PTR(ret);
+}
+
+/**
+ * scmi_get_protocol_instance - Protocol initialization helper.
+ * @handle: A reference to the SCMI platform instance.
+ * @protocol_id: The protocol being requested.
+ *
+ * In case the required protocol has never been requested before for this
+ * instance, allocate and initialize all the needed structures while handling
+ * resource allocation with a dedicated per-protocol devres subgroup.
+ *
+ * Return: A reference to an initialized protocol instance or error on failure:
+ * in particular returns -EPROBE_DEFER when the desired protocol could
+ * NOT be found.
+ */
+static struct scmi_protocol_instance * __must_check
+scmi_get_protocol_instance(const struct scmi_handle *handle, u8 protocol_id)
+{
+ struct scmi_protocol_instance *pi;
struct scmi_info *info = handle_to_scmi_info(handle);
+ mutex_lock(&info->protocols_mtx);
+ pi = idr_find(&info->protocols, protocol_id);
+
+ if (pi) {
+ refcount_inc(&pi->users);
+ } else {
+ const struct scmi_protocol *proto;
+
+ /* Fails if protocol not registered on bus */
+ proto = scmi_protocol_get(protocol_id);
+ if (proto)
+ pi = scmi_alloc_init_protocol_instance(info, proto);
+ else
+ pi = ERR_PTR(-EPROBE_DEFER);
+ }
+ mutex_unlock(&info->protocols_mtx);
+
+ return pi;
+}
+
+/**
+ * scmi_protocol_acquire - Protocol acquire
+ * @handle: A reference to the SCMI platform instance.
+ * @protocol_id: The protocol being requested.
+ *
+ * Register a new user for the requested protocol on the specified SCMI
+ * platform instance, possibly triggering its initialization on first user.
+ *
+ * Return: 0 if protocol was acquired successfully.
+ */
+int scmi_protocol_acquire(const struct scmi_handle *handle, u8 protocol_id)
+{
+ return PTR_ERR_OR_ZERO(scmi_get_protocol_instance(handle, protocol_id));
+}
+
+/**
+ * scmi_protocol_release - Protocol de-initialization helper.
+ * @handle: A reference to the SCMI platform instance.
+ * @protocol_id: The protocol being requested.
+ *
+ * Remove one user for the specified protocol and triggers de-initialization
+ * and resources de-allocation once the last user has gone.
+ */
+void scmi_protocol_release(const struct scmi_handle *handle, u8 protocol_id)
+{
+ struct scmi_info *info = handle_to_scmi_info(handle);
+ struct scmi_protocol_instance *pi;
+
+ mutex_lock(&info->protocols_mtx);
+ pi = idr_find(&info->protocols, protocol_id);
+ if (WARN_ON(!pi))
+ goto out;
+
+ if (refcount_dec_and_test(&pi->users)) {
+ void *gid = pi->gid;
+
+ if (pi->proto->events)
+ scmi_deregister_protocol_events(handle, protocol_id);
+
+ if (pi->proto->instance_deinit)
+ pi->proto->instance_deinit(&pi->ph);
+
+ idr_remove(&info->protocols, protocol_id);
+
+ scmi_protocol_put(protocol_id);
+
+ devres_release_group(handle->dev, gid);
+ dev_dbg(handle->dev, "De-Initialized protocol: 0x%X\n",
+ protocol_id);
+ }
+
+out:
+ mutex_unlock(&info->protocols_mtx);
+}
+
+void scmi_setup_protocol_implemented(const struct scmi_protocol_handle *ph,
+ u8 *prot_imp)
+{
+ const struct scmi_protocol_instance *pi = ph_to_pi(ph);
+ struct scmi_info *info = handle_to_scmi_info(pi->handle);
+
info->protocols_imp = prot_imp;
}
@@ -542,6 +858,102 @@ scmi_is_protocol_implemented(const struct scmi_handle *handle, u8 prot_id)
return false;
}
+struct scmi_protocol_devres {
+ const struct scmi_handle *handle;
+ u8 protocol_id;
+};
+
+static void scmi_devm_release_protocol(struct device *dev, void *res)
+{
+ struct scmi_protocol_devres *dres = res;
+
+ scmi_protocol_release(dres->handle, dres->protocol_id);
+}
+
+/**
+ * scmi_devm_protocol_get - Devres managed get protocol operations and handle
+ * @sdev: A reference to an scmi_device whose embedded struct device is to
+ * be used for devres accounting.
+ * @protocol_id: The protocol being requested.
+ * @ph: A pointer reference used to pass back the associated protocol handle.
+ *
+ * Get hold of a protocol accounting for its usage, eventually triggering its
+ * initialization, and returning the protocol specific operations and related
+ * protocol handle which will be used as first argument in most of the
+ * protocols operations methods.
+ * Being a devres based managed method, protocol hold will be automatically
+ * released, and possibly de-initialized on last user, once the SCMI driver
+ * owning the scmi_device is unbound from it.
+ *
+ * Return: A reference to the requested protocol operations or error.
+ * Must be checked for errors by caller.
+ */
+static const void __must_check *
+scmi_devm_protocol_get(struct scmi_device *sdev, u8 protocol_id,
+ struct scmi_protocol_handle **ph)
+{
+ struct scmi_protocol_instance *pi;
+ struct scmi_protocol_devres *dres;
+ struct scmi_handle *handle = sdev->handle;
+
+ if (!ph)
+ return ERR_PTR(-EINVAL);
+
+ dres = devres_alloc(scmi_devm_release_protocol,
+ sizeof(*dres), GFP_KERNEL);
+ if (!dres)
+ return ERR_PTR(-ENOMEM);
+
+ pi = scmi_get_protocol_instance(handle, protocol_id);
+ if (IS_ERR(pi)) {
+ devres_free(dres);
+ return pi;
+ }
+
+ dres->handle = handle;
+ dres->protocol_id = protocol_id;
+ devres_add(&sdev->dev, dres);
+
+ *ph = &pi->ph;
+
+ return pi->proto->ops;
+}
+
+static int scmi_devm_protocol_match(struct device *dev, void *res, void *data)
+{
+ struct scmi_protocol_devres *dres = res;
+
+ if (WARN_ON(!dres || !data))
+ return 0;
+
+ return dres->protocol_id == *((u8 *)data);
+}
+
+/**
+ * scmi_devm_protocol_put - Devres managed put protocol operations and handle
+ * @sdev: A reference to an scmi_device whose embedded struct device is to
+ * be used for devres accounting.
+ * @protocol_id: The protocol being requested.
+ *
+ * Explicitly release a protocol hold previously obtained calling the above
+ * @scmi_devm_protocol_get.
+ */
+static void scmi_devm_protocol_put(struct scmi_device *sdev, u8 protocol_id)
+{
+ int ret;
+
+ ret = devres_release(&sdev->dev, scmi_devm_release_protocol,
+ scmi_devm_protocol_match, &protocol_id);
+ WARN_ON(ret);
+}
+
+static inline
+struct scmi_handle *scmi_handle_get_from_info_unlocked(struct scmi_info *info)
+{
+ info->users++;
+ return &info->handle;
+}
+
/**
* scmi_handle_get() - Get the SCMI handle for a device
*
@@ -563,8 +975,7 @@ struct scmi_handle *scmi_handle_get(struct device *dev)
list_for_each(p, &scmi_list) {
info = list_entry(p, struct scmi_info, node);
if (dev->parent == info->dev) {
- handle = &info->handle;
- info->users++;
+ handle = scmi_handle_get_from_info_unlocked(info);
break;
}
}
@@ -707,63 +1118,268 @@ scmi_txrx_setup(struct scmi_info *info, struct device *dev, int prot_id)
return ret;
}
-static inline void
-scmi_create_protocol_device(struct device_node *np, struct scmi_info *info,
- int prot_id, const char *name)
+/**
+ * scmi_get_protocol_device - Helper to get/create an SCMI device.
+ *
+ * @np: A device node representing a valid active protocols for the referred
+ * SCMI instance.
+ * @info: The referred SCMI instance for which we are getting/creating this
+ * device.
+ * @prot_id: The protocol ID.
+ * @name: The device name.
+ *
+ * Referring to the specific SCMI instance identified by @info, this helper
+ * takes care to return a properly initialized device matching the requested
+ * @proto_id and @name: if device was still not existent it is created as a
+ * child of the specified SCMI instance @info and its transport properly
+ * initialized as usual.
+ */
+static inline struct scmi_device *
+scmi_get_protocol_device(struct device_node *np, struct scmi_info *info,
+ int prot_id, const char *name)
{
struct scmi_device *sdev;
+ /* Already created for this parent SCMI instance ? */
+ sdev = scmi_child_dev_find(info->dev, prot_id, name);
+ if (sdev)
+ return sdev;
+
+ pr_debug("Creating SCMI device (%s) for protocol %x\n", name, prot_id);
+
sdev = scmi_device_create(np, info->dev, prot_id, name);
if (!sdev) {
dev_err(info->dev, "failed to create %d protocol device\n",
prot_id);
- return;
+ return NULL;
}
if (scmi_txrx_setup(info, &sdev->dev, prot_id)) {
dev_err(&sdev->dev, "failed to setup transport\n");
scmi_device_destroy(sdev);
- return;
+ return NULL;
}
+ return sdev;
+}
+
+static inline void
+scmi_create_protocol_device(struct device_node *np, struct scmi_info *info,
+ int prot_id, const char *name)
+{
+ struct scmi_device *sdev;
+
+ sdev = scmi_get_protocol_device(np, info, prot_id, name);
+ if (!sdev)
+ return;
+
/* setup handle now as the transport is ready */
scmi_set_handle(sdev);
}
-#define MAX_SCMI_DEV_PER_PROTOCOL 2
-struct scmi_prot_devnames {
- int protocol_id;
- char *names[MAX_SCMI_DEV_PER_PROTOCOL];
-};
+/**
+ * scmi_create_protocol_devices - Create devices for all pending requests for
+ * this SCMI instance.
+ *
+ * @np: The device node describing the protocol
+ * @info: The SCMI instance descriptor
+ * @prot_id: The protocol ID
+ *
+ * All devices previously requested for this instance (if any) are found and
+ * created by scanning the proper @&scmi_requested_devices entry.
+ */
+static void scmi_create_protocol_devices(struct device_node *np,
+ struct scmi_info *info, int prot_id)
+{
+ struct list_head *phead;
-static struct scmi_prot_devnames devnames[] = {
- { SCMI_PROTOCOL_POWER, { "genpd" },},
- { SCMI_PROTOCOL_SYSTEM, { "syspower" },},
- { SCMI_PROTOCOL_PERF, { "cpufreq" },},
- { SCMI_PROTOCOL_CLOCK, { "clocks" },},
- { SCMI_PROTOCOL_SENSOR, { "hwmon" },},
- { SCMI_PROTOCOL_RESET, { "reset" },},
- { SCMI_PROTOCOL_VOLTAGE, { "regulator" },},
-};
+ mutex_lock(&scmi_requested_devices_mtx);
+ phead = idr_find(&scmi_requested_devices, prot_id);
+ if (phead) {
+ struct scmi_requested_dev *rdev;
-static inline void
-scmi_create_protocol_devices(struct device_node *np, struct scmi_info *info,
- int prot_id)
+ list_for_each_entry(rdev, phead, node)
+ scmi_create_protocol_device(np, info, prot_id,
+ rdev->id_table->name);
+ }
+ mutex_unlock(&scmi_requested_devices_mtx);
+}
+
+/**
+ * scmi_protocol_device_request - Helper to request a device
+ *
+ * @id_table: A protocol/name pair descriptor for the device to be created.
+ *
+ * This helper let an SCMI driver request specific devices identified by the
+ * @id_table to be created for each active SCMI instance.
+ *
+ * The requested device name MUST NOT be already existent for any protocol;
+ * at first the freshly requested @id_table is annotated in the IDR table
+ * @scmi_requested_devices, then a matching device is created for each already
+ * active SCMI instance. (if any)
+ *
+ * This way the requested device is created straight-away for all the already
+ * initialized(probed) SCMI instances (handles) and it remains also annotated
+ * as pending creation if the requesting SCMI driver was loaded before some
+ * SCMI instance and related transports were available: when such late instance
+ * is probed, its probe will take care to scan the list of pending requested
+ * devices and create those on its own (see @scmi_create_protocol_devices and
+ * its enclosing loop)
+ *
+ * Return: 0 on Success
+ */
+int scmi_protocol_device_request(const struct scmi_device_id *id_table)
{
- int loop, cnt;
+ int ret = 0;
+ unsigned int id = 0;
+ struct list_head *head, *phead = NULL;
+ struct scmi_requested_dev *rdev;
+ struct scmi_info *info;
- for (loop = 0; loop < ARRAY_SIZE(devnames); loop++) {
- if (devnames[loop].protocol_id != prot_id)
- continue;
+ pr_debug("Requesting SCMI device (%s) for protocol %x\n",
+ id_table->name, id_table->protocol_id);
- for (cnt = 0; cnt < ARRAY_SIZE(devnames[loop].names); cnt++) {
- const char *name = devnames[loop].names[cnt];
+ /*
+ * Search for the matching protocol rdev list and then search
+ * of any existent equally named device...fails if any duplicate found.
+ */
+ mutex_lock(&scmi_requested_devices_mtx);
+ idr_for_each_entry(&scmi_requested_devices, head, id) {
+ if (!phead) {
+ /* A list found registered in the IDR is never empty */
+ rdev = list_first_entry(head, struct scmi_requested_dev,
+ node);
+ if (rdev->id_table->protocol_id ==
+ id_table->protocol_id)
+ phead = head;
+ }
+ list_for_each_entry(rdev, head, node) {
+ if (!strcmp(rdev->id_table->name, id_table->name)) {
+ pr_err("Ignoring duplicate request [%d] %s\n",
+ rdev->id_table->protocol_id,
+ rdev->id_table->name);
+ ret = -EINVAL;
+ goto out;
+ }
+ }
+ }
+
+ /*
+ * No duplicate found for requested id_table, so let's create a new
+ * requested device entry for this new valid request.
+ */
+ rdev = kzalloc(sizeof(*rdev), GFP_KERNEL);
+ if (!rdev) {
+ ret = -ENOMEM;
+ goto out;
+ }
+ rdev->id_table = id_table;
- if (name)
- scmi_create_protocol_device(np, info, prot_id,
- name);
+ /*
+ * Append the new requested device table descriptor to the head of the
+ * related protocol list, eventually creating such head if not already
+ * there.
+ */
+ if (!phead) {
+ phead = kzalloc(sizeof(*phead), GFP_KERNEL);
+ if (!phead) {
+ kfree(rdev);
+ ret = -ENOMEM;
+ goto out;
+ }
+ INIT_LIST_HEAD(phead);
+
+ ret = idr_alloc(&scmi_requested_devices, (void *)phead,
+ id_table->protocol_id,
+ id_table->protocol_id + 1, GFP_KERNEL);
+ if (ret != id_table->protocol_id) {
+ pr_err("Failed to save SCMI device - ret:%d\n", ret);
+ kfree(rdev);
+ kfree(phead);
+ ret = -EINVAL;
+ goto out;
}
+ ret = 0;
}
+ list_add(&rdev->node, phead);
+
+ /*
+ * Now effectively create and initialize the requested device for every
+ * already initialized SCMI instance which has registered the requested
+ * protocol as a valid active one: i.e. defined in DT and supported by
+ * current platform FW.
+ */
+ mutex_lock(&scmi_list_mutex);
+ list_for_each_entry(info, &scmi_list, node) {
+ struct device_node *child;
+
+ child = idr_find(&info->active_protocols,
+ id_table->protocol_id);
+ if (child) {
+ struct scmi_device *sdev;
+
+ sdev = scmi_get_protocol_device(child, info,
+ id_table->protocol_id,
+ id_table->name);
+ /* Set handle if not already set: device existed */
+ if (sdev && !sdev->handle)
+ sdev->handle =
+ scmi_handle_get_from_info_unlocked(info);
+ } else {
+ dev_err(info->dev,
+ "Failed. SCMI protocol %d not active.\n",
+ id_table->protocol_id);
+ }
+ }
+ mutex_unlock(&scmi_list_mutex);
+
+out:
+ mutex_unlock(&scmi_requested_devices_mtx);
+
+ return ret;
+}
+
+/**
+ * scmi_protocol_device_unrequest - Helper to unrequest a device
+ *
+ * @id_table: A protocol/name pair descriptor for the device to be unrequested.
+ *
+ * An helper to let an SCMI driver release its request about devices; note that
+ * devices are created and initialized once the first SCMI driver request them
+ * but they destroyed only on SCMI core unloading/unbinding.
+ *
+ * The current SCMI transport layer uses such devices as internal references and
+ * as such they could be shared as same transport between multiple drivers so
+ * that cannot be safely destroyed till the whole SCMI stack is removed.
+ * (unless adding further burden of refcounting.)
+ */
+void scmi_protocol_device_unrequest(const struct scmi_device_id *id_table)
+{
+ struct list_head *phead;
+
+ pr_debug("Unrequesting SCMI device (%s) for protocol %x\n",
+ id_table->name, id_table->protocol_id);
+
+ mutex_lock(&scmi_requested_devices_mtx);
+ phead = idr_find(&scmi_requested_devices, id_table->protocol_id);
+ if (phead) {
+ struct scmi_requested_dev *victim, *tmp;
+
+ list_for_each_entry_safe(victim, tmp, phead, node) {
+ if (!strcmp(victim->id_table->name, id_table->name)) {
+ list_del(&victim->node);
+ kfree(victim);
+ break;
+ }
+ }
+
+ if (list_empty(phead)) {
+ idr_remove(&scmi_requested_devices,
+ id_table->protocol_id);
+ kfree(phead);
+ }
+ }
+ mutex_unlock(&scmi_requested_devices_mtx);
}
static int scmi_probe(struct platform_device *pdev)
@@ -786,6 +1402,9 @@ static int scmi_probe(struct platform_device *pdev)
info->dev = dev;
info->desc = desc;
INIT_LIST_HEAD(&info->node);
+ idr_init(&info->protocols);
+ mutex_init(&info->protocols_mtx);
+ idr_init(&info->active_protocols);
platform_set_drvdata(pdev, info);
idr_init(&info->tx_idr);
@@ -794,6 +1413,8 @@ static int scmi_probe(struct platform_device *pdev)
handle = &info->handle;
handle->dev = info->dev;
handle->version = &info->version;
+ handle->devm_protocol_get = scmi_devm_protocol_get;
+ handle->devm_protocol_put = scmi_devm_protocol_put;
ret = scmi_txrx_setup(info, dev, SCMI_PROTOCOL_BASE);
if (ret)
@@ -806,9 +1427,14 @@ static int scmi_probe(struct platform_device *pdev)
if (scmi_notification_init(handle))
dev_err(dev, "SCMI Notifications NOT available.\n");
- ret = scmi_base_protocol_init(handle);
+ /*
+ * Trigger SCMI Base protocol initialization.
+ * It's mandatory and won't be ever released/deinit until the
+ * SCMI stack is shutdown/unloaded as a whole.
+ */
+ ret = scmi_protocol_acquire(handle, SCMI_PROTOCOL_BASE);
if (ret) {
- dev_err(dev, "unable to communicate with SCMI(%d)\n", ret);
+ dev_err(dev, "unable to communicate with SCMI\n");
return ret;
}
@@ -831,6 +1457,19 @@ static int scmi_probe(struct platform_device *pdev)
continue;
}
+ /*
+ * Save this valid DT protocol descriptor amongst
+ * @active_protocols for this SCMI instance/
+ */
+ ret = idr_alloc(&info->active_protocols, child,
+ prot_id, prot_id + 1, GFP_KERNEL);
+ if (ret != prot_id) {
+ dev_err(dev, "SCMI protocol %d already activated. Skip\n",
+ prot_id);
+ continue;
+ }
+
+ of_node_get(child);
scmi_create_protocol_devices(child, info, prot_id);
}
@@ -844,9 +1483,10 @@ void scmi_free_channel(struct scmi_chan_info *cinfo, struct idr *idr, int id)
static int scmi_remove(struct platform_device *pdev)
{
- int ret = 0;
+ int ret = 0, id;
struct scmi_info *info = platform_get_drvdata(pdev);
struct idr *idr = &info->tx_idr;
+ struct device_node *child;
mutex_lock(&scmi_list_mutex);
if (info->users)
@@ -860,6 +1500,14 @@ static int scmi_remove(struct platform_device *pdev)
scmi_notification_exit(&info->handle);
+ mutex_lock(&info->protocols_mtx);
+ idr_destroy(&info->protocols);
+ mutex_unlock(&info->protocols_mtx);
+
+ idr_for_each_entry(&info->active_protocols, child, id)
+ of_node_put(child);
+ idr_destroy(&info->active_protocols);
+
/* Safe to free channels since no more users */
ret = idr_for_each(idr, info->desc->ops->chan_free, idr);
idr_destroy(&info->tx_idr);
@@ -942,6 +1590,8 @@ static int __init scmi_driver_init(void)
{
scmi_bus_init();
+ scmi_base_register();
+
scmi_clock_register();
scmi_perf_register();
scmi_power_register();
@@ -956,7 +1606,7 @@ subsys_initcall(scmi_driver_init);
static void __exit scmi_driver_exit(void)
{
- scmi_bus_exit();
+ scmi_base_unregister();
scmi_clock_unregister();
scmi_perf_unregister();
@@ -966,6 +1616,8 @@ static void __exit scmi_driver_exit(void)
scmi_voltage_unregister();
scmi_system_unregister();
+ scmi_bus_exit();
+
platform_driver_unregister(&scmi_driver);
}
module_exit(scmi_driver_exit);
diff --git a/drivers/firmware/arm_scmi/notify.c b/drivers/firmware/arm_scmi/notify.c
index 66196b293b6c..d860bebd984a 100644
--- a/drivers/firmware/arm_scmi/notify.c
+++ b/drivers/firmware/arm_scmi/notify.c
@@ -2,7 +2,7 @@
/*
* System Control and Management Interface (SCMI) Notification support
*
- * Copyright (C) 2020 ARM Ltd.
+ * Copyright (C) 2020-2021 ARM Ltd.
*/
/**
* DOC: Theory of operation
@@ -91,6 +91,7 @@
#include <linux/types.h>
#include <linux/workqueue.h>
+#include "common.h"
#include "notify.h"
#define SCMI_MAX_PROTO 256
@@ -177,7 +178,7 @@
#define REVT_NOTIFY_SET_STATUS(revt, eid, sid, state) \
({ \
typeof(revt) r = revt; \
- r->proto->ops->set_notify_enabled(r->proto->ni->handle, \
+ r->proto->ops->set_notify_enabled(r->proto->ph, \
(eid), (sid), (state)); \
})
@@ -190,7 +191,7 @@
#define REVT_FILL_REPORT(revt, ...) \
({ \
typeof(revt) r = revt; \
- r->proto->ops->fill_custom_report(r->proto->ni->handle, \
+ r->proto->ops->fill_custom_report(r->proto->ph, \
__VA_ARGS__); \
})
@@ -278,6 +279,7 @@ struct scmi_registered_event;
* events' descriptors, whose fixed-size is determined at
* compile time.
* @registered_mtx: A mutex to protect @registered_events_handlers
+ * @ph: SCMI protocol handle reference
* @registered_events_handlers: An hashtable containing all events' handlers
* descriptors registered for this protocol
*
@@ -302,6 +304,7 @@ struct scmi_registered_events_desc {
struct scmi_registered_event **registered_events;
/* mutex to protect registered_events_handlers */
struct mutex registered_mtx;
+ const struct scmi_protocol_handle *ph;
DECLARE_HASHTABLE(registered_events_handlers, SCMI_REGISTERED_HASH_SZ);
};
@@ -368,7 +371,7 @@ static struct scmi_event_handler *
scmi_get_active_handler(struct scmi_notify_instance *ni, u32 evt_key);
static void scmi_put_active_handler(struct scmi_notify_instance *ni,
struct scmi_event_handler *hndl);
-static void scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
+static bool scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
struct scmi_event_handler *hndl);
/**
@@ -579,11 +582,9 @@ int scmi_notify(const struct scmi_handle *handle, u8 proto_id, u8 evt_id,
struct scmi_event_header eh;
struct scmi_notify_instance *ni;
- /* Ensure notify_priv is updated */
- smp_rmb();
- if (!handle->notify_priv)
+ ni = scmi_notification_instance_data_get(handle);
+ if (!ni)
return 0;
- ni = handle->notify_priv;
r_evt = SCMI_GET_REVT(ni, proto_id, evt_id);
if (!r_evt)
@@ -732,14 +733,10 @@ scmi_allocate_registered_events_desc(struct scmi_notify_instance *ni,
/**
* scmi_register_protocol_events() - Register Protocol Events with the core
* @handle: The handle identifying the platform instance against which the
- * the protocol's events are registered
+ * protocol's events are registered
* @proto_id: Protocol ID
- * @queue_sz: Size in bytes of the associated queue to be allocated
- * @ops: Protocol specific event-related operations
- * @evt: Event descriptor array
- * @num_events: Number of events in @evt array
- * @num_sources: Number of possible sources for this protocol on this
- * platform.
+ * @ph: SCMI protocol handle.
+ * @ee: A structure describing the events supported by this protocol.
*
* Used by SCMI Protocols initialization code to register with the notification
* core the list of supported events and their descriptors: takes care to
@@ -748,60 +745,69 @@ scmi_allocate_registered_events_desc(struct scmi_notify_instance *ni,
*
* Return: 0 on Success
*/
-int scmi_register_protocol_events(const struct scmi_handle *handle,
- u8 proto_id, size_t queue_sz,
- const struct scmi_event_ops *ops,
- const struct scmi_event *evt, int num_events,
- int num_sources)
+int scmi_register_protocol_events(const struct scmi_handle *handle, u8 proto_id,
+ const struct scmi_protocol_handle *ph,
+ const struct scmi_protocol_events *ee)
{
int i;
+ unsigned int num_sources;
size_t payld_sz = 0;
struct scmi_registered_events_desc *pd;
struct scmi_notify_instance *ni;
+ const struct scmi_event *evt;
- if (!ops || !evt)
+ if (!ee || !ee->ops || !ee->evts || !ph ||
+ (!ee->num_sources && !ee->ops->get_num_sources))
return -EINVAL;
- /* Ensure notify_priv is updated */
- smp_rmb();
- if (!handle->notify_priv)
+ ni = scmi_notification_instance_data_get(handle);
+ if (!ni)
return -ENOMEM;
- ni = handle->notify_priv;
- /* Attach to the notification main devres group */
- if (!devres_open_group(ni->handle->dev, ni->gid, GFP_KERNEL))
- return -ENOMEM;
+ /* num_sources cannot be <= 0 */
+ if (ee->num_sources) {
+ num_sources = ee->num_sources;
+ } else {
+ int nsrc = ee->ops->get_num_sources(ph);
- for (i = 0; i < num_events; i++)
+ if (nsrc <= 0)
+ return -EINVAL;
+ num_sources = nsrc;
+ }
+
+ evt = ee->evts;
+ for (i = 0; i < ee->num_events; i++)
payld_sz = max_t(size_t, payld_sz, evt[i].max_payld_sz);
payld_sz += sizeof(struct scmi_event_header);
- pd = scmi_allocate_registered_events_desc(ni, proto_id, queue_sz,
- payld_sz, num_events, ops);
+ pd = scmi_allocate_registered_events_desc(ni, proto_id, ee->queue_sz,
+ payld_sz, ee->num_events,
+ ee->ops);
if (IS_ERR(pd))
- goto err;
+ return PTR_ERR(pd);
- for (i = 0; i < num_events; i++, evt++) {
+ pd->ph = ph;
+ for (i = 0; i < ee->num_events; i++, evt++) {
struct scmi_registered_event *r_evt;
r_evt = devm_kzalloc(ni->handle->dev, sizeof(*r_evt),
GFP_KERNEL);
if (!r_evt)
- goto err;
+ return -ENOMEM;
r_evt->proto = pd;
r_evt->evt = evt;
r_evt->sources = devm_kcalloc(ni->handle->dev, num_sources,
sizeof(refcount_t), GFP_KERNEL);
if (!r_evt->sources)
- goto err;
+ return -ENOMEM;
r_evt->num_sources = num_sources;
mutex_init(&r_evt->sources_mtx);
r_evt->report = devm_kzalloc(ni->handle->dev,
evt->max_report_sz, GFP_KERNEL);
if (!r_evt->report)
- goto err;
+ return -ENOMEM;
pd->registered_events[i] = r_evt;
/* Ensure events are updated */
@@ -815,8 +821,6 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
/* Ensure protocols are updated */
smp_wmb();
- devres_close_group(ni->handle->dev, ni->gid);
-
/*
* Finalize any pending events' handler which could have been waiting
* for this protocol's events registration.
@@ -824,13 +828,33 @@ int scmi_register_protocol_events(const struct scmi_handle *handle,
schedule_work(&ni->init_work);
return 0;
+}
-err:
- dev_warn(handle->dev, "Proto:%X - Registration Failed !\n", proto_id);
- /* A failing protocol registration does not trigger full failure */
- devres_close_group(ni->handle->dev, ni->gid);
+/**
+ * scmi_deregister_protocol_events - Deregister protocol events with the core
+ * @handle: The handle identifying the platform instance against which the
+ * protocol's events are registered
+ * @proto_id: Protocol ID
+ */
+void scmi_deregister_protocol_events(const struct scmi_handle *handle,
+ u8 proto_id)
+{
+ struct scmi_notify_instance *ni;
+ struct scmi_registered_events_desc *pd;
- return -ENOMEM;
+ ni = scmi_notification_instance_data_get(handle);
+ if (!ni)
+ return;
+
+ pd = ni->registered_protocols[proto_id];
+ if (!pd)
+ return;
+
+ ni->registered_protocols[proto_id] = NULL;
+ /* Ensure protocols are updated */
+ smp_wmb();
+
+ cancel_work_sync(&pd->equeue.notify_work);
}
/**
@@ -900,9 +924,21 @@ static inline int scmi_bind_event_handler(struct scmi_notify_instance *ni,
if (!r_evt)
return -EINVAL;
- /* Remove from pending and insert into registered */
+ /*
+ * Remove from pending and insert into registered while getting hold
+ * of protocol instance.
+ */
hash_del(&hndl->hash);
+ /*
+ * Acquire protocols only for NON pending handlers, so as NOT to trigger
+ * protocol initialization when a notifier is registered against a still
+ * not registered protocol, since it would make little sense to force init
+ * protocols for which still no SCMI driver user exists: they wouldn't
+ * emit any event anyway till some SCMI driver starts using it.
+ */
+ scmi_protocol_acquire(ni->handle, KEY_XTRACT_PROTO_ID(hndl->key));
hndl->r_evt = r_evt;
+
mutex_lock(&r_evt->proto->registered_mtx);
hash_add(r_evt->proto->registered_events_handlers,
&hndl->hash, hndl->key);
@@ -1193,41 +1229,65 @@ static int scmi_disable_events(struct scmi_event_handler *hndl)
* * unregister and free the handler itself
*
* Context: Assumes all the proper locking has been managed by the caller.
+ *
+ * Return: True if handler was freed (users dropped to zero)
*/
-static void scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
+static bool scmi_put_handler_unlocked(struct scmi_notify_instance *ni,
struct scmi_event_handler *hndl)
{
+ bool freed = false;
+
if (refcount_dec_and_test(&hndl->users)) {
if (!IS_HNDL_PENDING(hndl))
scmi_disable_events(hndl);
scmi_free_event_handler(hndl);
+ freed = true;
}
+
+ return freed;
}
static void scmi_put_handler(struct scmi_notify_instance *ni,
struct scmi_event_handler *hndl)
{
+ bool freed;
+ u8 protocol_id;
struct scmi_registered_event *r_evt = hndl->r_evt;
mutex_lock(&ni->pending_mtx);
- if (r_evt)
+ if (r_evt) {
+ protocol_id = r_evt->proto->id;
mutex_lock(&r_evt->proto->registered_mtx);
+ }
- scmi_put_handler_unlocked(ni, hndl);
+ freed = scmi_put_handler_unlocked(ni, hndl);
- if (r_evt)
+ if (r_evt) {
mutex_unlock(&r_evt->proto->registered_mtx);
+ /*
+ * Only registered handler acquired protocol; must be here
+ * released only AFTER unlocking registered_mtx, since
+ * releasing a protocol can trigger its de-initialization
+ * (ie. including r_evt and registered_mtx)
+ */
+ if (freed)
+ scmi_protocol_release(ni->handle, protocol_id);
+ }
mutex_unlock(&ni->pending_mtx);
}
static void scmi_put_active_handler(struct scmi_notify_instance *ni,
struct scmi_event_handler *hndl)
{
+ bool freed;
struct scmi_registered_event *r_evt = hndl->r_evt;
+ u8 protocol_id = r_evt->proto->id;
mutex_lock(&r_evt->proto->registered_mtx);
- scmi_put_handler_unlocked(ni, hndl);
+ freed = scmi_put_handler_unlocked(ni, hndl);
mutex_unlock(&r_evt->proto->registered_mtx);
+ if (freed)
+ scmi_protocol_release(ni->handle, protocol_id);
}
/**
@@ -1247,7 +1307,7 @@ static int scmi_event_handler_enable_events(struct scmi_event_handler *hndl)
}
/**
- * scmi_register_notifier() - Register a notifier_block for an event
+ * scmi_notifier_register() - Register a notifier_block for an event
* @handle: The handle identifying the platform instance against which the
* callback is registered
* @proto_id: Protocol ID
@@ -1279,8 +1339,8 @@ static int scmi_event_handler_enable_events(struct scmi_event_handler *hndl)
*
* Return: 0 on Success
*/
-static int scmi_register_notifier(const struct scmi_handle *handle,
- u8 proto_id, u8 evt_id, u32 *src_id,
+static int scmi_notifier_register(const struct scmi_handle *handle,
+ u8 proto_id, u8 evt_id, const u32 *src_id,
struct notifier_block *nb)
{
int ret = 0;
@@ -1288,11 +1348,9 @@ static int scmi_register_notifier(const struct scmi_handle *handle,
struct scmi_event_handler *hndl;
struct scmi_notify_instance *ni;
- /* Ensure notify_priv is updated */
- smp_rmb();
- if (!handle->notify_priv)
+ ni = scmi_notification_instance_data_get(handle);
+ if (!ni)
return -ENODEV;
- ni = handle->notify_priv;
evt_key = MAKE_HASH_KEY(proto_id, evt_id,
src_id ? *src_id : SRC_ID_MASK);
@@ -1313,7 +1371,7 @@ static int scmi_register_notifier(const struct scmi_handle *handle,
}
/**
- * scmi_unregister_notifier() - Unregister a notifier_block for an event
+ * scmi_notifier_unregister() - Unregister a notifier_block for an event
* @handle: The handle identifying the platform instance against which the
* callback is unregistered
* @proto_id: Protocol ID
@@ -1328,19 +1386,17 @@ static int scmi_register_notifier(const struct scmi_handle *handle,
*
* Return: 0 on Success
*/
-static int scmi_unregister_notifier(const struct scmi_handle *handle,
- u8 proto_id, u8 evt_id, u32 *src_id,
+static int scmi_notifier_unregister(const struct scmi_handle *handle,
+ u8 proto_id, u8 evt_id, const u32 *src_id,
struct notifier_block *nb)
{
u32 evt_key;
struct scmi_event_handler *hndl;
struct scmi_notify_instance *ni;
- /* Ensure notify_priv is updated */
- smp_rmb();
- if (!handle->notify_priv)
+ ni = scmi_notification_instance_data_get(handle);
+ if (!ni)
return -ENODEV;
- ni = handle->notify_priv;
evt_key = MAKE_HASH_KEY(proto_id, evt_id,
src_id ? *src_id : SRC_ID_MASK);
@@ -1356,7 +1412,7 @@ static int scmi_unregister_notifier(const struct scmi_handle *handle,
scmi_put_handler(ni, hndl);
/*
- * This balances the initial get issued in @scmi_register_notifier.
+ * This balances the initial get issued in @scmi_notifier_register.
* If this notifier_block happened to be the last known user callback
* for this event, the handler is here freed and the event's generation
* stopped.
@@ -1371,6 +1427,129 @@ static int scmi_unregister_notifier(const struct scmi_handle *handle,
return 0;
}
+struct scmi_notifier_devres {
+ const struct scmi_handle *handle;
+ u8 proto_id;
+ u8 evt_id;
+ u32 __src_id;
+ u32 *src_id;
+ struct notifier_block *nb;
+};
+
+static void scmi_devm_release_notifier(struct device *dev, void *res)
+{
+ struct scmi_notifier_devres *dres = res;
+
+ scmi_notifier_unregister(dres->handle, dres->proto_id, dres->evt_id,
+ dres->src_id, dres->nb);
+}
+
+/**
+ * scmi_devm_notifier_register() - Managed registration of a notifier_block
+ * for an event
+ * @sdev: A reference to an scmi_device whose embedded struct device is to
+ * be used for devres accounting.
+ * @proto_id: Protocol ID
+ * @evt_id: Event ID
+ * @src_id: Source ID, when NULL register for events coming form ALL possible
+ * sources
+ * @nb: A standard notifier block to register for the specified event
+ *
+ * Generic devres managed helper to register a notifier_block against a
+ * protocol event.
+ */
+static int scmi_devm_notifier_register(struct scmi_device *sdev,
+ u8 proto_id, u8 evt_id,
+ const u32 *src_id,
+ struct notifier_block *nb)
+{
+ int ret;
+ struct scmi_notifier_devres *dres;
+
+ dres = devres_alloc(scmi_devm_release_notifier,
+ sizeof(*dres), GFP_KERNEL);
+ if (!dres)
+ return -ENOMEM;
+
+ ret = scmi_notifier_register(sdev->handle, proto_id,
+ evt_id, src_id, nb);
+ if (ret) {
+ devres_free(dres);
+ return ret;
+ }
+
+ dres->handle = sdev->handle;
+ dres->proto_id = proto_id;
+ dres->evt_id = evt_id;
+ dres->nb = nb;
+ if (src_id) {
+ dres->__src_id = *src_id;
+ dres->src_id = &dres->__src_id;
+ } else {
+ dres->src_id = NULL;
+ }
+ devres_add(&sdev->dev, dres);
+
+ return ret;
+}
+
+static int scmi_devm_notifier_match(struct device *dev, void *res, void *data)
+{
+ struct scmi_notifier_devres *dres = res;
+ struct scmi_notifier_devres *xres = data;
+
+ if (WARN_ON(!dres || !xres))
+ return 0;
+
+ return dres->proto_id == xres->proto_id &&
+ dres->evt_id == xres->evt_id &&
+ dres->nb == xres->nb &&
+ ((!dres->src_id && !xres->src_id) ||
+ (dres->src_id && xres->src_id &&
+ dres->__src_id == xres->__src_id));
+}
+
+/**
+ * scmi_devm_notifier_unregister() - Managed un-registration of a
+ * notifier_block for an event
+ * @sdev: A reference to an scmi_device whose embedded struct device is to
+ * be used for devres accounting.
+ * @proto_id: Protocol ID
+ * @evt_id: Event ID
+ * @src_id: Source ID, when NULL register for events coming form ALL possible
+ * sources
+ * @nb: A standard notifier block to register for the specified event
+ *
+ * Generic devres managed helper to explicitly un-register a notifier_block
+ * against a protocol event, which was previously registered using the above
+ * @scmi_devm_notifier_register.
+ */
+static int scmi_devm_notifier_unregister(struct scmi_device *sdev,
+ u8 proto_id, u8 evt_id,
+ const u32 *src_id,
+ struct notifier_block *nb)
+{
+ int ret;
+ struct scmi_notifier_devres dres;
+
+ dres.handle = sdev->handle;
+ dres.proto_id = proto_id;
+ dres.evt_id = evt_id;
+ if (src_id) {
+ dres.__src_id = *src_id;
+ dres.src_id = &dres.__src_id;
+ } else {
+ dres.src_id = NULL;
+ }
+
+ ret = devres_release(&sdev->dev, scmi_devm_release_notifier,
+ scmi_devm_notifier_match, &dres);
+
+ WARN_ON(ret);
+
+ return ret;
+}
+
/**
* scmi_protocols_late_init() - Worker for late initialization
* @work: The work item to use associated to the proper SCMI instance
@@ -1428,8 +1607,10 @@ static void scmi_protocols_late_init(struct work_struct *work)
* directly from an scmi_driver to register its own notifiers.
*/
static const struct scmi_notify_ops notify_ops = {
- .register_event_notifier = scmi_register_notifier,
- .unregister_event_notifier = scmi_unregister_notifier,
+ .devm_event_notifier_register = scmi_devm_notifier_register,
+ .devm_event_notifier_unregister = scmi_devm_notifier_unregister,
+ .event_notifier_register = scmi_notifier_register,
+ .event_notifier_unregister = scmi_notifier_unregister,
};
/**
@@ -1490,8 +1671,8 @@ int scmi_notification_init(struct scmi_handle *handle)
INIT_WORK(&ni->init_work, scmi_protocols_late_init);
+ scmi_notification_instance_data_set(handle, ni);
handle->notify_ops = &notify_ops;
- handle->notify_priv = ni;
/* Ensure handle is up to date */
smp_wmb();
@@ -1503,7 +1684,7 @@ int scmi_notification_init(struct scmi_handle *handle)
err:
dev_warn(handle->dev, "Initialization Failed.\n");
- devres_release_group(handle->dev, NULL);
+ devres_release_group(handle->dev, gid);
return -ENOMEM;
}
@@ -1515,15 +1696,10 @@ void scmi_notification_exit(struct scmi_handle *handle)
{
struct scmi_notify_instance *ni;
- /* Ensure notify_priv is updated */
- smp_rmb();
- if (!handle->notify_priv)
+ ni = scmi_notification_instance_data_get(handle);
+ if (!ni)
return;
- ni = handle->notify_priv;
-
- handle->notify_priv = NULL;
- /* Ensure handle is up to date */
- smp_wmb();
+ scmi_notification_instance_data_set(handle, NULL);
/* Destroy while letting pending work complete */
destroy_workqueue(ni->notify_wq);
diff --git a/drivers/firmware/arm_scmi/notify.h b/drivers/firmware/arm_scmi/notify.h
index 3485f20fa70e..ce0324be6c71 100644
--- a/drivers/firmware/arm_scmi/notify.h
+++ b/drivers/firmware/arm_scmi/notify.h
@@ -4,7 +4,7 @@
* notification header file containing some definitions, structures
* and function prototypes related to SCMI Notification handling.
*
- * Copyright (C) 2020 ARM Ltd.
+ * Copyright (C) 2020-2021 ARM Ltd.
*/
#ifndef _SCMI_NOTIFY_H
#define _SCMI_NOTIFY_H
@@ -31,8 +31,12 @@ struct scmi_event {
size_t max_report_sz;
};
+struct scmi_protocol_handle;
+
/**
* struct scmi_event_ops - Protocol helpers called by the notification core.
+ * @get_num_sources: Returns the number of possible events' sources for this
+ * protocol
* @set_notify_enabled: Enable/disable the required evt_id/src_id notifications
* using the proper custom protocol commands.
* Return 0 on Success
@@ -46,22 +50,42 @@ struct scmi_event {
* process context.
*/
struct scmi_event_ops {
- int (*set_notify_enabled)(const struct scmi_handle *handle,
+ int (*get_num_sources)(const struct scmi_protocol_handle *ph);
+ int (*set_notify_enabled)(const struct scmi_protocol_handle *ph,
u8 evt_id, u32 src_id, bool enabled);
- void *(*fill_custom_report)(const struct scmi_handle *handle,
+ void *(*fill_custom_report)(const struct scmi_protocol_handle *ph,
u8 evt_id, ktime_t timestamp,
const void *payld, size_t payld_sz,
void *report, u32 *src_id);
};
+/**
+ * struct scmi_protocol_events - Per-protocol description of available events
+ * @queue_sz: Size in bytes of the per-protocol queue to use.
+ * @ops: Array of protocol-specific events operations.
+ * @evts: Array of supported protocol's events.
+ * @num_events: Number of supported protocol's events described in @evts.
+ * @num_sources: Number of protocol's sources, should be greater than 0; if not
+ * available at compile time, it will be provided at run-time via
+ * @get_num_sources.
+ */
+struct scmi_protocol_events {
+ size_t queue_sz;
+ const struct scmi_event_ops *ops;
+ const struct scmi_event *evts;
+ unsigned int num_events;
+ unsigned int num_sources;
+};
+
int scmi_notification_init(struct scmi_handle *handle);
void scmi_notification_exit(struct scmi_handle *handle);
-int scmi_register_protocol_events(const struct scmi_handle *handle,
- u8 proto_id, size_t queue_sz,
- const struct scmi_event_ops *ops,
- const struct scmi_event *evt, int num_events,
- int num_sources);
+struct scmi_protocol_handle;
+int scmi_register_protocol_events(const struct scmi_handle *handle, u8 proto_id,
+ const struct scmi_protocol_handle *ph,
+ const struct scmi_protocol_events *ee);
+void scmi_deregister_protocol_events(const struct scmi_handle *handle,
+ u8 proto_id);
int scmi_notify(const struct scmi_handle *handle, u8 proto_id, u8 evt_id,
const void *buf, size_t len, ktime_t ts);
diff --git a/drivers/firmware/arm_scmi/perf.c b/drivers/firmware/arm_scmi/perf.c
index e374b1125fca..f4cd5193b961 100644
--- a/drivers/firmware/arm_scmi/perf.c
+++ b/drivers/firmware/arm_scmi/perf.c
@@ -2,7 +2,7 @@
/*
* System Control and Management Interface (SCMI) Performance Protocol
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#define pr_fmt(fmt) "SCMI Notifications PERF - " fmt
@@ -11,6 +11,7 @@
#include <linux/of.h>
#include <linux/io.h>
#include <linux/io-64-nonatomic-hi-lo.h>
+#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/pm_opp.h>
#include <linux/scmi_protocol.h>
@@ -175,21 +176,21 @@ static enum scmi_performance_protocol_cmd evt_2_cmd[] = {
PERF_NOTIFY_LEVEL,
};
-static int scmi_perf_attributes_get(const struct scmi_handle *handle,
+static int scmi_perf_attributes_get(const struct scmi_protocol_handle *ph,
struct scmi_perf_info *pi)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_perf_attributes *attr;
- ret = scmi_xfer_get_init(handle, PROTOCOL_ATTRIBUTES,
- SCMI_PROTOCOL_PERF, 0, sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, PROTOCOL_ATTRIBUTES, 0,
+ sizeof(*attr), &t);
if (ret)
return ret;
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
u16 flags = le16_to_cpu(attr->flags);
@@ -200,28 +201,27 @@ static int scmi_perf_attributes_get(const struct scmi_handle *handle,
pi->stats_size = le32_to_cpu(attr->stats_size);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
static int
-scmi_perf_domain_attributes_get(const struct scmi_handle *handle, u32 domain,
- struct perf_dom_info *dom_info)
+scmi_perf_domain_attributes_get(const struct scmi_protocol_handle *ph,
+ u32 domain, struct perf_dom_info *dom_info)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_perf_domain_attributes *attr;
- ret = scmi_xfer_get_init(handle, PERF_DOMAIN_ATTRIBUTES,
- SCMI_PROTOCOL_PERF, sizeof(domain),
- sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, PERF_DOMAIN_ATTRIBUTES,
+ sizeof(domain), sizeof(*attr), &t);
if (ret)
return ret;
put_unaligned_le32(domain, t->tx.buf);
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
u32 flags = le32_to_cpu(attr->flags);
@@ -245,7 +245,7 @@ scmi_perf_domain_attributes_get(const struct scmi_handle *handle, u32 domain,
strlcpy(dom_info->name, attr->name, SCMI_MAX_STR_SIZE);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -257,7 +257,7 @@ static int opp_cmp_func(const void *opp1, const void *opp2)
}
static int
-scmi_perf_describe_levels_get(const struct scmi_handle *handle, u32 domain,
+scmi_perf_describe_levels_get(const struct scmi_protocol_handle *ph, u32 domain,
struct perf_dom_info *perf_dom)
{
int ret, cnt;
@@ -268,8 +268,8 @@ scmi_perf_describe_levels_get(const struct scmi_handle *handle, u32 domain,
struct scmi_msg_perf_describe_levels *dom_info;
struct scmi_msg_resp_perf_describe_levels *level_info;
- ret = scmi_xfer_get_init(handle, PERF_DESCRIBE_LEVELS,
- SCMI_PROTOCOL_PERF, sizeof(*dom_info), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, PERF_DESCRIBE_LEVELS,
+ sizeof(*dom_info), 0, &t);
if (ret)
return ret;
@@ -281,14 +281,14 @@ scmi_perf_describe_levels_get(const struct scmi_handle *handle, u32 domain,
/* Set the number of OPPs to be skipped/already read */
dom_info->level_index = cpu_to_le32(tot_opp_cnt);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (ret)
break;
num_returned = le16_to_cpu(level_info->num_returned);
num_remaining = le16_to_cpu(level_info->num_remaining);
if (tot_opp_cnt + num_returned > MAX_OPPS) {
- dev_err(handle->dev, "No. of OPPs exceeded MAX_OPPS");
+ dev_err(ph->dev, "No. of OPPs exceeded MAX_OPPS");
break;
}
@@ -299,13 +299,13 @@ scmi_perf_describe_levels_get(const struct scmi_handle *handle, u32 domain,
opp->trans_latency_us = le16_to_cpu
(level_info->opp[cnt].transition_latency_us);
- dev_dbg(handle->dev, "Level %d Power %d Latency %dus\n",
+ dev_dbg(ph->dev, "Level %d Power %d Latency %dus\n",
opp->perf, opp->power, opp->trans_latency_us);
}
tot_opp_cnt += num_returned;
- scmi_reset_rx_to_maxsz(handle, t);
+ ph->xops->reset_rx_to_maxsz(ph, t);
/*
* check for both returned and remaining to avoid infinite
* loop due to buggy firmware
@@ -313,7 +313,7 @@ scmi_perf_describe_levels_get(const struct scmi_handle *handle, u32 domain,
} while (num_returned && num_remaining);
perf_dom->opp_count = tot_opp_cnt;
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
sort(perf_dom->opp, tot_opp_cnt, sizeof(*opp), opp_cmp_func, NULL);
return ret;
@@ -353,15 +353,15 @@ static void scmi_perf_fc_ring_db(struct scmi_fc_db_info *db)
#endif
}
-static int scmi_perf_mb_limits_set(const struct scmi_handle *handle, u32 domain,
- u32 max_perf, u32 min_perf)
+static int scmi_perf_mb_limits_set(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 max_perf, u32 min_perf)
{
int ret;
struct scmi_xfer *t;
struct scmi_perf_set_limits *limits;
- ret = scmi_xfer_get_init(handle, PERF_LIMITS_SET, SCMI_PROTOCOL_PERF,
- sizeof(*limits), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, PERF_LIMITS_SET,
+ sizeof(*limits), 0, &t);
if (ret)
return ret;
@@ -370,16 +370,16 @@ static int scmi_perf_mb_limits_set(const struct scmi_handle *handle, u32 domain,
limits->max_level = cpu_to_le32(max_perf);
limits->min_level = cpu_to_le32(min_perf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_perf_limits_set(const struct scmi_handle *handle, u32 domain,
- u32 max_perf, u32 min_perf)
+static int scmi_perf_limits_set(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 max_perf, u32 min_perf)
{
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
struct perf_dom_info *dom = pi->dom_info + domain;
if (dom->fc_info && dom->fc_info->limit_set_addr) {
@@ -389,24 +389,24 @@ static int scmi_perf_limits_set(const struct scmi_handle *handle, u32 domain,
return 0;
}
- return scmi_perf_mb_limits_set(handle, domain, max_perf, min_perf);
+ return scmi_perf_mb_limits_set(ph, domain, max_perf, min_perf);
}
-static int scmi_perf_mb_limits_get(const struct scmi_handle *handle, u32 domain,
- u32 *max_perf, u32 *min_perf)
+static int scmi_perf_mb_limits_get(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 *max_perf, u32 *min_perf)
{
int ret;
struct scmi_xfer *t;
struct scmi_perf_get_limits *limits;
- ret = scmi_xfer_get_init(handle, PERF_LIMITS_GET, SCMI_PROTOCOL_PERF,
- sizeof(__le32), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, PERF_LIMITS_GET,
+ sizeof(__le32), 0, &t);
if (ret)
return ret;
put_unaligned_le32(domain, t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
limits = t->rx.buf;
@@ -414,14 +414,14 @@ static int scmi_perf_mb_limits_get(const struct scmi_handle *handle, u32 domain,
*min_perf = le32_to_cpu(limits->min_level);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_perf_limits_get(const struct scmi_handle *handle, u32 domain,
- u32 *max_perf, u32 *min_perf)
+static int scmi_perf_limits_get(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 *max_perf, u32 *min_perf)
{
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
struct perf_dom_info *dom = pi->dom_info + domain;
if (dom->fc_info && dom->fc_info->limit_get_addr) {
@@ -430,18 +430,17 @@ static int scmi_perf_limits_get(const struct scmi_handle *handle, u32 domain,
return 0;
}
- return scmi_perf_mb_limits_get(handle, domain, max_perf, min_perf);
+ return scmi_perf_mb_limits_get(ph, domain, max_perf, min_perf);
}
-static int scmi_perf_mb_level_set(const struct scmi_handle *handle, u32 domain,
- u32 level, bool poll)
+static int scmi_perf_mb_level_set(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 level, bool poll)
{
int ret;
struct scmi_xfer *t;
struct scmi_perf_set_level *lvl;
- ret = scmi_xfer_get_init(handle, PERF_LEVEL_SET, SCMI_PROTOCOL_PERF,
- sizeof(*lvl), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, PERF_LEVEL_SET, sizeof(*lvl), 0, &t);
if (ret)
return ret;
@@ -450,16 +449,16 @@ static int scmi_perf_mb_level_set(const struct scmi_handle *handle, u32 domain,
lvl->domain = cpu_to_le32(domain);
lvl->level = cpu_to_le32(level);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_perf_level_set(const struct scmi_handle *handle, u32 domain,
- u32 level, bool poll)
+static int scmi_perf_level_set(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 level, bool poll)
{
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
struct perf_dom_info *dom = pi->dom_info + domain;
if (dom->fc_info && dom->fc_info->level_set_addr) {
@@ -468,35 +467,35 @@ static int scmi_perf_level_set(const struct scmi_handle *handle, u32 domain,
return 0;
}
- return scmi_perf_mb_level_set(handle, domain, level, poll);
+ return scmi_perf_mb_level_set(ph, domain, level, poll);
}
-static int scmi_perf_mb_level_get(const struct scmi_handle *handle, u32 domain,
- u32 *level, bool poll)
+static int scmi_perf_mb_level_get(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 *level, bool poll)
{
int ret;
struct scmi_xfer *t;
- ret = scmi_xfer_get_init(handle, PERF_LEVEL_GET, SCMI_PROTOCOL_PERF,
- sizeof(u32), sizeof(u32), &t);
+ ret = ph->xops->xfer_get_init(ph, PERF_LEVEL_GET,
+ sizeof(u32), sizeof(u32), &t);
if (ret)
return ret;
t->hdr.poll_completion = poll;
put_unaligned_le32(domain, t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
*level = get_unaligned_le32(t->rx.buf);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_perf_level_get(const struct scmi_handle *handle, u32 domain,
- u32 *level, bool poll)
+static int scmi_perf_level_get(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 *level, bool poll)
{
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
struct perf_dom_info *dom = pi->dom_info + domain;
if (dom->fc_info && dom->fc_info->level_get_addr) {
@@ -504,10 +503,10 @@ static int scmi_perf_level_get(const struct scmi_handle *handle, u32 domain,
return 0;
}
- return scmi_perf_mb_level_get(handle, domain, level, poll);
+ return scmi_perf_mb_level_get(ph, domain, level, poll);
}
-static int scmi_perf_level_limits_notify(const struct scmi_handle *handle,
+static int scmi_perf_level_limits_notify(const struct scmi_protocol_handle *ph,
u32 domain, int message_id,
bool enable)
{
@@ -515,8 +514,7 @@ static int scmi_perf_level_limits_notify(const struct scmi_handle *handle,
struct scmi_xfer *t;
struct scmi_perf_notify_level_or_limits *notify;
- ret = scmi_xfer_get_init(handle, message_id, SCMI_PROTOCOL_PERF,
- sizeof(*notify), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, message_id, sizeof(*notify), 0, &t);
if (ret)
return ret;
@@ -524,9 +522,9 @@ static int scmi_perf_level_limits_notify(const struct scmi_handle *handle,
notify->domain = cpu_to_le32(domain);
notify->notify_enable = enable ? cpu_to_le32(BIT(0)) : 0;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -540,7 +538,7 @@ static bool scmi_perf_fc_size_is_valid(u32 msg, u32 size)
}
static void
-scmi_perf_domain_desc_fc(const struct scmi_handle *handle, u32 domain,
+scmi_perf_domain_desc_fc(const struct scmi_protocol_handle *ph, u32 domain,
u32 message_id, void __iomem **p_addr,
struct scmi_fc_db_info **p_db)
{
@@ -557,9 +555,8 @@ scmi_perf_domain_desc_fc(const struct scmi_handle *handle, u32 domain,
if (!p_addr)
return;
- ret = scmi_xfer_get_init(handle, PERF_DESCRIBE_FASTCHANNEL,
- SCMI_PROTOCOL_PERF,
- sizeof(*info), sizeof(*resp), &t);
+ ret = ph->xops->xfer_get_init(ph, PERF_DESCRIBE_FASTCHANNEL,
+ sizeof(*info), sizeof(*resp), &t);
if (ret)
return;
@@ -567,7 +564,7 @@ scmi_perf_domain_desc_fc(const struct scmi_handle *handle, u32 domain,
info->domain = cpu_to_le32(domain);
info->message_id = cpu_to_le32(message_id);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (ret)
goto err_xfer;
@@ -579,20 +576,20 @@ scmi_perf_domain_desc_fc(const struct scmi_handle *handle, u32 domain,
phys_addr = le32_to_cpu(resp->chan_addr_low);
phys_addr |= (u64)le32_to_cpu(resp->chan_addr_high) << 32;
- addr = devm_ioremap(handle->dev, phys_addr, size);
+ addr = devm_ioremap(ph->dev, phys_addr, size);
if (!addr)
goto err_xfer;
*p_addr = addr;
if (p_db && SUPPORTS_DOORBELL(flags)) {
- db = devm_kzalloc(handle->dev, sizeof(*db), GFP_KERNEL);
+ db = devm_kzalloc(ph->dev, sizeof(*db), GFP_KERNEL);
if (!db)
goto err_xfer;
size = 1 << DOORBELL_REG_WIDTH(flags);
phys_addr = le32_to_cpu(resp->db_addr_low);
phys_addr |= (u64)le32_to_cpu(resp->db_addr_high) << 32;
- addr = devm_ioremap(handle->dev, phys_addr, size);
+ addr = devm_ioremap(ph->dev, phys_addr, size);
if (!addr)
goto err_xfer;
@@ -605,25 +602,25 @@ scmi_perf_domain_desc_fc(const struct scmi_handle *handle, u32 domain,
*p_db = db;
}
err_xfer:
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
}
-static void scmi_perf_domain_init_fc(const struct scmi_handle *handle,
+static void scmi_perf_domain_init_fc(const struct scmi_protocol_handle *ph,
u32 domain, struct scmi_fc_info **p_fc)
{
struct scmi_fc_info *fc;
- fc = devm_kzalloc(handle->dev, sizeof(*fc), GFP_KERNEL);
+ fc = devm_kzalloc(ph->dev, sizeof(*fc), GFP_KERNEL);
if (!fc)
return;
- scmi_perf_domain_desc_fc(handle, domain, PERF_LEVEL_SET,
+ scmi_perf_domain_desc_fc(ph, domain, PERF_LEVEL_SET,
&fc->level_set_addr, &fc->level_set_db);
- scmi_perf_domain_desc_fc(handle, domain, PERF_LEVEL_GET,
+ scmi_perf_domain_desc_fc(ph, domain, PERF_LEVEL_GET,
&fc->level_get_addr, NULL);
- scmi_perf_domain_desc_fc(handle, domain, PERF_LIMITS_SET,
+ scmi_perf_domain_desc_fc(ph, domain, PERF_LIMITS_SET,
&fc->limit_set_addr, &fc->limit_set_db);
- scmi_perf_domain_desc_fc(handle, domain, PERF_LIMITS_GET,
+ scmi_perf_domain_desc_fc(ph, domain, PERF_LIMITS_GET,
&fc->limit_get_addr, NULL);
*p_fc = fc;
}
@@ -640,14 +637,14 @@ static int scmi_dev_domain_id(struct device *dev)
return clkspec.args[0];
}
-static int scmi_dvfs_device_opps_add(const struct scmi_handle *handle,
+static int scmi_dvfs_device_opps_add(const struct scmi_protocol_handle *ph,
struct device *dev)
{
int idx, ret, domain;
unsigned long freq;
struct scmi_opp *opp;
struct perf_dom_info *dom;
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
domain = scmi_dev_domain_id(dev);
if (domain < 0)
@@ -672,11 +669,12 @@ static int scmi_dvfs_device_opps_add(const struct scmi_handle *handle,
return 0;
}
-static int scmi_dvfs_transition_latency_get(const struct scmi_handle *handle,
- struct device *dev)
+static int
+scmi_dvfs_transition_latency_get(const struct scmi_protocol_handle *ph,
+ struct device *dev)
{
struct perf_dom_info *dom;
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
int domain = scmi_dev_domain_id(dev);
if (domain < 0)
@@ -687,35 +685,35 @@ static int scmi_dvfs_transition_latency_get(const struct scmi_handle *handle,
return dom->opp[dom->opp_count - 1].trans_latency_us * 1000;
}
-static int scmi_dvfs_freq_set(const struct scmi_handle *handle, u32 domain,
+static int scmi_dvfs_freq_set(const struct scmi_protocol_handle *ph, u32 domain,
unsigned long freq, bool poll)
{
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
struct perf_dom_info *dom = pi->dom_info + domain;
- return scmi_perf_level_set(handle, domain, freq / dom->mult_factor,
- poll);
+ return scmi_perf_level_set(ph, domain, freq / dom->mult_factor, poll);
}
-static int scmi_dvfs_freq_get(const struct scmi_handle *handle, u32 domain,
+static int scmi_dvfs_freq_get(const struct scmi_protocol_handle *ph, u32 domain,
unsigned long *freq, bool poll)
{
int ret;
u32 level;
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
struct perf_dom_info *dom = pi->dom_info + domain;
- ret = scmi_perf_level_get(handle, domain, &level, poll);
+ ret = scmi_perf_level_get(ph, domain, &level, poll);
if (!ret)
*freq = level * dom->mult_factor;
return ret;
}
-static int scmi_dvfs_est_power_get(const struct scmi_handle *handle, u32 domain,
- unsigned long *freq, unsigned long *power)
+static int scmi_dvfs_est_power_get(const struct scmi_protocol_handle *ph,
+ u32 domain, unsigned long *freq,
+ unsigned long *power)
{
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
struct perf_dom_info *dom;
unsigned long opp_freq;
int idx, ret = -EINVAL;
@@ -739,25 +737,25 @@ static int scmi_dvfs_est_power_get(const struct scmi_handle *handle, u32 domain,
return ret;
}
-static bool scmi_fast_switch_possible(const struct scmi_handle *handle,
+static bool scmi_fast_switch_possible(const struct scmi_protocol_handle *ph,
struct device *dev)
{
struct perf_dom_info *dom;
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
dom = pi->dom_info + scmi_dev_domain_id(dev);
return dom->fc_info && dom->fc_info->level_set_addr;
}
-static bool scmi_power_scale_mw_get(const struct scmi_handle *handle)
+static bool scmi_power_scale_mw_get(const struct scmi_protocol_handle *ph)
{
- struct scmi_perf_info *pi = handle->perf_priv;
+ struct scmi_perf_info *pi = ph->get_priv(ph);
return pi->power_scale_mw;
}
-static const struct scmi_perf_ops perf_ops = {
+static const struct scmi_perf_proto_ops perf_proto_ops = {
.limits_set = scmi_perf_limits_set,
.limits_get = scmi_perf_limits_get,
.level_set = scmi_perf_level_set,
@@ -772,7 +770,7 @@ static const struct scmi_perf_ops perf_ops = {
.power_scale_mw_get = scmi_power_scale_mw_get,
};
-static int scmi_perf_set_notify_enabled(const struct scmi_handle *handle,
+static int scmi_perf_set_notify_enabled(const struct scmi_protocol_handle *ph,
u8 evt_id, u32 src_id, bool enable)
{
int ret, cmd_id;
@@ -781,7 +779,7 @@ static int scmi_perf_set_notify_enabled(const struct scmi_handle *handle,
return -EINVAL;
cmd_id = evt_2_cmd[evt_id];
- ret = scmi_perf_level_limits_notify(handle, src_id, cmd_id, enable);
+ ret = scmi_perf_level_limits_notify(ph, src_id, cmd_id, enable);
if (ret)
pr_debug("FAIL_ENABLED - evt[%X] dom[%d] - ret:%d\n",
evt_id, src_id, ret);
@@ -789,7 +787,7 @@ static int scmi_perf_set_notify_enabled(const struct scmi_handle *handle,
return ret;
}
-static void *scmi_perf_fill_custom_report(const struct scmi_handle *handle,
+static void *scmi_perf_fill_custom_report(const struct scmi_protocol_handle *ph,
u8 evt_id, ktime_t timestamp,
const void *payld, size_t payld_sz,
void *report, u32 *src_id)
@@ -837,6 +835,16 @@ static void *scmi_perf_fill_custom_report(const struct scmi_handle *handle,
return rep;
}
+static int scmi_perf_get_num_sources(const struct scmi_protocol_handle *ph)
+{
+ struct scmi_perf_info *pi = ph->get_priv(ph);
+
+ if (!pi)
+ return -EINVAL;
+
+ return pi->num_domains;
+}
+
static const struct scmi_event perf_events[] = {
{
.id = SCMI_EVENT_PERFORMANCE_LIMITS_CHANGED,
@@ -851,28 +859,36 @@ static const struct scmi_event perf_events[] = {
};
static const struct scmi_event_ops perf_event_ops = {
+ .get_num_sources = scmi_perf_get_num_sources,
.set_notify_enabled = scmi_perf_set_notify_enabled,
.fill_custom_report = scmi_perf_fill_custom_report,
};
-static int scmi_perf_protocol_init(struct scmi_handle *handle)
+static const struct scmi_protocol_events perf_protocol_events = {
+ .queue_sz = SCMI_PROTO_QUEUE_SZ,
+ .ops = &perf_event_ops,
+ .evts = perf_events,
+ .num_events = ARRAY_SIZE(perf_events),
+};
+
+static int scmi_perf_protocol_init(const struct scmi_protocol_handle *ph)
{
int domain;
u32 version;
struct scmi_perf_info *pinfo;
- scmi_version_get(handle, SCMI_PROTOCOL_PERF, &version);
+ ph->xops->version_get(ph, &version);
- dev_dbg(handle->dev, "Performance Version %d.%d\n",
+ dev_dbg(ph->dev, "Performance Version %d.%d\n",
PROTOCOL_REV_MAJOR(version), PROTOCOL_REV_MINOR(version));
- pinfo = devm_kzalloc(handle->dev, sizeof(*pinfo), GFP_KERNEL);
+ pinfo = devm_kzalloc(ph->dev, sizeof(*pinfo), GFP_KERNEL);
if (!pinfo)
return -ENOMEM;
- scmi_perf_attributes_get(handle, pinfo);
+ scmi_perf_attributes_get(ph, pinfo);
- pinfo->dom_info = devm_kcalloc(handle->dev, pinfo->num_domains,
+ pinfo->dom_info = devm_kcalloc(ph->dev, pinfo->num_domains,
sizeof(*pinfo->dom_info), GFP_KERNEL);
if (!pinfo->dom_info)
return -ENOMEM;
@@ -880,24 +896,24 @@ static int scmi_perf_protocol_init(struct scmi_handle *handle)
for (domain = 0; domain < pinfo->num_domains; domain++) {
struct perf_dom_info *dom = pinfo->dom_info + domain;
- scmi_perf_domain_attributes_get(handle, domain, dom);
- scmi_perf_describe_levels_get(handle, domain, dom);
+ scmi_perf_domain_attributes_get(ph, domain, dom);
+ scmi_perf_describe_levels_get(ph, domain, dom);
if (dom->perf_fastchannels)
- scmi_perf_domain_init_fc(handle, domain, &dom->fc_info);
+ scmi_perf_domain_init_fc(ph, domain, &dom->fc_info);
}
- scmi_register_protocol_events(handle,
- SCMI_PROTOCOL_PERF, SCMI_PROTO_QUEUE_SZ,
- &perf_event_ops, perf_events,
- ARRAY_SIZE(perf_events),
- pinfo->num_domains);
-
pinfo->version = version;
- handle->perf_ops = &perf_ops;
- handle->perf_priv = pinfo;
- return 0;
+ return ph->set_priv(ph, pinfo);
}
-DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(SCMI_PROTOCOL_PERF, perf)
+static const struct scmi_protocol scmi_perf = {
+ .id = SCMI_PROTOCOL_PERF,
+ .owner = THIS_MODULE,
+ .instance_init = &scmi_perf_protocol_init,
+ .ops = &perf_proto_ops,
+ .events = &perf_protocol_events,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(perf, scmi_perf)
diff --git a/drivers/firmware/arm_scmi/power.c b/drivers/firmware/arm_scmi/power.c
index 1f37258e9bee..ad2ab080f344 100644
--- a/drivers/firmware/arm_scmi/power.c
+++ b/drivers/firmware/arm_scmi/power.c
@@ -2,11 +2,12 @@
/*
* System Control and Management Interface (SCMI) Power Protocol
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#define pr_fmt(fmt) "SCMI Notifications POWER - " fmt
+#include <linux/module.h>
#include <linux/scmi_protocol.h>
#include "common.h"
@@ -68,21 +69,21 @@ struct scmi_power_info {
struct power_dom_info *dom_info;
};
-static int scmi_power_attributes_get(const struct scmi_handle *handle,
+static int scmi_power_attributes_get(const struct scmi_protocol_handle *ph,
struct scmi_power_info *pi)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_power_attributes *attr;
- ret = scmi_xfer_get_init(handle, PROTOCOL_ATTRIBUTES,
- SCMI_PROTOCOL_POWER, 0, sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, PROTOCOL_ATTRIBUTES,
+ 0, sizeof(*attr), &t);
if (ret)
return ret;
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
pi->num_domains = le16_to_cpu(attr->num_domains);
pi->stats_addr = le32_to_cpu(attr->stats_addr_low) |
@@ -90,28 +91,27 @@ static int scmi_power_attributes_get(const struct scmi_handle *handle,
pi->stats_size = le32_to_cpu(attr->stats_size);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
static int
-scmi_power_domain_attributes_get(const struct scmi_handle *handle, u32 domain,
- struct power_dom_info *dom_info)
+scmi_power_domain_attributes_get(const struct scmi_protocol_handle *ph,
+ u32 domain, struct power_dom_info *dom_info)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_power_domain_attributes *attr;
- ret = scmi_xfer_get_init(handle, POWER_DOMAIN_ATTRIBUTES,
- SCMI_PROTOCOL_POWER, sizeof(domain),
- sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, POWER_DOMAIN_ATTRIBUTES,
+ sizeof(domain), sizeof(*attr), &t);
if (ret)
return ret;
put_unaligned_le32(domain, t->tx.buf);
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
u32 flags = le32_to_cpu(attr->flags);
@@ -121,19 +121,18 @@ scmi_power_domain_attributes_get(const struct scmi_handle *handle, u32 domain,
strlcpy(dom_info->name, attr->name, SCMI_MAX_STR_SIZE);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int
-scmi_power_state_set(const struct scmi_handle *handle, u32 domain, u32 state)
+static int scmi_power_state_set(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 state)
{
int ret;
struct scmi_xfer *t;
struct scmi_power_set_state *st;
- ret = scmi_xfer_get_init(handle, POWER_STATE_SET, SCMI_PROTOCOL_POWER,
- sizeof(*st), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, POWER_STATE_SET, sizeof(*st), 0, &t);
if (ret)
return ret;
@@ -142,64 +141,64 @@ scmi_power_state_set(const struct scmi_handle *handle, u32 domain, u32 state)
st->domain = cpu_to_le32(domain);
st->state = cpu_to_le32(state);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int
-scmi_power_state_get(const struct scmi_handle *handle, u32 domain, u32 *state)
+static int scmi_power_state_get(const struct scmi_protocol_handle *ph,
+ u32 domain, u32 *state)
{
int ret;
struct scmi_xfer *t;
- ret = scmi_xfer_get_init(handle, POWER_STATE_GET, SCMI_PROTOCOL_POWER,
- sizeof(u32), sizeof(u32), &t);
+ ret = ph->xops->xfer_get_init(ph, POWER_STATE_GET, sizeof(u32), sizeof(u32), &t);
if (ret)
return ret;
put_unaligned_le32(domain, t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
*state = get_unaligned_le32(t->rx.buf);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_power_num_domains_get(const struct scmi_handle *handle)
+static int scmi_power_num_domains_get(const struct scmi_protocol_handle *ph)
{
- struct scmi_power_info *pi = handle->power_priv;
+ struct scmi_power_info *pi = ph->get_priv(ph);
return pi->num_domains;
}
-static char *scmi_power_name_get(const struct scmi_handle *handle, u32 domain)
+static char *scmi_power_name_get(const struct scmi_protocol_handle *ph,
+ u32 domain)
{
- struct scmi_power_info *pi = handle->power_priv;
+ struct scmi_power_info *pi = ph->get_priv(ph);
struct power_dom_info *dom = pi->dom_info + domain;
return dom->name;
}
-static const struct scmi_power_ops power_ops = {
+static const struct scmi_power_proto_ops power_proto_ops = {
.num_domains_get = scmi_power_num_domains_get,
.name_get = scmi_power_name_get,
.state_set = scmi_power_state_set,
.state_get = scmi_power_state_get,
};
-static int scmi_power_request_notify(const struct scmi_handle *handle,
+static int scmi_power_request_notify(const struct scmi_protocol_handle *ph,
u32 domain, bool enable)
{
int ret;
struct scmi_xfer *t;
struct scmi_power_state_notify *notify;
- ret = scmi_xfer_get_init(handle, POWER_STATE_NOTIFY,
- SCMI_PROTOCOL_POWER, sizeof(*notify), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, POWER_STATE_NOTIFY,
+ sizeof(*notify), 0, &t);
if (ret)
return ret;
@@ -207,18 +206,18 @@ static int scmi_power_request_notify(const struct scmi_handle *handle,
notify->domain = cpu_to_le32(domain);
notify->notify_enable = enable ? cpu_to_le32(BIT(0)) : 0;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_power_set_notify_enabled(const struct scmi_handle *handle,
+static int scmi_power_set_notify_enabled(const struct scmi_protocol_handle *ph,
u8 evt_id, u32 src_id, bool enable)
{
int ret;
- ret = scmi_power_request_notify(handle, src_id, enable);
+ ret = scmi_power_request_notify(ph, src_id, enable);
if (ret)
pr_debug("FAIL_ENABLE - evt[%X] dom[%d] - ret:%d\n",
evt_id, src_id, ret);
@@ -226,10 +225,11 @@ static int scmi_power_set_notify_enabled(const struct scmi_handle *handle,
return ret;
}
-static void *scmi_power_fill_custom_report(const struct scmi_handle *handle,
- u8 evt_id, ktime_t timestamp,
- const void *payld, size_t payld_sz,
- void *report, u32 *src_id)
+static void *
+scmi_power_fill_custom_report(const struct scmi_protocol_handle *ph,
+ u8 evt_id, ktime_t timestamp,
+ const void *payld, size_t payld_sz,
+ void *report, u32 *src_id)
{
const struct scmi_power_state_notify_payld *p = payld;
struct scmi_power_state_changed_report *r = report;
@@ -246,6 +246,16 @@ static void *scmi_power_fill_custom_report(const struct scmi_handle *handle,
return r;
}
+static int scmi_power_get_num_sources(const struct scmi_protocol_handle *ph)
+{
+ struct scmi_power_info *pinfo = ph->get_priv(ph);
+
+ if (!pinfo)
+ return -EINVAL;
+
+ return pinfo->num_domains;
+}
+
static const struct scmi_event power_events[] = {
{
.id = SCMI_EVENT_POWER_STATE_CHANGED,
@@ -256,28 +266,36 @@ static const struct scmi_event power_events[] = {
};
static const struct scmi_event_ops power_event_ops = {
+ .get_num_sources = scmi_power_get_num_sources,
.set_notify_enabled = scmi_power_set_notify_enabled,
.fill_custom_report = scmi_power_fill_custom_report,
};
-static int scmi_power_protocol_init(struct scmi_handle *handle)
+static const struct scmi_protocol_events power_protocol_events = {
+ .queue_sz = SCMI_PROTO_QUEUE_SZ,
+ .ops = &power_event_ops,
+ .evts = power_events,
+ .num_events = ARRAY_SIZE(power_events),
+};
+
+static int scmi_power_protocol_init(const struct scmi_protocol_handle *ph)
{
int domain;
u32 version;
struct scmi_power_info *pinfo;
- scmi_version_get(handle, SCMI_PROTOCOL_POWER, &version);
+ ph->xops->version_get(ph, &version);
- dev_dbg(handle->dev, "Power Version %d.%d\n",
+ dev_dbg(ph->dev, "Power Version %d.%d\n",
PROTOCOL_REV_MAJOR(version), PROTOCOL_REV_MINOR(version));
- pinfo = devm_kzalloc(handle->dev, sizeof(*pinfo), GFP_KERNEL);
+ pinfo = devm_kzalloc(ph->dev, sizeof(*pinfo), GFP_KERNEL);
if (!pinfo)
return -ENOMEM;
- scmi_power_attributes_get(handle, pinfo);
+ scmi_power_attributes_get(ph, pinfo);
- pinfo->dom_info = devm_kcalloc(handle->dev, pinfo->num_domains,
+ pinfo->dom_info = devm_kcalloc(ph->dev, pinfo->num_domains,
sizeof(*pinfo->dom_info), GFP_KERNEL);
if (!pinfo->dom_info)
return -ENOMEM;
@@ -285,20 +303,20 @@ static int scmi_power_protocol_init(struct scmi_handle *handle)
for (domain = 0; domain < pinfo->num_domains; domain++) {
struct power_dom_info *dom = pinfo->dom_info + domain;
- scmi_power_domain_attributes_get(handle, domain, dom);
+ scmi_power_domain_attributes_get(ph, domain, dom);
}
- scmi_register_protocol_events(handle,
- SCMI_PROTOCOL_POWER, SCMI_PROTO_QUEUE_SZ,
- &power_event_ops, power_events,
- ARRAY_SIZE(power_events),
- pinfo->num_domains);
-
pinfo->version = version;
- handle->power_ops = &power_ops;
- handle->power_priv = pinfo;
- return 0;
+ return ph->set_priv(ph, pinfo);
}
-DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(SCMI_PROTOCOL_POWER, power)
+static const struct scmi_protocol scmi_power = {
+ .id = SCMI_PROTOCOL_POWER,
+ .owner = THIS_MODULE,
+ .instance_init = &scmi_power_protocol_init,
+ .ops = &power_proto_ops,
+ .events = &power_protocol_events,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(power, scmi_power)
diff --git a/drivers/firmware/arm_scmi/reset.c b/drivers/firmware/arm_scmi/reset.c
index a981a22cfe89..9bf2478ec6d1 100644
--- a/drivers/firmware/arm_scmi/reset.c
+++ b/drivers/firmware/arm_scmi/reset.c
@@ -2,11 +2,12 @@
/*
* System Control and Management Interface (SCMI) Reset Protocol
*
- * Copyright (C) 2019 ARM Ltd.
+ * Copyright (C) 2019-2021 ARM Ltd.
*/
#define pr_fmt(fmt) "SCMI Notifications RESET - " fmt
+#include <linux/module.h>
#include <linux/scmi_protocol.h>
#include "common.h"
@@ -64,46 +65,45 @@ struct scmi_reset_info {
struct reset_dom_info *dom_info;
};
-static int scmi_reset_attributes_get(const struct scmi_handle *handle,
+static int scmi_reset_attributes_get(const struct scmi_protocol_handle *ph,
struct scmi_reset_info *pi)
{
int ret;
struct scmi_xfer *t;
u32 attr;
- ret = scmi_xfer_get_init(handle, PROTOCOL_ATTRIBUTES,
- SCMI_PROTOCOL_RESET, 0, sizeof(attr), &t);
+ ret = ph->xops->xfer_get_init(ph, PROTOCOL_ATTRIBUTES,
+ 0, sizeof(attr), &t);
if (ret)
return ret;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
attr = get_unaligned_le32(t->rx.buf);
pi->num_domains = attr & NUM_RESET_DOMAIN_MASK;
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
static int
-scmi_reset_domain_attributes_get(const struct scmi_handle *handle, u32 domain,
- struct reset_dom_info *dom_info)
+scmi_reset_domain_attributes_get(const struct scmi_protocol_handle *ph,
+ u32 domain, struct reset_dom_info *dom_info)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_reset_domain_attributes *attr;
- ret = scmi_xfer_get_init(handle, RESET_DOMAIN_ATTRIBUTES,
- SCMI_PROTOCOL_RESET, sizeof(domain),
- sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, RESET_DOMAIN_ATTRIBUTES,
+ sizeof(domain), sizeof(*attr), &t);
if (ret)
return ret;
put_unaligned_le32(domain, t->tx.buf);
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
u32 attributes = le32_to_cpu(attr->attributes);
@@ -115,47 +115,49 @@ scmi_reset_domain_attributes_get(const struct scmi_handle *handle, u32 domain,
strlcpy(dom_info->name, attr->name, SCMI_MAX_STR_SIZE);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_reset_num_domains_get(const struct scmi_handle *handle)
+static int scmi_reset_num_domains_get(const struct scmi_protocol_handle *ph)
{
- struct scmi_reset_info *pi = handle->reset_priv;
+ struct scmi_reset_info *pi = ph->get_priv(ph);
return pi->num_domains;
}
-static char *scmi_reset_name_get(const struct scmi_handle *handle, u32 domain)
+static char *scmi_reset_name_get(const struct scmi_protocol_handle *ph,
+ u32 domain)
{
- struct scmi_reset_info *pi = handle->reset_priv;
+ struct scmi_reset_info *pi = ph->get_priv(ph);
+
struct reset_dom_info *dom = pi->dom_info + domain;
return dom->name;
}
-static int scmi_reset_latency_get(const struct scmi_handle *handle, u32 domain)
+static int scmi_reset_latency_get(const struct scmi_protocol_handle *ph,
+ u32 domain)
{
- struct scmi_reset_info *pi = handle->reset_priv;
+ struct scmi_reset_info *pi = ph->get_priv(ph);
struct reset_dom_info *dom = pi->dom_info + domain;
return dom->latency_us;
}
-static int scmi_domain_reset(const struct scmi_handle *handle, u32 domain,
+static int scmi_domain_reset(const struct scmi_protocol_handle *ph, u32 domain,
u32 flags, u32 state)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_reset_domain_reset *dom;
- struct scmi_reset_info *pi = handle->reset_priv;
+ struct scmi_reset_info *pi = ph->get_priv(ph);
struct reset_dom_info *rdom = pi->dom_info + domain;
if (rdom->async_reset)
flags |= ASYNCHRONOUS_RESET;
- ret = scmi_xfer_get_init(handle, RESET, SCMI_PROTOCOL_RESET,
- sizeof(*dom), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, RESET, sizeof(*dom), 0, &t);
if (ret)
return ret;
@@ -165,34 +167,35 @@ static int scmi_domain_reset(const struct scmi_handle *handle, u32 domain,
dom->reset_state = cpu_to_le32(state);
if (rdom->async_reset)
- ret = scmi_do_xfer_with_response(handle, t);
+ ret = ph->xops->do_xfer_with_response(ph, t);
else
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_reset_domain_reset(const struct scmi_handle *handle, u32 domain)
+static int scmi_reset_domain_reset(const struct scmi_protocol_handle *ph,
+ u32 domain)
{
- return scmi_domain_reset(handle, domain, AUTONOMOUS_RESET,
+ return scmi_domain_reset(ph, domain, AUTONOMOUS_RESET,
ARCH_COLD_RESET);
}
static int
-scmi_reset_domain_assert(const struct scmi_handle *handle, u32 domain)
+scmi_reset_domain_assert(const struct scmi_protocol_handle *ph, u32 domain)
{
- return scmi_domain_reset(handle, domain, EXPLICIT_RESET_ASSERT,
+ return scmi_domain_reset(ph, domain, EXPLICIT_RESET_ASSERT,
ARCH_COLD_RESET);
}
static int
-scmi_reset_domain_deassert(const struct scmi_handle *handle, u32 domain)
+scmi_reset_domain_deassert(const struct scmi_protocol_handle *ph, u32 domain)
{
- return scmi_domain_reset(handle, domain, 0, ARCH_COLD_RESET);
+ return scmi_domain_reset(ph, domain, 0, ARCH_COLD_RESET);
}
-static const struct scmi_reset_ops reset_ops = {
+static const struct scmi_reset_proto_ops reset_proto_ops = {
.num_domains_get = scmi_reset_num_domains_get,
.name_get = scmi_reset_name_get,
.latency_get = scmi_reset_latency_get,
@@ -201,16 +204,15 @@ static const struct scmi_reset_ops reset_ops = {
.deassert = scmi_reset_domain_deassert,
};
-static int scmi_reset_notify(const struct scmi_handle *handle, u32 domain_id,
- bool enable)
+static int scmi_reset_notify(const struct scmi_protocol_handle *ph,
+ u32 domain_id, bool enable)
{
int ret;
u32 evt_cntl = enable ? RESET_TP_NOTIFY_ALL : 0;
struct scmi_xfer *t;
struct scmi_msg_reset_notify *cfg;
- ret = scmi_xfer_get_init(handle, RESET_NOTIFY,
- SCMI_PROTOCOL_RESET, sizeof(*cfg), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, RESET_NOTIFY, sizeof(*cfg), 0, &t);
if (ret)
return ret;
@@ -218,18 +220,18 @@ static int scmi_reset_notify(const struct scmi_handle *handle, u32 domain_id,
cfg->id = cpu_to_le32(domain_id);
cfg->event_control = cpu_to_le32(evt_cntl);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_reset_set_notify_enabled(const struct scmi_handle *handle,
+static int scmi_reset_set_notify_enabled(const struct scmi_protocol_handle *ph,
u8 evt_id, u32 src_id, bool enable)
{
int ret;
- ret = scmi_reset_notify(handle, src_id, enable);
+ ret = scmi_reset_notify(ph, src_id, enable);
if (ret)
pr_debug("FAIL_ENABLED - evt[%X] dom[%d] - ret:%d\n",
evt_id, src_id, ret);
@@ -237,10 +239,11 @@ static int scmi_reset_set_notify_enabled(const struct scmi_handle *handle,
return ret;
}
-static void *scmi_reset_fill_custom_report(const struct scmi_handle *handle,
- u8 evt_id, ktime_t timestamp,
- const void *payld, size_t payld_sz,
- void *report, u32 *src_id)
+static void *
+scmi_reset_fill_custom_report(const struct scmi_protocol_handle *ph,
+ u8 evt_id, ktime_t timestamp,
+ const void *payld, size_t payld_sz,
+ void *report, u32 *src_id)
{
const struct scmi_reset_issued_notify_payld *p = payld;
struct scmi_reset_issued_report *r = report;
@@ -257,6 +260,16 @@ static void *scmi_reset_fill_custom_report(const struct scmi_handle *handle,
return r;
}
+static int scmi_reset_get_num_sources(const struct scmi_protocol_handle *ph)
+{
+ struct scmi_reset_info *pinfo = ph->get_priv(ph);
+
+ if (!pinfo)
+ return -EINVAL;
+
+ return pinfo->num_domains;
+}
+
static const struct scmi_event reset_events[] = {
{
.id = SCMI_EVENT_RESET_ISSUED,
@@ -266,28 +279,36 @@ static const struct scmi_event reset_events[] = {
};
static const struct scmi_event_ops reset_event_ops = {
+ .get_num_sources = scmi_reset_get_num_sources,
.set_notify_enabled = scmi_reset_set_notify_enabled,
.fill_custom_report = scmi_reset_fill_custom_report,
};
-static int scmi_reset_protocol_init(struct scmi_handle *handle)
+static const struct scmi_protocol_events reset_protocol_events = {
+ .queue_sz = SCMI_PROTO_QUEUE_SZ,
+ .ops = &reset_event_ops,
+ .evts = reset_events,
+ .num_events = ARRAY_SIZE(reset_events),
+};
+
+static int scmi_reset_protocol_init(const struct scmi_protocol_handle *ph)
{
int domain;
u32 version;
struct scmi_reset_info *pinfo;
- scmi_version_get(handle, SCMI_PROTOCOL_RESET, &version);
+ ph->xops->version_get(ph, &version);
- dev_dbg(handle->dev, "Reset Version %d.%d\n",
+ dev_dbg(ph->dev, "Reset Version %d.%d\n",
PROTOCOL_REV_MAJOR(version), PROTOCOL_REV_MINOR(version));
- pinfo = devm_kzalloc(handle->dev, sizeof(*pinfo), GFP_KERNEL);
+ pinfo = devm_kzalloc(ph->dev, sizeof(*pinfo), GFP_KERNEL);
if (!pinfo)
return -ENOMEM;
- scmi_reset_attributes_get(handle, pinfo);
+ scmi_reset_attributes_get(ph, pinfo);
- pinfo->dom_info = devm_kcalloc(handle->dev, pinfo->num_domains,
+ pinfo->dom_info = devm_kcalloc(ph->dev, pinfo->num_domains,
sizeof(*pinfo->dom_info), GFP_KERNEL);
if (!pinfo->dom_info)
return -ENOMEM;
@@ -295,20 +316,19 @@ static int scmi_reset_protocol_init(struct scmi_handle *handle)
for (domain = 0; domain < pinfo->num_domains; domain++) {
struct reset_dom_info *dom = pinfo->dom_info + domain;
- scmi_reset_domain_attributes_get(handle, domain, dom);
+ scmi_reset_domain_attributes_get(ph, domain, dom);
}
- scmi_register_protocol_events(handle,
- SCMI_PROTOCOL_RESET, SCMI_PROTO_QUEUE_SZ,
- &reset_event_ops, reset_events,
- ARRAY_SIZE(reset_events),
- pinfo->num_domains);
-
pinfo->version = version;
- handle->reset_ops = &reset_ops;
- handle->reset_priv = pinfo;
-
- return 0;
+ return ph->set_priv(ph, pinfo);
}
-DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(SCMI_PROTOCOL_RESET, reset)
+static const struct scmi_protocol scmi_reset = {
+ .id = SCMI_PROTOCOL_RESET,
+ .owner = THIS_MODULE,
+ .instance_init = &scmi_reset_protocol_init,
+ .ops = &reset_proto_ops,
+ .events = &reset_protocol_events,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(reset, scmi_reset)
diff --git a/drivers/firmware/arm_scmi/scmi_pm_domain.c b/drivers/firmware/arm_scmi/scmi_pm_domain.c
index 9e44479f0284..9d36d5c0622d 100644
--- a/drivers/firmware/arm_scmi/scmi_pm_domain.c
+++ b/drivers/firmware/arm_scmi/scmi_pm_domain.c
@@ -2,7 +2,7 @@
/*
* SCMI Generic power domain support.
*
- * Copyright (C) 2018 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#include <linux/err.h>
@@ -11,9 +11,11 @@
#include <linux/pm_domain.h>
#include <linux/scmi_protocol.h>
+static const struct scmi_power_proto_ops *power_ops;
+
struct scmi_pm_domain {
struct generic_pm_domain genpd;
- const struct scmi_handle *handle;
+ const struct scmi_protocol_handle *ph;
const char *name;
u32 domain;
};
@@ -25,16 +27,15 @@ static int scmi_pd_power(struct generic_pm_domain *domain, bool power_on)
int ret;
u32 state, ret_state;
struct scmi_pm_domain *pd = to_scmi_pd(domain);
- const struct scmi_power_ops *ops = pd->handle->power_ops;
if (power_on)
state = SCMI_POWER_STATE_GENERIC_ON;
else
state = SCMI_POWER_STATE_GENERIC_OFF;
- ret = ops->state_set(pd->handle, pd->domain, state);
+ ret = power_ops->state_set(pd->ph, pd->domain, state);
if (!ret)
- ret = ops->state_get(pd->handle, pd->domain, &ret_state);
+ ret = power_ops->state_get(pd->ph, pd->domain, &ret_state);
if (!ret && state != ret_state)
return -EIO;
@@ -60,11 +61,16 @@ static int scmi_pm_domain_probe(struct scmi_device *sdev)
struct genpd_onecell_data *scmi_pd_data;
struct generic_pm_domain **domains;
const struct scmi_handle *handle = sdev->handle;
+ struct scmi_protocol_handle *ph;
- if (!handle || !handle->power_ops)
+ if (!handle)
return -ENODEV;
- num_domains = handle->power_ops->num_domains_get(handle);
+ power_ops = handle->devm_protocol_get(sdev, SCMI_PROTOCOL_POWER, &ph);
+ if (IS_ERR(power_ops))
+ return PTR_ERR(power_ops);
+
+ num_domains = power_ops->num_domains_get(ph);
if (num_domains < 0) {
dev_err(dev, "number of domains not found\n");
return num_domains;
@@ -85,14 +91,14 @@ static int scmi_pm_domain_probe(struct scmi_device *sdev)
for (i = 0; i < num_domains; i++, scmi_pd++) {
u32 state;
- if (handle->power_ops->state_get(handle, i, &state)) {
+ if (power_ops->state_get(ph, i, &state)) {
dev_warn(dev, "failed to get state for domain %d\n", i);
continue;
}
scmi_pd->domain = i;
- scmi_pd->handle = handle;
- scmi_pd->name = handle->power_ops->name_get(handle, i);
+ scmi_pd->ph = ph;
+ scmi_pd->name = power_ops->name_get(ph, i);
scmi_pd->genpd.name = scmi_pd->name;
scmi_pd->genpd.power_off = scmi_pd_power_off;
scmi_pd->genpd.power_on = scmi_pd_power_on;
diff --git a/drivers/firmware/arm_scmi/sensors.c b/drivers/firmware/arm_scmi/sensors.c
index 4541b891b733..2c88aa221559 100644
--- a/drivers/firmware/arm_scmi/sensors.c
+++ b/drivers/firmware/arm_scmi/sensors.c
@@ -2,12 +2,13 @@
/*
* System Control and Management Interface (SCMI) Sensor Protocol
*
- * Copyright (C) 2018-2020 ARM Ltd.
+ * Copyright (C) 2018-2021 ARM Ltd.
*/
#define pr_fmt(fmt) "SCMI Notifications SENSOR - " fmt
#include <linux/bitfield.h>
+#include <linux/module.h>
#include <linux/scmi_protocol.h>
#include "common.h"
@@ -201,21 +202,21 @@ struct sensors_info {
struct scmi_sensor_info *sensors;
};
-static int scmi_sensor_attributes_get(const struct scmi_handle *handle,
+static int scmi_sensor_attributes_get(const struct scmi_protocol_handle *ph,
struct sensors_info *si)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_resp_sensor_attributes *attr;
- ret = scmi_xfer_get_init(handle, PROTOCOL_ATTRIBUTES,
- SCMI_PROTOCOL_SENSOR, 0, sizeof(*attr), &t);
+ ret = ph->xops->xfer_get_init(ph, PROTOCOL_ATTRIBUTES,
+ 0, sizeof(*attr), &t);
if (ret)
return ret;
attr = t->rx.buf;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
si->num_sensors = le16_to_cpu(attr->num_sensors);
si->max_requests = attr->max_requests;
@@ -224,7 +225,7 @@ static int scmi_sensor_attributes_get(const struct scmi_handle *handle,
si->reg_size = le32_to_cpu(attr->reg_size);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -235,7 +236,7 @@ static inline void scmi_parse_range_attrs(struct scmi_range_attrs *out,
out->max_range = get_unaligned_le64((void *)&in->max_range_low);
}
-static int scmi_sensor_update_intervals(const struct scmi_handle *handle,
+static int scmi_sensor_update_intervals(const struct scmi_protocol_handle *ph,
struct scmi_sensor_info *s)
{
int ret, cnt;
@@ -245,8 +246,8 @@ static int scmi_sensor_update_intervals(const struct scmi_handle *handle,
struct scmi_msg_resp_sensor_list_update_intervals *buf;
struct scmi_msg_sensor_list_update_intervals *msg;
- ret = scmi_xfer_get_init(handle, SENSOR_LIST_UPDATE_INTERVALS,
- SCMI_PROTOCOL_SENSOR, sizeof(*msg), 0, &ti);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_LIST_UPDATE_INTERVALS,
+ sizeof(*msg), 0, &ti);
if (ret)
return ret;
@@ -259,7 +260,7 @@ static int scmi_sensor_update_intervals(const struct scmi_handle *handle,
msg->id = cpu_to_le32(s->id);
msg->index = cpu_to_le32(desc_index);
- ret = scmi_do_xfer(handle, ti);
+ ret = ph->xops->do_xfer(ph, ti);
if (ret)
break;
@@ -277,7 +278,7 @@ static int scmi_sensor_update_intervals(const struct scmi_handle *handle,
/* segmented intervals are reported in one triplet */
if (s->intervals.segmented &&
(num_remaining || num_returned != 3)) {
- dev_err(handle->dev,
+ dev_err(ph->dev,
"Sensor ID:%d advertises an invalid segmented interval (%d)\n",
s->id, s->intervals.count);
s->intervals.segmented = false;
@@ -288,7 +289,7 @@ static int scmi_sensor_update_intervals(const struct scmi_handle *handle,
/* Direct allocation when exceeding pre-allocated */
if (s->intervals.count >= SCMI_MAX_PREALLOC_POOL) {
s->intervals.desc =
- devm_kcalloc(handle->dev,
+ devm_kcalloc(ph->dev,
s->intervals.count,
sizeof(*s->intervals.desc),
GFP_KERNEL);
@@ -300,7 +301,7 @@ static int scmi_sensor_update_intervals(const struct scmi_handle *handle,
}
}
} else if (desc_index + num_returned > s->intervals.count) {
- dev_err(handle->dev,
+ dev_err(ph->dev,
"No. of update intervals can't exceed %d\n",
s->intervals.count);
ret = -EINVAL;
@@ -313,18 +314,18 @@ static int scmi_sensor_update_intervals(const struct scmi_handle *handle,
desc_index += num_returned;
- scmi_reset_rx_to_maxsz(handle, ti);
+ ph->xops->reset_rx_to_maxsz(ph, ti);
/*
* check for both returned and remaining to avoid infinite
* loop due to buggy firmware
*/
} while (num_returned && num_remaining);
- scmi_xfer_put(handle, ti);
+ ph->xops->xfer_put(ph, ti);
return ret;
}
-static int scmi_sensor_axis_description(const struct scmi_handle *handle,
+static int scmi_sensor_axis_description(const struct scmi_protocol_handle *ph,
struct scmi_sensor_info *s)
{
int ret, cnt;
@@ -334,13 +335,13 @@ static int scmi_sensor_axis_description(const struct scmi_handle *handle,
struct scmi_msg_resp_sensor_axis_description *buf;
struct scmi_msg_sensor_axis_description_get *msg;
- s->axis = devm_kcalloc(handle->dev, s->num_axis,
+ s->axis = devm_kcalloc(ph->dev, s->num_axis,
sizeof(*s->axis), GFP_KERNEL);
if (!s->axis)
return -ENOMEM;
- ret = scmi_xfer_get_init(handle, SENSOR_AXIS_DESCRIPTION_GET,
- SCMI_PROTOCOL_SENSOR, sizeof(*msg), 0, &te);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_AXIS_DESCRIPTION_GET,
+ sizeof(*msg), 0, &te);
if (ret)
return ret;
@@ -354,7 +355,7 @@ static int scmi_sensor_axis_description(const struct scmi_handle *handle,
msg->id = cpu_to_le32(s->id);
msg->axis_desc_index = cpu_to_le32(desc_index);
- ret = scmi_do_xfer(handle, te);
+ ret = ph->xops->do_xfer(ph, te);
if (ret)
break;
@@ -363,7 +364,7 @@ static int scmi_sensor_axis_description(const struct scmi_handle *handle,
num_remaining = NUM_AXIS_REMAINING(flags);
if (desc_index + num_returned > s->num_axis) {
- dev_err(handle->dev, "No. of axis can't exceed %d\n",
+ dev_err(ph->dev, "No. of axis can't exceed %d\n",
s->num_axis);
break;
}
@@ -405,18 +406,18 @@ static int scmi_sensor_axis_description(const struct scmi_handle *handle,
desc_index += num_returned;
- scmi_reset_rx_to_maxsz(handle, te);
+ ph->xops->reset_rx_to_maxsz(ph, te);
/*
* check for both returned and remaining to avoid infinite
* loop due to buggy firmware
*/
} while (num_returned && num_remaining);
- scmi_xfer_put(handle, te);
+ ph->xops->xfer_put(ph, te);
return ret;
}
-static int scmi_sensor_description_get(const struct scmi_handle *handle,
+static int scmi_sensor_description_get(const struct scmi_protocol_handle *ph,
struct sensors_info *si)
{
int ret, cnt;
@@ -425,8 +426,8 @@ static int scmi_sensor_description_get(const struct scmi_handle *handle,
struct scmi_xfer *t;
struct scmi_msg_resp_sensor_description *buf;
- ret = scmi_xfer_get_init(handle, SENSOR_DESCRIPTION_GET,
- SCMI_PROTOCOL_SENSOR, sizeof(__le32), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_DESCRIPTION_GET,
+ sizeof(__le32), 0, &t);
if (ret)
return ret;
@@ -437,7 +438,8 @@ static int scmi_sensor_description_get(const struct scmi_handle *handle,
/* Set the number of sensors to be skipped/already read */
put_unaligned_le32(desc_index, t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+
+ ret = ph->xops->do_xfer(ph, t);
if (ret)
break;
@@ -445,7 +447,7 @@ static int scmi_sensor_description_get(const struct scmi_handle *handle,
num_remaining = le16_to_cpu(buf->num_remaining);
if (desc_index + num_returned > si->num_sensors) {
- dev_err(handle->dev, "No. of sensors can't exceed %d",
+ dev_err(ph->dev, "No. of sensors can't exceed %d",
si->num_sensors);
break;
}
@@ -500,8 +502,8 @@ static int scmi_sensor_description_get(const struct scmi_handle *handle,
* Since the command is optional, on error carry
* on without any update interval.
*/
- if (scmi_sensor_update_intervals(handle, s))
- dev_dbg(handle->dev,
+ if (scmi_sensor_update_intervals(ph, s))
+ dev_dbg(ph->dev,
"Update Intervals not available for sensor ID:%d\n",
s->id);
}
@@ -535,7 +537,7 @@ static int scmi_sensor_description_get(const struct scmi_handle *handle,
}
}
if (s->num_axis > 0) {
- ret = scmi_sensor_axis_description(handle, s);
+ ret = scmi_sensor_axis_description(ph, s);
if (ret)
goto out;
}
@@ -545,7 +547,7 @@ static int scmi_sensor_description_get(const struct scmi_handle *handle,
desc_index += num_returned;
- scmi_reset_rx_to_maxsz(handle, t);
+ ph->xops->reset_rx_to_maxsz(ph, t);
/*
* check for both returned and remaining to avoid infinite
* loop due to buggy firmware
@@ -553,12 +555,12 @@ static int scmi_sensor_description_get(const struct scmi_handle *handle,
} while (num_returned && num_remaining);
out:
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
static inline int
-scmi_sensor_request_notify(const struct scmi_handle *handle, u32 sensor_id,
+scmi_sensor_request_notify(const struct scmi_protocol_handle *ph, u32 sensor_id,
u8 message_id, bool enable)
{
int ret;
@@ -566,8 +568,7 @@ scmi_sensor_request_notify(const struct scmi_handle *handle, u32 sensor_id,
struct scmi_xfer *t;
struct scmi_msg_sensor_request_notify *cfg;
- ret = scmi_xfer_get_init(handle, message_id,
- SCMI_PROTOCOL_SENSOR, sizeof(*cfg), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, message_id, sizeof(*cfg), 0, &t);
if (ret)
return ret;
@@ -575,40 +576,40 @@ scmi_sensor_request_notify(const struct scmi_handle *handle, u32 sensor_id,
cfg->id = cpu_to_le32(sensor_id);
cfg->event_control = cpu_to_le32(evt_cntl);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_sensor_trip_point_notify(const struct scmi_handle *handle,
+static int scmi_sensor_trip_point_notify(const struct scmi_protocol_handle *ph,
u32 sensor_id, bool enable)
{
- return scmi_sensor_request_notify(handle, sensor_id,
+ return scmi_sensor_request_notify(ph, sensor_id,
SENSOR_TRIP_POINT_NOTIFY,
enable);
}
static int
-scmi_sensor_continuous_update_notify(const struct scmi_handle *handle,
+scmi_sensor_continuous_update_notify(const struct scmi_protocol_handle *ph,
u32 sensor_id, bool enable)
{
- return scmi_sensor_request_notify(handle, sensor_id,
+ return scmi_sensor_request_notify(ph, sensor_id,
SENSOR_CONTINUOUS_UPDATE_NOTIFY,
enable);
}
static int
-scmi_sensor_trip_point_config(const struct scmi_handle *handle, u32 sensor_id,
- u8 trip_id, u64 trip_value)
+scmi_sensor_trip_point_config(const struct scmi_protocol_handle *ph,
+ u32 sensor_id, u8 trip_id, u64 trip_value)
{
int ret;
u32 evt_cntl = SENSOR_TP_BOTH;
struct scmi_xfer *t;
struct scmi_msg_set_sensor_trip_point *trip;
- ret = scmi_xfer_get_init(handle, SENSOR_TRIP_POINT_CONFIG,
- SCMI_PROTOCOL_SENSOR, sizeof(*trip), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_TRIP_POINT_CONFIG,
+ sizeof(*trip), 0, &t);
if (ret)
return ret;
@@ -618,47 +619,46 @@ scmi_sensor_trip_point_config(const struct scmi_handle *handle, u32 sensor_id,
trip->value_low = cpu_to_le32(trip_value & 0xffffffff);
trip->value_high = cpu_to_le32(trip_value >> 32);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_sensor_config_get(const struct scmi_handle *handle,
+static int scmi_sensor_config_get(const struct scmi_protocol_handle *ph,
u32 sensor_id, u32 *sensor_config)
{
int ret;
struct scmi_xfer *t;
- ret = scmi_xfer_get_init(handle, SENSOR_CONFIG_GET,
- SCMI_PROTOCOL_SENSOR, sizeof(__le32),
- sizeof(__le32), &t);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_CONFIG_GET,
+ sizeof(__le32), sizeof(__le32), &t);
if (ret)
return ret;
put_unaligned_le32(cpu_to_le32(sensor_id), t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
- struct sensors_info *si = handle->sensor_priv;
+ struct sensors_info *si = ph->get_priv(ph);
struct scmi_sensor_info *s = si->sensors + sensor_id;
*sensor_config = get_unaligned_le64(t->rx.buf);
s->sensor_config = *sensor_config;
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_sensor_config_set(const struct scmi_handle *handle,
+static int scmi_sensor_config_set(const struct scmi_protocol_handle *ph,
u32 sensor_id, u32 sensor_config)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_sensor_config_set *msg;
- ret = scmi_xfer_get_init(handle, SENSOR_CONFIG_SET,
- SCMI_PROTOCOL_SENSOR, sizeof(*msg), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_CONFIG_SET,
+ sizeof(*msg), 0, &t);
if (ret)
return ret;
@@ -666,21 +666,21 @@ static int scmi_sensor_config_set(const struct scmi_handle *handle,
msg->id = cpu_to_le32(sensor_id);
msg->sensor_config = cpu_to_le32(sensor_config);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
- struct sensors_info *si = handle->sensor_priv;
+ struct sensors_info *si = ph->get_priv(ph);
struct scmi_sensor_info *s = si->sensors + sensor_id;
s->sensor_config = sensor_config;
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
/**
* scmi_sensor_reading_get - Read scalar sensor value
- * @handle: Platform handle
+ * @ph: Protocol handle
* @sensor_id: Sensor ID
* @value: The 64bit value sensor reading
*
@@ -693,17 +693,17 @@ static int scmi_sensor_config_set(const struct scmi_handle *handle,
*
* Return: 0 on Success
*/
-static int scmi_sensor_reading_get(const struct scmi_handle *handle,
+static int scmi_sensor_reading_get(const struct scmi_protocol_handle *ph,
u32 sensor_id, u64 *value)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_sensor_reading_get *sensor;
- struct sensors_info *si = handle->sensor_priv;
+ struct sensors_info *si = ph->get_priv(ph);
struct scmi_sensor_info *s = si->sensors + sensor_id;
- ret = scmi_xfer_get_init(handle, SENSOR_READING_GET,
- SCMI_PROTOCOL_SENSOR, sizeof(*sensor), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_READING_GET,
+ sizeof(*sensor), 0, &t);
if (ret)
return ret;
@@ -711,7 +711,7 @@ static int scmi_sensor_reading_get(const struct scmi_handle *handle,
sensor->id = cpu_to_le32(sensor_id);
if (s->async) {
sensor->flags = cpu_to_le32(SENSOR_READ_ASYNC);
- ret = scmi_do_xfer_with_response(handle, t);
+ ret = ph->xops->do_xfer_with_response(ph, t);
if (!ret) {
struct scmi_resp_sensor_reading_complete *resp;
@@ -723,12 +723,12 @@ static int scmi_sensor_reading_get(const struct scmi_handle *handle,
}
} else {
sensor->flags = cpu_to_le32(0);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
*value = get_unaligned_le64(t->rx.buf);
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -742,7 +742,7 @@ scmi_parse_sensor_readings(struct scmi_sensor_reading *out,
/**
* scmi_sensor_reading_get_timestamped - Read multiple-axis timestamped values
- * @handle: Platform handle
+ * @ph: Protocol handle
* @sensor_id: Sensor ID
* @count: The length of the provided @readings array
* @readings: An array of elements each representing a timestamped per-axis
@@ -755,22 +755,22 @@ scmi_parse_sensor_readings(struct scmi_sensor_reading *out,
* Return: 0 on Success
*/
static int
-scmi_sensor_reading_get_timestamped(const struct scmi_handle *handle,
+scmi_sensor_reading_get_timestamped(const struct scmi_protocol_handle *ph,
u32 sensor_id, u8 count,
struct scmi_sensor_reading *readings)
{
int ret;
struct scmi_xfer *t;
struct scmi_msg_sensor_reading_get *sensor;
- struct sensors_info *si = handle->sensor_priv;
+ struct sensors_info *si = ph->get_priv(ph);
struct scmi_sensor_info *s = si->sensors + sensor_id;
if (!count || !readings ||
(!s->num_axis && count > 1) || (s->num_axis && count > s->num_axis))
return -EINVAL;
- ret = scmi_xfer_get_init(handle, SENSOR_READING_GET,
- SCMI_PROTOCOL_SENSOR, sizeof(*sensor), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, SENSOR_READING_GET,
+ sizeof(*sensor), 0, &t);
if (ret)
return ret;
@@ -778,7 +778,7 @@ scmi_sensor_reading_get_timestamped(const struct scmi_handle *handle,
sensor->id = cpu_to_le32(sensor_id);
if (s->async) {
sensor->flags = cpu_to_le32(SENSOR_READ_ASYNC);
- ret = scmi_do_xfer_with_response(handle, t);
+ ret = ph->xops->do_xfer_with_response(ph, t);
if (!ret) {
int i;
struct scmi_resp_sensor_reading_complete_v3 *resp;
@@ -794,7 +794,7 @@ scmi_sensor_reading_get_timestamped(const struct scmi_handle *handle,
}
} else {
sensor->flags = cpu_to_le32(0);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret) {
int i;
struct scmi_sensor_reading_resp *resp_readings;
@@ -806,26 +806,26 @@ scmi_sensor_reading_get_timestamped(const struct scmi_handle *handle,
}
}
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
static const struct scmi_sensor_info *
-scmi_sensor_info_get(const struct scmi_handle *handle, u32 sensor_id)
+scmi_sensor_info_get(const struct scmi_protocol_handle *ph, u32 sensor_id)
{
- struct sensors_info *si = handle->sensor_priv;
+ struct sensors_info *si = ph->get_priv(ph);
return si->sensors + sensor_id;
}
-static int scmi_sensor_count_get(const struct scmi_handle *handle)
+static int scmi_sensor_count_get(const struct scmi_protocol_handle *ph)
{
- struct sensors_info *si = handle->sensor_priv;
+ struct sensors_info *si = ph->get_priv(ph);
return si->num_sensors;
}
-static const struct scmi_sensor_ops sensor_ops = {
+static const struct scmi_sensor_proto_ops sensor_proto_ops = {
.count_get = scmi_sensor_count_get,
.info_get = scmi_sensor_info_get,
.trip_point_config = scmi_sensor_trip_point_config,
@@ -835,18 +835,17 @@ static const struct scmi_sensor_ops sensor_ops = {
.config_set = scmi_sensor_config_set,
};
-static int scmi_sensor_set_notify_enabled(const struct scmi_handle *handle,
+static int scmi_sensor_set_notify_enabled(const struct scmi_protocol_handle *ph,
u8 evt_id, u32 src_id, bool enable)
{
int ret;
switch (evt_id) {
case SCMI_EVENT_SENSOR_TRIP_POINT_EVENT:
- ret = scmi_sensor_trip_point_notify(handle, src_id, enable);
+ ret = scmi_sensor_trip_point_notify(ph, src_id, enable);
break;
case SCMI_EVENT_SENSOR_UPDATE:
- ret = scmi_sensor_continuous_update_notify(handle, src_id,
- enable);
+ ret = scmi_sensor_continuous_update_notify(ph, src_id, enable);
break;
default:
ret = -EINVAL;
@@ -860,10 +859,11 @@ static int scmi_sensor_set_notify_enabled(const struct scmi_handle *handle,
return ret;
}
-static void *scmi_sensor_fill_custom_report(const struct scmi_handle *handle,
- u8 evt_id, ktime_t timestamp,
- const void *payld, size_t payld_sz,
- void *report, u32 *src_id)
+static void *
+scmi_sensor_fill_custom_report(const struct scmi_protocol_handle *ph,
+ u8 evt_id, ktime_t timestamp,
+ const void *payld, size_t payld_sz,
+ void *report, u32 *src_id)
{
void *rep = NULL;
@@ -890,7 +890,7 @@ static void *scmi_sensor_fill_custom_report(const struct scmi_handle *handle,
struct scmi_sensor_info *s;
const struct scmi_sensor_update_notify_payld *p = payld;
struct scmi_sensor_update_report *r = report;
- struct sensors_info *sinfo = handle->sensor_priv;
+ struct sensors_info *sinfo = ph->get_priv(ph);
/* payld_sz is variable for this event */
r->sensor_id = le32_to_cpu(p->sensor_id);
@@ -920,6 +920,13 @@ static void *scmi_sensor_fill_custom_report(const struct scmi_handle *handle,
return rep;
}
+static int scmi_sensor_get_num_sources(const struct scmi_protocol_handle *ph)
+{
+ struct sensors_info *si = ph->get_priv(ph);
+
+ return si->num_sensors;
+}
+
static const struct scmi_event sensor_events[] = {
{
.id = SCMI_EVENT_SENSOR_TRIP_POINT_EVENT,
@@ -939,48 +946,55 @@ static const struct scmi_event sensor_events[] = {
};
static const struct scmi_event_ops sensor_event_ops = {
+ .get_num_sources = scmi_sensor_get_num_sources,
.set_notify_enabled = scmi_sensor_set_notify_enabled,
.fill_custom_report = scmi_sensor_fill_custom_report,
};
-static int scmi_sensors_protocol_init(struct scmi_handle *handle)
+static const struct scmi_protocol_events sensor_protocol_events = {
+ .queue_sz = SCMI_PROTO_QUEUE_SZ,
+ .ops = &sensor_event_ops,
+ .evts = sensor_events,
+ .num_events = ARRAY_SIZE(sensor_events),
+};
+
+static int scmi_sensors_protocol_init(const struct scmi_protocol_handle *ph)
{
u32 version;
int ret;
struct sensors_info *sinfo;
- scmi_version_get(handle, SCMI_PROTOCOL_SENSOR, &version);
+ ph->xops->version_get(ph, &version);
- dev_dbg(handle->dev, "Sensor Version %d.%d\n",
+ dev_dbg(ph->dev, "Sensor Version %d.%d\n",
PROTOCOL_REV_MAJOR(version), PROTOCOL_REV_MINOR(version));
- sinfo = devm_kzalloc(handle->dev, sizeof(*sinfo), GFP_KERNEL);
+ sinfo = devm_kzalloc(ph->dev, sizeof(*sinfo), GFP_KERNEL);
if (!sinfo)
return -ENOMEM;
sinfo->version = version;
- ret = scmi_sensor_attributes_get(handle, sinfo);
+ ret = scmi_sensor_attributes_get(ph, sinfo);
if (ret)
return ret;
- sinfo->sensors = devm_kcalloc(handle->dev, sinfo->num_sensors,
+ sinfo->sensors = devm_kcalloc(ph->dev, sinfo->num_sensors,
sizeof(*sinfo->sensors), GFP_KERNEL);
if (!sinfo->sensors)
return -ENOMEM;
- ret = scmi_sensor_description_get(handle, sinfo);
+ ret = scmi_sensor_description_get(ph, sinfo);
if (ret)
return ret;
- scmi_register_protocol_events(handle,
- SCMI_PROTOCOL_SENSOR, SCMI_PROTO_QUEUE_SZ,
- &sensor_event_ops, sensor_events,
- ARRAY_SIZE(sensor_events),
- sinfo->num_sensors);
-
- handle->sensor_priv = sinfo;
- handle->sensor_ops = &sensor_ops;
-
- return 0;
+ return ph->set_priv(ph, sinfo);
}
-DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(SCMI_PROTOCOL_SENSOR, sensors)
+static const struct scmi_protocol scmi_sensors = {
+ .id = SCMI_PROTOCOL_SENSOR,
+ .owner = THIS_MODULE,
+ .instance_init = &scmi_sensors_protocol_init,
+ .ops = &sensor_proto_ops,
+ .events = &sensor_protocol_events,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(sensors, scmi_sensors)
diff --git a/drivers/firmware/arm_scmi/system.c b/drivers/firmware/arm_scmi/system.c
index 283e12d5f24b..e5175ef73b40 100644
--- a/drivers/firmware/arm_scmi/system.c
+++ b/drivers/firmware/arm_scmi/system.c
@@ -2,11 +2,12 @@
/*
* System Control and Management Interface (SCMI) System Power Protocol
*
- * Copyright (C) 2020 ARM Ltd.
+ * Copyright (C) 2020-2021 ARM Ltd.
*/
#define pr_fmt(fmt) "SCMI Notifications SYSTEM - " fmt
+#include <linux/module.h>
#include <linux/scmi_protocol.h>
#include "common.h"
@@ -32,43 +33,44 @@ struct scmi_system_info {
u32 version;
};
-static int scmi_system_request_notify(const struct scmi_handle *handle,
+static int scmi_system_request_notify(const struct scmi_protocol_handle *ph,
bool enable)
{
int ret;
struct scmi_xfer *t;
struct scmi_system_power_state_notify *notify;
- ret = scmi_xfer_get_init(handle, SYSTEM_POWER_STATE_NOTIFY,
- SCMI_PROTOCOL_SYSTEM, sizeof(*notify), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, SYSTEM_POWER_STATE_NOTIFY,
+ sizeof(*notify), 0, &t);
if (ret)
return ret;
notify = t->tx.buf;
notify->notify_enable = enable ? cpu_to_le32(BIT(0)) : 0;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_system_set_notify_enabled(const struct scmi_handle *handle,
+static int scmi_system_set_notify_enabled(const struct scmi_protocol_handle *ph,
u8 evt_id, u32 src_id, bool enable)
{
int ret;
- ret = scmi_system_request_notify(handle, enable);
+ ret = scmi_system_request_notify(ph, enable);
if (ret)
pr_debug("FAIL_ENABLE - evt[%X] - ret:%d\n", evt_id, ret);
return ret;
}
-static void *scmi_system_fill_custom_report(const struct scmi_handle *handle,
- u8 evt_id, ktime_t timestamp,
- const void *payld, size_t payld_sz,
- void *report, u32 *src_id)
+static void *
+scmi_system_fill_custom_report(const struct scmi_protocol_handle *ph,
+ u8 evt_id, ktime_t timestamp,
+ const void *payld, size_t payld_sz,
+ void *report, u32 *src_id)
{
const struct scmi_system_power_state_notifier_payld *p = payld;
struct scmi_system_power_state_notifier_report *r = report;
@@ -101,31 +103,38 @@ static const struct scmi_event_ops system_event_ops = {
.fill_custom_report = scmi_system_fill_custom_report,
};
-static int scmi_system_protocol_init(struct scmi_handle *handle)
+static const struct scmi_protocol_events system_protocol_events = {
+ .queue_sz = SCMI_PROTO_QUEUE_SZ,
+ .ops = &system_event_ops,
+ .evts = system_events,
+ .num_events = ARRAY_SIZE(system_events),
+ .num_sources = SCMI_SYSTEM_NUM_SOURCES,
+};
+
+static int scmi_system_protocol_init(const struct scmi_protocol_handle *ph)
{
u32 version;
struct scmi_system_info *pinfo;
- scmi_version_get(handle, SCMI_PROTOCOL_SYSTEM, &version);
+ ph->xops->version_get(ph, &version);
- dev_dbg(handle->dev, "System Power Version %d.%d\n",
+ dev_dbg(ph->dev, "System Power Version %d.%d\n",
PROTOCOL_REV_MAJOR(version), PROTOCOL_REV_MINOR(version));
- pinfo = devm_kzalloc(handle->dev, sizeof(*pinfo), GFP_KERNEL);
+ pinfo = devm_kzalloc(ph->dev, sizeof(*pinfo), GFP_KERNEL);
if (!pinfo)
return -ENOMEM;
- scmi_register_protocol_events(handle,
- SCMI_PROTOCOL_SYSTEM, SCMI_PROTO_QUEUE_SZ,
- &system_event_ops,
- system_events,
- ARRAY_SIZE(system_events),
- SCMI_SYSTEM_NUM_SOURCES);
-
pinfo->version = version;
- handle->system_priv = pinfo;
-
- return 0;
+ return ph->set_priv(ph, pinfo);
}
-DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(SCMI_PROTOCOL_SYSTEM, system)
+static const struct scmi_protocol scmi_system = {
+ .id = SCMI_PROTOCOL_SYSTEM,
+ .owner = THIS_MODULE,
+ .instance_init = &scmi_system_protocol_init,
+ .ops = NULL,
+ .events = &system_protocol_events,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(system, scmi_system)
diff --git a/drivers/firmware/arm_scmi/voltage.c b/drivers/firmware/arm_scmi/voltage.c
index e794e4349ae6..a5048956a0be 100644
--- a/drivers/firmware/arm_scmi/voltage.c
+++ b/drivers/firmware/arm_scmi/voltage.c
@@ -2,9 +2,10 @@
/*
* System Control and Management Interface (SCMI) Voltage Protocol
*
- * Copyright (C) 2020 ARM Ltd.
+ * Copyright (C) 2020-2021 ARM Ltd.
*/
+#include <linux/module.h>
#include <linux/scmi_protocol.h>
#include "common.h"
@@ -59,23 +60,23 @@ struct voltage_info {
struct scmi_voltage_info *domains;
};
-static int scmi_protocol_attributes_get(const struct scmi_handle *handle,
+static int scmi_protocol_attributes_get(const struct scmi_protocol_handle *ph,
struct voltage_info *vinfo)
{
int ret;
struct scmi_xfer *t;
- ret = scmi_xfer_get_init(handle, PROTOCOL_ATTRIBUTES,
- SCMI_PROTOCOL_VOLTAGE, 0, sizeof(__le32), &t);
+ ret = ph->xops->xfer_get_init(ph, PROTOCOL_ATTRIBUTES, 0,
+ sizeof(__le32), &t);
if (ret)
return ret;
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
vinfo->num_domains =
NUM_VOLTAGE_DOMAINS(get_unaligned_le32(t->rx.buf));
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
@@ -109,24 +110,23 @@ static int scmi_init_voltage_levels(struct device *dev,
return 0;
}
-static int scmi_voltage_descriptors_get(const struct scmi_handle *handle,
+static int scmi_voltage_descriptors_get(const struct scmi_protocol_handle *ph,
struct voltage_info *vinfo)
{
int ret, dom;
struct scmi_xfer *td, *tl;
- struct device *dev = handle->dev;
+ struct device *dev = ph->dev;
struct scmi_msg_resp_domain_attributes *resp_dom;
struct scmi_msg_resp_describe_levels *resp_levels;
- ret = scmi_xfer_get_init(handle, VOLTAGE_DOMAIN_ATTRIBUTES,
- SCMI_PROTOCOL_VOLTAGE, sizeof(__le32),
- sizeof(*resp_dom), &td);
+ ret = ph->xops->xfer_get_init(ph, VOLTAGE_DOMAIN_ATTRIBUTES,
+ sizeof(__le32), sizeof(*resp_dom), &td);
if (ret)
return ret;
resp_dom = td->rx.buf;
- ret = scmi_xfer_get_init(handle, VOLTAGE_DESCRIBE_LEVELS,
- SCMI_PROTOCOL_VOLTAGE, sizeof(__le64), 0, &tl);
+ ret = ph->xops->xfer_get_init(ph, VOLTAGE_DESCRIBE_LEVELS,
+ sizeof(__le64), 0, &tl);
if (ret)
goto outd;
resp_levels = tl->rx.buf;
@@ -139,7 +139,7 @@ static int scmi_voltage_descriptors_get(const struct scmi_handle *handle,
/* Retrieve domain attributes at first ... */
put_unaligned_le32(dom, td->tx.buf);
- ret = scmi_do_xfer(handle, td);
+ ret = ph->xops->do_xfer(ph, td);
/* Skip domain on comms error */
if (ret)
continue;
@@ -157,7 +157,7 @@ static int scmi_voltage_descriptors_get(const struct scmi_handle *handle,
cmd->domain_id = cpu_to_le32(v->id);
cmd->level_index = desc_index;
- ret = scmi_do_xfer(handle, tl);
+ ret = ph->xops->do_xfer(ph, tl);
if (ret)
break;
@@ -176,7 +176,7 @@ static int scmi_voltage_descriptors_get(const struct scmi_handle *handle,
}
if (desc_index + num_returned > v->num_levels) {
- dev_err(handle->dev,
+ dev_err(ph->dev,
"No. of voltage levels can't exceed %d\n",
v->num_levels);
ret = -EINVAL;
@@ -195,7 +195,7 @@ static int scmi_voltage_descriptors_get(const struct scmi_handle *handle,
desc_index += num_returned;
- scmi_reset_rx_to_maxsz(handle, tl);
+ ph->xops->reset_rx_to_maxsz(ph, tl);
/* check both to avoid infinite loop due to buggy fw */
} while (num_returned && num_remaining);
@@ -204,55 +204,52 @@ static int scmi_voltage_descriptors_get(const struct scmi_handle *handle,
devm_kfree(dev, v->levels_uv);
}
- scmi_reset_rx_to_maxsz(handle, td);
+ ph->xops->reset_rx_to_maxsz(ph, td);
}
- scmi_xfer_put(handle, tl);
+ ph->xops->xfer_put(ph, tl);
outd:
- scmi_xfer_put(handle, td);
+ ph->xops->xfer_put(ph, td);
return ret;
}
-static int __scmi_voltage_get_u32(const struct scmi_handle *handle,
+static int __scmi_voltage_get_u32(const struct scmi_protocol_handle *ph,
u8 cmd_id, u32 domain_id, u32 *value)
{
int ret;
struct scmi_xfer *t;
- struct voltage_info *vinfo = handle->voltage_priv;
+ struct voltage_info *vinfo = ph->get_priv(ph);
if (domain_id >= vinfo->num_domains)
return -EINVAL;
- ret = scmi_xfer_get_init(handle, cmd_id,
- SCMI_PROTOCOL_VOLTAGE,
- sizeof(__le32), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, cmd_id, sizeof(__le32), 0, &t);
if (ret)
return ret;
put_unaligned_le32(domain_id, t->tx.buf);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
if (!ret)
*value = get_unaligned_le32(t->rx.buf);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_voltage_config_set(const struct scmi_handle *handle,
+static int scmi_voltage_config_set(const struct scmi_protocol_handle *ph,
u32 domain_id, u32 config)
{
int ret;
struct scmi_xfer *t;
- struct voltage_info *vinfo = handle->voltage_priv;
+ struct voltage_info *vinfo = ph->get_priv(ph);
struct scmi_msg_cmd_config_set *cmd;
if (domain_id >= vinfo->num_domains)
return -EINVAL;
- ret = scmi_xfer_get_init(handle, VOLTAGE_CONFIG_SET,
- SCMI_PROTOCOL_VOLTAGE,
- sizeof(*cmd), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, VOLTAGE_CONFIG_SET,
+ sizeof(*cmd), 0, &t);
if (ret)
return ret;
@@ -260,33 +257,32 @@ static int scmi_voltage_config_set(const struct scmi_handle *handle,
cmd->domain_id = cpu_to_le32(domain_id);
cmd->config = cpu_to_le32(config & GENMASK(3, 0));
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_voltage_config_get(const struct scmi_handle *handle,
+static int scmi_voltage_config_get(const struct scmi_protocol_handle *ph,
u32 domain_id, u32 *config)
{
- return __scmi_voltage_get_u32(handle, VOLTAGE_CONFIG_GET,
+ return __scmi_voltage_get_u32(ph, VOLTAGE_CONFIG_GET,
domain_id, config);
}
-static int scmi_voltage_level_set(const struct scmi_handle *handle,
+static int scmi_voltage_level_set(const struct scmi_protocol_handle *ph,
u32 domain_id, u32 flags, s32 volt_uV)
{
int ret;
struct scmi_xfer *t;
- struct voltage_info *vinfo = handle->voltage_priv;
+ struct voltage_info *vinfo = ph->get_priv(ph);
struct scmi_msg_cmd_level_set *cmd;
if (domain_id >= vinfo->num_domains)
return -EINVAL;
- ret = scmi_xfer_get_init(handle, VOLTAGE_LEVEL_SET,
- SCMI_PROTOCOL_VOLTAGE,
- sizeof(*cmd), 0, &t);
+ ret = ph->xops->xfer_get_init(ph, VOLTAGE_LEVEL_SET,
+ sizeof(*cmd), 0, &t);
if (ret)
return ret;
@@ -295,23 +291,23 @@ static int scmi_voltage_level_set(const struct scmi_handle *handle,
cmd->flags = cpu_to_le32(flags);
cmd->voltage_level = cpu_to_le32(volt_uV);
- ret = scmi_do_xfer(handle, t);
+ ret = ph->xops->do_xfer(ph, t);
- scmi_xfer_put(handle, t);
+ ph->xops->xfer_put(ph, t);
return ret;
}
-static int scmi_voltage_level_get(const struct scmi_handle *handle,
+static int scmi_voltage_level_get(const struct scmi_protocol_handle *ph,
u32 domain_id, s32 *volt_uV)
{
- return __scmi_voltage_get_u32(handle, VOLTAGE_LEVEL_GET,
+ return __scmi_voltage_get_u32(ph, VOLTAGE_LEVEL_GET,
domain_id, (u32 *)volt_uV);
}
static const struct scmi_voltage_info * __must_check
-scmi_voltage_info_get(const struct scmi_handle *handle, u32 domain_id)
+scmi_voltage_info_get(const struct scmi_protocol_handle *ph, u32 domain_id)
{
- struct voltage_info *vinfo = handle->voltage_priv;
+ struct voltage_info *vinfo = ph->get_priv(ph);
if (domain_id >= vinfo->num_domains ||
!vinfo->domains[domain_id].num_levels)
@@ -320,14 +316,14 @@ scmi_voltage_info_get(const struct scmi_handle *handle, u32 domain_id)
return vinfo->domains + domain_id;
}
-static int scmi_voltage_domains_num_get(const struct scmi_handle *handle)
+static int scmi_voltage_domains_num_get(const struct scmi_protocol_handle *ph)
{
- struct voltage_info *vinfo = handle->voltage_priv;
+ struct voltage_info *vinfo = ph->get_priv(ph);
return vinfo->num_domains;
}
-static struct scmi_voltage_ops voltage_ops = {
+static struct scmi_voltage_proto_ops voltage_proto_ops = {
.num_domains_get = scmi_voltage_domains_num_get,
.info_get = scmi_voltage_info_get,
.config_set = scmi_voltage_config_set,
@@ -336,45 +332,49 @@ static struct scmi_voltage_ops voltage_ops = {
.level_get = scmi_voltage_level_get,
};
-static int scmi_voltage_protocol_init(struct scmi_handle *handle)
+static int scmi_voltage_protocol_init(const struct scmi_protocol_handle *ph)
{
int ret;
u32 version;
struct voltage_info *vinfo;
- ret = scmi_version_get(handle, SCMI_PROTOCOL_VOLTAGE, &version);
+ ret = ph->xops->version_get(ph, &version);
if (ret)
return ret;
- dev_dbg(handle->dev, "Voltage Version %d.%d\n",
+ dev_dbg(ph->dev, "Voltage Version %d.%d\n",
PROTOCOL_REV_MAJOR(version), PROTOCOL_REV_MINOR(version));
- vinfo = devm_kzalloc(handle->dev, sizeof(*vinfo), GFP_KERNEL);
+ vinfo = devm_kzalloc(ph->dev, sizeof(*vinfo), GFP_KERNEL);
if (!vinfo)
return -ENOMEM;
vinfo->version = version;
- ret = scmi_protocol_attributes_get(handle, vinfo);
+ ret = scmi_protocol_attributes_get(ph, vinfo);
if (ret)
return ret;
if (vinfo->num_domains) {
- vinfo->domains = devm_kcalloc(handle->dev, vinfo->num_domains,
+ vinfo->domains = devm_kcalloc(ph->dev, vinfo->num_domains,
sizeof(*vinfo->domains),
GFP_KERNEL);
if (!vinfo->domains)
return -ENOMEM;
- ret = scmi_voltage_descriptors_get(handle, vinfo);
+ ret = scmi_voltage_descriptors_get(ph, vinfo);
if (ret)
return ret;
} else {
- dev_warn(handle->dev, "No Voltage domains found.\n");
+ dev_warn(ph->dev, "No Voltage domains found.\n");
}
- handle->voltage_ops = &voltage_ops;
- handle->voltage_priv = vinfo;
-
- return 0;
+ return ph->set_priv(ph, vinfo);
}
-DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(SCMI_PROTOCOL_VOLTAGE, voltage)
+static const struct scmi_protocol scmi_voltage = {
+ .id = SCMI_PROTOCOL_VOLTAGE,
+ .owner = THIS_MODULE,
+ .instance_init = &scmi_voltage_protocol_init,
+ .ops = &voltage_proto_ops,
+};
+
+DEFINE_SCMI_PROTOCOL_REGISTER_UNREGISTER(voltage, scmi_voltage)
diff --git a/drivers/firmware/broadcom/bcm47xx_nvram.c b/drivers/firmware/broadcom/bcm47xx_nvram.c
index 835ece9c00f1..bd235833b687 100644
--- a/drivers/firmware/broadcom/bcm47xx_nvram.c
+++ b/drivers/firmware/broadcom/bcm47xx_nvram.c
@@ -34,26 +34,46 @@ static char nvram_buf[NVRAM_SPACE];
static size_t nvram_len;
static const u32 nvram_sizes[] = {0x6000, 0x8000, 0xF000, 0x10000};
-static u32 find_nvram_size(void __iomem *end)
+/**
+ * bcm47xx_nvram_is_valid - check for a valid NVRAM at specified memory
+ */
+static bool bcm47xx_nvram_is_valid(void __iomem *nvram)
{
- struct nvram_header __iomem *header;
- int i;
+ return ((struct nvram_header *)nvram)->magic == NVRAM_MAGIC;
+}
+
+/**
+ * bcm47xx_nvram_copy - copy NVRAM to internal buffer
+ */
+static void bcm47xx_nvram_copy(void __iomem *nvram_start, size_t res_size)
+{
+ struct nvram_header __iomem *header = nvram_start;
+ size_t copy_size;
- for (i = 0; i < ARRAY_SIZE(nvram_sizes); i++) {
- header = (struct nvram_header *)(end - nvram_sizes[i]);
- if (header->magic == NVRAM_MAGIC)
- return nvram_sizes[i];
+ copy_size = header->len;
+ if (copy_size > res_size) {
+ pr_err("The nvram size according to the header seems to be bigger than the partition on flash\n");
+ copy_size = res_size;
+ }
+ if (copy_size >= NVRAM_SPACE) {
+ pr_err("nvram on flash (%zu bytes) is bigger than the reserved space in memory, will just copy the first %i bytes\n",
+ copy_size, NVRAM_SPACE - 1);
+ copy_size = NVRAM_SPACE - 1;
}
- return 0;
+ __ioread32_copy(nvram_buf, nvram_start, DIV_ROUND_UP(copy_size, 4));
+ nvram_buf[NVRAM_SPACE - 1] = '\0';
+ nvram_len = copy_size;
}
-/* Probe for NVRAM header */
-static int nvram_find_and_copy(void __iomem *iobase, u32 lim)
+/**
+ * bcm47xx_nvram_find_and_copy - find NVRAM on flash mapping & copy it
+ */
+static int bcm47xx_nvram_find_and_copy(void __iomem *flash_start, size_t res_size)
{
- struct nvram_header __iomem *header;
- u32 off;
- u32 size;
+ size_t flash_size;
+ size_t offset;
+ int i;
if (nvram_len) {
pr_warn("nvram already initialized\n");
@@ -61,49 +81,31 @@ static int nvram_find_and_copy(void __iomem *iobase, u32 lim)
}
/* TODO: when nvram is on nand flash check for bad blocks first. */
- off = FLASH_MIN;
- while (off <= lim) {
- /* Windowed flash access */
- size = find_nvram_size(iobase + off);
- if (size) {
- header = (struct nvram_header *)(iobase + off - size);
- goto found;
+
+ /* Try every possible flash size and check for NVRAM at its end */
+ for (flash_size = FLASH_MIN; flash_size <= res_size; flash_size <<= 1) {
+ for (i = 0; i < ARRAY_SIZE(nvram_sizes); i++) {
+ offset = flash_size - nvram_sizes[i];
+ if (bcm47xx_nvram_is_valid(flash_start + offset))
+ goto found;
}
- off <<= 1;
}
/* Try embedded NVRAM at 4 KB and 1 KB as last resorts */
- header = (struct nvram_header *)(iobase + 4096);
- if (header->magic == NVRAM_MAGIC) {
- size = NVRAM_SPACE;
+
+ offset = 4096;
+ if (bcm47xx_nvram_is_valid(flash_start + offset))
goto found;
- }
- header = (struct nvram_header *)(iobase + 1024);
- if (header->magic == NVRAM_MAGIC) {
- size = NVRAM_SPACE;
+ offset = 1024;
+ if (bcm47xx_nvram_is_valid(flash_start + offset))
goto found;
- }
pr_err("no nvram found\n");
return -ENXIO;
found:
- __ioread32_copy(nvram_buf, header, sizeof(*header) / 4);
- nvram_len = ((struct nvram_header *)(nvram_buf))->len;
- if (nvram_len > size) {
- pr_err("The nvram size according to the header seems to be bigger than the partition on flash\n");
- nvram_len = size;
- }
- if (nvram_len >= NVRAM_SPACE) {
- pr_err("nvram on flash (%zu bytes) is bigger than the reserved space in memory, will just copy the first %i bytes\n",
- nvram_len, NVRAM_SPACE - 1);
- nvram_len = NVRAM_SPACE - 1;
- }
- /* proceed reading data after header */
- __ioread32_copy(nvram_buf + sizeof(*header), header + 1,
- DIV_ROUND_UP(nvram_len, 4));
- nvram_buf[NVRAM_SPACE - 1] = '\0';
+ bcm47xx_nvram_copy(flash_start + offset, res_size - offset);
return 0;
}
@@ -124,7 +126,7 @@ int bcm47xx_nvram_init_from_mem(u32 base, u32 lim)
if (!iobase)
return -ENOMEM;
- err = nvram_find_and_copy(iobase, lim);
+ err = bcm47xx_nvram_find_and_copy(iobase, lim);
iounmap(iobase);
diff --git a/drivers/firmware/dmi-id.c b/drivers/firmware/dmi-id.c
index 86d71b0212b1..4d5421d14a41 100644
--- a/drivers/firmware/dmi-id.c
+++ b/drivers/firmware/dmi-id.c
@@ -85,6 +85,7 @@ static ssize_t get_modalias(char *buffer, size_t buffer_size)
{ "svn", DMI_SYS_VENDOR },
{ "pn", DMI_PRODUCT_NAME },
{ "pvr", DMI_PRODUCT_VERSION },
+ { "sku", DMI_PRODUCT_SKU },
{ "rvn", DMI_BOARD_VENDOR },
{ "rn", DMI_BOARD_NAME },
{ "rvr", DMI_BOARD_VERSION },
diff --git a/drivers/firmware/dmi_scan.c b/drivers/firmware/dmi_scan.c
index d51ca0428bb8..f191a1f901ac 100644
--- a/drivers/firmware/dmi_scan.c
+++ b/drivers/firmware/dmi_scan.c
@@ -166,6 +166,7 @@ static int __init dmi_checksum(const u8 *buf, u8 len)
static const char *dmi_ident[DMI_STRING_MAX];
static LIST_HEAD(dmi_devices);
int dmi_available;
+EXPORT_SYMBOL_GPL(dmi_available);
/*
* Save a DMI string
diff --git a/drivers/firmware/efi/efi.c b/drivers/firmware/efi/efi.c
index df3f9bcab581..4b7ee3fa9224 100644
--- a/drivers/firmware/efi/efi.c
+++ b/drivers/firmware/efi/efi.c
@@ -927,7 +927,7 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
}
/* first try to find a slot in an existing linked list entry */
- for (prsv = efi_memreserve_root->next; prsv; prsv = rsv->next) {
+ for (prsv = efi_memreserve_root->next; prsv; ) {
rsv = memremap(prsv, sizeof(*rsv), MEMREMAP_WB);
index = atomic_fetch_add_unless(&rsv->count, 1, rsv->size);
if (index < rsv->size) {
@@ -937,6 +937,7 @@ int __ref efi_mem_reserve_persistent(phys_addr_t addr, u64 size)
memunmap(rsv);
return efi_mem_reserve_iomem(addr, size);
}
+ prsv = rsv->next;
memunmap(rsv);
}
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index c23466e05e60..d0537573501e 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -13,7 +13,8 @@ cflags-$(CONFIG_X86) += -m$(BITS) -D__KERNEL__ \
-Wno-pointer-sign \
$(call cc-disable-warning, address-of-packed-member) \
$(call cc-disable-warning, gnu) \
- -fno-asynchronous-unwind-tables
+ -fno-asynchronous-unwind-tables \
+ $(CLANG_FLAGS)
# arm64 uses the full KBUILD_CFLAGS so it's necessary to explicitly
# disable the stackleak plugin
diff --git a/drivers/firmware/efi/libstub/arm64-stub.c b/drivers/firmware/efi/libstub/arm64-stub.c
index b69d63143e0d..7bf0a7acae5e 100644
--- a/drivers/firmware/efi/libstub/arm64-stub.c
+++ b/drivers/firmware/efi/libstub/arm64-stub.c
@@ -24,7 +24,7 @@ efi_status_t check_platform_features(void)
return EFI_SUCCESS;
tg = (read_cpuid(ID_AA64MMFR0_EL1) >> ID_AA64MMFR0_TGRAN_SHIFT) & 0xf;
- if (tg != ID_AA64MMFR0_TGRAN_SUPPORTED) {
+ if (tg < ID_AA64MMFR0_TGRAN_SUPPORTED_MIN || tg > ID_AA64MMFR0_TGRAN_SUPPORTED_MAX) {
if (IS_ENABLED(CONFIG_ARM64_64K_PAGES))
efi_err("This 64 KB granular kernel is not supported by your CPU\n");
else
diff --git a/drivers/firmware/efi/libstub/efi-stub.c b/drivers/firmware/efi/libstub/efi-stub.c
index ec2f3985bef3..26e69788f27a 100644
--- a/drivers/firmware/efi/libstub/efi-stub.c
+++ b/drivers/firmware/efi/libstub/efi-stub.c
@@ -96,6 +96,18 @@ static void install_memreserve_table(void)
efi_err("Failed to install memreserve config table!\n");
}
+static u32 get_supported_rt_services(void)
+{
+ const efi_rt_properties_table_t *rt_prop_table;
+ u32 supported = EFI_RT_SUPPORTED_ALL;
+
+ rt_prop_table = get_efi_config_table(EFI_RT_PROPERTIES_TABLE_GUID);
+ if (rt_prop_table)
+ supported &= rt_prop_table->runtime_services_supported;
+
+ return supported;
+}
+
/*
* EFI entry point for the arm/arm64 EFI stubs. This is the entrypoint
* that is described in the PE/COFF header. Most of the code is the same
@@ -250,6 +262,10 @@ efi_status_t __efiapi efi_pe_entry(efi_handle_t handle,
(prop_tbl->memory_protection_attribute &
EFI_PROPERTIES_RUNTIME_MEMORY_PROTECTION_NON_EXECUTABLE_PE_DATA);
+ /* force efi_novamap if SetVirtualAddressMap() is unsupported */
+ efi_novamap |= !(get_supported_rt_services() &
+ EFI_RT_SUPPORTED_SET_VIRTUAL_ADDRESS_MAP);
+
/* hibernation expects the runtime regions to stay in the same place */
if (!IS_ENABLED(CONFIG_HIBERNATION) && !efi_nokaslr && !flat_va_mapping) {
/*
diff --git a/drivers/firmware/efi/vars.c b/drivers/firmware/efi/vars.c
index 41c1d00bf933..abdc8a6a3963 100644
--- a/drivers/firmware/efi/vars.c
+++ b/drivers/firmware/efi/vars.c
@@ -485,6 +485,10 @@ int efivar_init(int (*func)(efi_char16_t *, efi_guid_t, unsigned long, void *),
}
break;
+ case EFI_UNSUPPORTED:
+ err = -EOPNOTSUPP;
+ status = EFI_NOT_FOUND;
+ break;
case EFI_NOT_FOUND:
break;
default:
diff --git a/drivers/firmware/google/gsmi.c b/drivers/firmware/google/gsmi.c
index 3d77f26c1e8c..bb6e77ee3898 100644
--- a/drivers/firmware/google/gsmi.c
+++ b/drivers/firmware/google/gsmi.c
@@ -136,12 +136,16 @@ MODULE_PARM_DESC(spincount,
"The number of loop iterations to use when using the spin handshake.");
/*
- * Platforms might not support S0ix logging in their GSMI handlers. In order to
- * avoid any side-effects of generating an SMI for S0ix logging, use the S0ix
- * related GSMI commands only for those platforms that explicitly enable this
- * option.
+ * Some older platforms with Apollo Lake chipsets do not support S0ix logging
+ * in their GSMI handlers, and behaved poorly when resuming via power button
+ * press if the logging was attempted. Updated firmware with proper behavior
+ * has long since shipped, removing the need for this opt-in parameter. It
+ * now exists as an opt-out parameter for folks defiantly running old
+ * firmware, or unforeseen circumstances. After the change from opt-in to
+ * opt-out has baked sufficiently, this parameter should probably be removed
+ * entirely.
*/
-static bool s0ix_logging_enable;
+static bool s0ix_logging_enable = true;
module_param(s0ix_logging_enable, bool, 0600);
static struct gsmi_buf *gsmi_buf_alloc(void)
diff --git a/drivers/firmware/imx/scu-pd.c b/drivers/firmware/imx/scu-pd.c
index 08533ee67626..ff6569c4a53b 100644
--- a/drivers/firmware/imx/scu-pd.c
+++ b/drivers/firmware/imx/scu-pd.c
@@ -29,6 +29,10 @@
* The framework needs some proper extension to support multi power
* domain cases.
*
+ * Update: Genpd assigns the ->of_node for the virtual device before it
+ * invokes ->attach_dev() callback, hence parsing for device resources via
+ * DT should work fine.
+ *
* 2. It also breaks most of current drivers as the driver probe sequence
* behavior changed if removing ->power_on|off() callback and use
* ->start() and ->stop() instead. genpd_dev_pm_attach will only power
@@ -39,8 +43,11 @@
* domain enabled will trigger a HW access error. That means we need fix
* most drivers probe sequence with proper runtime pm.
*
- * In summary, we need fix above two issue before being able to switch to
- * the "single global power domain" way.
+ * Update: Runtime PM support isn't necessary. Instead, this can easily be
+ * fixed in drivers by adding a call to dev_pm_domain_start() during probe.
+ *
+ * In summary, the second part needs to be addressed via minor updates to the
+ * relevant drivers, before the "single global power domain" model can be used.
*
*/
@@ -86,6 +93,8 @@ struct imx_sc_pd_soc {
u8 num_ranges;
};
+static int imx_con_rsrc;
+
static const struct imx_sc_pd_range imx8qxp_scu_pd_ranges[] = {
/* LSIO SS */
{ "pwm", IMX_SC_R_PWM_0, 8, true, 0 },
@@ -134,7 +143,7 @@ static const struct imx_sc_pd_range imx8qxp_scu_pd_ranges[] = {
{ "can", IMX_SC_R_CAN_0, 3, true, 0 },
{ "ftm", IMX_SC_R_FTM_0, 2, true, 0 },
{ "lpi2c", IMX_SC_R_I2C_0, 4, true, 0 },
- { "adc", IMX_SC_R_ADC_0, 1, true, 0 },
+ { "adc", IMX_SC_R_ADC_0, 2, true, 0 },
{ "lcd", IMX_SC_R_LCD_0, 1, true, 0 },
{ "lcd0-pwm", IMX_SC_R_LCD_0_PWM_0, 1, true, 0 },
{ "lpuart", IMX_SC_R_UART_0, 4, true, 0 },
@@ -207,6 +216,23 @@ to_imx_sc_pd(struct generic_pm_domain *genpd)
return container_of(genpd, struct imx_sc_pm_domain, pd);
}
+static void imx_sc_pd_get_console_rsrc(void)
+{
+ struct of_phandle_args specs;
+ int ret;
+
+ if (!of_stdout)
+ return;
+
+ ret = of_parse_phandle_with_args(of_stdout, "power-domains",
+ "#power-domain-cells",
+ 0, &specs);
+ if (ret)
+ return;
+
+ imx_con_rsrc = specs.args[0];
+}
+
static int imx_sc_pd_power(struct generic_pm_domain *domain, bool power_on)
{
struct imx_sc_msg_req_set_resource_power_mode msg;
@@ -267,6 +293,7 @@ imx_scu_add_pm_domain(struct device *dev, int idx,
const struct imx_sc_pd_range *pd_ranges)
{
struct imx_sc_pm_domain *sc_pd;
+ bool is_off = true;
int ret;
if (!imx_sc_rm_is_resource_owned(pm_ipc_handle, pd_ranges->rsrc + idx))
@@ -288,6 +315,10 @@ imx_scu_add_pm_domain(struct device *dev, int idx,
"%s", pd_ranges->name);
sc_pd->pd.name = sc_pd->name;
+ if (imx_con_rsrc == sc_pd->rsrc) {
+ sc_pd->pd.flags = GENPD_FLAG_RPM_ALWAYS_ON;
+ is_off = false;
+ }
if (sc_pd->rsrc >= IMX_SC_R_LAST) {
dev_warn(dev, "invalid pd %s rsrc id %d found",
@@ -297,7 +328,7 @@ imx_scu_add_pm_domain(struct device *dev, int idx,
return NULL;
}
- ret = pm_genpd_init(&sc_pd->pd, NULL, true);
+ ret = pm_genpd_init(&sc_pd->pd, NULL, is_off);
if (ret) {
dev_warn(dev, "failed to init pd %s rsrc id %d",
sc_pd->name, sc_pd->rsrc);
@@ -363,6 +394,8 @@ static int imx_sc_pd_probe(struct platform_device *pdev)
if (!pd_soc)
return -ENODEV;
+ imx_sc_pd_get_console_rsrc();
+
return imx_scu_init_pm_domains(&pdev->dev, pd_soc);
}
diff --git a/drivers/firmware/psci/psci.c b/drivers/firmware/psci/psci.c
index f5fc429cae3f..3c1c5daf6df2 100644
--- a/drivers/firmware/psci/psci.c
+++ b/drivers/firmware/psci/psci.c
@@ -23,6 +23,7 @@
#include <asm/cpuidle.h>
#include <asm/cputype.h>
+#include <asm/hypervisor.h>
#include <asm/system_misc.h>
#include <asm/smp_plat.h>
#include <asm/suspend.h>
@@ -138,7 +139,7 @@ static int psci_to_linux_errno(int errno)
return -EINVAL;
case PSCI_RET_DENIED:
return -EPERM;
- };
+ }
return -EINVAL;
}
@@ -325,8 +326,9 @@ static int __init psci_features(u32 psci_func_id)
static int psci_suspend_finisher(unsigned long state)
{
u32 power_state = state;
+ phys_addr_t pa_cpu_resume = __pa_symbol(function_nocfi(cpu_resume));
- return psci_ops.cpu_suspend(power_state, __pa_symbol(cpu_resume));
+ return psci_ops.cpu_suspend(power_state, pa_cpu_resume);
}
int psci_cpu_suspend_enter(u32 state)
@@ -344,8 +346,10 @@ int psci_cpu_suspend_enter(u32 state)
static int psci_system_suspend(unsigned long unused)
{
+ phys_addr_t pa_cpu_resume = __pa_symbol(function_nocfi(cpu_resume));
+
return invoke_psci_fn(PSCI_FN_NATIVE(1_0, SYSTEM_SUSPEND),
- __pa_symbol(cpu_resume), 0, 0);
+ pa_cpu_resume, 0, 0);
}
static int psci_system_suspend_enter(suspend_state_t state)
@@ -498,6 +502,7 @@ static int __init psci_probe(void)
psci_init_cpu_suspend();
psci_init_system_suspend();
psci_init_system_reset2();
+ kvm_init_hyp_services();
}
return 0;
diff --git a/drivers/firmware/qcom_scm-legacy.c b/drivers/firmware/qcom_scm-legacy.c
index eba6b60bfb61..1829ba220576 100644
--- a/drivers/firmware/qcom_scm-legacy.c
+++ b/drivers/firmware/qcom_scm-legacy.c
@@ -118,7 +118,7 @@ static void __scm_legacy_do(const struct arm_smccc_args *smc,
}
/**
- * qcom_scm_call() - Sends a command to the SCM and waits for the command to
+ * scm_legacy_call() - Sends a command to the SCM and waits for the command to
* finish processing.
*
* A note on cache maintenance:
@@ -209,7 +209,7 @@ out:
(n & 0xf))
/**
- * qcom_scm_call_atomic() - Send an atomic SCM command with up to 5 arguments
+ * scm_legacy_call_atomic() - Send an atomic SCM command with up to 5 arguments
* and 3 return values
* @desc: SCM call descriptor containing arguments
* @res: SCM call return values
diff --git a/drivers/firmware/qcom_scm-smc.c b/drivers/firmware/qcom_scm-smc.c
index 497c13ba98d6..d111833364ba 100644
--- a/drivers/firmware/qcom_scm-smc.c
+++ b/drivers/firmware/qcom_scm-smc.c
@@ -77,8 +77,10 @@ static void __scm_smc_do(const struct arm_smccc_args *smc,
} while (res->a0 == QCOM_SCM_V2_EBUSY);
}
-int scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
- struct qcom_scm_res *res, bool atomic)
+
+int __scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
+ enum qcom_scm_convention qcom_convention,
+ struct qcom_scm_res *res, bool atomic)
{
int arglen = desc->arginfo & 0xf;
int i;
@@ -87,9 +89,8 @@ int scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
size_t alloc_len;
gfp_t flag = atomic ? GFP_ATOMIC : GFP_KERNEL;
u32 smccc_call_type = atomic ? ARM_SMCCC_FAST_CALL : ARM_SMCCC_STD_CALL;
- u32 qcom_smccc_convention =
- (qcom_scm_convention == SMC_CONVENTION_ARM_32) ?
- ARM_SMCCC_SMC_32 : ARM_SMCCC_SMC_64;
+ u32 qcom_smccc_convention = (qcom_convention == SMC_CONVENTION_ARM_32) ?
+ ARM_SMCCC_SMC_32 : ARM_SMCCC_SMC_64;
struct arm_smccc_res smc_res;
struct arm_smccc_args smc = {0};
@@ -148,4 +149,5 @@ int scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
}
return (long)smc_res.a0 ? qcom_scm_remap_error(smc_res.a0) : 0;
+
}
diff --git a/drivers/firmware/qcom_scm.c b/drivers/firmware/qcom_scm.c
index f57779fc7ee9..ee9cb545e73b 100644
--- a/drivers/firmware/qcom_scm.c
+++ b/drivers/firmware/qcom_scm.c
@@ -113,14 +113,10 @@ static void qcom_scm_clk_disable(void)
clk_disable_unprepare(__scm->bus_clk);
}
-static int __qcom_scm_is_call_available(struct device *dev, u32 svc_id,
- u32 cmd_id);
+enum qcom_scm_convention qcom_scm_convention = SMC_CONVENTION_UNKNOWN;
+static DEFINE_SPINLOCK(scm_query_lock);
-enum qcom_scm_convention qcom_scm_convention;
-static bool has_queried __read_mostly;
-static DEFINE_SPINLOCK(query_lock);
-
-static void __query_convention(void)
+static enum qcom_scm_convention __get_convention(void)
{
unsigned long flags;
struct qcom_scm_desc desc = {
@@ -133,36 +129,50 @@ static void __query_convention(void)
.owner = ARM_SMCCC_OWNER_SIP,
};
struct qcom_scm_res res;
+ enum qcom_scm_convention probed_convention;
int ret;
+ bool forced = false;
- spin_lock_irqsave(&query_lock, flags);
- if (has_queried)
- goto out;
+ if (likely(qcom_scm_convention != SMC_CONVENTION_UNKNOWN))
+ return qcom_scm_convention;
- qcom_scm_convention = SMC_CONVENTION_ARM_64;
- // Device isn't required as there is only one argument - no device
- // needed to dma_map_single to secure world
- ret = scm_smc_call(NULL, &desc, &res, true);
+ /*
+ * Device isn't required as there is only one argument - no device
+ * needed to dma_map_single to secure world
+ */
+ probed_convention = SMC_CONVENTION_ARM_64;
+ ret = __scm_smc_call(NULL, &desc, probed_convention, &res, true);
if (!ret && res.result[0] == 1)
- goto out;
+ goto found;
+
+ /*
+ * Some SC7180 firmwares didn't implement the
+ * QCOM_SCM_INFO_IS_CALL_AVAIL call, so we fallback to forcing ARM_64
+ * calling conventions on these firmwares. Luckily we don't make any
+ * early calls into the firmware on these SoCs so the device pointer
+ * will be valid here to check if the compatible matches.
+ */
+ if (of_device_is_compatible(__scm ? __scm->dev->of_node : NULL, "qcom,scm-sc7180")) {
+ forced = true;
+ goto found;
+ }
- qcom_scm_convention = SMC_CONVENTION_ARM_32;
- ret = scm_smc_call(NULL, &desc, &res, true);
+ probed_convention = SMC_CONVENTION_ARM_32;
+ ret = __scm_smc_call(NULL, &desc, probed_convention, &res, true);
if (!ret && res.result[0] == 1)
- goto out;
-
- qcom_scm_convention = SMC_CONVENTION_LEGACY;
-out:
- has_queried = true;
- spin_unlock_irqrestore(&query_lock, flags);
- pr_info("qcom_scm: convention: %s\n",
- qcom_scm_convention_names[qcom_scm_convention]);
-}
+ goto found;
+
+ probed_convention = SMC_CONVENTION_LEGACY;
+found:
+ spin_lock_irqsave(&scm_query_lock, flags);
+ if (probed_convention != qcom_scm_convention) {
+ qcom_scm_convention = probed_convention;
+ pr_info("qcom_scm: convention: %s%s\n",
+ qcom_scm_convention_names[qcom_scm_convention],
+ forced ? " (forced)" : "");
+ }
+ spin_unlock_irqrestore(&scm_query_lock, flags);
-static inline enum qcom_scm_convention __get_convention(void)
-{
- if (unlikely(!has_queried))
- __query_convention();
return qcom_scm_convention;
}
@@ -219,8 +229,8 @@ static int qcom_scm_call_atomic(struct device *dev,
}
}
-static int __qcom_scm_is_call_available(struct device *dev, u32 svc_id,
- u32 cmd_id)
+static bool __qcom_scm_is_call_available(struct device *dev, u32 svc_id,
+ u32 cmd_id)
{
int ret;
struct qcom_scm_desc desc = {
@@ -247,7 +257,7 @@ static int __qcom_scm_is_call_available(struct device *dev, u32 svc_id,
ret = qcom_scm_call(dev, &desc, &res);
- return ret ? : res.result[0];
+ return ret ? false : !!res.result[0];
}
/**
@@ -585,9 +595,8 @@ bool qcom_scm_pas_supported(u32 peripheral)
};
struct qcom_scm_res res;
- ret = __qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_PIL,
- QCOM_SCM_PIL_PAS_IS_SUPPORTED);
- if (ret <= 0)
+ if (!__qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_PIL,
+ QCOM_SCM_PIL_PAS_IS_SUPPORTED))
return false;
ret = qcom_scm_call(__scm->dev, &desc, &res);
@@ -1060,17 +1069,18 @@ EXPORT_SYMBOL(qcom_scm_ice_set_key);
*/
bool qcom_scm_hdcp_available(void)
{
+ bool avail;
int ret = qcom_scm_clk_enable();
if (ret)
return ret;
- ret = __qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_HDCP,
+ avail = __qcom_scm_is_call_available(__scm->dev, QCOM_SCM_SVC_HDCP,
QCOM_SCM_HDCP_INVOKE);
qcom_scm_clk_disable();
- return ret > 0;
+ return avail;
}
EXPORT_SYMBOL(qcom_scm_hdcp_available);
@@ -1242,7 +1252,7 @@ static int qcom_scm_probe(struct platform_device *pdev)
__scm = scm;
__scm->dev = &pdev->dev;
- __query_convention();
+ __get_convention();
/*
* If requested enable "download mode", from this point on warmboot
@@ -1291,6 +1301,7 @@ static struct platform_driver qcom_scm_driver = {
.driver = {
.name = "qcom_scm",
.of_match_table = qcom_scm_dt_match,
+ .suppress_bind_attrs = true,
},
.probe = qcom_scm_probe,
.shutdown = qcom_scm_shutdown,
diff --git a/drivers/firmware/qcom_scm.h b/drivers/firmware/qcom_scm.h
index 95cd1ac30ab0..632fe3142462 100644
--- a/drivers/firmware/qcom_scm.h
+++ b/drivers/firmware/qcom_scm.h
@@ -61,8 +61,11 @@ struct qcom_scm_res {
};
#define SCM_SMC_FNID(s, c) ((((s) & 0xFF) << 8) | ((c) & 0xFF))
-extern int scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
- struct qcom_scm_res *res, bool atomic);
+extern int __scm_smc_call(struct device *dev, const struct qcom_scm_desc *desc,
+ enum qcom_scm_convention qcom_convention,
+ struct qcom_scm_res *res, bool atomic);
+#define scm_smc_call(dev, desc, res, atomic) \
+ __scm_smc_call((dev), (desc), qcom_scm_convention, (res), (atomic))
#define SCM_LEGACY_FNID(s, c) (((s) << 10) | ((c) & 0x3ff))
extern int scm_legacy_call_atomic(struct device *dev,
diff --git a/drivers/firmware/raspberrypi.c b/drivers/firmware/raspberrypi.c
index 30259dc9b805..250e01680742 100644
--- a/drivers/firmware/raspberrypi.c
+++ b/drivers/firmware/raspberrypi.c
@@ -7,6 +7,7 @@
*/
#include <linux/dma-mapping.h>
+#include <linux/kref.h>
#include <linux/mailbox_client.h>
#include <linux/module.h>
#include <linux/of_platform.h>
@@ -27,6 +28,8 @@ struct rpi_firmware {
struct mbox_chan *chan; /* The property channel. */
struct completion c;
u32 enabled;
+
+ struct kref consumers;
};
static DEFINE_MUTEX(transaction_lock);
@@ -225,12 +228,38 @@ static void rpi_register_clk_driver(struct device *dev)
-1, NULL, 0);
}
+static void rpi_firmware_delete(struct kref *kref)
+{
+ struct rpi_firmware *fw = container_of(kref, struct rpi_firmware,
+ consumers);
+
+ mbox_free_channel(fw->chan);
+ kfree(fw);
+}
+
+void rpi_firmware_put(struct rpi_firmware *fw)
+{
+ kref_put(&fw->consumers, rpi_firmware_delete);
+}
+EXPORT_SYMBOL_GPL(rpi_firmware_put);
+
+static void devm_rpi_firmware_put(void *data)
+{
+ struct rpi_firmware *fw = data;
+
+ rpi_firmware_put(fw);
+}
+
static int rpi_firmware_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct rpi_firmware *fw;
- fw = devm_kzalloc(dev, sizeof(*fw), GFP_KERNEL);
+ /*
+ * Memory will be freed by rpi_firmware_delete() once all users have
+ * released their firmware handles. Don't use devm_kzalloc() here.
+ */
+ fw = kzalloc(sizeof(*fw), GFP_KERNEL);
if (!fw)
return -ENOMEM;
@@ -247,6 +276,7 @@ static int rpi_firmware_probe(struct platform_device *pdev)
}
init_completion(&fw->c);
+ kref_init(&fw->consumers);
platform_set_drvdata(pdev, fw);
@@ -275,7 +305,8 @@ static int rpi_firmware_remove(struct platform_device *pdev)
rpi_hwmon = NULL;
platform_device_unregister(rpi_clk);
rpi_clk = NULL;
- mbox_free_channel(fw->chan);
+
+ rpi_firmware_put(fw);
return 0;
}
@@ -284,19 +315,51 @@ static int rpi_firmware_remove(struct platform_device *pdev)
* rpi_firmware_get - Get pointer to rpi_firmware structure.
* @firmware_node: Pointer to the firmware Device Tree node.
*
+ * The reference to rpi_firmware has to be released with rpi_firmware_put().
+ *
* Returns NULL is the firmware device is not ready.
*/
struct rpi_firmware *rpi_firmware_get(struct device_node *firmware_node)
{
struct platform_device *pdev = of_find_device_by_node(firmware_node);
+ struct rpi_firmware *fw;
if (!pdev)
return NULL;
- return platform_get_drvdata(pdev);
+ fw = platform_get_drvdata(pdev);
+ if (!fw)
+ return NULL;
+
+ if (!kref_get_unless_zero(&fw->consumers))
+ return NULL;
+
+ return fw;
}
EXPORT_SYMBOL_GPL(rpi_firmware_get);
+/**
+ * devm_rpi_firmware_get - Get pointer to rpi_firmware structure.
+ * @firmware_node: Pointer to the firmware Device Tree node.
+ *
+ * Returns NULL is the firmware device is not ready.
+ */
+struct rpi_firmware *devm_rpi_firmware_get(struct device *dev,
+ struct device_node *firmware_node)
+{
+ struct rpi_firmware *fw;
+
+ fw = rpi_firmware_get(firmware_node);
+ if (!fw)
+ return NULL;
+
+ if (devm_add_action_or_reset(dev, devm_rpi_firmware_put, fw))
+ return NULL;
+
+ return fw;
+}
+EXPORT_SYMBOL_GPL(devm_rpi_firmware_get);
+
static const struct of_device_id rpi_firmware_of_match[] = {
{ .compatible = "raspberrypi,bcm2835-firmware", },
{},
diff --git a/drivers/firmware/smccc/Makefile b/drivers/firmware/smccc/Makefile
index 72ab84042832..40d19144a860 100644
--- a/drivers/firmware/smccc/Makefile
+++ b/drivers/firmware/smccc/Makefile
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: GPL-2.0
#
-obj-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smccc.o
+obj-$(CONFIG_HAVE_ARM_SMCCC_DISCOVERY) += smccc.o kvm_guest.o
obj-$(CONFIG_ARM_SMCCC_SOC_ID) += soc_id.o
diff --git a/drivers/firmware/smccc/kvm_guest.c b/drivers/firmware/smccc/kvm_guest.c
new file mode 100644
index 000000000000..2d3e866decaa
--- /dev/null
+++ b/drivers/firmware/smccc/kvm_guest.c
@@ -0,0 +1,50 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#define pr_fmt(fmt) "smccc: KVM: " fmt
+
+#include <linux/arm-smccc.h>
+#include <linux/bitmap.h>
+#include <linux/kernel.h>
+#include <linux/string.h>
+
+#include <asm/hypervisor.h>
+
+static DECLARE_BITMAP(__kvm_arm_hyp_services, ARM_SMCCC_KVM_NUM_FUNCS) __ro_after_init = { };
+
+void __init kvm_init_hyp_services(void)
+{
+ struct arm_smccc_res res;
+ u32 val[4];
+
+ if (arm_smccc_1_1_get_conduit() != SMCCC_CONDUIT_HVC)
+ return;
+
+ arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_CALL_UID_FUNC_ID, &res);
+ if (res.a0 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_0 ||
+ res.a1 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_1 ||
+ res.a2 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_2 ||
+ res.a3 != ARM_SMCCC_VENDOR_HYP_UID_KVM_REG_3)
+ return;
+
+ memset(&res, 0, sizeof(res));
+ arm_smccc_1_1_invoke(ARM_SMCCC_VENDOR_HYP_KVM_FEATURES_FUNC_ID, &res);
+
+ val[0] = lower_32_bits(res.a0);
+ val[1] = lower_32_bits(res.a1);
+ val[2] = lower_32_bits(res.a2);
+ val[3] = lower_32_bits(res.a3);
+
+ bitmap_from_arr32(__kvm_arm_hyp_services, val, ARM_SMCCC_KVM_NUM_FUNCS);
+
+ pr_info("hypervisor services detected (0x%08lx 0x%08lx 0x%08lx 0x%08lx)\n",
+ res.a3, res.a2, res.a1, res.a0);
+}
+
+bool kvm_arm_hyp_service_available(u32 func_id)
+{
+ if (func_id >= ARM_SMCCC_KVM_NUM_FUNCS)
+ return false;
+
+ return test_bit(func_id, __kvm_arm_hyp_services);
+}
+EXPORT_SYMBOL_GPL(kvm_arm_hyp_service_available);
diff --git a/drivers/firmware/smccc/smccc.c b/drivers/firmware/smccc/smccc.c
index d52bfc5ed5e4..028f81d702cc 100644
--- a/drivers/firmware/smccc/smccc.c
+++ b/drivers/firmware/smccc/smccc.c
@@ -8,6 +8,7 @@
#include <linux/cache.h>
#include <linux/init.h>
#include <linux/arm-smccc.h>
+#include <linux/kernel.h>
#include <asm/archrandom.h>
static u32 smccc_version = ARM_SMCCC_VERSION_1_0;
diff --git a/drivers/firmware/turris-mox-rwtm.c b/drivers/firmware/turris-mox-rwtm.c
index 50bb2a6d6ccf..62f0d1a5dd32 100644
--- a/drivers/firmware/turris-mox-rwtm.c
+++ b/drivers/firmware/turris-mox-rwtm.c
@@ -2,7 +2,7 @@
/*
* Turris Mox rWTM firmware driver
*
- * Copyright (C) 2019 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2019 Marek Behún <kabel@kernel.org>
*/
#include <linux/armada-37xx-rwtm-mailbox.h>
@@ -547,4 +547,4 @@ module_platform_driver(turris_mox_rwtm_driver);
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("Turris Mox rWTM firmware driver");
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
diff --git a/drivers/firmware/xilinx/zynqmp.c b/drivers/firmware/xilinx/zynqmp.c
index 7eb9958662dd..15b138326ecc 100644
--- a/drivers/firmware/xilinx/zynqmp.c
+++ b/drivers/firmware/xilinx/zynqmp.c
@@ -2,7 +2,7 @@
/*
* Xilinx Zynq MPSoC Firmware layer
*
- * Copyright (C) 2014-2020 Xilinx, Inc.
+ * Copyright (C) 2014-2021 Xilinx, Inc.
*
* Michal Simek <michal.simek@xilinx.com>
* Davorin Mista <davorin.mista@aggios.com>
@@ -812,6 +812,120 @@ int zynqmp_pm_fpga_get_status(u32 *value)
EXPORT_SYMBOL_GPL(zynqmp_pm_fpga_get_status);
/**
+ * zynqmp_pm_pinctrl_request - Request Pin from firmware
+ * @pin: Pin number to request
+ *
+ * This function requests pin from firmware.
+ *
+ * Return: Returns status, either success or error+reason.
+ */
+int zynqmp_pm_pinctrl_request(const u32 pin)
+{
+ return zynqmp_pm_invoke_fn(PM_PINCTRL_REQUEST, pin, 0, 0, 0, NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_pinctrl_request);
+
+/**
+ * zynqmp_pm_pinctrl_release - Inform firmware that Pin control is released
+ * @pin: Pin number to release
+ *
+ * This function release pin from firmware.
+ *
+ * Return: Returns status, either success or error+reason.
+ */
+int zynqmp_pm_pinctrl_release(const u32 pin)
+{
+ return zynqmp_pm_invoke_fn(PM_PINCTRL_RELEASE, pin, 0, 0, 0, NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_pinctrl_release);
+
+/**
+ * zynqmp_pm_pinctrl_get_function - Read function id set for the given pin
+ * @pin: Pin number
+ * @id: Buffer to store function ID
+ *
+ * This function provides the function currently set for the given pin.
+ *
+ * Return: Returns status, either success or error+reason
+ */
+int zynqmp_pm_pinctrl_get_function(const u32 pin, u32 *id)
+{
+ u32 ret_payload[PAYLOAD_ARG_CNT];
+ int ret;
+
+ if (!id)
+ return -EINVAL;
+
+ ret = zynqmp_pm_invoke_fn(PM_PINCTRL_GET_FUNCTION, pin, 0,
+ 0, 0, ret_payload);
+ *id = ret_payload[1];
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_pinctrl_get_function);
+
+/**
+ * zynqmp_pm_pinctrl_set_function - Set requested function for the pin
+ * @pin: Pin number
+ * @id: Function ID to set
+ *
+ * This function sets requested function for the given pin.
+ *
+ * Return: Returns status, either success or error+reason.
+ */
+int zynqmp_pm_pinctrl_set_function(const u32 pin, const u32 id)
+{
+ return zynqmp_pm_invoke_fn(PM_PINCTRL_SET_FUNCTION, pin, id,
+ 0, 0, NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_pinctrl_set_function);
+
+/**
+ * zynqmp_pm_pinctrl_get_config - Get configuration parameter for the pin
+ * @pin: Pin number
+ * @param: Parameter to get
+ * @value: Buffer to store parameter value
+ *
+ * This function gets requested configuration parameter for the given pin.
+ *
+ * Return: Returns status, either success or error+reason.
+ */
+int zynqmp_pm_pinctrl_get_config(const u32 pin, const u32 param,
+ u32 *value)
+{
+ u32 ret_payload[PAYLOAD_ARG_CNT];
+ int ret;
+
+ if (!value)
+ return -EINVAL;
+
+ ret = zynqmp_pm_invoke_fn(PM_PINCTRL_CONFIG_PARAM_GET, pin, param,
+ 0, 0, ret_payload);
+ *value = ret_payload[1];
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_pinctrl_get_config);
+
+/**
+ * zynqmp_pm_pinctrl_set_config - Set configuration parameter for the pin
+ * @pin: Pin number
+ * @param: Parameter to set
+ * @value: Parameter value to set
+ *
+ * This function sets requested configuration parameter for the given pin.
+ *
+ * Return: Returns status, either success or error+reason.
+ */
+int zynqmp_pm_pinctrl_set_config(const u32 pin, const u32 param,
+ u32 value)
+{
+ return zynqmp_pm_invoke_fn(PM_PINCTRL_CONFIG_PARAM_SET, pin,
+ param, value, 0, NULL);
+}
+EXPORT_SYMBOL_GPL(zynqmp_pm_pinctrl_set_config);
+
+/**
* zynqmp_pm_init_finalize() - PM call to inform firmware that the caller
* master has initialized its own power management
*
@@ -1280,12 +1394,13 @@ static int zynqmp_firmware_probe(struct platform_device *pdev)
static int zynqmp_firmware_remove(struct platform_device *pdev)
{
struct pm_api_feature_data *feature_data;
+ struct hlist_node *tmp;
int i;
mfd_remove_devices(&pdev->dev);
zynqmp_pm_api_debugfs_exit();
- hash_for_each(pm_api_features_map, i, feature_data, hentry) {
+ hash_for_each_safe(pm_api_features_map, i, tmp, feature_data, hentry) {
hash_del(&feature_data->hentry);
kfree(feature_data);
}
diff --git a/drivers/fpga/Kconfig b/drivers/fpga/Kconfig
index 5ff9438b7b46..33e15058d0dc 100644
--- a/drivers/fpga/Kconfig
+++ b/drivers/fpga/Kconfig
@@ -14,13 +14,13 @@ if FPGA
config FPGA_MGR_SOCFPGA
tristate "Altera SOCFPGA FPGA Manager"
- depends on ARCH_SOCFPGA || COMPILE_TEST
+ depends on ARCH_INTEL_SOCFPGA || COMPILE_TEST
help
FPGA manager driver support for Altera SOCFPGA.
config FPGA_MGR_SOCFPGA_A10
tristate "Altera SoCFPGA Arria10"
- depends on ARCH_SOCFPGA || COMPILE_TEST
+ depends on ARCH_INTEL_SOCFPGA || COMPILE_TEST
select REGMAP_MMIO
help
FPGA manager driver support for Altera Arria10 SoCFPGA.
@@ -60,7 +60,7 @@ config FPGA_MGR_ZYNQ_FPGA
config FPGA_MGR_STRATIX10_SOC
tristate "Intel Stratix10 SoC FPGA Manager"
- depends on (ARCH_STRATIX10 && INTEL_STRATIX10_SERVICE)
+ depends on (ARCH_INTEL_SOCFPGA && INTEL_STRATIX10_SERVICE)
help
FPGA manager driver support for the Intel Stratix10 SoC.
@@ -99,7 +99,7 @@ config FPGA_BRIDGE
config SOCFPGA_FPGA_BRIDGE
tristate "Altera SoCFPGA FPGA Bridges"
- depends on ARCH_SOCFPGA && FPGA_BRIDGE
+ depends on ARCH_INTEL_SOCFPGA && FPGA_BRIDGE
help
Say Y to enable drivers for FPGA bridges for Altera SOCFPGA
devices.
@@ -118,10 +118,17 @@ config XILINX_PR_DECOUPLER
depends on FPGA_BRIDGE
depends on HAS_IOMEM
help
- Say Y to enable drivers for Xilinx LogiCORE PR Decoupler.
+ Say Y to enable drivers for Xilinx LogiCORE PR Decoupler
+ or Xilinx Dynamic Function eXchnage AIX Shutdown Manager.
The PR Decoupler exists in the FPGA fabric to isolate one
region of the FPGA from the busses while that region is
being reprogrammed during partial reconfig.
+ The Dynamic Function eXchange AXI shutdown manager prevents
+ AXI traffic from passing through the bridge. The controller
+ safely handles AXI4MM and AXI4-Lite interfaces on a
+ Reconfigurable Partition when it is undergoing dynamic
+ reconfiguration, preventing the system deadlock that can
+ occur if AXI transactions are interrupted by DFX.
config FPGA_REGION
tristate "FPGA Region"
diff --git a/drivers/fpga/dfl-afu-error.c b/drivers/fpga/dfl-afu-error.c
index c4691187cca9..ab7be6217368 100644
--- a/drivers/fpga/dfl-afu-error.c
+++ b/drivers/fpga/dfl-afu-error.c
@@ -52,7 +52,7 @@ static int afu_port_err_clear(struct device *dev, u64 err)
struct dfl_feature_platform_data *pdata = dev_get_platdata(dev);
struct platform_device *pdev = to_platform_device(dev);
void __iomem *base_err, *base_hdr;
- int ret = -EBUSY;
+ int enable_ret = 0, ret = -EBUSY;
u64 v;
base_err = dfl_get_feature_ioaddr_by_id(dev, PORT_FEATURE_ID_ERROR);
@@ -96,18 +96,20 @@ static int afu_port_err_clear(struct device *dev, u64 err)
v = readq(base_err + PORT_FIRST_ERROR);
writeq(v, base_err + PORT_FIRST_ERROR);
} else {
+ dev_warn(dev, "%s: received 0x%llx, expected 0x%llx\n",
+ __func__, v, err);
ret = -EINVAL;
}
/* Clear mask */
__afu_port_err_mask(dev, false);
- /* Enable the Port by clear the reset */
- __afu_port_enable(pdev);
+ /* Enable the Port by clearing the reset */
+ enable_ret = __afu_port_enable(pdev);
done:
mutex_unlock(&pdata->lock);
- return ret;
+ return enable_ret ? enable_ret : ret;
}
static ssize_t errors_show(struct device *dev, struct device_attribute *attr,
diff --git a/drivers/fpga/dfl-afu-main.c b/drivers/fpga/dfl-afu-main.c
index 753cda4b2568..7f621e96d3b8 100644
--- a/drivers/fpga/dfl-afu-main.c
+++ b/drivers/fpga/dfl-afu-main.c
@@ -21,6 +21,9 @@
#include "dfl-afu.h"
+#define RST_POLL_INVL 10 /* us */
+#define RST_POLL_TIMEOUT 1000 /* us */
+
/**
* __afu_port_enable - enable a port by clear reset
* @pdev: port platform device.
@@ -32,7 +35,7 @@
*
* The caller needs to hold lock for protection.
*/
-void __afu_port_enable(struct platform_device *pdev)
+int __afu_port_enable(struct platform_device *pdev)
{
struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
void __iomem *base;
@@ -41,7 +44,7 @@ void __afu_port_enable(struct platform_device *pdev)
WARN_ON(!pdata->disable_count);
if (--pdata->disable_count != 0)
- return;
+ return 0;
base = dfl_get_feature_ioaddr_by_id(&pdev->dev, PORT_FEATURE_ID_HEADER);
@@ -49,10 +52,20 @@ void __afu_port_enable(struct platform_device *pdev)
v = readq(base + PORT_HDR_CTRL);
v &= ~PORT_CTRL_SFTRST;
writeq(v, base + PORT_HDR_CTRL);
-}
-#define RST_POLL_INVL 10 /* us */
-#define RST_POLL_TIMEOUT 1000 /* us */
+ /*
+ * HW clears the ack bit to indicate that the port is fully out
+ * of reset.
+ */
+ if (readq_poll_timeout(base + PORT_HDR_CTRL, v,
+ !(v & PORT_CTRL_SFTRST_ACK),
+ RST_POLL_INVL, RST_POLL_TIMEOUT)) {
+ dev_err(&pdev->dev, "timeout, failure to enable device\n");
+ return -ETIMEDOUT;
+ }
+
+ return 0;
+}
/**
* __afu_port_disable - disable a port by hold reset
@@ -86,7 +99,7 @@ int __afu_port_disable(struct platform_device *pdev)
if (readq_poll_timeout(base + PORT_HDR_CTRL, v,
v & PORT_CTRL_SFTRST_ACK,
RST_POLL_INVL, RST_POLL_TIMEOUT)) {
- dev_err(&pdev->dev, "timeout, fail to reset device\n");
+ dev_err(&pdev->dev, "timeout, failure to disable device\n");
return -ETIMEDOUT;
}
@@ -110,10 +123,10 @@ static int __port_reset(struct platform_device *pdev)
int ret;
ret = __afu_port_disable(pdev);
- if (!ret)
- __afu_port_enable(pdev);
+ if (ret)
+ return ret;
- return ret;
+ return __afu_port_enable(pdev);
}
static int port_reset(struct platform_device *pdev)
@@ -872,11 +885,11 @@ static int afu_dev_destroy(struct platform_device *pdev)
static int port_enable_set(struct platform_device *pdev, bool enable)
{
struct dfl_feature_platform_data *pdata = dev_get_platdata(&pdev->dev);
- int ret = 0;
+ int ret;
mutex_lock(&pdata->lock);
if (enable)
- __afu_port_enable(pdev);
+ ret = __afu_port_enable(pdev);
else
ret = __afu_port_disable(pdev);
mutex_unlock(&pdata->lock);
diff --git a/drivers/fpga/dfl-afu.h b/drivers/fpga/dfl-afu.h
index 576e94960086..e5020e2b1f3d 100644
--- a/drivers/fpga/dfl-afu.h
+++ b/drivers/fpga/dfl-afu.h
@@ -80,7 +80,7 @@ struct dfl_afu {
};
/* hold pdata->lock when call __afu_port_enable/disable */
-void __afu_port_enable(struct platform_device *pdev);
+int __afu_port_enable(struct platform_device *pdev);
int __afu_port_disable(struct platform_device *pdev);
void afu_mmio_region_init(struct dfl_feature_platform_data *pdata);
diff --git a/drivers/fpga/dfl-pci.c b/drivers/fpga/dfl-pci.c
index 04e47e266f26..b44523ea8c91 100644
--- a/drivers/fpga/dfl-pci.c
+++ b/drivers/fpga/dfl-pci.c
@@ -69,14 +69,16 @@ static void cci_pci_free_irq(struct pci_dev *pcidev)
}
/* PCI Device ID */
-#define PCIE_DEVICE_ID_PF_INT_5_X 0xBCBD
-#define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0
-#define PCIE_DEVICE_ID_PF_DSC_1_X 0x09C4
-#define PCIE_DEVICE_ID_INTEL_PAC_N3000 0x0B30
+#define PCIE_DEVICE_ID_PF_INT_5_X 0xBCBD
+#define PCIE_DEVICE_ID_PF_INT_6_X 0xBCC0
+#define PCIE_DEVICE_ID_PF_DSC_1_X 0x09C4
+#define PCIE_DEVICE_ID_INTEL_PAC_N3000 0x0B30
+#define PCIE_DEVICE_ID_INTEL_PAC_D5005 0x0B2B
/* VF Device */
-#define PCIE_DEVICE_ID_VF_INT_5_X 0xBCBF
-#define PCIE_DEVICE_ID_VF_INT_6_X 0xBCC1
-#define PCIE_DEVICE_ID_VF_DSC_1_X 0x09C5
+#define PCIE_DEVICE_ID_VF_INT_5_X 0xBCBF
+#define PCIE_DEVICE_ID_VF_INT_6_X 0xBCC1
+#define PCIE_DEVICE_ID_VF_DSC_1_X 0x09C5
+#define PCIE_DEVICE_ID_INTEL_PAC_D5005_VF 0x0B2C
static struct pci_device_id cci_pcie_id_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_PF_INT_5_X),},
@@ -86,6 +88,8 @@ static struct pci_device_id cci_pcie_id_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_PF_DSC_1_X),},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_VF_DSC_1_X),},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_INTEL_PAC_N3000),},
+ {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_INTEL_PAC_D5005),},
+ {PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCIE_DEVICE_ID_INTEL_PAC_D5005_VF),},
{0,}
};
MODULE_DEVICE_TABLE(pci, cci_pcie_id_tbl);
diff --git a/drivers/fpga/xilinx-pr-decoupler.c b/drivers/fpga/xilinx-pr-decoupler.c
index 7d69af230567..ea2bde6e5bc4 100644
--- a/drivers/fpga/xilinx-pr-decoupler.c
+++ b/drivers/fpga/xilinx-pr-decoupler.c
@@ -1,7 +1,7 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* Copyright (c) 2017, National Instruments Corp.
- * Copyright (c) 2017, Xilix Inc
+ * Copyright (c) 2017, Xilinx Inc
*
* FPGA Bridge Driver for the Xilinx LogiCORE Partial Reconfiguration
* Decoupler IP Core.
@@ -18,7 +18,12 @@
#define CTRL_CMD_COUPLE 0
#define CTRL_OFFSET 0
+struct xlnx_config_data {
+ const char *name;
+};
+
struct xlnx_pr_decoupler_data {
+ const struct xlnx_config_data *ipconfig;
void __iomem *io_base;
struct clk *clk;
};
@@ -76,15 +81,28 @@ static const struct fpga_bridge_ops xlnx_pr_decoupler_br_ops = {
.enable_show = xlnx_pr_decoupler_enable_show,
};
+static const struct xlnx_config_data decoupler_config = {
+ .name = "Xilinx PR Decoupler",
+};
+
+static const struct xlnx_config_data shutdown_config = {
+ .name = "Xilinx DFX AXI Shutdown Manager",
+};
+
static const struct of_device_id xlnx_pr_decoupler_of_match[] = {
- { .compatible = "xlnx,pr-decoupler-1.00", },
- { .compatible = "xlnx,pr-decoupler", },
+ { .compatible = "xlnx,pr-decoupler-1.00", .data = &decoupler_config },
+ { .compatible = "xlnx,pr-decoupler", .data = &decoupler_config },
+ { .compatible = "xlnx,dfx-axi-shutdown-manager-1.00",
+ .data = &shutdown_config },
+ { .compatible = "xlnx,dfx-axi-shutdown-manager",
+ .data = &shutdown_config },
{},
};
MODULE_DEVICE_TABLE(of, xlnx_pr_decoupler_of_match);
static int xlnx_pr_decoupler_probe(struct platform_device *pdev)
{
+ struct device_node *np = pdev->dev.of_node;
struct xlnx_pr_decoupler_data *priv;
struct fpga_bridge *br;
int err;
@@ -94,17 +112,23 @@ static int xlnx_pr_decoupler_probe(struct platform_device *pdev)
if (!priv)
return -ENOMEM;
+ if (np) {
+ const struct of_device_id *match;
+
+ match = of_match_node(xlnx_pr_decoupler_of_match, np);
+ if (match && match->data)
+ priv->ipconfig = match->data;
+ }
+
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
priv->io_base = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(priv->io_base))
return PTR_ERR(priv->io_base);
priv->clk = devm_clk_get(&pdev->dev, "aclk");
- if (IS_ERR(priv->clk)) {
- if (PTR_ERR(priv->clk) != -EPROBE_DEFER)
- dev_err(&pdev->dev, "input clock not found\n");
- return PTR_ERR(priv->clk);
- }
+ if (IS_ERR(priv->clk))
+ return dev_err_probe(&pdev->dev, PTR_ERR(priv->clk),
+ "input clock not found\n");
err = clk_prepare_enable(priv->clk);
if (err) {
@@ -114,7 +138,7 @@ static int xlnx_pr_decoupler_probe(struct platform_device *pdev)
clk_disable(priv->clk);
- br = devm_fpga_bridge_create(&pdev->dev, "Xilinx PR Decoupler",
+ br = devm_fpga_bridge_create(&pdev->dev, priv->ipconfig->name,
&xlnx_pr_decoupler_br_ops, priv);
if (!br) {
err = -ENOMEM;
@@ -125,7 +149,8 @@ static int xlnx_pr_decoupler_probe(struct platform_device *pdev)
err = fpga_bridge_register(br);
if (err) {
- dev_err(&pdev->dev, "unable to register Xilinx PR Decoupler");
+ dev_err(&pdev->dev, "unable to register %s",
+ priv->ipconfig->name);
goto err_clk;
}
diff --git a/drivers/fpga/xilinx-spi.c b/drivers/fpga/xilinx-spi.c
index 27defa98092d..fee4d0abf6bf 100644
--- a/drivers/fpga/xilinx-spi.c
+++ b/drivers/fpga/xilinx-spi.c
@@ -233,25 +233,19 @@ static int xilinx_spi_probe(struct spi_device *spi)
/* PROGRAM_B is active low */
conf->prog_b = devm_gpiod_get(&spi->dev, "prog_b", GPIOD_OUT_LOW);
- if (IS_ERR(conf->prog_b)) {
- dev_err(&spi->dev, "Failed to get PROGRAM_B gpio: %ld\n",
- PTR_ERR(conf->prog_b));
- return PTR_ERR(conf->prog_b);
- }
+ if (IS_ERR(conf->prog_b))
+ return dev_err_probe(&spi->dev, PTR_ERR(conf->prog_b),
+ "Failed to get PROGRAM_B gpio\n");
conf->init_b = devm_gpiod_get_optional(&spi->dev, "init-b", GPIOD_IN);
- if (IS_ERR(conf->init_b)) {
- dev_err(&spi->dev, "Failed to get INIT_B gpio: %ld\n",
- PTR_ERR(conf->init_b));
- return PTR_ERR(conf->init_b);
- }
+ if (IS_ERR(conf->init_b))
+ return dev_err_probe(&spi->dev, PTR_ERR(conf->init_b),
+ "Failed to get INIT_B gpio\n");
conf->done = devm_gpiod_get(&spi->dev, "done", GPIOD_IN);
- if (IS_ERR(conf->done)) {
- dev_err(&spi->dev, "Failed to get DONE gpio: %ld\n",
- PTR_ERR(conf->done));
- return PTR_ERR(conf->done);
- }
+ if (IS_ERR(conf->done))
+ return dev_err_probe(&spi->dev, PTR_ERR(conf->done),
+ "Failed to get DONE gpio\n");
mgr = devm_fpga_mgr_create(&spi->dev,
"Xilinx Slave Serial FPGA Manager",
diff --git a/drivers/gpio/Kconfig b/drivers/gpio/Kconfig
index e3607ec4c2e8..1dd0ec6727fd 100644
--- a/drivers/gpio/Kconfig
+++ b/drivers/gpio/Kconfig
@@ -321,9 +321,8 @@ config GPIO_HLWD
config GPIO_ICH
tristate "Intel ICH GPIO"
- depends on PCI && X86
- select MFD_CORE
- select LPC_ICH
+ depends on X86
+ depends on LPC_ICH
help
Say yes here to support the GPIO functionality of a number of Intel
ICH-based chipsets. Currently supported devices: ICH6, ICH7, ICH8
@@ -502,6 +501,19 @@ config GPIO_RDA
help
Say Y here to support RDA Micro GPIO controller.
+config GPIO_REALTEK_OTTO
+ tristate "Realtek Otto GPIO support"
+ depends on MACH_REALTEK_RTL
+ default MACH_REALTEK_RTL
+ select GPIO_GENERIC
+ select GPIOLIB_IRQCHIP
+ help
+ The GPIO controller on the Otto MIPS platform supports up to two
+ banks of 32 GPIOs, with edge triggered interrupts. The 32 GPIOs
+ are grouped in four 8-bit wide ports.
+
+ When built as a module, the module will be called realtek_otto_gpio.
+
config GPIO_REG
bool
help
@@ -847,9 +859,9 @@ config GPIO_IT87
config GPIO_SCH
tristate "Intel SCH/TunnelCreek/Centerton/Quark X1000 GPIO"
- depends on (X86 || COMPILE_TEST) && PCI
- select MFD_CORE
- select LPC_SCH
+ depends on (X86 || COMPILE_TEST) && ACPI
+ depends on LPC_SCH
+ select GPIOLIB_IRQCHIP
help
Say yes here to support GPIO interface on Intel Poulsbo SCH,
Intel Tunnel Creek processor, Intel Centerton processor or
@@ -1105,6 +1117,16 @@ config GPIO_BD70528
This driver can also be built as a module. If so, the module
will be called gpio-bd70528.
+config GPIO_BD71815
+ tristate "ROHM BD71815 PMIC GPIO support"
+ depends on MFD_ROHM_BD71828
+ help
+ Support for GPO(s) on ROHM BD71815 PMIC. There are two GPOs
+ available on the ROHM PMIC.
+
+ This driver can also be built as a module. If so, the module
+ will be called gpio-bd71815.
+
config GPIO_BD71828
tristate "ROHM BD71828 GPIO support"
depends on MFD_ROHM_BD71828
diff --git a/drivers/gpio/Makefile b/drivers/gpio/Makefile
index c58a90a3c3b1..d7c81e1611a4 100644
--- a/drivers/gpio/Makefile
+++ b/drivers/gpio/Makefile
@@ -39,6 +39,7 @@ obj-$(CONFIG_GPIO_ATH79) += gpio-ath79.o
obj-$(CONFIG_GPIO_BCM_KONA) += gpio-bcm-kona.o
obj-$(CONFIG_GPIO_BCM_XGS_IPROC) += gpio-xgs-iproc.o
obj-$(CONFIG_GPIO_BD70528) += gpio-bd70528.o
+obj-$(CONFIG_GPIO_BD71815) += gpio-bd71815.o
obj-$(CONFIG_GPIO_BD71828) += gpio-bd71828.o
obj-$(CONFIG_GPIO_BD9571MWV) += gpio-bd9571mwv.o
obj-$(CONFIG_GPIO_BRCMSTB) += gpio-brcmstb.o
@@ -124,6 +125,7 @@ obj-$(CONFIG_GPIO_RC5T583) += gpio-rc5t583.o
obj-$(CONFIG_GPIO_RCAR) += gpio-rcar.o
obj-$(CONFIG_GPIO_RDA) += gpio-rda.o
obj-$(CONFIG_GPIO_RDC321X) += gpio-rdc321x.o
+obj-$(CONFIG_GPIO_REALTEK_OTTO) += gpio-realtek-otto.o
obj-$(CONFIG_GPIO_REG) += gpio-reg.o
obj-$(CONFIG_ARCH_SA1100) += gpio-sa1100.o
obj-$(CONFIG_GPIO_SAMA5D2_PIOBU) += gpio-sama5d2-piobu.o
diff --git a/drivers/gpio/gpio-104-dio-48e.c b/drivers/gpio/gpio-104-dio-48e.c
index 7a9021c4fa48..71c0bea34d7b 100644
--- a/drivers/gpio/gpio-104-dio-48e.c
+++ b/drivers/gpio/gpio-104-dio-48e.c
@@ -49,15 +49,15 @@ struct dio48e_gpio {
unsigned char out_state[6];
unsigned char control[2];
raw_spinlock_t lock;
- unsigned base;
+ unsigned int base;
unsigned char irq_mask;
};
-static int dio48e_gpio_get_direction(struct gpio_chip *chip, unsigned offset)
+static int dio48e_gpio_get_direction(struct gpio_chip *chip, unsigned int offset)
{
struct dio48e_gpio *const dio48egpio = gpiochip_get_data(chip);
- const unsigned port = offset / 8;
- const unsigned mask = BIT(offset % 8);
+ const unsigned int port = offset / 8;
+ const unsigned int mask = BIT(offset % 8);
if (dio48egpio->io_state[port] & mask)
return GPIO_LINE_DIRECTION_IN;
@@ -65,14 +65,14 @@ static int dio48e_gpio_get_direction(struct gpio_chip *chip, unsigned offset)
return GPIO_LINE_DIRECTION_OUT;
}
-static int dio48e_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
+static int dio48e_gpio_direction_input(struct gpio_chip *chip, unsigned int offset)
{
struct dio48e_gpio *const dio48egpio = gpiochip_get_data(chip);
- const unsigned io_port = offset / 8;
+ const unsigned int io_port = offset / 8;
const unsigned int control_port = io_port / 3;
- const unsigned control_addr = dio48egpio->base + 3 + control_port*4;
+ const unsigned int control_addr = dio48egpio->base + 3 + control_port * 4;
unsigned long flags;
- unsigned control;
+ unsigned int control;
raw_spin_lock_irqsave(&dio48egpio->lock, flags);
@@ -104,17 +104,17 @@ static int dio48e_gpio_direction_input(struct gpio_chip *chip, unsigned offset)
return 0;
}
-static int dio48e_gpio_direction_output(struct gpio_chip *chip, unsigned offset,
- int value)
+static int dio48e_gpio_direction_output(struct gpio_chip *chip, unsigned int offset,
+ int value)
{
struct dio48e_gpio *const dio48egpio = gpiochip_get_data(chip);
- const unsigned io_port = offset / 8;
+ const unsigned int io_port = offset / 8;
const unsigned int control_port = io_port / 3;
- const unsigned mask = BIT(offset % 8);
- const unsigned control_addr = dio48egpio->base + 3 + control_port*4;
- const unsigned out_port = (io_port > 2) ? io_port + 1 : io_port;
+ const unsigned int mask = BIT(offset % 8);
+ const unsigned int control_addr = dio48egpio->base + 3 + control_port * 4;
+ const unsigned int out_port = (io_port > 2) ? io_port + 1 : io_port;
unsigned long flags;
- unsigned control;
+ unsigned int control;
raw_spin_lock_irqsave(&dio48egpio->lock, flags);
@@ -154,14 +154,14 @@ static int dio48e_gpio_direction_output(struct gpio_chip *chip, unsigned offset,
return 0;
}
-static int dio48e_gpio_get(struct gpio_chip *chip, unsigned offset)
+static int dio48e_gpio_get(struct gpio_chip *chip, unsigned int offset)
{
struct dio48e_gpio *const dio48egpio = gpiochip_get_data(chip);
- const unsigned port = offset / 8;
- const unsigned mask = BIT(offset % 8);
- const unsigned in_port = (port > 2) ? port + 1 : port;
+ const unsigned int port = offset / 8;
+ const unsigned int mask = BIT(offset % 8);
+ const unsigned int in_port = (port > 2) ? port + 1 : port;
unsigned long flags;
- unsigned port_state;
+ unsigned int port_state;
raw_spin_lock_irqsave(&dio48egpio->lock, flags);
@@ -202,12 +202,12 @@ static int dio48e_gpio_get_multiple(struct gpio_chip *chip, unsigned long *mask,
return 0;
}
-static void dio48e_gpio_set(struct gpio_chip *chip, unsigned offset, int value)
+static void dio48e_gpio_set(struct gpio_chip *chip, unsigned int offset, int value)
{
struct dio48e_gpio *const dio48egpio = gpiochip_get_data(chip);
- const unsigned port = offset / 8;
- const unsigned mask = BIT(offset % 8);
- const unsigned out_port = (port > 2) ? port + 1 : port;
+ const unsigned int port = offset / 8;
+ const unsigned int mask = BIT(offset % 8);
+ const unsigned int out_port = (port > 2) ? port + 1 : port;
unsigned long flags;
raw_spin_lock_irqsave(&dio48egpio->lock, flags);
@@ -306,7 +306,7 @@ static void dio48e_irq_unmask(struct irq_data *data)
raw_spin_unlock_irqrestore(&dio48egpio->lock, flags);
}
-static int dio48e_irq_set_type(struct irq_data *data, unsigned flow_type)
+static int dio48e_irq_set_type(struct irq_data *data, unsigned int flow_type)
{
const unsigned long offset = irqd_to_hwirq(data);
diff --git a/drivers/gpio/gpio-aggregator.c b/drivers/gpio/gpio-aggregator.c
index 08171431bb8f..34e35b64dcdc 100644
--- a/drivers/gpio/gpio-aggregator.c
+++ b/drivers/gpio/gpio-aggregator.c
@@ -37,31 +37,6 @@ struct gpio_aggregator {
static DEFINE_MUTEX(gpio_aggregator_lock); /* protects idr */
static DEFINE_IDR(gpio_aggregator_idr);
-static char *get_arg(char **args)
-{
- char *start, *end;
-
- start = skip_spaces(*args);
- if (!*start)
- return NULL;
-
- if (*start == '"') {
- /* Quoted arg */
- end = strchr(++start, '"');
- if (!end)
- return ERR_PTR(-EINVAL);
- } else {
- /* Unquoted arg */
- for (end = start; *end && !isspace(*end); end++) ;
- }
-
- if (*end)
- *end++ = '\0';
-
- *args = end;
- return start;
-}
-
static int aggr_add_gpio(struct gpio_aggregator *aggr, const char *key,
int hwnum, unsigned int *n)
{
@@ -83,8 +58,8 @@ static int aggr_add_gpio(struct gpio_aggregator *aggr, const char *key,
static int aggr_parse(struct gpio_aggregator *aggr)
{
+ char *args = skip_spaces(aggr->args);
char *name, *offsets, *p;
- char *args = aggr->args;
unsigned long *bitmap;
unsigned int i, n = 0;
int error = 0;
@@ -93,13 +68,9 @@ static int aggr_parse(struct gpio_aggregator *aggr)
if (!bitmap)
return -ENOMEM;
- for (name = get_arg(&args), offsets = get_arg(&args); name;
- offsets = get_arg(&args)) {
- if (IS_ERR(name)) {
- pr_err("Cannot get GPIO specifier: %pe\n", name);
- error = PTR_ERR(name);
- goto free_bitmap;
- }
+ args = next_arg(args, &name, &p);
+ while (*args) {
+ args = next_arg(args, &offsets, &p);
p = get_options(offsets, 0, &error);
if (error == 0 || *p) {
@@ -125,7 +96,7 @@ static int aggr_parse(struct gpio_aggregator *aggr)
goto free_bitmap;
}
- name = get_arg(&args);
+ args = next_arg(args, &name, &p);
}
if (!n) {
diff --git a/drivers/gpio/gpio-bd71815.c b/drivers/gpio/gpio-bd71815.c
new file mode 100644
index 000000000000..08ff2857256f
--- /dev/null
+++ b/drivers/gpio/gpio-bd71815.c
@@ -0,0 +1,185 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Support to GPOs on ROHM BD71815
+ * Copyright 2021 ROHM Semiconductors.
+ * Author: Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>
+ *
+ * Copyright 2014 Embest Technology Co. Ltd. Inc.
+ * Author: yanglsh@embest-tech.com
+ */
+
+#include <linux/gpio/driver.h>
+#include <linux/init.h>
+#include <linux/irq.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+/* For the BD71815 register definitions */
+#include <linux/mfd/rohm-bd71815.h>
+
+struct bd71815_gpio {
+ /* chip.parent points the MFD which provides DT node and regmap */
+ struct gpio_chip chip;
+ /* dev points to the platform device for devm and prints */
+ struct device *dev;
+ struct regmap *regmap;
+};
+
+static int bd71815gpo_get(struct gpio_chip *chip, unsigned int offset)
+{
+ struct bd71815_gpio *bd71815 = gpiochip_get_data(chip);
+ int ret, val;
+
+ ret = regmap_read(bd71815->regmap, BD71815_REG_GPO, &val);
+ if (ret)
+ return ret;
+
+ return (val >> offset) & 1;
+}
+
+static void bd71815gpo_set(struct gpio_chip *chip, unsigned int offset,
+ int value)
+{
+ struct bd71815_gpio *bd71815 = gpiochip_get_data(chip);
+ int ret, bit;
+
+ bit = BIT(offset);
+
+ if (value)
+ ret = regmap_set_bits(bd71815->regmap, BD71815_REG_GPO, bit);
+ else
+ ret = regmap_clear_bits(bd71815->regmap, BD71815_REG_GPO, bit);
+
+ if (ret)
+ dev_warn(bd71815->dev, "failed to toggle GPO\n");
+}
+
+static int bd71815_gpio_set_config(struct gpio_chip *chip, unsigned int offset,
+ unsigned long config)
+{
+ struct bd71815_gpio *bdgpio = gpiochip_get_data(chip);
+
+ switch (pinconf_to_config_param(config)) {
+ case PIN_CONFIG_DRIVE_OPEN_DRAIN:
+ return regmap_update_bits(bdgpio->regmap,
+ BD71815_REG_GPO,
+ BD71815_GPIO_DRIVE_MASK << offset,
+ BD71815_GPIO_OPEN_DRAIN << offset);
+ case PIN_CONFIG_DRIVE_PUSH_PULL:
+ return regmap_update_bits(bdgpio->regmap,
+ BD71815_REG_GPO,
+ BD71815_GPIO_DRIVE_MASK << offset,
+ BD71815_GPIO_CMOS << offset);
+ default:
+ break;
+ }
+ return -ENOTSUPP;
+}
+
+/* BD71815 GPIO is actually GPO */
+static int bd71815gpo_direction_get(struct gpio_chip *gc, unsigned int offset)
+{
+ return GPIO_LINE_DIRECTION_OUT;
+}
+
+/* Template for GPIO chip */
+static const struct gpio_chip bd71815gpo_chip = {
+ .label = "bd71815",
+ .owner = THIS_MODULE,
+ .get = bd71815gpo_get,
+ .get_direction = bd71815gpo_direction_get,
+ .set = bd71815gpo_set,
+ .set_config = bd71815_gpio_set_config,
+ .can_sleep = true,
+};
+
+#define BD71815_TWO_GPIOS GENMASK(1, 0)
+#define BD71815_ONE_GPIO BIT(0)
+
+/*
+ * Sigh. The BD71815 and BD71817 were originally designed to support two GPO
+ * pins. At some point it was noticed the second GPO pin which is the E5 pin
+ * located at the center of IC is hard to use on PCB (due to the location). It
+ * was decided to not promote this second GPO and the pin is marked as GND in
+ * the datasheet. The functionality is still there though! I guess driving a GPO
+ * connected to the ground is a bad idea. Thus we do not support it by default.
+ * OTOH - the original driver written by colleagues at Embest did support
+ * controlling this second GPO. It is thus possible this is used in some of the
+ * products.
+ *
+ * This driver does not by default support configuring this second GPO
+ * but allows using it by providing the DT property
+ * "rohm,enable-hidden-gpo".
+ */
+static int bd71815_init_valid_mask(struct gpio_chip *gc,
+ unsigned long *valid_mask,
+ unsigned int ngpios)
+{
+ if (ngpios != 2)
+ return 0;
+
+ if (gc->parent && device_property_present(gc->parent,
+ "rohm,enable-hidden-gpo"))
+ *valid_mask = BD71815_TWO_GPIOS;
+ else
+ *valid_mask = BD71815_ONE_GPIO;
+
+ return 0;
+}
+
+static int gpo_bd71815_probe(struct platform_device *pdev)
+{
+ struct bd71815_gpio *g;
+ struct device *parent, *dev;
+
+ /*
+ * Bind devm lifetime to this platform device => use dev for devm.
+ * also the prints should originate from this device.
+ */
+ dev = &pdev->dev;
+ /* The device-tree and regmap come from MFD => use parent for that */
+ parent = dev->parent;
+
+ g = devm_kzalloc(dev, sizeof(*g), GFP_KERNEL);
+ if (!g)
+ return -ENOMEM;
+
+ g->chip = bd71815gpo_chip;
+
+ /*
+ * FIXME: As writing of this the sysfs interface for GPIO control does
+ * not respect the valid_mask. Do not trust it but rather set the ngpios
+ * to 1 if "rohm,enable-hidden-gpo" is not given.
+ *
+ * This check can be removed later if the sysfs export is fixed and
+ * if the fix is backported.
+ *
+ * For now it is safest to just set the ngpios though.
+ */
+ if (device_property_present(parent, "rohm,enable-hidden-gpo"))
+ g->chip.ngpio = 2;
+ else
+ g->chip.ngpio = 1;
+
+ g->chip.init_valid_mask = bd71815_init_valid_mask;
+ g->chip.base = -1;
+ g->chip.parent = parent;
+ g->regmap = dev_get_regmap(parent, NULL);
+ g->dev = dev;
+
+ return devm_gpiochip_add_data(dev, &g->chip, g);
+}
+
+static struct platform_driver gpo_bd71815_driver = {
+ .driver = {
+ .name = "bd71815-gpo",
+ },
+ .probe = gpo_bd71815_probe,
+};
+module_platform_driver(gpo_bd71815_driver);
+
+MODULE_ALIAS("platform:bd71815-gpo");
+MODULE_AUTHOR("Matti Vaittinen <matti.vaittinen@fi.rohmeurope.com>");
+MODULE_AUTHOR("Peter Yang <yanglsh@embest-tech.com>");
+MODULE_DESCRIPTION("GPO interface for BD71815");
+MODULE_LICENSE("GPL");
diff --git a/drivers/gpio/gpio-ich.c b/drivers/gpio/gpio-ich.c
index de56c013a658..3b31f5e9bf40 100644
--- a/drivers/gpio/gpio-ich.c
+++ b/drivers/gpio/gpio-ich.c
@@ -5,13 +5,11 @@
* Copyright (C) 2010 Extreme Engineering Solutions.
*/
-
#include <linux/bitops.h>
#include <linux/gpio/driver.h>
#include <linux/ioport.h>
#include <linux/mfd/lpc_ich.h>
#include <linux/module.h>
-#include <linux/pci.h>
#include <linux/platform_device.h>
#define DRV_NAME "gpio_ich"
diff --git a/drivers/gpio/gpio-it87.c b/drivers/gpio/gpio-it87.c
index 8f1be34953ce..f332341fd4c8 100644
--- a/drivers/gpio/gpio-it87.c
+++ b/drivers/gpio/gpio-it87.c
@@ -125,14 +125,6 @@ static inline int superio_inw(int reg)
return val;
}
-static inline void superio_outw(int val, int reg)
-{
- outb(reg++, REG);
- outb(val >> 8, VAL);
- outb(reg, REG);
- outb(val, VAL);
-}
-
static inline void superio_set_mask(int mask, int reg)
{
u8 curr_val = superio_inb(reg);
diff --git a/drivers/gpio/gpio-mockup.c b/drivers/gpio/gpio-mockup.c
index 28b757d34046..d7e73876a3b9 100644
--- a/drivers/gpio/gpio-mockup.c
+++ b/drivers/gpio/gpio-mockup.c
@@ -479,15 +479,10 @@ static struct platform_device *gpio_mockup_pdevs[GPIO_MOCKUP_MAX_GC];
static void gpio_mockup_unregister_pdevs(void)
{
- struct platform_device *pdev;
int i;
- for (i = 0; i < GPIO_MOCKUP_MAX_GC; i++) {
- pdev = gpio_mockup_pdevs[i];
-
- if (pdev)
- platform_device_unregister(pdev);
- }
+ for (i = 0; i < GPIO_MOCKUP_MAX_GC; i++)
+ platform_device_unregister(gpio_mockup_pdevs[i]);
}
static __init char **gpio_mockup_make_line_names(const char *label,
diff --git a/drivers/gpio/gpio-moxtet.c b/drivers/gpio/gpio-moxtet.c
index 8299909318f4..61f9efd6c64f 100644
--- a/drivers/gpio/gpio-moxtet.c
+++ b/drivers/gpio/gpio-moxtet.c
@@ -2,7 +2,7 @@
/*
* Turris Mox Moxtet GPIO expander
*
- * Copyright (C) 2018 Marek Behun <marek.behun@nic.cz>
+ * Copyright (C) 2018 Marek Behún <kabel@kernel.org>
*/
#include <linux/bitops.h>
@@ -174,6 +174,6 @@ static struct moxtet_driver moxtet_gpio_driver = {
};
module_moxtet_driver(moxtet_gpio_driver);
-MODULE_AUTHOR("Marek Behun <marek.behun@nic.cz>");
+MODULE_AUTHOR("Marek Behun <kabel@kernel.org>");
MODULE_DESCRIPTION("Turris Mox Moxtet GPIO expander");
MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpio/gpio-mpc8xxx.c b/drivers/gpio/gpio-mpc8xxx.c
index 6dfca83bcd90..4b9157a69fca 100644
--- a/drivers/gpio/gpio-mpc8xxx.c
+++ b/drivers/gpio/gpio-mpc8xxx.c
@@ -9,6 +9,7 @@
* kind, whether express or implied.
*/
+#include <linux/acpi.h>
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/spinlock.h>
@@ -18,6 +19,8 @@
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include <linux/of_platform.h>
+#include <linux/property.h>
+#include <linux/mod_devicetable.h>
#include <linux/slab.h>
#include <linux/irq.h>
#include <linux/gpio/driver.h>
@@ -303,8 +306,8 @@ static int mpc8xxx_probe(struct platform_device *pdev)
struct device_node *np = pdev->dev.of_node;
struct mpc8xxx_gpio_chip *mpc8xxx_gc;
struct gpio_chip *gc;
- const struct mpc8xxx_gpio_devtype *devtype =
- of_device_get_match_data(&pdev->dev);
+ const struct mpc8xxx_gpio_devtype *devtype = NULL;
+ struct fwnode_handle *fwnode;
int ret;
mpc8xxx_gc = devm_kzalloc(&pdev->dev, sizeof(*mpc8xxx_gc), GFP_KERNEL);
@@ -315,14 +318,14 @@ static int mpc8xxx_probe(struct platform_device *pdev)
raw_spin_lock_init(&mpc8xxx_gc->lock);
- mpc8xxx_gc->regs = of_iomap(np, 0);
- if (!mpc8xxx_gc->regs)
- return -ENOMEM;
+ mpc8xxx_gc->regs = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(mpc8xxx_gc->regs))
+ return PTR_ERR(mpc8xxx_gc->regs);
gc = &mpc8xxx_gc->gc;
gc->parent = &pdev->dev;
- if (of_property_read_bool(np, "little-endian")) {
+ if (device_property_read_bool(&pdev->dev, "little-endian")) {
ret = bgpio_init(gc, &pdev->dev, 4,
mpc8xxx_gc->regs + GPIO_DAT,
NULL, NULL,
@@ -345,6 +348,7 @@ static int mpc8xxx_probe(struct platform_device *pdev)
mpc8xxx_gc->direction_output = gc->direction_output;
+ devtype = device_get_match_data(&pdev->dev);
if (!devtype)
devtype = &mpc8xxx_gpio_devtype_default;
@@ -369,24 +373,29 @@ static int mpc8xxx_probe(struct platform_device *pdev)
* associated input enable must be set (GPIOxGPIE[IEn]=1) to propagate
* the port value to the GPIO Data Register.
*/
+ fwnode = dev_fwnode(&pdev->dev);
if (of_device_is_compatible(np, "fsl,qoriq-gpio") ||
of_device_is_compatible(np, "fsl,ls1028a-gpio") ||
- of_device_is_compatible(np, "fsl,ls1088a-gpio"))
+ of_device_is_compatible(np, "fsl,ls1088a-gpio") ||
+ is_acpi_node(fwnode))
gc->write_reg(mpc8xxx_gc->regs + GPIO_IBE, 0xffffffff);
ret = gpiochip_add_data(gc, mpc8xxx_gc);
if (ret) {
- pr_err("%pOF: GPIO chip registration failed with status %d\n",
- np, ret);
+ dev_err(&pdev->dev,
+ "GPIO chip registration failed with status %d\n", ret);
goto err;
}
- mpc8xxx_gc->irqn = irq_of_parse_and_map(np, 0);
+ mpc8xxx_gc->irqn = platform_get_irq(pdev, 0);
if (!mpc8xxx_gc->irqn)
return 0;
- mpc8xxx_gc->irq = irq_domain_add_linear(np, MPC8XXX_GPIO_PINS,
- &mpc8xxx_gpio_irq_ops, mpc8xxx_gc);
+ mpc8xxx_gc->irq = irq_domain_create_linear(fwnode,
+ MPC8XXX_GPIO_PINS,
+ &mpc8xxx_gpio_irq_ops,
+ mpc8xxx_gc);
+
if (!mpc8xxx_gc->irq)
return 0;
@@ -399,8 +408,9 @@ static int mpc8xxx_probe(struct platform_device *pdev)
IRQF_SHARED, "gpio-cascade",
mpc8xxx_gc);
if (ret) {
- dev_err(&pdev->dev, "%s: failed to devm_request_irq(%d), ret = %d\n",
- np->full_name, mpc8xxx_gc->irqn, ret);
+ dev_err(&pdev->dev,
+ "failed to devm_request_irq(%d), ret = %d\n",
+ mpc8xxx_gc->irqn, ret);
goto err;
}
@@ -425,12 +435,21 @@ static int mpc8xxx_remove(struct platform_device *pdev)
return 0;
}
+#ifdef CONFIG_ACPI
+static const struct acpi_device_id gpio_acpi_ids[] = {
+ {"NXP0031",},
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, gpio_acpi_ids);
+#endif
+
static struct platform_driver mpc8xxx_plat_driver = {
.probe = mpc8xxx_probe,
.remove = mpc8xxx_remove,
.driver = {
.name = "gpio-mpc8xxx",
.of_match_table = mpc8xxx_gpio_ids,
+ .acpi_match_table = ACPI_PTR(gpio_acpi_ids),
},
};
diff --git a/drivers/gpio/gpio-mxs.c b/drivers/gpio/gpio-mxs.c
index dfc0c1eb1b33..524b668eb1ac 100644
--- a/drivers/gpio/gpio-mxs.c
+++ b/drivers/gpio/gpio-mxs.c
@@ -60,11 +60,6 @@ static inline int is_imx23_gpio(struct mxs_gpio_port *port)
return port->devid == IMX23_GPIO;
}
-static inline int is_imx28_gpio(struct mxs_gpio_port *port)
-{
- return port->devid == IMX28_GPIO;
-}
-
/* Note: This driver assumes 32 GPIOs are handled in one register */
static int mxs_gpio_set_irq_type(struct irq_data *d, unsigned int type)
diff --git a/drivers/gpio/gpio-omap.c b/drivers/gpio/gpio-omap.c
index 41952bb818ad..ca23f72165ca 100644
--- a/drivers/gpio/gpio-omap.c
+++ b/drivers/gpio/gpio-omap.c
@@ -29,6 +29,7 @@
#define OMAP4_GPIO_DEBOUNCINGTIME_MASK 0xFF
struct gpio_regs {
+ u32 sysconfig;
u32 irqenable1;
u32 irqenable2;
u32 wake_en;
@@ -1069,6 +1070,7 @@ static void omap_gpio_init_context(struct gpio_bank *p)
const struct omap_gpio_reg_offs *regs = p->regs;
void __iomem *base = p->base;
+ p->context.sysconfig = readl_relaxed(base + regs->sysconfig);
p->context.ctrl = readl_relaxed(base + regs->ctrl);
p->context.oe = readl_relaxed(base + regs->direction);
p->context.wake_en = readl_relaxed(base + regs->wkup_en);
@@ -1088,6 +1090,7 @@ static void omap_gpio_restore_context(struct gpio_bank *bank)
const struct omap_gpio_reg_offs *regs = bank->regs;
void __iomem *base = bank->base;
+ writel_relaxed(bank->context.sysconfig, base + regs->sysconfig);
writel_relaxed(bank->context.wake_en, base + regs->wkup_en);
writel_relaxed(bank->context.ctrl, base + regs->ctrl);
writel_relaxed(bank->context.leveldetect0, base + regs->leveldetect0);
@@ -1115,6 +1118,10 @@ static void omap_gpio_idle(struct gpio_bank *bank, bool may_lose_context)
bank->saved_datain = readl_relaxed(base + bank->regs->datain);
+ /* Save syconfig, it's runtime value can be different from init value */
+ if (bank->loses_context)
+ bank->context.sysconfig = readl_relaxed(base + bank->regs->sysconfig);
+
if (!bank->enabled_non_wakeup_gpios)
goto update_gpio_context_count;
@@ -1279,6 +1286,7 @@ out_unlock:
static const struct omap_gpio_reg_offs omap2_gpio_regs = {
.revision = OMAP24XX_GPIO_REVISION,
+ .sysconfig = OMAP24XX_GPIO_SYSCONFIG,
.direction = OMAP24XX_GPIO_OE,
.datain = OMAP24XX_GPIO_DATAIN,
.dataout = OMAP24XX_GPIO_DATAOUT,
@@ -1302,6 +1310,7 @@ static const struct omap_gpio_reg_offs omap2_gpio_regs = {
static const struct omap_gpio_reg_offs omap4_gpio_regs = {
.revision = OMAP4_GPIO_REVISION,
+ .sysconfig = OMAP4_GPIO_SYSCONFIG,
.direction = OMAP4_GPIO_OE,
.datain = OMAP4_GPIO_DATAIN,
.dataout = OMAP4_GPIO_DATAOUT,
@@ -1364,15 +1373,14 @@ static int omap_gpio_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct device_node *node = dev->of_node;
- const struct of_device_id *match;
const struct omap_gpio_platform_data *pdata;
struct gpio_bank *bank;
struct irq_chip *irqc;
int ret;
- match = of_match_device(of_match_ptr(omap_gpio_match), dev);
+ pdata = device_get_match_data(dev);
- pdata = match ? match->data : dev_get_platdata(dev);
+ pdata = pdata ?: dev_get_platdata(dev);
if (!pdata)
return -EINVAL;
diff --git a/drivers/gpio/gpio-pca953x.c b/drivers/gpio/gpio-pca953x.c
index 5ea09fd01544..c91d05651596 100644
--- a/drivers/gpio/gpio-pca953x.c
+++ b/drivers/gpio/gpio-pca953x.c
@@ -113,8 +113,29 @@ MODULE_DEVICE_TABLE(i2c, pca953x_id);
#ifdef CONFIG_GPIO_PCA953X_IRQ
#include <linux/dmi.h>
-#include <linux/gpio.h>
-#include <linux/list.h>
+
+static const struct acpi_gpio_params pca953x_irq_gpios = { 0, 0, true };
+
+static const struct acpi_gpio_mapping pca953x_acpi_irq_gpios[] = {
+ { "irq-gpios", &pca953x_irq_gpios, 1, ACPI_GPIO_QUIRK_ABSOLUTE_NUMBER },
+ { }
+};
+
+static int pca953x_acpi_get_irq(struct device *dev)
+{
+ int ret;
+
+ ret = devm_acpi_dev_add_driver_gpios(dev, pca953x_acpi_irq_gpios);
+ if (ret)
+ dev_warn(dev, "can't add GPIO ACPI mapping\n");
+
+ ret = acpi_dev_gpio_irq_get_by(ACPI_COMPANION(dev), "irq-gpios", 0);
+ if (ret < 0)
+ return ret;
+
+ dev_info(dev, "ACPI interrupt quirk (IRQ %d)\n", ret);
+ return ret;
+}
static const struct dmi_system_id pca953x_dmi_acpi_irq_info[] = {
{
@@ -133,59 +154,6 @@ static const struct dmi_system_id pca953x_dmi_acpi_irq_info[] = {
},
{}
};
-
-#ifdef CONFIG_ACPI
-static int pca953x_acpi_get_pin(struct acpi_resource *ares, void *data)
-{
- struct acpi_resource_gpio *agpio;
- int *pin = data;
-
- if (acpi_gpio_get_irq_resource(ares, &agpio))
- *pin = agpio->pin_table[0];
- return 1;
-}
-
-static int pca953x_acpi_find_pin(struct device *dev)
-{
- struct acpi_device *adev = ACPI_COMPANION(dev);
- int pin = -ENOENT, ret;
- LIST_HEAD(r);
-
- ret = acpi_dev_get_resources(adev, &r, pca953x_acpi_get_pin, &pin);
- acpi_dev_free_resource_list(&r);
- if (ret < 0)
- return ret;
-
- return pin;
-}
-#else
-static inline int pca953x_acpi_find_pin(struct device *dev) { return -ENXIO; }
-#endif
-
-static int pca953x_acpi_get_irq(struct device *dev)
-{
- int pin, ret;
-
- pin = pca953x_acpi_find_pin(dev);
- if (pin < 0)
- return pin;
-
- dev_info(dev, "Applying ACPI interrupt quirk (GPIO %d)\n", pin);
-
- if (!gpio_is_valid(pin))
- return -EINVAL;
-
- ret = gpio_request(pin, "pca953x interrupt");
- if (ret)
- return ret;
-
- ret = gpio_to_irq(pin);
-
- /* When pin is used as an IRQ, no need to keep it requested */
- gpio_free(pin);
-
- return ret;
-}
#endif
static const struct acpi_device_id pca953x_acpi_ids[] = {
diff --git a/drivers/gpio/gpio-raspberrypi-exp.c b/drivers/gpio/gpio-raspberrypi-exp.c
index bb100e0124e6..64a552ecc2ad 100644
--- a/drivers/gpio/gpio-raspberrypi-exp.c
+++ b/drivers/gpio/gpio-raspberrypi-exp.c
@@ -208,7 +208,7 @@ static int rpi_exp_gpio_probe(struct platform_device *pdev)
return -ENOENT;
}
- fw = rpi_firmware_get(fw_node);
+ fw = devm_rpi_firmware_get(&pdev->dev, fw_node);
of_node_put(fw_node);
if (!fw)
return -EPROBE_DEFER;
diff --git a/drivers/gpio/gpio-realtek-otto.c b/drivers/gpio/gpio-realtek-otto.c
new file mode 100644
index 000000000000..cb64fb5a51aa
--- /dev/null
+++ b/drivers/gpio/gpio-realtek-otto.c
@@ -0,0 +1,325 @@
+// SPDX-License-Identifier: GPL-2.0-only
+
+#include <linux/gpio/driver.h>
+#include <linux/irq.h>
+#include <linux/minmax.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/property.h>
+
+/*
+ * Total register block size is 0x1C for one bank of four ports (A, B, C, D).
+ * An optional second bank, with ports E, F, G, and H, may be present, starting
+ * at register offset 0x1C.
+ */
+
+/*
+ * Pin select: (0) "normal", (1) "dedicate peripheral"
+ * Not used on RTL8380/RTL8390, peripheral selection is managed by control bits
+ * in the peripheral registers.
+ */
+#define REALTEK_GPIO_REG_CNR 0x00
+/* Clear bit (0) for input, set bit (1) for output */
+#define REALTEK_GPIO_REG_DIR 0x08
+#define REALTEK_GPIO_REG_DATA 0x0C
+/* Read bit for IRQ status, write 1 to clear IRQ */
+#define REALTEK_GPIO_REG_ISR 0x10
+/* Two bits per GPIO in IMR registers */
+#define REALTEK_GPIO_REG_IMR 0x14
+#define REALTEK_GPIO_REG_IMR_AB 0x14
+#define REALTEK_GPIO_REG_IMR_CD 0x18
+#define REALTEK_GPIO_IMR_LINE_MASK GENMASK(1, 0)
+#define REALTEK_GPIO_IRQ_EDGE_FALLING 1
+#define REALTEK_GPIO_IRQ_EDGE_RISING 2
+#define REALTEK_GPIO_IRQ_EDGE_BOTH 3
+
+#define REALTEK_GPIO_MAX 32
+#define REALTEK_GPIO_PORTS_PER_BANK 4
+
+/**
+ * realtek_gpio_ctrl - Realtek Otto GPIO driver data
+ *
+ * @gc: Associated gpio_chip instance
+ * @base: Base address of the register block for a GPIO bank
+ * @lock: Lock for accessing the IRQ registers and values
+ * @intr_mask: Mask for interrupts lines
+ * @intr_type: Interrupt type selection
+ *
+ * Because the interrupt mask register (IMR) combines the function of IRQ type
+ * selection and masking, two extra values are stored. @intr_mask is used to
+ * mask/unmask the interrupts for a GPIO port, and @intr_type is used to store
+ * the selected interrupt types. The logical AND of these values is written to
+ * IMR on changes.
+ */
+struct realtek_gpio_ctrl {
+ struct gpio_chip gc;
+ void __iomem *base;
+ raw_spinlock_t lock;
+ u16 intr_mask[REALTEK_GPIO_PORTS_PER_BANK];
+ u16 intr_type[REALTEK_GPIO_PORTS_PER_BANK];
+};
+
+/* Expand with more flags as devices with other quirks are added */
+enum realtek_gpio_flags {
+ /*
+ * Allow disabling interrupts, for cases where the port order is
+ * unknown. This may result in a port mismatch between ISR and IMR.
+ * An interrupt would appear to come from a different line than the
+ * line the IRQ handler was assigned to, causing uncaught interrupts.
+ */
+ GPIO_INTERRUPTS_DISABLED = BIT(0),
+};
+
+static struct realtek_gpio_ctrl *irq_data_to_ctrl(struct irq_data *data)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(data);
+
+ return container_of(gc, struct realtek_gpio_ctrl, gc);
+}
+
+/*
+ * Normal port order register access
+ *
+ * Port information is stored with the first port at offset 0, followed by the
+ * second, etc. Most registers store one bit per GPIO and use a u8 value per
+ * port. The two interrupt mask registers store two bits per GPIO, so use u16
+ * values.
+ */
+static void realtek_gpio_write_imr(struct realtek_gpio_ctrl *ctrl,
+ unsigned int port, u16 irq_type, u16 irq_mask)
+{
+ iowrite16(irq_type & irq_mask, ctrl->base + REALTEK_GPIO_REG_IMR + 2 * port);
+}
+
+static void realtek_gpio_clear_isr(struct realtek_gpio_ctrl *ctrl,
+ unsigned int port, u8 mask)
+{
+ iowrite8(mask, ctrl->base + REALTEK_GPIO_REG_ISR + port);
+}
+
+static u8 realtek_gpio_read_isr(struct realtek_gpio_ctrl *ctrl, unsigned int port)
+{
+ return ioread8(ctrl->base + REALTEK_GPIO_REG_ISR + port);
+}
+
+/* Set the rising and falling edge mask bits for a GPIO port pin */
+static u16 realtek_gpio_imr_bits(unsigned int pin, u16 value)
+{
+ return (value & REALTEK_GPIO_IMR_LINE_MASK) << 2 * pin;
+}
+
+static void realtek_gpio_irq_ack(struct irq_data *data)
+{
+ struct realtek_gpio_ctrl *ctrl = irq_data_to_ctrl(data);
+ irq_hw_number_t line = irqd_to_hwirq(data);
+ unsigned int port = line / 8;
+ unsigned int port_pin = line % 8;
+
+ realtek_gpio_clear_isr(ctrl, port, BIT(port_pin));
+}
+
+static void realtek_gpio_irq_unmask(struct irq_data *data)
+{
+ struct realtek_gpio_ctrl *ctrl = irq_data_to_ctrl(data);
+ unsigned int line = irqd_to_hwirq(data);
+ unsigned int port = line / 8;
+ unsigned int port_pin = line % 8;
+ unsigned long flags;
+ u16 m;
+
+ raw_spin_lock_irqsave(&ctrl->lock, flags);
+ m = ctrl->intr_mask[port];
+ m |= realtek_gpio_imr_bits(port_pin, REALTEK_GPIO_IMR_LINE_MASK);
+ ctrl->intr_mask[port] = m;
+ realtek_gpio_write_imr(ctrl, port, ctrl->intr_type[port], m);
+ raw_spin_unlock_irqrestore(&ctrl->lock, flags);
+}
+
+static void realtek_gpio_irq_mask(struct irq_data *data)
+{
+ struct realtek_gpio_ctrl *ctrl = irq_data_to_ctrl(data);
+ unsigned int line = irqd_to_hwirq(data);
+ unsigned int port = line / 8;
+ unsigned int port_pin = line % 8;
+ unsigned long flags;
+ u16 m;
+
+ raw_spin_lock_irqsave(&ctrl->lock, flags);
+ m = ctrl->intr_mask[port];
+ m &= ~realtek_gpio_imr_bits(port_pin, REALTEK_GPIO_IMR_LINE_MASK);
+ ctrl->intr_mask[port] = m;
+ realtek_gpio_write_imr(ctrl, port, ctrl->intr_type[port], m);
+ raw_spin_unlock_irqrestore(&ctrl->lock, flags);
+}
+
+static int realtek_gpio_irq_set_type(struct irq_data *data, unsigned int flow_type)
+{
+ struct realtek_gpio_ctrl *ctrl = irq_data_to_ctrl(data);
+ unsigned int line = irqd_to_hwirq(data);
+ unsigned int port = line / 8;
+ unsigned int port_pin = line % 8;
+ unsigned long flags;
+ u16 type, t;
+
+ switch (flow_type & IRQ_TYPE_SENSE_MASK) {
+ case IRQ_TYPE_EDGE_FALLING:
+ type = REALTEK_GPIO_IRQ_EDGE_FALLING;
+ break;
+ case IRQ_TYPE_EDGE_RISING:
+ type = REALTEK_GPIO_IRQ_EDGE_RISING;
+ break;
+ case IRQ_TYPE_EDGE_BOTH:
+ type = REALTEK_GPIO_IRQ_EDGE_BOTH;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ irq_set_handler_locked(data, handle_edge_irq);
+
+ raw_spin_lock_irqsave(&ctrl->lock, flags);
+ t = ctrl->intr_type[port];
+ t &= ~realtek_gpio_imr_bits(port_pin, REALTEK_GPIO_IMR_LINE_MASK);
+ t |= realtek_gpio_imr_bits(port_pin, type);
+ ctrl->intr_type[port] = t;
+ realtek_gpio_write_imr(ctrl, port, t, ctrl->intr_mask[port]);
+ raw_spin_unlock_irqrestore(&ctrl->lock, flags);
+
+ return 0;
+}
+
+static void realtek_gpio_irq_handler(struct irq_desc *desc)
+{
+ struct gpio_chip *gc = irq_desc_get_handler_data(desc);
+ struct realtek_gpio_ctrl *ctrl = gpiochip_get_data(gc);
+ struct irq_chip *irq_chip = irq_desc_get_chip(desc);
+ unsigned int lines_done;
+ unsigned int port_pin_count;
+ unsigned int irq;
+ unsigned long status;
+ int offset;
+
+ chained_irq_enter(irq_chip, desc);
+
+ for (lines_done = 0; lines_done < gc->ngpio; lines_done += 8) {
+ status = realtek_gpio_read_isr(ctrl, lines_done / 8);
+ port_pin_count = min(gc->ngpio - lines_done, 8U);
+ for_each_set_bit(offset, &status, port_pin_count) {
+ irq = irq_find_mapping(gc->irq.domain, offset);
+ generic_handle_irq(irq);
+ }
+ }
+
+ chained_irq_exit(irq_chip, desc);
+}
+
+static int realtek_gpio_irq_init(struct gpio_chip *gc)
+{
+ struct realtek_gpio_ctrl *ctrl = gpiochip_get_data(gc);
+ unsigned int port;
+
+ for (port = 0; (port * 8) < gc->ngpio; port++) {
+ realtek_gpio_write_imr(ctrl, port, 0, 0);
+ realtek_gpio_clear_isr(ctrl, port, GENMASK(7, 0));
+ }
+
+ return 0;
+}
+
+static struct irq_chip realtek_gpio_irq_chip = {
+ .name = "realtek-otto-gpio",
+ .irq_ack = realtek_gpio_irq_ack,
+ .irq_mask = realtek_gpio_irq_mask,
+ .irq_unmask = realtek_gpio_irq_unmask,
+ .irq_set_type = realtek_gpio_irq_set_type,
+};
+
+static const struct of_device_id realtek_gpio_of_match[] = {
+ {
+ .compatible = "realtek,otto-gpio",
+ .data = (void *)GPIO_INTERRUPTS_DISABLED,
+ },
+ {
+ .compatible = "realtek,rtl8380-gpio",
+ },
+ {
+ .compatible = "realtek,rtl8390-gpio",
+ },
+ {}
+};
+MODULE_DEVICE_TABLE(of, realtek_gpio_of_match);
+
+static int realtek_gpio_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ unsigned int dev_flags;
+ struct gpio_irq_chip *girq;
+ struct realtek_gpio_ctrl *ctrl;
+ u32 ngpios;
+ int err, irq;
+
+ ctrl = devm_kzalloc(dev, sizeof(*ctrl), GFP_KERNEL);
+ if (!ctrl)
+ return -ENOMEM;
+
+ dev_flags = (unsigned int) device_get_match_data(dev);
+
+ ngpios = REALTEK_GPIO_MAX;
+ device_property_read_u32(dev, "ngpios", &ngpios);
+
+ if (ngpios > REALTEK_GPIO_MAX) {
+ dev_err(&pdev->dev, "invalid ngpios (max. %d)\n",
+ REALTEK_GPIO_MAX);
+ return -EINVAL;
+ }
+
+ ctrl->base = devm_platform_ioremap_resource(pdev, 0);
+ if (IS_ERR(ctrl->base))
+ return PTR_ERR(ctrl->base);
+
+ raw_spin_lock_init(&ctrl->lock);
+
+ err = bgpio_init(&ctrl->gc, dev, 4,
+ ctrl->base + REALTEK_GPIO_REG_DATA, NULL, NULL,
+ ctrl->base + REALTEK_GPIO_REG_DIR, NULL,
+ BGPIOF_BIG_ENDIAN_BYTE_ORDER);
+ if (err) {
+ dev_err(dev, "unable to init generic GPIO");
+ return err;
+ }
+
+ ctrl->gc.ngpio = ngpios;
+ ctrl->gc.owner = THIS_MODULE;
+
+ irq = platform_get_irq_optional(pdev, 0);
+ if (!(dev_flags & GPIO_INTERRUPTS_DISABLED) && irq > 0) {
+ girq = &ctrl->gc.irq;
+ girq->chip = &realtek_gpio_irq_chip;
+ girq->default_type = IRQ_TYPE_NONE;
+ girq->handler = handle_bad_irq;
+ girq->parent_handler = realtek_gpio_irq_handler;
+ girq->num_parents = 1;
+ girq->parents = devm_kcalloc(dev, girq->num_parents,
+ sizeof(*girq->parents), GFP_KERNEL);
+ if (!girq->parents)
+ return -ENOMEM;
+ girq->parents[0] = irq;
+ girq->init_hw = realtek_gpio_irq_init;
+ }
+
+ return devm_gpiochip_add_data(dev, &ctrl->gc, ctrl);
+}
+
+static struct platform_driver realtek_gpio_driver = {
+ .driver = {
+ .name = "realtek-otto-gpio",
+ .of_match_table = realtek_gpio_of_match,
+ },
+ .probe = realtek_gpio_probe,
+};
+module_platform_driver(realtek_gpio_driver);
+
+MODULE_DESCRIPTION("Realtek Otto GPIO support");
+MODULE_AUTHOR("Sander Vanheule <sander@svanheule.net>");
+MODULE_LICENSE("GPL v2");
diff --git a/drivers/gpio/gpio-regmap.c b/drivers/gpio/gpio-regmap.c
index 5412cb3b0b2a..134cedf151a7 100644
--- a/drivers/gpio/gpio-regmap.c
+++ b/drivers/gpio/gpio-regmap.c
@@ -254,6 +254,11 @@ struct gpio_regmap *gpio_regmap_register(const struct gpio_regmap_config *config
chip->names = config->names;
chip->label = config->label ?: dev_name(config->parent);
+#if defined(CONFIG_OF_GPIO)
+ /* gpiolib will use of_node of the parent if chip->of_node is NULL */
+ chip->of_node = to_of_node(config->fwnode);
+#endif /* CONFIG_OF_GPIO */
+
/*
* If our regmap is fast_io we should probably set can_sleep to false.
* Right now, the regmap doesn't save this property, nor is there any
diff --git a/drivers/gpio/gpio-sch.c b/drivers/gpio/gpio-sch.c
index 3a1b1adb08c6..a6f0421d6e50 100644
--- a/drivers/gpio/gpio-sch.c
+++ b/drivers/gpio/gpio-sch.c
@@ -7,33 +7,55 @@
*/
#include <linux/acpi.h>
+#include <linux/bitops.h>
#include <linux/errno.h>
#include <linux/gpio/driver.h>
#include <linux/io.h>
+#include <linux/irq.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/pci_ids.h>
#include <linux/platform_device.h>
+#include <linux/types.h>
#define GEN 0x00
#define GIO 0x04
#define GLV 0x08
+#define GTPE 0x0c
+#define GTNE 0x10
+#define GGPE 0x14
+#define GSMI 0x18
+#define GTS 0x1c
+
+#define CORE_BANK_OFFSET 0x00
+#define RESUME_BANK_OFFSET 0x20
+
+/*
+ * iLB datasheet describes GPE0BLK registers, in particular GPE0E.GPIO bit.
+ * Document Number: 328195-001
+ */
+#define GPE0E_GPIO 14
struct sch_gpio {
struct gpio_chip chip;
+ struct irq_chip irqchip;
spinlock_t lock;
unsigned short iobase;
unsigned short resume_base;
+
+ /* GPE handling */
+ u32 gpe;
+ acpi_gpe_handler gpe_handler;
};
static unsigned int sch_gpio_offset(struct sch_gpio *sch, unsigned int gpio,
unsigned int reg)
{
- unsigned int base = 0;
+ unsigned int base = CORE_BANK_OFFSET;
if (gpio >= sch->resume_base) {
gpio -= sch->resume_base;
- base += 0x20;
+ base = RESUME_BANK_OFFSET;
}
return base + reg + gpio / 8;
@@ -79,10 +101,11 @@ static void sch_gpio_reg_set(struct sch_gpio *sch, unsigned int gpio, unsigned i
static int sch_gpio_direction_in(struct gpio_chip *gc, unsigned int gpio_num)
{
struct sch_gpio *sch = gpiochip_get_data(gc);
+ unsigned long flags;
- spin_lock(&sch->lock);
+ spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GIO, 1);
- spin_unlock(&sch->lock);
+ spin_unlock_irqrestore(&sch->lock, flags);
return 0;
}
@@ -96,20 +119,22 @@ static int sch_gpio_get(struct gpio_chip *gc, unsigned int gpio_num)
static void sch_gpio_set(struct gpio_chip *gc, unsigned int gpio_num, int val)
{
struct sch_gpio *sch = gpiochip_get_data(gc);
+ unsigned long flags;
- spin_lock(&sch->lock);
+ spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GLV, val);
- spin_unlock(&sch->lock);
+ spin_unlock_irqrestore(&sch->lock, flags);
}
static int sch_gpio_direction_out(struct gpio_chip *gc, unsigned int gpio_num,
int val)
{
struct sch_gpio *sch = gpiochip_get_data(gc);
+ unsigned long flags;
- spin_lock(&sch->lock);
+ spin_lock_irqsave(&sch->lock, flags);
sch_gpio_reg_set(sch, gpio_num, GIO, 0);
- spin_unlock(&sch->lock);
+ spin_unlock_irqrestore(&sch->lock, flags);
/*
* according to the datasheet, writing to the level register has no
@@ -144,10 +169,145 @@ static const struct gpio_chip sch_gpio_chip = {
.get_direction = sch_gpio_get_direction,
};
+static int sch_irq_type(struct irq_data *d, unsigned int type)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct sch_gpio *sch = gpiochip_get_data(gc);
+ irq_hw_number_t gpio_num = irqd_to_hwirq(d);
+ unsigned long flags;
+ int rising, falling;
+
+ switch (type & IRQ_TYPE_SENSE_MASK) {
+ case IRQ_TYPE_EDGE_RISING:
+ rising = 1;
+ falling = 0;
+ break;
+ case IRQ_TYPE_EDGE_FALLING:
+ rising = 0;
+ falling = 1;
+ break;
+ case IRQ_TYPE_EDGE_BOTH:
+ rising = 1;
+ falling = 1;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ spin_lock_irqsave(&sch->lock, flags);
+
+ sch_gpio_reg_set(sch, gpio_num, GTPE, rising);
+ sch_gpio_reg_set(sch, gpio_num, GTNE, falling);
+
+ irq_set_handler_locked(d, handle_edge_irq);
+
+ spin_unlock_irqrestore(&sch->lock, flags);
+
+ return 0;
+}
+
+static void sch_irq_ack(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct sch_gpio *sch = gpiochip_get_data(gc);
+ irq_hw_number_t gpio_num = irqd_to_hwirq(d);
+ unsigned long flags;
+
+ spin_lock_irqsave(&sch->lock, flags);
+ sch_gpio_reg_set(sch, gpio_num, GTS, 1);
+ spin_unlock_irqrestore(&sch->lock, flags);
+}
+
+static void sch_irq_mask_unmask(struct irq_data *d, int val)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct sch_gpio *sch = gpiochip_get_data(gc);
+ irq_hw_number_t gpio_num = irqd_to_hwirq(d);
+ unsigned long flags;
+
+ spin_lock_irqsave(&sch->lock, flags);
+ sch_gpio_reg_set(sch, gpio_num, GGPE, val);
+ spin_unlock_irqrestore(&sch->lock, flags);
+}
+
+static void sch_irq_mask(struct irq_data *d)
+{
+ sch_irq_mask_unmask(d, 0);
+}
+
+static void sch_irq_unmask(struct irq_data *d)
+{
+ sch_irq_mask_unmask(d, 1);
+}
+
+static u32 sch_gpio_gpe_handler(acpi_handle gpe_device, u32 gpe, void *context)
+{
+ struct sch_gpio *sch = context;
+ struct gpio_chip *gc = &sch->chip;
+ unsigned long core_status, resume_status;
+ unsigned long pending;
+ unsigned long flags;
+ int offset;
+ u32 ret;
+
+ spin_lock_irqsave(&sch->lock, flags);
+
+ core_status = inl(sch->iobase + CORE_BANK_OFFSET + GTS);
+ resume_status = inl(sch->iobase + RESUME_BANK_OFFSET + GTS);
+
+ spin_unlock_irqrestore(&sch->lock, flags);
+
+ pending = (resume_status << sch->resume_base) | core_status;
+ for_each_set_bit(offset, &pending, sch->chip.ngpio)
+ generic_handle_irq(irq_find_mapping(gc->irq.domain, offset));
+
+ /* Set returning value depending on whether we handled an interrupt */
+ ret = pending ? ACPI_INTERRUPT_HANDLED : ACPI_INTERRUPT_NOT_HANDLED;
+
+ /* Acknowledge GPE to ACPICA */
+ ret |= ACPI_REENABLE_GPE;
+
+ return ret;
+}
+
+static void sch_gpio_remove_gpe_handler(void *data)
+{
+ struct sch_gpio *sch = data;
+
+ acpi_disable_gpe(NULL, sch->gpe);
+ acpi_remove_gpe_handler(NULL, sch->gpe, sch->gpe_handler);
+}
+
+static int sch_gpio_install_gpe_handler(struct sch_gpio *sch)
+{
+ struct device *dev = sch->chip.parent;
+ acpi_status status;
+
+ status = acpi_install_gpe_handler(NULL, sch->gpe, ACPI_GPE_LEVEL_TRIGGERED,
+ sch->gpe_handler, sch);
+ if (ACPI_FAILURE(status)) {
+ dev_err(dev, "Failed to install GPE handler for %u: %s\n",
+ sch->gpe, acpi_format_exception(status));
+ return -ENODEV;
+ }
+
+ status = acpi_enable_gpe(NULL, sch->gpe);
+ if (ACPI_FAILURE(status)) {
+ dev_err(dev, "Failed to enable GPE handler for %u: %s\n",
+ sch->gpe, acpi_format_exception(status));
+ acpi_remove_gpe_handler(NULL, sch->gpe, sch->gpe_handler);
+ return -ENODEV;
+ }
+
+ return devm_add_action_or_reset(dev, sch_gpio_remove_gpe_handler, sch);
+}
+
static int sch_gpio_probe(struct platform_device *pdev)
{
+ struct gpio_irq_chip *girq;
struct sch_gpio *sch;
struct resource *res;
+ int ret;
sch = devm_kzalloc(&pdev->dev, sizeof(*sch), GFP_KERNEL);
if (!sch)
@@ -207,6 +367,28 @@ static int sch_gpio_probe(struct platform_device *pdev)
platform_set_drvdata(pdev, sch);
+ sch->irqchip.name = "sch_gpio";
+ sch->irqchip.irq_ack = sch_irq_ack;
+ sch->irqchip.irq_mask = sch_irq_mask;
+ sch->irqchip.irq_unmask = sch_irq_unmask;
+ sch->irqchip.irq_set_type = sch_irq_type;
+
+ girq = &sch->chip.irq;
+ girq->chip = &sch->irqchip;
+ girq->num_parents = 0;
+ girq->parents = NULL;
+ girq->parent_handler = NULL;
+ girq->default_type = IRQ_TYPE_NONE;
+ girq->handler = handle_bad_irq;
+
+ /* GPE setup is optional */
+ sch->gpe = GPE0E_GPIO;
+ sch->gpe_handler = sch_gpio_gpe_handler;
+
+ ret = sch_gpio_install_gpe_handler(sch);
+ if (ret)
+ dev_warn(&pdev->dev, "Can't setup GPE, no IRQ support\n");
+
return devm_gpiochip_add_data(&pdev->dev, &sch->chip, sch);
}
diff --git a/drivers/gpio/gpiolib-acpi.c b/drivers/gpio/gpiolib-acpi.c
index e37a57d0a2f0..3ef22a3c104d 100644
--- a/drivers/gpio/gpiolib-acpi.c
+++ b/drivers/gpio/gpiolib-acpi.c
@@ -174,7 +174,7 @@ static void acpi_gpiochip_request_irq(struct acpi_gpio_chip *acpi_gpio,
int ret, value;
ret = request_threaded_irq(event->irq, NULL, event->handler,
- event->irqflags, "ACPI:Event", event);
+ event->irqflags | IRQF_ONESHOT, "ACPI:Event", event);
if (ret) {
dev_err(acpi_gpio->chip->parent,
"Failed to setup interrupt handler for %d\n",
@@ -677,6 +677,7 @@ static int acpi_populate_gpio_lookup(struct acpi_resource *ares, void *data)
if (!lookup->desc) {
const struct acpi_resource_gpio *agpio = &ares->data.gpio;
bool gpioint = agpio->connection_type == ACPI_RESOURCE_GPIO_TYPE_INT;
+ struct gpio_desc *desc;
u16 pin_index;
if (lookup->info.quirks & ACPI_GPIO_QUIRK_ONLY_GPIOIO && gpioint)
@@ -689,8 +690,12 @@ static int acpi_populate_gpio_lookup(struct acpi_resource *ares, void *data)
if (pin_index >= agpio->pin_table_length)
return 1;
- lookup->desc = acpi_get_gpiod(agpio->resource_source.string_ptr,
+ if (lookup->info.quirks & ACPI_GPIO_QUIRK_ABSOLUTE_NUMBER)
+ desc = gpio_to_desc(agpio->pin_table[pin_index]);
+ else
+ desc = acpi_get_gpiod(agpio->resource_source.string_ptr,
agpio->pin_table[pin_index]);
+ lookup->desc = desc;
lookup->info.pin_config = agpio->pin_config;
lookup->info.debounce = agpio->debounce_timeout;
lookup->info.gpioint = gpioint;
@@ -940,8 +945,9 @@ struct gpio_desc *acpi_node_get_gpiod(struct fwnode_handle *fwnode,
}
/**
- * acpi_dev_gpio_irq_get() - Find GpioInt and translate it to Linux IRQ number
+ * acpi_dev_gpio_irq_get_by() - Find GpioInt and translate it to Linux IRQ number
* @adev: pointer to a ACPI device to get IRQ from
+ * @name: optional name of GpioInt resource
* @index: index of GpioInt resource (starting from %0)
*
* If the device has one or more GpioInt resources, this function can be
@@ -951,9 +957,12 @@ struct gpio_desc *acpi_node_get_gpiod(struct fwnode_handle *fwnode,
* The function is idempotent, though each time it runs it will configure GPIO
* pin direction according to the flags in GpioInt resource.
*
+ * The function takes optional @name parameter. If the resource has a property
+ * name, then only those will be taken into account.
+ *
* Return: Linux IRQ number (> %0) on success, negative errno on failure.
*/
-int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index)
+int acpi_dev_gpio_irq_get_by(struct acpi_device *adev, const char *name, int index)
{
int idx, i;
unsigned int irq_flags;
@@ -963,7 +972,7 @@ int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index)
struct acpi_gpio_info info;
struct gpio_desc *desc;
- desc = acpi_get_gpiod_by_index(adev, NULL, i, &info);
+ desc = acpi_get_gpiod_by_index(adev, name, i, &info);
/* Ignore -EPROBE_DEFER, it only matters if idx matches */
if (IS_ERR(desc) && PTR_ERR(desc) != -EPROBE_DEFER)
@@ -1008,7 +1017,7 @@ int acpi_dev_gpio_irq_get(struct acpi_device *adev, int index)
}
return -ENOENT;
}
-EXPORT_SYMBOL_GPL(acpi_dev_gpio_irq_get);
+EXPORT_SYMBOL_GPL(acpi_dev_gpio_irq_get_by);
static acpi_status
acpi_gpio_adr_space_handler(u32 function, acpi_physical_address address,
@@ -1282,6 +1291,13 @@ void acpi_gpiochip_remove(struct gpio_chip *chip)
kfree(acpi_gpio);
}
+void acpi_gpio_dev_init(struct gpio_chip *gc, struct gpio_device *gdev)
+{
+ /* Set default fwnode to parent's one if present */
+ if (gc->parent)
+ ACPI_COMPANION_SET(&gdev->dev, ACPI_COMPANION(gc->parent));
+}
+
static int acpi_gpio_package_count(const union acpi_object *obj)
{
const union acpi_object *element = obj->package.elements;
@@ -1431,6 +1447,20 @@ static const struct dmi_system_id gpiolib_acpi_quirks[] __initconst = {
},
{
/*
+ * The Dell Venue 10 Pro 5055, with Bay Trail SoC + TI PMIC uses an
+ * external embedded-controller connected via I2C + an ACPI GPIO
+ * event handler on INT33FFC:02 pin 12, causing spurious wakeups.
+ */
+ .matches = {
+ DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
+ DMI_MATCH(DMI_PRODUCT_NAME, "Venue 10 Pro 5055"),
+ },
+ .driver_data = &(struct acpi_gpiolib_dmi_quirk) {
+ .ignore_wake = "INT33FC:02@12",
+ },
+ },
+ {
+ /*
* HP X2 10 models with Cherry Trail SoC + TI PMIC use an
* external embedded-controller connected via I2C + an ACPI GPIO
* event handler on INT33FF:01 pin 0, causing spurious wakeups.
diff --git a/drivers/gpio/gpiolib-acpi.h b/drivers/gpio/gpiolib-acpi.h
index e2edb632b2cc..e476558d9471 100644
--- a/drivers/gpio/gpiolib-acpi.h
+++ b/drivers/gpio/gpiolib-acpi.h
@@ -36,6 +36,8 @@ struct acpi_gpio_info {
void acpi_gpiochip_add(struct gpio_chip *chip);
void acpi_gpiochip_remove(struct gpio_chip *chip);
+void acpi_gpio_dev_init(struct gpio_chip *gc, struct gpio_device *gdev);
+
void acpi_gpiochip_request_interrupts(struct gpio_chip *chip);
void acpi_gpiochip_free_interrupts(struct gpio_chip *chip);
@@ -58,6 +60,8 @@ int acpi_gpio_count(struct device *dev, const char *con_id);
static inline void acpi_gpiochip_add(struct gpio_chip *chip) { }
static inline void acpi_gpiochip_remove(struct gpio_chip *chip) { }
+static inline void acpi_gpio_dev_init(struct gpio_chip *gc, struct gpio_device *gdev) { }
+
static inline void
acpi_gpiochip_request_interrupts(struct gpio_chip *chip) { }
diff --git a/drivers/gpio/gpiolib-of.c b/drivers/gpio/gpiolib-of.c
index baf0153b7bca..bbcc7c073f63 100644
--- a/drivers/gpio/gpiolib-of.c
+++ b/drivers/gpio/gpiolib-of.c
@@ -1042,11 +1042,13 @@ void of_gpiochip_remove(struct gpio_chip *chip)
void of_gpio_dev_init(struct gpio_chip *gc, struct gpio_device *gdev)
{
+ /* Set default OF node to parent's one if present */
+ if (gc->parent)
+ gdev->dev.of_node = gc->parent->of_node;
+
/* If the gpiochip has an assigned OF node this takes precedence */
if (gc->of_node)
gdev->dev.of_node = gc->of_node;
else
gc->of_node = gdev->dev.of_node;
- if (gdev->dev.of_node)
- gdev->dev.fwnode = of_fwnode_handle(gdev->dev.of_node);
}
diff --git a/drivers/gpio/gpiolib-sysfs.c b/drivers/gpio/gpiolib-sysfs.c
index 26c5466b8179..ae49bb23c6ed 100644
--- a/drivers/gpio/gpiolib-sysfs.c
+++ b/drivers/gpio/gpiolib-sysfs.c
@@ -458,6 +458,8 @@ static ssize_t export_store(struct class *class,
long gpio;
struct gpio_desc *desc;
int status;
+ struct gpio_chip *gc;
+ int offset;
status = kstrtol(buf, 0, &gpio);
if (status < 0)
@@ -469,6 +471,12 @@ static ssize_t export_store(struct class *class,
pr_warn("%s: invalid GPIO %ld\n", __func__, gpio);
return -EINVAL;
}
+ gc = desc->gdev->chip;
+ offset = gpio_chip_hwgpio(desc);
+ if (!gpiochip_line_is_valid(gc, offset)) {
+ pr_warn("%s: GPIO %ld masked\n", __func__, gpio);
+ return -EINVAL;
+ }
/* No extra locking here; FLAG_SYSFS just signifies that the
* request and export were done by on behalf of userspace, so
diff --git a/drivers/gpio/gpiolib.c b/drivers/gpio/gpiolib.c
index adf55db080d8..1427c1be749b 100644
--- a/drivers/gpio/gpiolib.c
+++ b/drivers/gpio/gpiolib.c
@@ -367,22 +367,18 @@ static int gpiochip_set_desc_names(struct gpio_chip *gc)
*
* Looks for device property "gpio-line-names" and if it exists assigns
* GPIO line names for the chip. The memory allocated for the assigned
- * names belong to the underlying software node and should not be released
+ * names belong to the underlying firmware node and should not be released
* by the caller.
*/
static int devprop_gpiochip_set_names(struct gpio_chip *chip)
{
struct gpio_device *gdev = chip->gpiodev;
- struct device *dev = chip->parent;
+ struct fwnode_handle *fwnode = dev_fwnode(&gdev->dev);
const char **names;
int ret, i;
int count;
- /* GPIO chip may not have a parent device whose properties we inspect. */
- if (!dev)
- return 0;
-
- count = device_property_string_array_count(dev, "gpio-line-names");
+ count = fwnode_property_string_array_count(fwnode, "gpio-line-names");
if (count < 0)
return 0;
@@ -396,7 +392,7 @@ static int devprop_gpiochip_set_names(struct gpio_chip *chip)
if (!names)
return -ENOMEM;
- ret = device_property_read_string_array(dev, "gpio-line-names",
+ ret = fwnode_property_read_string_array(fwnode, "gpio-line-names",
names, count);
if (ret < 0) {
dev_warn(&gdev->dev, "failed to read GPIO line names\n");
@@ -474,9 +470,13 @@ EXPORT_SYMBOL_GPL(gpiochip_line_is_valid);
static void gpiodevice_release(struct device *dev)
{
- struct gpio_device *gdev = dev_get_drvdata(dev);
+ struct gpio_device *gdev = container_of(dev, struct gpio_device, dev);
+ unsigned long flags;
+ spin_lock_irqsave(&gpio_lock, flags);
list_del(&gdev->list);
+ spin_unlock_irqrestore(&gpio_lock, flags);
+
ida_free(&gpio_ida, gdev->id);
kfree_const(gdev->label);
kfree(gdev->descs);
@@ -571,6 +571,7 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data,
struct lock_class_key *lock_key,
struct lock_class_key *request_key)
{
+ struct fwnode_handle *fwnode = gc->parent ? dev_fwnode(gc->parent) : NULL;
unsigned long flags;
int ret = 0;
unsigned i;
@@ -585,14 +586,18 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data,
if (!gdev)
return -ENOMEM;
gdev->dev.bus = &gpio_bus_type;
+ gdev->dev.parent = gc->parent;
gdev->chip = gc;
gc->gpiodev = gdev;
- if (gc->parent) {
- gdev->dev.parent = gc->parent;
- gdev->dev.of_node = gc->parent->of_node;
- }
of_gpio_dev_init(gc, gdev);
+ acpi_gpio_dev_init(gc, gdev);
+
+ /*
+ * Assign fwnode depending on the result of the previous calls,
+ * if none of them succeed, assign it to the parent's one.
+ */
+ gdev->dev.fwnode = dev_fwnode(&gdev->dev) ?: fwnode;
gdev->id = ida_alloc(&gpio_ida, GFP_KERNEL);
if (gdev->id < 0) {
@@ -605,7 +610,6 @@ int gpiochip_add_data_with_key(struct gpio_chip *gc, void *data,
goto err_free_ida;
device_initialize(&gdev->dev);
- dev_set_drvdata(&gdev->dev, gdev);
if (gc->parent && gc->parent->driver)
gdev->owner = gc->parent->driver->owner;
else if (gc->owner)
@@ -1459,9 +1463,8 @@ static int gpiochip_add_irqchip(struct gpio_chip *gc,
struct lock_class_key *lock_key,
struct lock_class_key *request_key)
{
+ struct fwnode_handle *fwnode = dev_fwnode(&gc->gpiodev->dev);
struct irq_chip *irqchip = gc->irq.chip;
- const struct irq_domain_ops *ops = NULL;
- struct device_node *np;
unsigned int type;
unsigned int i;
@@ -1473,7 +1476,6 @@ static int gpiochip_add_irqchip(struct gpio_chip *gc,
return -EINVAL;
}
- np = gc->gpiodev->dev.of_node;
type = gc->irq.default_type;
/*
@@ -1481,16 +1483,10 @@ static int gpiochip_add_irqchip(struct gpio_chip *gc,
* used to configure the interrupts, as you may end up with
* conflicting triggers. Tell the user, and reset to NONE.
*/
- if (WARN(np && type != IRQ_TYPE_NONE,
- "%s: Ignoring %u default trigger\n", np->full_name, type))
+ if (WARN(fwnode && type != IRQ_TYPE_NONE,
+ "%pfw: Ignoring %u default trigger\n", fwnode, type))
type = IRQ_TYPE_NONE;
- if (has_acpi_companion(gc->parent) && type != IRQ_TYPE_NONE) {
- acpi_handle_warn(ACPI_HANDLE(gc->parent),
- "Ignoring %u default trigger\n", type);
- type = IRQ_TYPE_NONE;
- }
-
if (gc->to_irq)
chip_warn(gc, "to_irq is redefined in %s and you shouldn't rely on it\n", __func__);
@@ -1506,15 +1502,11 @@ static int gpiochip_add_irqchip(struct gpio_chip *gc,
return ret;
} else {
/* Some drivers provide custom irqdomain ops */
- if (gc->irq.domain_ops)
- ops = gc->irq.domain_ops;
-
- if (!ops)
- ops = &gpiochip_domain_ops;
- gc->irq.domain = irq_domain_add_simple(np,
+ gc->irq.domain = irq_domain_create_simple(fwnode,
gc->ngpio,
gc->irq.first,
- ops, gc);
+ gc->irq.domain_ops ?: &gpiochip_domain_ops,
+ gc);
if (!gc->irq.domain)
return -EINVAL;
}
@@ -3678,11 +3670,12 @@ EXPORT_SYMBOL_GPL(fwnode_gpiod_get_index);
*/
int gpiod_count(struct device *dev, const char *con_id)
{
+ const struct fwnode_handle *fwnode = dev ? dev_fwnode(dev) : NULL;
int count = -ENOENT;
- if (IS_ENABLED(CONFIG_OF) && dev && dev->of_node)
+ if (is_of_node(fwnode))
count = of_gpio_get_count(dev, con_id);
- else if (IS_ENABLED(CONFIG_ACPI) && dev && ACPI_HANDLE(dev))
+ else if (is_acpi_node(fwnode))
count = acpi_gpio_count(dev, con_id);
if (count < 0)
@@ -3820,18 +3813,17 @@ struct gpio_desc *__must_check gpiod_get_index(struct device *dev,
int ret;
/* Maybe we have a device name, maybe not */
const char *devname = dev ? dev_name(dev) : "?";
+ const struct fwnode_handle *fwnode = dev ? dev_fwnode(dev) : NULL;
dev_dbg(dev, "GPIO lookup for consumer %s\n", con_id);
- if (dev) {
- /* Using device tree? */
- if (IS_ENABLED(CONFIG_OF) && dev->of_node) {
- dev_dbg(dev, "using device tree for GPIO lookup\n");
- desc = of_find_gpio(dev, con_id, idx, &lookupflags);
- } else if (ACPI_COMPANION(dev)) {
- dev_dbg(dev, "using ACPI for GPIO lookup\n");
- desc = acpi_find_gpio(dev, con_id, idx, &flags, &lookupflags);
- }
+ /* Using device tree? */
+ if (is_of_node(fwnode)) {
+ dev_dbg(dev, "using device tree for GPIO lookup\n");
+ desc = of_find_gpio(dev, con_id, idx, &lookupflags);
+ } else if (is_acpi_node(fwnode)) {
+ dev_dbg(dev, "using ACPI for GPIO lookup\n");
+ desc = acpi_find_gpio(dev, con_id, idx, &flags, &lookupflags);
}
/*
@@ -3915,9 +3907,6 @@ struct gpio_desc *fwnode_get_named_gpiod(struct fwnode_handle *fwnode,
struct gpio_desc *desc = ERR_PTR(-ENODEV);
int ret;
- if (!fwnode)
- return ERR_PTR(-EINVAL);
-
if (is_of_node(fwnode)) {
desc = gpiod_get_from_of_node(to_of_node(fwnode),
propname, index,
@@ -3933,7 +3922,8 @@ struct gpio_desc *fwnode_get_named_gpiod(struct fwnode_handle *fwnode,
acpi_gpio_update_gpiod_flags(&dflags, &info);
acpi_gpio_update_gpiod_lookup_flags(&lflags, &info);
- }
+ } else
+ return ERR_PTR(-EINVAL);
/* Currently only ACPI takes this path */
ret = gpiod_request(desc, label);
@@ -4214,11 +4204,13 @@ EXPORT_SYMBOL_GPL(gpiod_put_array);
static int gpio_bus_match(struct device *dev, struct device_driver *drv)
{
+ struct fwnode_handle *fwnode = dev_fwnode(dev);
+
/*
* Only match if the fwnode doesn't already have a proper struct device
* created for it.
*/
- if (dev->fwnode && dev->fwnode->dev != dev)
+ if (fwnode && fwnode->dev != dev)
return 0;
return 1;
}
@@ -4257,7 +4249,8 @@ static int __init gpiolib_dev_init(void)
return ret;
}
- if (driver_register(&gpio_stub_drv) < 0) {
+ ret = driver_register(&gpio_stub_drv);
+ if (ret < 0) {
pr_err("gpiolib: could not register GPIO stub driver\n");
bus_unregister(&gpio_bus_type);
return ret;
diff --git a/drivers/gpu/drm/Kconfig b/drivers/gpu/drm/Kconfig
index e392a90ca687..3c16bd1afd87 100644
--- a/drivers/gpu/drm/Kconfig
+++ b/drivers/gpu/drm/Kconfig
@@ -228,6 +228,7 @@ source "drivers/gpu/drm/arm/Kconfig"
config DRM_RADEON
tristate "ATI Radeon"
depends on DRM && PCI && MMU
+ depends on AGP || !AGP
select FW_LOADER
select DRM_KMS_HELPER
select DRM_TTM
@@ -351,8 +352,6 @@ source "drivers/gpu/drm/vc4/Kconfig"
source "drivers/gpu/drm/etnaviv/Kconfig"
-source "drivers/gpu/drm/arc/Kconfig"
-
source "drivers/gpu/drm/hisilicon/Kconfig"
source "drivers/gpu/drm/mediatek/Kconfig"
@@ -385,6 +384,8 @@ source "drivers/gpu/drm/tidss/Kconfig"
source "drivers/gpu/drm/xlnx/Kconfig"
+source "drivers/gpu/drm/gud/Kconfig"
+
# Keep legacy drivers last
menuconfig DRM_LEGACY
diff --git a/drivers/gpu/drm/Makefile b/drivers/gpu/drm/Makefile
index 926adef289db..5279db4392df 100644
--- a/drivers/gpu/drm/Makefile
+++ b/drivers/gpu/drm/Makefile
@@ -7,7 +7,7 @@ drm-y := drm_auth.o drm_cache.o \
drm_file.o drm_gem.o drm_ioctl.o drm_irq.o \
drm_drv.o \
drm_sysfs.o drm_hashtab.o drm_mm.o \
- drm_crtc.o drm_fourcc.o drm_modes.o drm_edid.o \
+ drm_crtc.o drm_fourcc.o drm_modes.o drm_edid.o drm_displayid.o \
drm_encoder_slave.o \
drm_trace_points.o drm_prime.o \
drm_rect.o drm_vma_manager.o drm_flip_work.o \
@@ -44,7 +44,8 @@ drm_kms_helper-y := drm_bridge_connector.o drm_crtc_helper.o drm_dp_helper.o \
drm_plane_helper.o drm_dp_mst_topology.o drm_atomic_helper.o \
drm_kms_helper_common.o drm_dp_dual_mode_helper.o \
drm_simple_kms_helper.o drm_modeset_helper.o \
- drm_scdc_helper.o drm_gem_framebuffer_helper.o \
+ drm_scdc_helper.o drm_gem_atomic_helper.o \
+ drm_gem_framebuffer_helper.o \
drm_atomic_state_helper.o drm_damage_helper.o \
drm_format_helper.o drm_self_refresh_helper.o
@@ -110,7 +111,6 @@ obj-y += panel/
obj-y += bridge/
obj-$(CONFIG_DRM_FSL_DCU) += fsl-dcu/
obj-$(CONFIG_DRM_ETNAVIV) += etnaviv/
-obj-$(CONFIG_DRM_ARCPGU)+= arc/
obj-y += hisilicon/
obj-$(CONFIG_DRM_ZTE) += zte/
obj-$(CONFIG_DRM_MXSFB) += mxsfb/
@@ -125,3 +125,4 @@ obj-$(CONFIG_DRM_ASPEED_GFX) += aspeed/
obj-$(CONFIG_DRM_MCDE) += mcde/
obj-$(CONFIG_DRM_TIDSS) += tidss/
obj-y += xlnx/
+obj-y += gud/
diff --git a/drivers/gpu/drm/amd/amdgpu/Kconfig b/drivers/gpu/drm/amd/amdgpu/Kconfig
index 9375e7f12420..74a8105fd2c0 100644
--- a/drivers/gpu/drm/amd/amdgpu/Kconfig
+++ b/drivers/gpu/drm/amd/amdgpu/Kconfig
@@ -34,15 +34,6 @@ config DRM_AMDGPU_USERPTR
This option selects CONFIG_HMM and CONFIG_HMM_MIRROR if it
isn't already selected to enabled full userptr support.
-config DRM_AMDGPU_GART_DEBUGFS
- bool "Allow GART access through debugfs"
- depends on DRM_AMDGPU
- depends on DEBUG_FS
- default n
- help
- Selecting this option creates a debugfs file to inspect the mapped
- pages. Uses more memory for housekeeping, enable only for debugging.
-
source "drivers/gpu/drm/amd/acp/Kconfig"
source "drivers/gpu/drm/amd/display/Kconfig"
source "drivers/gpu/drm/amd/amdkfd/Kconfig"
diff --git a/drivers/gpu/drm/amd/amdgpu/Makefile b/drivers/gpu/drm/amd/amdgpu/Makefile
index 13ebb1f71e49..ee85e8aba636 100644
--- a/drivers/gpu/drm/amd/amdgpu/Makefile
+++ b/drivers/gpu/drm/amd/amdgpu/Makefile
@@ -71,7 +71,7 @@ amdgpu-y += \
vi.o mxgpu_vi.o nbio_v6_1.o soc15.o emu_soc.o mxgpu_ai.o nbio_v7_0.o vega10_reg_init.o \
vega20_reg_init.o nbio_v7_4.o nbio_v2_3.o nv.o navi10_reg_init.o navi14_reg_init.o \
arct_reg_init.o navi12_reg_init.o mxgpu_nv.o sienna_cichlid_reg_init.o vangogh_reg_init.o \
- nbio_v7_2.o dimgrey_cavefish_reg_init.o hdp_v4_0.o hdp_v5_0.o
+ nbio_v7_2.o dimgrey_cavefish_reg_init.o hdp_v4_0.o hdp_v5_0.o aldebaran_reg_init.o aldebaran.o
# add DF block
amdgpu-y += \
@@ -83,11 +83,12 @@ amdgpu-y += \
gmc_v7_0.o \
gmc_v8_0.o \
gfxhub_v1_0.o mmhub_v1_0.o gmc_v9_0.o gfxhub_v1_1.o mmhub_v9_4.o \
- gfxhub_v2_0.o mmhub_v2_0.o gmc_v10_0.o gfxhub_v2_1.o mmhub_v2_3.o
+ gfxhub_v2_0.o mmhub_v2_0.o gmc_v10_0.o gfxhub_v2_1.o mmhub_v2_3.o \
+ mmhub_v1_7.o
# add UMC block
amdgpu-y += \
- umc_v6_1.o umc_v6_0.o umc_v8_7.o
+ umc_v6_0.o umc_v6_1.o umc_v6_7.o umc_v8_7.o
# add IH block
amdgpu-y += \
@@ -106,7 +107,8 @@ amdgpu-y += \
psp_v3_1.o \
psp_v10_0.o \
psp_v11_0.o \
- psp_v12_0.o
+ psp_v12_0.o \
+ psp_v13_0.o
# add DCE block
amdgpu-y += \
@@ -121,6 +123,7 @@ amdgpu-y += \
gfx_v8_0.o \
gfx_v9_0.o \
gfx_v9_4.o \
+ gfx_v9_4_2.o \
gfx_v10_0.o
# add async DMA block
@@ -129,6 +132,7 @@ amdgpu-y += \
sdma_v2_4.o \
sdma_v3_0.o \
sdma_v4_0.o \
+ sdma_v4_4.o \
sdma_v5_0.o \
sdma_v5_2.o
@@ -172,11 +176,17 @@ amdgpu-y += \
amdgpu-y += \
smuio_v9_0.o \
smuio_v11_0.o \
- smuio_v11_0_6.o
+ smuio_v11_0_6.o \
+ smuio_v13_0.o
+
+# add reset block
+amdgpu-y += \
+ amdgpu_reset.o
# add amdkfd interfaces
amdgpu-y += amdgpu_amdkfd.o
+
ifneq ($(CONFIG_HSA_AMD),)
AMDKFD_PATH := ../amdkfd
include $(FULL_AMD_PATH)/amdkfd/Makefile
@@ -187,6 +197,7 @@ amdgpu-y += \
amdgpu_amdkfd_gfx_v8.o \
amdgpu_amdkfd_gfx_v9.o \
amdgpu_amdkfd_arcturus.o \
+ amdgpu_amdkfd_aldebaran.o \
amdgpu_amdkfd_gfx_v10.o \
amdgpu_amdkfd_gfx_v10_3.o
diff --git a/drivers/gpu/drm/amd/amdgpu/aldebaran.c b/drivers/gpu/drm/amd/amdgpu/aldebaran.c
new file mode 100644
index 000000000000..65b1dca4b02e
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/aldebaran.c
@@ -0,0 +1,407 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "aldebaran.h"
+#include "amdgpu_reset.h"
+#include "amdgpu_amdkfd.h"
+#include "amdgpu_dpm.h"
+#include "amdgpu_job.h"
+#include "amdgpu_ring.h"
+#include "amdgpu_ras.h"
+#include "amdgpu_psp.h"
+#include "amdgpu_xgmi.h"
+
+static struct amdgpu_reset_handler *
+aldebaran_get_reset_handler(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ struct amdgpu_reset_handler *handler;
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+
+ if (reset_context->method != AMD_RESET_METHOD_NONE) {
+ dev_dbg(adev->dev, "Getting reset handler for method %d\n",
+ reset_context->method);
+ list_for_each_entry(handler, &reset_ctl->reset_handlers,
+ handler_list) {
+ if (handler->reset_method == reset_context->method)
+ return handler;
+ }
+ }
+
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ list_for_each_entry(handler, &reset_ctl->reset_handlers,
+ handler_list) {
+ if (handler->reset_method == AMD_RESET_METHOD_MODE2) {
+ reset_context->method = AMD_RESET_METHOD_MODE2;
+ return handler;
+ }
+ }
+ }
+
+ dev_dbg(adev->dev, "Reset handler not found!\n");
+
+ return NULL;
+}
+
+static int aldebaran_mode2_suspend_ip(struct amdgpu_device *adev)
+{
+ int r, i;
+
+ amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
+ amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
+
+ for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
+ if (!(adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_GFX ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_SDMA))
+ continue;
+
+ r = adev->ip_blocks[i].version->funcs->suspend(adev);
+
+ if (r) {
+ dev_err(adev->dev,
+ "suspend of IP block <%s> failed %d\n",
+ adev->ip_blocks[i].version->funcs->name, r);
+ return r;
+ }
+
+ adev->ip_blocks[i].status.hw = false;
+ }
+
+ return r;
+}
+
+static int
+aldebaran_mode2_prepare_hwcontext(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ int r = 0;
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+
+ dev_dbg(adev->dev, "Aldebaran prepare hw context\n");
+ /* Don't suspend on bare metal if we are not going to HW reset the ASIC */
+ if (!amdgpu_sriov_vf(adev))
+ r = aldebaran_mode2_suspend_ip(adev);
+
+ return r;
+}
+
+static void aldebaran_async_reset(struct work_struct *work)
+{
+ struct amdgpu_reset_handler *handler;
+ struct amdgpu_reset_control *reset_ctl =
+ container_of(work, struct amdgpu_reset_control, reset_work);
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+
+ list_for_each_entry(handler, &reset_ctl->reset_handlers,
+ handler_list) {
+ if (handler->reset_method == reset_ctl->active_reset) {
+ dev_dbg(adev->dev, "Resetting device\n");
+ handler->do_reset(adev);
+ break;
+ }
+ }
+}
+
+static int aldebaran_mode2_reset(struct amdgpu_device *adev)
+{
+ /* disable BM */
+ pci_clear_master(adev->pdev);
+ adev->asic_reset_res = amdgpu_dpm_mode2_reset(adev);
+ return adev->asic_reset_res;
+}
+
+static int
+aldebaran_mode2_perform_reset(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ struct amdgpu_device *tmp_adev = NULL;
+ struct amdgpu_device *adev = (struct amdgpu_device *)reset_ctl->handle;
+ int r = 0;
+
+ dev_dbg(adev->dev, "aldebaran perform hw reset\n");
+ if (reset_context->hive == NULL) {
+ /* Wrong context, return error */
+ return -EINVAL;
+ }
+
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ mutex_lock(&tmp_adev->reset_cntl->reset_lock);
+ tmp_adev->reset_cntl->active_reset = AMD_RESET_METHOD_MODE2;
+ }
+ /*
+ * Mode2 reset doesn't need any sync between nodes in XGMI hive, instead launch
+ * them together so that they can be completed asynchronously on multiple nodes
+ */
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ /* For XGMI run all resets in parallel to speed up the process */
+ if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
+ if (!queue_work(system_unbound_wq,
+ &tmp_adev->reset_cntl->reset_work))
+ r = -EALREADY;
+ } else
+ r = aldebaran_mode2_reset(tmp_adev);
+ if (r) {
+ dev_err(tmp_adev->dev,
+ "ASIC reset failed with error, %d for drm dev, %s",
+ r, adev_to_drm(tmp_adev)->unique);
+ break;
+ }
+ }
+
+ /* For XGMI wait for all resets to complete before proceed */
+ if (!r) {
+ list_for_each_entry(tmp_adev,
+ &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
+ flush_work(&tmp_adev->reset_cntl->reset_work);
+ r = tmp_adev->asic_reset_res;
+ if (r)
+ break;
+ }
+ }
+ }
+
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ mutex_unlock(&tmp_adev->reset_cntl->reset_lock);
+ tmp_adev->reset_cntl->active_reset = AMD_RESET_METHOD_NONE;
+ }
+
+ return r;
+}
+
+static int aldebaran_mode2_restore_ip(struct amdgpu_device *adev)
+{
+ struct amdgpu_firmware_info *ucode_list[AMDGPU_UCODE_ID_MAXIMUM];
+ struct amdgpu_firmware_info *ucode;
+ struct amdgpu_ip_block *cmn_block;
+ int ucode_count = 0;
+ int i, r;
+
+ dev_dbg(adev->dev, "Reloading ucodes after reset\n");
+ for (i = 0; i < adev->firmware.max_ucodes; i++) {
+ ucode = &adev->firmware.ucode[i];
+ if (!ucode->fw)
+ continue;
+ switch (ucode->ucode_id) {
+ case AMDGPU_UCODE_ID_SDMA0:
+ case AMDGPU_UCODE_ID_SDMA1:
+ case AMDGPU_UCODE_ID_SDMA2:
+ case AMDGPU_UCODE_ID_SDMA3:
+ case AMDGPU_UCODE_ID_SDMA4:
+ case AMDGPU_UCODE_ID_SDMA5:
+ case AMDGPU_UCODE_ID_SDMA6:
+ case AMDGPU_UCODE_ID_SDMA7:
+ case AMDGPU_UCODE_ID_CP_MEC1:
+ case AMDGPU_UCODE_ID_CP_MEC1_JT:
+ case AMDGPU_UCODE_ID_RLC_RESTORE_LIST_CNTL:
+ case AMDGPU_UCODE_ID_RLC_RESTORE_LIST_GPM_MEM:
+ case AMDGPU_UCODE_ID_RLC_RESTORE_LIST_SRM_MEM:
+ case AMDGPU_UCODE_ID_RLC_G:
+ ucode_list[ucode_count++] = ucode;
+ break;
+ default:
+ break;
+ };
+ }
+
+ /* Reinit NBIF block */
+ cmn_block =
+ amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_COMMON);
+ if (unlikely(!cmn_block)) {
+ dev_err(adev->dev, "Failed to get BIF handle\n");
+ return -EINVAL;
+ }
+ r = cmn_block->version->funcs->resume(adev);
+ if (r)
+ return r;
+
+ /* Reinit GFXHUB */
+ adev->gfxhub.funcs->init(adev);
+ r = adev->gfxhub.funcs->gart_enable(adev);
+ if (r) {
+ dev_err(adev->dev, "GFXHUB gart reenable failed after reset\n");
+ return r;
+ }
+
+ /* Reload GFX firmware */
+ r = psp_load_fw_list(&adev->psp, ucode_list, ucode_count);
+ if (r) {
+ dev_err(adev->dev, "GFX ucode load failed after reset\n");
+ return r;
+ }
+
+ /* Resume RLC, FW needs RLC alive to complete reset process */
+ adev->gfx.rlc.funcs->resume(adev);
+
+ /* Wait for FW reset event complete */
+ r = smu_wait_for_event(adev, SMU_EVENT_RESET_COMPLETE, 0);
+ if (r) {
+ dev_err(adev->dev,
+ "Failed to get response from firmware after reset\n");
+ return r;
+ }
+
+ for (i = 0; i < adev->num_ip_blocks; i++) {
+ if (!(adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_GFX ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_SDMA))
+ continue;
+ r = adev->ip_blocks[i].version->funcs->resume(adev);
+ if (r) {
+ dev_err(adev->dev,
+ "resume of IP block <%s> failed %d\n",
+ adev->ip_blocks[i].version->funcs->name, r);
+ return r;
+ }
+
+ adev->ip_blocks[i].status.hw = true;
+ }
+
+ for (i = 0; i < adev->num_ip_blocks; i++) {
+ if (!(adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_GFX ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_SDMA ||
+ adev->ip_blocks[i].version->type ==
+ AMD_IP_BLOCK_TYPE_COMMON))
+ continue;
+
+ if (adev->ip_blocks[i].version->funcs->late_init) {
+ r = adev->ip_blocks[i].version->funcs->late_init(
+ (void *)adev);
+ if (r) {
+ dev_err(adev->dev,
+ "late_init of IP block <%s> failed %d after reset\n",
+ adev->ip_blocks[i].version->funcs->name,
+ r);
+ return r;
+ }
+ }
+ adev->ip_blocks[i].status.late_initialized = true;
+ }
+
+ amdgpu_device_set_cg_state(adev, AMD_CG_STATE_GATE);
+ amdgpu_device_set_pg_state(adev, AMD_PG_STATE_GATE);
+
+ return r;
+}
+
+static int
+aldebaran_mode2_restore_hwcontext(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *reset_context)
+{
+ int r;
+ struct amdgpu_device *tmp_adev = NULL;
+
+ if (reset_context->hive == NULL) {
+ /* Wrong context, return error */
+ return -EINVAL;
+ }
+
+ list_for_each_entry(tmp_adev, &reset_context->hive->device_list,
+ gmc.xgmi.head) {
+ dev_info(tmp_adev->dev,
+ "GPU reset succeeded, trying to resume\n");
+ r = aldebaran_mode2_restore_ip(tmp_adev);
+ if (r)
+ goto end;
+
+ /*
+ * Add this ASIC as tracked as reset was already
+ * complete successfully.
+ */
+ amdgpu_register_gpu_instance(tmp_adev);
+
+ /* Resume RAS */
+ amdgpu_ras_resume(tmp_adev);
+
+ /* Update PSP FW topology after reset */
+ if (reset_context->hive &&
+ tmp_adev->gmc.xgmi.num_physical_nodes > 1)
+ r = amdgpu_xgmi_update_topology(reset_context->hive,
+ tmp_adev);
+
+ if (!r) {
+ amdgpu_irq_gpu_reset_resume_helper(tmp_adev);
+
+ r = amdgpu_ib_ring_tests(tmp_adev);
+ if (r) {
+ dev_err(tmp_adev->dev,
+ "ib ring test failed (%d).\n", r);
+ r = -EAGAIN;
+ tmp_adev->asic_reset_res = r;
+ goto end;
+ }
+ }
+ }
+
+end:
+ return r;
+}
+
+static struct amdgpu_reset_handler aldebaran_mode2_handler = {
+ .reset_method = AMD_RESET_METHOD_MODE2,
+ .prepare_env = NULL,
+ .prepare_hwcontext = aldebaran_mode2_prepare_hwcontext,
+ .perform_reset = aldebaran_mode2_perform_reset,
+ .restore_hwcontext = aldebaran_mode2_restore_hwcontext,
+ .restore_env = NULL,
+ .do_reset = aldebaran_mode2_reset,
+};
+
+int aldebaran_reset_init(struct amdgpu_device *adev)
+{
+ struct amdgpu_reset_control *reset_ctl;
+
+ reset_ctl = kzalloc(sizeof(*reset_ctl), GFP_KERNEL);
+ if (!reset_ctl)
+ return -ENOMEM;
+
+ reset_ctl->handle = adev;
+ reset_ctl->async_reset = aldebaran_async_reset;
+ reset_ctl->active_reset = AMD_RESET_METHOD_NONE;
+ reset_ctl->get_reset_handler = aldebaran_get_reset_handler;
+
+ INIT_LIST_HEAD(&reset_ctl->reset_handlers);
+ INIT_WORK(&reset_ctl->reset_work, reset_ctl->async_reset);
+ /* Only mode2 is handled through reset control now */
+ amdgpu_reset_add_handler(reset_ctl, &aldebaran_mode2_handler);
+
+ adev->reset_cntl = reset_ctl;
+
+ return 0;
+}
+
+int aldebaran_reset_fini(struct amdgpu_device *adev)
+{
+ kfree(adev->reset_cntl);
+ adev->reset_cntl = NULL;
+ return 0;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/aldebaran.h b/drivers/gpu/drm/amd/amdgpu/aldebaran.h
new file mode 100644
index 000000000000..a07db5454d49
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/aldebaran.h
@@ -0,0 +1,32 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __ALDEBARAN_H__
+#define __ALDEBARAN_H__
+
+#include "amdgpu.h"
+
+int aldebaran_reset_init(struct amdgpu_device *adev);
+int aldebaran_reset_fini(struct amdgpu_device *adev);
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/aldebaran_reg_init.c b/drivers/gpu/drm/amd/amdgpu/aldebaran_reg_init.c
new file mode 100644
index 000000000000..28e6c9ab8767
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/aldebaran_reg_init.c
@@ -0,0 +1,54 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "amdgpu.h"
+#include "soc15.h"
+
+#include "soc15_common.h"
+#include "aldebaran_ip_offset.h"
+
+int aldebaran_reg_base_init(struct amdgpu_device *adev)
+{
+ /* HW has more IP blocks, only initialized the block needed by our driver */
+ uint32_t i;
+ for (i = 0 ; i < MAX_INSTANCE ; ++i) {
+ adev->reg_offset[GC_HWIP][i] = (uint32_t *)(&(GC_BASE.instance[i]));
+ adev->reg_offset[HDP_HWIP][i] = (uint32_t *)(&(HDP_BASE.instance[i]));
+ adev->reg_offset[MMHUB_HWIP][i] = (uint32_t *)(&(MMHUB_BASE.instance[i]));
+ adev->reg_offset[ATHUB_HWIP][i] = (uint32_t *)(&(ATHUB_BASE.instance[i]));
+ adev->reg_offset[NBIO_HWIP][i] = (uint32_t *)(&(NBIO_BASE.instance[i]));
+ adev->reg_offset[MP0_HWIP][i] = (uint32_t *)(&(MP0_BASE.instance[i]));
+ adev->reg_offset[MP1_HWIP][i] = (uint32_t *)(&(MP1_BASE.instance[i]));
+ adev->reg_offset[DF_HWIP][i] = (uint32_t *)(&(DF_BASE.instance[i]));
+ adev->reg_offset[OSSSYS_HWIP][i] = (uint32_t *)(&(OSSSYS_BASE.instance[i]));
+ adev->reg_offset[SDMA0_HWIP][i] = (uint32_t *)(&(SDMA0_BASE.instance[i]));
+ adev->reg_offset[SDMA1_HWIP][i] = (uint32_t *)(&(SDMA1_BASE.instance[i]));
+ adev->reg_offset[SDMA2_HWIP][i] = (uint32_t *)(&(SDMA2_BASE.instance[i]));
+ adev->reg_offset[SDMA3_HWIP][i] = (uint32_t *)(&(SDMA3_BASE.instance[i]));
+ adev->reg_offset[SDMA4_HWIP][i] = (uint32_t *)(&(SDMA4_BASE.instance[i]));
+ adev->reg_offset[SMUIO_HWIP][i] = (uint32_t *)(&(SMUIO_BASE.instance[i]));
+ adev->reg_offset[THM_HWIP][i] = (uint32_t *)(&(THM_BASE.instance[i]));
+ adev->reg_offset[UMC_HWIP][i] = (uint32_t *)(&(UMC_BASE.instance[i]));
+ adev->reg_offset[VCN_HWIP][i] = (uint32_t *)(&(VCN_BASE.instance[i]));
+ }
+ return 0;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index b6879d97c9c9..264176a01e16 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -107,7 +107,6 @@
#include "amdgpu_gfxhub.h"
#include "amdgpu_df.h"
#include "amdgpu_smuio.h"
-#include "amdgpu_hdp.h"
#define MAX_GPU_INSTANCE 16
@@ -124,6 +123,16 @@ struct amdgpu_mgpu_info
uint32_t num_gpu;
uint32_t num_dgpu;
uint32_t num_apu;
+
+ /* delayed reset_func for XGMI configuration if necessary */
+ struct delayed_work delayed_reset_work;
+ bool pending_reset;
+};
+
+struct amdgpu_watchdog_timer
+{
+ bool timeout_fatal_disable;
+ uint32_t period; /* maxCycles = (1 << period), the number of cycles before a timeout */
};
#define AMDGPU_MAX_TIMEOUT_PARAM_LENGTH 256
@@ -177,13 +186,17 @@ extern int amdgpu_compute_multipipe;
extern int amdgpu_gpu_recovery;
extern int amdgpu_emu_mode;
extern uint amdgpu_smu_memory_pool_size;
+extern int amdgpu_smu_pptable_id;
extern uint amdgpu_dc_feature_mask;
+extern uint amdgpu_freesync_vid_mode;
extern uint amdgpu_dc_debug_mask;
extern uint amdgpu_dm_abm_level;
+extern int amdgpu_backlight;
extern struct amdgpu_mgpu_info mgpu_info;
extern int amdgpu_ras_enable;
extern uint amdgpu_ras_mask;
extern int amdgpu_bad_page_threshold;
+extern struct amdgpu_watchdog_timer amdgpu_watchdog_timer;
extern int amdgpu_async_gfx_ring;
extern int amdgpu_mcbp;
extern int amdgpu_discovery;
@@ -257,6 +270,8 @@ struct amdgpu_bo_va_mapping;
struct amdgpu_atif;
struct kfd_vm_fault_info;
struct amdgpu_hive_info;
+struct amdgpu_reset_context;
+struct amdgpu_reset_control;
enum amdgpu_cp_irq {
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP = 0,
@@ -575,6 +590,7 @@ struct amdgpu_allowed_register_entry {
};
enum amd_reset_method {
+ AMD_RESET_METHOD_NONE = -1,
AMD_RESET_METHOD_LEGACY = 0,
AMD_RESET_METHOD_MODE0,
AMD_RESET_METHOD_MODE1,
@@ -583,6 +599,19 @@ enum amd_reset_method {
AMD_RESET_METHOD_PCI,
};
+struct amdgpu_video_codec_info {
+ u32 codec_type;
+ u32 max_width;
+ u32 max_height;
+ u32 max_pixels_per_frame;
+ u32 max_level;
+};
+
+struct amdgpu_video_codecs {
+ const u32 codec_count;
+ const struct amdgpu_video_codec_info *codec_array;
+};
+
/*
* ASIC specific functions.
*/
@@ -627,6 +656,9 @@ struct amdgpu_asic_funcs {
void (*pre_asic_init)(struct amdgpu_device *adev);
/* enter/exit umd stable pstate */
int (*update_umd_stable_pstate)(struct amdgpu_device *adev, bool enter);
+ /* query video codecs */
+ int (*query_video_codecs)(struct amdgpu_device *adev, bool encode,
+ const struct amdgpu_video_codecs **codecs);
};
/*
@@ -791,12 +823,7 @@ struct amdgpu_device {
bool accel_working;
struct notifier_block acpi_nb;
struct amdgpu_i2c_chan *i2c_bus[AMDGPU_MAX_I2C_BUS];
- struct amdgpu_debugfs debugfs[AMDGPU_DEBUGFS_MAX_COMPONENTS];
- unsigned debugfs_count;
-#if defined(CONFIG_DEBUG_FS)
- struct dentry *debugfs_preempt;
- struct dentry *debugfs_regs[AMDGPU_DEBUGFS_MAX_COMPONENTS];
-#endif
+ struct debugfs_blob_wrapper debugfs_vbios_blob;
struct amdgpu_atif *atif;
struct amdgpu_atcs atcs;
struct mutex srbm_mutex;
@@ -852,8 +879,6 @@ struct amdgpu_device {
spinlock_t audio_endpt_idx_lock;
amdgpu_block_rreg_t audio_endpt_rreg;
amdgpu_block_wreg_t audio_endpt_wreg;
- void __iomem *rio_mem;
- resource_size_t rio_mem_size;
struct amdgpu_doorbell doorbell;
/* clock/pll info */
@@ -896,6 +921,8 @@ struct amdgpu_device {
struct amdgpu_irq_src vupdate_irq;
struct amdgpu_irq_src pageflip_irq;
struct amdgpu_irq_src hpd_irq;
+ struct amdgpu_irq_src dmub_trace_irq;
+ struct amdgpu_irq_src dmub_outbox_irq;
/* rings */
u64 fence_context;
@@ -979,6 +1006,7 @@ struct amdgpu_device {
struct amdgpu_df df;
struct amdgpu_ip_block ip_blocks[AMDGPU_MAX_IP_NUM];
+ uint32_t harvest_ip_mask;
int num_ip_blocks;
struct mutex mn_lock;
DECLARE_HASHTABLE(mn_hash, 7);
@@ -1006,13 +1034,9 @@ struct amdgpu_device {
/* s3/s4 mask */
bool in_suspend;
- bool in_hibernate;
-
- /*
- * The combination flag in_poweroff_reboot_com used to identify the poweroff
- * and reboot opt in the s0i3 system-wide suspend.
- */
- bool in_poweroff_reboot_com;
+ bool in_s3;
+ bool in_s4;
+ bool in_s0ix;
atomic_t in_gpu_reset;
enum pp_mp1_state mp1_state;
@@ -1023,6 +1047,7 @@ struct amdgpu_device {
int asic_reset_res;
struct work_struct xgmi_reset_work;
+ struct list_head reset_list;
long gfx_timeout;
long sdma_timeout;
@@ -1053,6 +1078,8 @@ struct amdgpu_device {
bool in_pci_err_recovery;
struct pci_saved_state *pci_state;
+
+ struct amdgpu_reset_control *reset_cntl;
};
static inline struct amdgpu_device *drm_to_adev(struct drm_device *ddev)
@@ -1065,7 +1092,7 @@ static inline struct drm_device *adev_to_drm(struct amdgpu_device *adev)
return &adev->ddev;
}
-static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_bo_device *bdev)
+static inline struct amdgpu_device *amdgpu_ttm_adev(struct ttm_device *bdev)
{
return container_of(bdev, struct amdgpu_device, mman.bdev);
}
@@ -1087,9 +1114,6 @@ void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value);
uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset);
-u32 amdgpu_io_rreg(struct amdgpu_device *adev, u32 reg);
-void amdgpu_io_wreg(struct amdgpu_device *adev, u32 reg, u32 v);
-
u32 amdgpu_device_indirect_rreg(struct amdgpu_device *adev,
u32 pcie_index, u32 pcie_data,
u32 reg_addr);
@@ -1106,6 +1130,12 @@ void amdgpu_device_indirect_wreg64(struct amdgpu_device *adev,
bool amdgpu_device_asic_has_dc_support(enum amd_asic_type asic_type);
bool amdgpu_device_has_dc_support(struct amdgpu_device *adev);
+int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context);
+
+int amdgpu_do_asic_reset(struct list_head *device_list_handle,
+ struct amdgpu_reset_context *reset_context);
+
int emu_soc_asic_init(struct amdgpu_device *adev);
/*
@@ -1171,8 +1201,6 @@ int emu_soc_asic_init(struct amdgpu_device *adev);
} while (0)
#define DREG32_SYS(sqf, adev, reg) seq_printf((sqf), #reg " : 0x%08X\n", amdgpu_device_rreg((adev), (reg), false))
-#define RREG32_IO(reg) amdgpu_io_rreg(adev, (reg))
-#define WREG32_IO(reg, v) amdgpu_io_wreg(adev, (reg), (v))
#define REG_FIELD_SHIFT(reg, field) reg##__##field##__SHIFT
#define REG_FIELD_MASK(reg, field) reg##__##field##_MASK
@@ -1226,6 +1254,7 @@ int emu_soc_asic_init(struct amdgpu_device *adev);
#define amdgpu_asic_pre_asic_init(adev) (adev)->asic_funcs->pre_asic_init((adev))
#define amdgpu_asic_update_umd_stable_pstate(adev, enter) \
((adev)->asic_funcs->update_umd_stable_pstate ? (adev)->asic_funcs->update_umd_stable_pstate((adev), (enter)) : 0)
+#define amdgpu_asic_query_video_codecs(adev, e, c) (adev)->asic_funcs->query_video_codecs((adev), (e), (c))
#define amdgpu_inc_vram_lost(adev) atomic_inc(&((adev)->vram_lost_counter));
@@ -1245,7 +1274,9 @@ void amdgpu_device_program_register_sequence(struct amdgpu_device *adev,
const u32 *registers,
const u32 array_size);
+int amdgpu_device_mode1_reset(struct amdgpu_device *adev);
bool amdgpu_device_supports_atpx(struct drm_device *dev);
+bool amdgpu_device_supports_px(struct drm_device *dev);
bool amdgpu_device_supports_boco(struct drm_device *dev);
bool amdgpu_device_supports_baco(struct drm_device *dev);
bool amdgpu_device_is_peer_accessible(struct amdgpu_device *adev,
@@ -1359,6 +1390,13 @@ void amdgpu_pci_resume(struct pci_dev *pdev);
bool amdgpu_device_cache_pci_state(struct pci_dev *pdev);
bool amdgpu_device_load_pci_state(struct pci_dev *pdev);
+bool amdgpu_device_skip_hw_access(struct amdgpu_device *adev);
+
+int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
+ enum amd_clockgating_state state);
+int amdgpu_device_set_pg_state(struct amdgpu_device *adev,
+ enum amd_powergating_state state);
+
#include "amdgpu_object.h"
static inline bool amdgpu_is_tmz(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
index 36a741d63ddc..2e9b16fb3fcd 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_acpi.c
@@ -903,7 +903,7 @@ void amdgpu_acpi_fini(struct amdgpu_device *adev)
*/
bool amdgpu_acpi_is_s0ix_supported(struct amdgpu_device *adev)
{
-#if defined(CONFIG_AMD_PMC)
+#if defined(CONFIG_AMD_PMC) || defined(CONFIG_AMD_PMC_MODULE)
if (acpi_gbl_FADT.flags & ACPI_FADT_LOW_POWER_S0) {
if (adev->flags & AMD_IS_APU)
return true;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
index c5343a5eecbe..5f6696a3c778 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -44,7 +44,7 @@ int amdgpu_amdkfd_init(void)
int ret;
si_meminfo(&si);
- amdgpu_amdkfd_total_mem_size = si.totalram - si.totalhigh;
+ amdgpu_amdkfd_total_mem_size = si.freeram - si.freehigh;
amdgpu_amdkfd_total_mem_size *= si.mem_unit;
ret = kgd2kfd_init();
@@ -165,7 +165,8 @@ void amdgpu_amdkfd_device_init(struct amdgpu_device *adev)
adev->doorbell_index.last_non_cp;
}
- kgd2kfd_device_init(adev->kfd.dev, adev_to_drm(adev), &gpu_resources);
+ adev->kfd.init_complete = kgd2kfd_device_init(adev->kfd.dev,
+ adev_to_drm(adev), &gpu_resources);
}
}
@@ -245,6 +246,7 @@ int amdgpu_amdkfd_alloc_gtt_mem(struct kgd_dev *kgd, size_t size,
bp.flags = AMDGPU_GEM_CREATE_CPU_GTT_USWC;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
if (cp_mqd_gfx9)
bp.flags |= AMDGPU_GEM_CREATE_CP_MQD_GFX9;
@@ -316,6 +318,7 @@ int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size,
{
struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
struct amdgpu_bo *bo = NULL;
+ struct amdgpu_bo_user *ubo;
struct amdgpu_bo_param bp;
int r;
@@ -326,14 +329,16 @@ int amdgpu_amdkfd_alloc_gws(struct kgd_dev *kgd, size_t size,
bp.flags = AMDGPU_GEM_CREATE_NO_CPU_ACCESS;
bp.type = ttm_bo_type_device;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
- r = amdgpu_bo_create(adev, &bp, &bo);
+ r = amdgpu_bo_create_user(adev, &bp, &ubo);
if (r) {
dev_err(adev->dev,
"failed to allocate gws BO for amdkfd (%d)\n", r);
return r;
}
+ bo = &ubo->bo;
*mem_obj = bo;
return 0;
}
@@ -494,8 +499,6 @@ int amdgpu_amdkfd_get_dmabuf_info(struct kgd_dev *kgd, int dma_buf_fd,
*dma_buf_kgd = (struct kgd_dev *)adev;
if (bo_size)
*bo_size = amdgpu_bo_size(bo);
- if (metadata_size)
- *metadata_size = bo->metadata_size;
if (metadata_buffer)
r = amdgpu_bo_get_metadata(bo, metadata_buffer, buffer_size,
metadata_size, &metadata_flags);
@@ -638,13 +641,6 @@ void amdgpu_amdkfd_set_compute_idle(struct kgd_dev *kgd, bool idle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)kgd;
- /* Temp workaround to fix the soft hang observed in certain compute
- * applications if GFXOFF is enabled.
- */
- if (adev->asic_type == CHIP_SIENNA_CICHLID) {
- pr_debug("GFXOFF is %s\n", idle ? "enabled" : "disabled");
- amdgpu_gfx_off_ctrl(adev, idle);
- }
amdgpu_dpm_switch_power_profile(adev,
PP_SMC_POWER_PROFILE_COMPUTE,
!idle);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index a81d9cacf9b8..5ffb07b02810 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -80,6 +80,7 @@ struct amdgpu_amdkfd_fence {
struct amdgpu_kfd_dev {
struct kfd_dev *dev;
uint64_t vram_used;
+ bool init_complete;
};
enum kgd_engine_type {
@@ -233,14 +234,10 @@ uint8_t amdgpu_amdkfd_get_xgmi_hops_count(struct kgd_dev *dst, struct kgd_dev *s
})
/* GPUVM API */
-int amdgpu_amdkfd_gpuvm_create_process_vm(struct kgd_dev *kgd, u32 pasid,
- void **vm, void **process_info,
- struct dma_fence **ef);
int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct kgd_dev *kgd,
struct file *filp, u32 pasid,
void **vm, void **process_info,
struct dma_fence **ef);
-void amdgpu_amdkfd_gpuvm_destroy_process_vm(struct kgd_dev *kgd, void *vm);
void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *vm);
uint64_t amdgpu_amdkfd_gpuvm_get_process_page_dir(void *vm);
int amdgpu_amdkfd_gpuvm_alloc_memory_of_gpu(
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_aldebaran.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_aldebaran.c
new file mode 100644
index 000000000000..a5434b713856
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_aldebaran.c
@@ -0,0 +1,47 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+#include "amdgpu.h"
+#include "amdgpu_amdkfd.h"
+#include "amdgpu_amdkfd_arcturus.h"
+#include "amdgpu_amdkfd_gfx_v9.h"
+
+const struct kfd2kgd_calls aldebaran_kfd2kgd = {
+ .program_sh_mem_settings = kgd_gfx_v9_program_sh_mem_settings,
+ .set_pasid_vmid_mapping = kgd_gfx_v9_set_pasid_vmid_mapping,
+ .init_interrupts = kgd_gfx_v9_init_interrupts,
+ .hqd_load = kgd_gfx_v9_hqd_load,
+ .hiq_mqd_load = kgd_gfx_v9_hiq_mqd_load,
+ .hqd_sdma_load = kgd_arcturus_hqd_sdma_load,
+ .hqd_dump = kgd_gfx_v9_hqd_dump,
+ .hqd_sdma_dump = kgd_arcturus_hqd_sdma_dump,
+ .hqd_is_occupied = kgd_gfx_v9_hqd_is_occupied,
+ .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied,
+ .hqd_destroy = kgd_gfx_v9_hqd_destroy,
+ .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy,
+ .address_watch_disable = kgd_gfx_v9_address_watch_disable,
+ .address_watch_execute = kgd_gfx_v9_address_watch_execute,
+ .wave_control_execute = kgd_gfx_v9_wave_control_execute,
+ .address_watch_get_offset = kgd_gfx_v9_address_watch_get_offset,
+ .get_atc_vmid_pasid_mapping_info =
+ kgd_gfx_v9_get_atc_vmid_pasid_mapping_info,
+ .set_vm_context_page_table_base = kgd_gfx_v9_set_vm_context_page_table_base,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c
index 604757a1e440..9ef9f3ddad48 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.c
@@ -122,7 +122,7 @@ static uint32_t get_sdma_rlc_reg_offset(struct amdgpu_device *adev,
return sdma_rlc_reg_offset;
}
-static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
uint32_t __user *wptr, struct mm_struct *mm)
{
struct amdgpu_device *adev = get_amdgpu_device(kgd);
@@ -192,7 +192,7 @@ static int kgd_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
return 0;
}
-static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
+int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd,
uint32_t engine_id, uint32_t queue_id,
uint32_t (**dump)[2], uint32_t *n_regs)
{
@@ -224,7 +224,7 @@ static int kgd_hqd_sdma_dump(struct kgd_dev *kgd,
return 0;
}
-static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
+bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
{
struct amdgpu_device *adev = get_amdgpu_device(kgd);
struct v9_sdma_mqd *m;
@@ -243,7 +243,7 @@ static bool kgd_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd)
return false;
}
-static int kgd_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+int kgd_arcturus_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
unsigned int utimeout)
{
struct amdgpu_device *adev = get_amdgpu_device(kgd);
@@ -289,13 +289,13 @@ const struct kfd2kgd_calls arcturus_kfd2kgd = {
.init_interrupts = kgd_gfx_v9_init_interrupts,
.hqd_load = kgd_gfx_v9_hqd_load,
.hiq_mqd_load = kgd_gfx_v9_hiq_mqd_load,
- .hqd_sdma_load = kgd_hqd_sdma_load,
+ .hqd_sdma_load = kgd_arcturus_hqd_sdma_load,
.hqd_dump = kgd_gfx_v9_hqd_dump,
- .hqd_sdma_dump = kgd_hqd_sdma_dump,
+ .hqd_sdma_dump = kgd_arcturus_hqd_sdma_dump,
.hqd_is_occupied = kgd_gfx_v9_hqd_is_occupied,
- .hqd_sdma_is_occupied = kgd_hqd_sdma_is_occupied,
+ .hqd_sdma_is_occupied = kgd_arcturus_hqd_sdma_is_occupied,
.hqd_destroy = kgd_gfx_v9_hqd_destroy,
- .hqd_sdma_destroy = kgd_hqd_sdma_destroy,
+ .hqd_sdma_destroy = kgd_arcturus_hqd_sdma_destroy,
.address_watch_disable = kgd_gfx_v9_address_watch_disable,
.address_watch_execute = kgd_gfx_v9_address_watch_execute,
.wave_control_execute = kgd_gfx_v9_wave_control_execute,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h
new file mode 100644
index 000000000000..ce08131b7b5f
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_arcturus.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ */
+
+int kgd_arcturus_hqd_sdma_load(struct kgd_dev *kgd, void *mqd,
+ uint32_t __user *wptr, struct mm_struct *mm);
+int kgd_arcturus_hqd_sdma_dump(struct kgd_dev *kgd,
+ uint32_t engine_id, uint32_t queue_id,
+ uint32_t (**dump)[2], uint32_t *n_regs);
+bool kgd_arcturus_hqd_sdma_is_occupied(struct kgd_dev *kgd, void *mqd);
+int kgd_arcturus_hqd_sdma_destroy(struct kgd_dev *kgd, void *mqd,
+ unsigned int utimeout);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
index 3107b9575929..5af464933976 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_fence.c
@@ -40,13 +40,13 @@ static atomic_t fence_seq = ATOMIC_INIT(0);
* All the BOs in a process share an eviction fence. When process X wants
* to map VRAM memory but TTM can't find enough space, TTM will attempt to
* evict BOs from its LRU list. TTM checks if the BO is valuable to evict
- * by calling ttm_bo_driver->eviction_valuable().
+ * by calling ttm_device_funcs->eviction_valuable().
*
- * ttm_bo_driver->eviction_valuable() - will return false if the BO belongs
+ * ttm_device_funcs->eviction_valuable() - will return false if the BO belongs
* to process X. Otherwise, it will return true to indicate BO can be
* evicted by TTM.
*
- * If ttm_bo_driver->eviction_valuable returns true, then TTM will continue
+ * If ttm_device_funcs->eviction_valuable returns true, then TTM will continue
* the evcition process for that BO by calling ttm_bo_evict --> amdgpu_bo_move
* --> amdgpu_copy_buffer(). This sets up job in GPU scheduler.
*
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
index ac0a432a9bf7..7d4118c8128a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd_gpuvm.c
@@ -31,6 +31,7 @@
#include "amdgpu_amdkfd.h"
#include "amdgpu_dma_buf.h"
#include <uapi/linux/kfd_ioctl.h>
+#include "amdgpu_xgmi.h"
/* BO flag to indicate a KFD userptr BO */
#define AMDGPU_AMDKFD_USERPTR_BO (1ULL << 63)
@@ -96,7 +97,7 @@ void amdgpu_amdkfd_gpuvm_init_mem_limits(void)
uint64_t mem;
si_meminfo(&si);
- mem = si.totalram - si.totalhigh;
+ mem = si.freeram - si.freehigh;
mem *= si.mem_unit;
spin_lock_init(&kfd_mem_limit.mem_limit_lock);
@@ -119,6 +120,16 @@ void amdgpu_amdkfd_gpuvm_init_mem_limits(void)
*/
#define ESTIMATE_PT_SIZE(mem_size) ((mem_size) >> 14)
+static size_t amdgpu_amdkfd_acc_size(uint64_t size)
+{
+ size >>= PAGE_SHIFT;
+ size *= sizeof(dma_addr_t) + sizeof(void *);
+
+ return __roundup_pow_of_two(sizeof(struct amdgpu_bo)) +
+ __roundup_pow_of_two(sizeof(struct ttm_tt)) +
+ PAGE_ALIGN(size);
+}
+
static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
uint64_t size, u32 domain, bool sg)
{
@@ -127,8 +138,7 @@ static int amdgpu_amdkfd_reserve_mem_limit(struct amdgpu_device *adev,
size_t acc_size, system_mem_needed, ttm_mem_needed, vram_needed;
int ret = 0;
- acc_size = ttm_bo_dma_acc_size(&adev->mman.bdev, size,
- sizeof(struct amdgpu_bo));
+ acc_size = amdgpu_amdkfd_acc_size(size);
vram_needed = 0;
if (domain == AMDGPU_GEM_DOMAIN_GTT) {
@@ -175,8 +185,7 @@ static void unreserve_mem_limit(struct amdgpu_device *adev,
{
size_t acc_size;
- acc_size = ttm_bo_dma_acc_size(&adev->mman.bdev, size,
- sizeof(struct amdgpu_bo));
+ acc_size = amdgpu_amdkfd_acc_size(size);
spin_lock(&kfd_mem_limit.mem_limit_lock);
if (domain == AMDGPU_GEM_DOMAIN_GTT) {
@@ -404,7 +413,10 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem)
{
struct amdgpu_device *bo_adev = amdgpu_ttm_adev(mem->bo->tbo.bdev);
bool coherent = mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_COHERENT;
+ bool uncached = mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_UNCACHED;
uint32_t mapping_flags;
+ uint64_t pte_flags;
+ bool snoop = false;
mapping_flags = AMDGPU_VM_PAGE_READABLE;
if (mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_WRITABLE)
@@ -425,12 +437,38 @@ static uint64_t get_pte_flags(struct amdgpu_device *adev, struct kgd_mem *mem)
AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC;
}
break;
+ case CHIP_ALDEBARAN:
+ if (coherent && uncached) {
+ if (adev->gmc.xgmi.connected_to_cpu ||
+ !(mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM))
+ snoop = true;
+ mapping_flags |= AMDGPU_VM_MTYPE_UC;
+ } else if (mem->alloc_flags & KFD_IOC_ALLOC_MEM_FLAGS_VRAM) {
+ if (bo_adev == adev) {
+ mapping_flags |= coherent ?
+ AMDGPU_VM_MTYPE_CC : AMDGPU_VM_MTYPE_RW;
+ if (adev->gmc.xgmi.connected_to_cpu)
+ snoop = true;
+ } else {
+ mapping_flags |= AMDGPU_VM_MTYPE_UC;
+ if (amdgpu_xgmi_same_hive(adev, bo_adev))
+ snoop = true;
+ }
+ } else {
+ snoop = true;
+ mapping_flags |= coherent ?
+ AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC;
+ }
+ break;
default:
mapping_flags |= coherent ?
AMDGPU_VM_MTYPE_UC : AMDGPU_VM_MTYPE_NC;
}
- return amdgpu_gem_va_map_flags(adev, mapping_flags);
+ pte_flags = amdgpu_gem_va_map_flags(adev, mapping_flags);
+ pte_flags |= snoop ? AMDGPU_PTE_SNOOPED : 0;
+
+ return pte_flags;
}
/* add_bo_to_vm - Add a BO to a VM
@@ -996,41 +1034,6 @@ create_evict_fence_fail:
return ret;
}
-int amdgpu_amdkfd_gpuvm_create_process_vm(struct kgd_dev *kgd, u32 pasid,
- void **vm, void **process_info,
- struct dma_fence **ef)
-{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
- struct amdgpu_vm *new_vm;
- int ret;
-
- new_vm = kzalloc(sizeof(*new_vm), GFP_KERNEL);
- if (!new_vm)
- return -ENOMEM;
-
- /* Initialize AMDGPU part of the VM */
- ret = amdgpu_vm_init(adev, new_vm, AMDGPU_VM_CONTEXT_COMPUTE, pasid);
- if (ret) {
- pr_err("Failed init vm ret %d\n", ret);
- goto amdgpu_vm_init_fail;
- }
-
- /* Initialize KFD part of the VM and process info */
- ret = init_kfd_vm(new_vm, process_info, ef);
- if (ret)
- goto init_kfd_vm_fail;
-
- *vm = (void *) new_vm;
-
- return 0;
-
-init_kfd_vm_fail:
- amdgpu_vm_fini(adev, new_vm);
-amdgpu_vm_init_fail:
- kfree(new_vm);
- return ret;
-}
-
int amdgpu_amdkfd_gpuvm_acquire_process_vm(struct kgd_dev *kgd,
struct file *filp, u32 pasid,
void **vm, void **process_info,
@@ -1097,21 +1100,6 @@ void amdgpu_amdkfd_gpuvm_destroy_cb(struct amdgpu_device *adev,
}
}
-void amdgpu_amdkfd_gpuvm_destroy_process_vm(struct kgd_dev *kgd, void *vm)
-{
- struct amdgpu_device *adev = get_amdgpu_device(kgd);
- struct amdgpu_vm *avm = (struct amdgpu_vm *)vm;
-
- if (WARN_ON(!kgd || !vm))
- return;
-
- pr_debug("Destroying process vm %p\n", vm);
-
- /* Release the VM context */
- amdgpu_vm_fini(adev, avm);
- kfree(vm);
-}
-
void amdgpu_amdkfd_gpuvm_release_process_vm(struct kgd_dev *kgd, void *vm)
{
struct amdgpu_device *adev = get_amdgpu_device(kgd);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
index 86add0f4ea4d..494b2e1717d5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.c
@@ -1232,157 +1232,6 @@ int amdgpu_atombios_get_leakage_vddc_based_on_leakage_idx(struct amdgpu_device *
return amdgpu_atombios_get_max_vddc(adev, VOLTAGE_TYPE_VDDC, leakage_idx, voltage);
}
-int amdgpu_atombios_get_leakage_id_from_vbios(struct amdgpu_device *adev,
- u16 *leakage_id)
-{
- union set_voltage args;
- int index = GetIndexIntoMasterTable(COMMAND, SetVoltage);
- u8 frev, crev;
-
- if (!amdgpu_atom_parse_cmd_header(adev->mode_info.atom_context, index, &frev, &crev))
- return -EINVAL;
-
- switch (crev) {
- case 3:
- case 4:
- args.v3.ucVoltageType = 0;
- args.v3.ucVoltageMode = ATOM_GET_LEAKAGE_ID;
- args.v3.usVoltageLevel = 0;
-
- amdgpu_atom_execute_table(adev->mode_info.atom_context, index, (uint32_t *)&args);
-
- *leakage_id = le16_to_cpu(args.v3.usVoltageLevel);
- break;
- default:
- DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
- return -EINVAL;
- }
-
- return 0;
-}
-
-int amdgpu_atombios_get_leakage_vddc_based_on_leakage_params(struct amdgpu_device *adev,
- u16 *vddc, u16 *vddci,
- u16 virtual_voltage_id,
- u16 vbios_voltage_id)
-{
- int index = GetIndexIntoMasterTable(DATA, ASIC_ProfilingInfo);
- u8 frev, crev;
- u16 data_offset, size;
- int i, j;
- ATOM_ASIC_PROFILING_INFO_V2_1 *profile;
- u16 *leakage_bin, *vddc_id_buf, *vddc_buf, *vddci_id_buf, *vddci_buf;
-
- *vddc = 0;
- *vddci = 0;
-
- if (!amdgpu_atom_parse_data_header(adev->mode_info.atom_context, index, &size,
- &frev, &crev, &data_offset))
- return -EINVAL;
-
- profile = (ATOM_ASIC_PROFILING_INFO_V2_1 *)
- (adev->mode_info.atom_context->bios + data_offset);
-
- switch (frev) {
- case 1:
- return -EINVAL;
- case 2:
- switch (crev) {
- case 1:
- if (size < sizeof(ATOM_ASIC_PROFILING_INFO_V2_1))
- return -EINVAL;
- leakage_bin = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usLeakageBinArrayOffset));
- vddc_id_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDC_IdArrayOffset));
- vddc_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDC_LevelArrayOffset));
- vddci_id_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDCI_IdArrayOffset));
- vddci_buf = (u16 *)
- (adev->mode_info.atom_context->bios + data_offset +
- le16_to_cpu(profile->usElbVDDCI_LevelArrayOffset));
-
- if (profile->ucElbVDDC_Num > 0) {
- for (i = 0; i < profile->ucElbVDDC_Num; i++) {
- if (vddc_id_buf[i] == virtual_voltage_id) {
- for (j = 0; j < profile->ucLeakageBinNum; j++) {
- if (vbios_voltage_id <= leakage_bin[j]) {
- *vddc = vddc_buf[j * profile->ucElbVDDC_Num + i];
- break;
- }
- }
- break;
- }
- }
- }
- if (profile->ucElbVDDCI_Num > 0) {
- for (i = 0; i < profile->ucElbVDDCI_Num; i++) {
- if (vddci_id_buf[i] == virtual_voltage_id) {
- for (j = 0; j < profile->ucLeakageBinNum; j++) {
- if (vbios_voltage_id <= leakage_bin[j]) {
- *vddci = vddci_buf[j * profile->ucElbVDDCI_Num + i];
- break;
- }
- }
- break;
- }
- }
- }
- break;
- default:
- DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
- return -EINVAL;
- }
- break;
- default:
- DRM_ERROR("Unknown table version %d, %d\n", frev, crev);
- return -EINVAL;
- }
-
- return 0;
-}
-
-union get_voltage_info {
- struct _GET_VOLTAGE_INFO_INPUT_PARAMETER_V1_2 in;
- struct _GET_EVV_VOLTAGE_INFO_OUTPUT_PARAMETER_V1_2 evv_out;
-};
-
-int amdgpu_atombios_get_voltage_evv(struct amdgpu_device *adev,
- u16 virtual_voltage_id,
- u16 *voltage)
-{
- int index = GetIndexIntoMasterTable(COMMAND, GetVoltageInfo);
- u32 entry_id;
- u32 count = adev->pm.dpm.dyn_state.vddc_dependency_on_sclk.count;
- union get_voltage_info args;
-
- for (entry_id = 0; entry_id < count; entry_id++) {
- if (adev->pm.dpm.dyn_state.vddc_dependency_on_sclk.entries[entry_id].v ==
- virtual_voltage_id)
- break;
- }
-
- if (entry_id >= count)
- return -EINVAL;
-
- args.in.ucVoltageType = VOLTAGE_TYPE_VDDC;
- args.in.ucVoltageMode = ATOM_GET_VOLTAGE_EVV_VOLTAGE;
- args.in.usVoltageLevel = cpu_to_le16(virtual_voltage_id);
- args.in.ulSCLKFreq =
- cpu_to_le32(adev->pm.dpm.dyn_state.vddc_dependency_on_sclk.entries[entry_id].clk);
-
- amdgpu_atom_execute_table(adev->mode_info.atom_context, index, (uint32_t *)&args);
-
- *voltage = le16_to_cpu(args.evv_out.usVoltageLevel);
-
- return 0;
-}
-
union voltage_object_info {
struct _ATOM_VOLTAGE_OBJECT_INFO v1;
struct _ATOM_VOLTAGE_OBJECT_INFO_V2 v2;
@@ -1905,40 +1754,6 @@ static uint32_t cail_reg_read(struct card_info *info, uint32_t reg)
return r;
}
-/**
- * cail_ioreg_write - write IO register
- *
- * @info: atom card_info pointer
- * @reg: IO register offset
- * @val: value to write to the pll register
- *
- * Provides a IO register accessor for the atom interpreter (r4xx+).
- */
-static void cail_ioreg_write(struct card_info *info, uint32_t reg, uint32_t val)
-{
- struct amdgpu_device *adev = drm_to_adev(info->dev);
-
- WREG32_IO(reg, val);
-}
-
-/**
- * cail_ioreg_read - read IO register
- *
- * @info: atom card_info pointer
- * @reg: IO register offset
- *
- * Provides an IO register accessor for the atom interpreter (r4xx+).
- * Returns the value of the IO register.
- */
-static uint32_t cail_ioreg_read(struct card_info *info, uint32_t reg)
-{
- struct amdgpu_device *adev = drm_to_adev(info->dev);
- uint32_t r;
-
- r = RREG32_IO(reg);
- return r;
-}
-
static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev,
struct device_attribute *attr,
char *buf)
@@ -1947,7 +1762,7 @@ static ssize_t amdgpu_atombios_get_vbios_version(struct device *dev,
struct amdgpu_device *adev = drm_to_adev(ddev);
struct atom_context *ctx = adev->mode_info.atom_context;
- return snprintf(buf, PAGE_SIZE, "%s\n", ctx->vbios_version);
+ return sysfs_emit(buf, "%s\n", ctx->vbios_version);
}
static DEVICE_ATTR(vbios_version, 0444, amdgpu_atombios_get_vbios_version,
@@ -1998,15 +1813,6 @@ int amdgpu_atombios_init(struct amdgpu_device *adev)
atom_card_info->dev = adev_to_drm(adev);
atom_card_info->reg_read = cail_reg_read;
atom_card_info->reg_write = cail_reg_write;
- /* needed for iio ops */
- if (adev->rio_mem) {
- atom_card_info->ioreg_read = cail_ioreg_read;
- atom_card_info->ioreg_write = cail_ioreg_write;
- } else {
- DRM_DEBUG("PCI I/O BAR is not found. Using MMIO to access ATOM BIOS\n");
- atom_card_info->ioreg_read = cail_reg_read;
- atom_card_info->ioreg_write = cail_reg_write;
- }
atom_card_info->mc_read = cail_mc_read;
atom_card_info->mc_write = cail_mc_write;
atom_card_info->pll_read = cail_pll_read;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h
index 1321ec09c734..8cc0222dba19 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atombios.h
@@ -168,18 +168,6 @@ int amdgpu_atombios_get_memory_pll_dividers(struct amdgpu_device *adev,
void amdgpu_atombios_set_engine_dram_timings(struct amdgpu_device *adev,
u32 eng_clock, u32 mem_clock);
-int amdgpu_atombios_get_leakage_id_from_vbios(struct amdgpu_device *adev,
- u16 *leakage_id);
-
-int amdgpu_atombios_get_leakage_vddc_based_on_leakage_params(struct amdgpu_device *adev,
- u16 *vddc, u16 *vddci,
- u16 virtual_voltage_id,
- u16 vbios_voltage_id);
-
-int amdgpu_atombios_get_voltage_evv(struct amdgpu_device *adev,
- u16 virtual_voltage_id,
- u16 *voltage);
-
bool
amdgpu_atombios_is_voltage_gpio(struct amdgpu_device *adev,
u8 voltage_type, u8 voltage_mode);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
index 6107ac91db25..60716b35444b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_atomfirmware.c
@@ -117,12 +117,15 @@ union igp_info {
union umc_info {
struct atom_umc_info_v3_1 v31;
+ struct atom_umc_info_v3_2 v32;
+ struct atom_umc_info_v3_3 v33;
};
union vram_info {
struct atom_vram_info_header_v2_3 v23;
struct atom_vram_info_header_v2_4 v24;
struct atom_vram_info_header_v2_5 v25;
+ struct atom_vram_info_header_v2_6 v26;
};
union vram_module {
@@ -164,6 +167,7 @@ static int convert_atom_mem_type_to_vram_type(struct amdgpu_device *adev,
vram_type = AMDGPU_VRAM_TYPE_GDDR5;
break;
case ATOM_DGPU_VRAM_TYPE_HBM2:
+ case ATOM_DGPU_VRAM_TYPE_HBM2E:
vram_type = AMDGPU_VRAM_TYPE_HBM;
break;
case ATOM_DGPU_VRAM_TYPE_GDDR6:
@@ -315,6 +319,26 @@ amdgpu_atomfirmware_get_vram_info(struct amdgpu_device *adev,
if (vram_vendor)
*vram_vendor = mem_vendor;
break;
+ case 6:
+ if (module_id > vram_info->v26.vram_module_num)
+ module_id = 0;
+ vram_module = (union vram_module *)vram_info->v26.vram_module;
+ while (i < module_id) {
+ vram_module = (union vram_module *)
+ ((u8 *)vram_module + vram_module->v9.vram_module_size);
+ i++;
+ }
+ mem_type = vram_module->v9.memory_type;
+ if (vram_type)
+ *vram_type = convert_atom_mem_type_to_vram_type(adev, mem_type);
+ mem_channel_number = vram_module->v9.channel_num;
+ mem_channel_width = vram_module->v9.channel_width;
+ if (vram_width)
+ *vram_width = mem_channel_number * (1 << mem_channel_width);
+ mem_vendor = (vram_module->v9.vender_rev_id) & 0xF;
+ if (vram_vendor)
+ *vram_vendor = mem_vendor;
+ break;
default:
return -EINVAL;
}
@@ -337,19 +361,39 @@ bool amdgpu_atomfirmware_mem_ecc_supported(struct amdgpu_device *adev)
union umc_info *umc_info;
u8 frev, crev;
bool ecc_default_enabled = false;
+ u8 umc_config;
+ u32 umc_config1;
index = get_index_into_master_table(atom_master_list_of_data_tables_v2_1,
umc_info);
if (amdgpu_atom_parse_data_header(mode_info->atom_context,
index, &size, &frev, &crev, &data_offset)) {
- /* support umc_info 3.1+ */
- if ((frev == 3 && crev >= 1) || (frev > 3)) {
+ if (frev == 3) {
umc_info = (union umc_info *)
(mode_info->atom_context->bios + data_offset);
- ecc_default_enabled =
- (le32_to_cpu(umc_info->v31.umc_config) &
- UMC_CONFIG__DEFAULT_MEM_ECC_ENABLE) ? true : false;
+ switch (crev) {
+ case 1:
+ umc_config = le32_to_cpu(umc_info->v31.umc_config);
+ ecc_default_enabled =
+ (umc_config & UMC_CONFIG__DEFAULT_MEM_ECC_ENABLE) ? true : false;
+ break;
+ case 2:
+ umc_config = le32_to_cpu(umc_info->v32.umc_config);
+ ecc_default_enabled =
+ (umc_config & UMC_CONFIG__DEFAULT_MEM_ECC_ENABLE) ? true : false;
+ break;
+ case 3:
+ umc_config = le32_to_cpu(umc_info->v33.umc_config);
+ umc_config1 = le32_to_cpu(umc_info->v33.umc_config1);
+ ecc_default_enabled =
+ ((umc_config & UMC_CONFIG__DEFAULT_MEM_ECC_ENABLE) ||
+ (umc_config1 & UMC_CONFIG1__ENABLE_ECC_CAPABLE)) ? true : false;
+ break;
+ default:
+ /* unsupported crev */
+ return false;
+ }
}
}
@@ -479,7 +523,8 @@ int amdgpu_atomfirmware_get_clock_info(struct amdgpu_device *adev)
}
union gfx_info {
- struct atom_gfx_info_v2_4 v24;
+ struct atom_gfx_info_v2_4 v24;
+ struct atom_gfx_info_v2_7 v27;
};
int amdgpu_atomfirmware_get_gfx_info(struct amdgpu_device *adev)
@@ -514,6 +559,22 @@ int amdgpu_atomfirmware_get_gfx_info(struct amdgpu_device *adev)
adev->gfx.cu_info.max_scratch_slots_per_cu = gfx_info->v24.gc_max_scratch_slots_per_cu;
adev->gfx.cu_info.lds_size = le16_to_cpu(gfx_info->v24.gc_lds_size);
return 0;
+ case 7:
+ adev->gfx.config.max_shader_engines = gfx_info->v27.max_shader_engines;
+ adev->gfx.config.max_cu_per_sh = gfx_info->v27.max_cu_per_sh;
+ adev->gfx.config.max_sh_per_se = gfx_info->v27.max_sh_per_se;
+ adev->gfx.config.max_backends_per_se = gfx_info->v27.max_backends_per_se;
+ adev->gfx.config.max_texture_channel_caches = gfx_info->v27.max_texture_channel_caches;
+ adev->gfx.config.max_gprs = le16_to_cpu(gfx_info->v27.gc_num_gprs);
+ adev->gfx.config.max_gs_threads = gfx_info->v27.gc_num_max_gs_thds;
+ adev->gfx.config.gs_vgt_table_depth = gfx_info->v27.gc_gs_table_depth;
+ adev->gfx.config.gs_prim_buffer_depth = le16_to_cpu(gfx_info->v27.gc_gsprim_buff_depth);
+ adev->gfx.config.double_offchip_lds_buf = gfx_info->v27.gc_double_offchip_lds_buffer;
+ adev->gfx.cu_info.wave_front_size = le16_to_cpu(gfx_info->v27.gc_wave_size);
+ adev->gfx.cu_info.max_waves_per_simd = le16_to_cpu(gfx_info->v27.gc_max_waves_per_simd);
+ adev->gfx.cu_info.max_scratch_slots_per_cu = gfx_info->v27.gc_max_scratch_slots_per_cu;
+ adev->gfx.cu_info.lds_size = le16_to_cpu(gfx_info->v27.gc_lds_size);
+ return 0;
default:
return -EINVAL;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c
index d9b35df33806..313517f7cf10 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_benchmark.c
@@ -85,6 +85,8 @@ static void amdgpu_benchmark_move(struct amdgpu_device *adev, unsigned size,
bp.flags = 0;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
n = AMDGPU_BENCHMARK_ITERATIONS;
r = amdgpu_bo_create(adev, &bp, &sobj);
if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
index cfb1a9a04477..27b19503773b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_bios.c
@@ -97,6 +97,10 @@ static bool igp_read_bios_from_vram(struct amdgpu_device *adev)
if (amdgpu_device_need_post(adev))
return false;
+ /* FB BAR not enabled */
+ if (pci_resource_len(adev->pdev, 0) == 0)
+ return false;
+
adev->bios = NULL;
vram_base = pci_resource_start(adev->pdev, 0);
bios = ioremap_wc(vram_base, size);
@@ -316,7 +320,7 @@ static bool amdgpu_atrm_get_bios(struct amdgpu_device *adev)
adev->bios = kmalloc(size, GFP_KERNEL);
if (!adev->bios) {
- DRM_ERROR("Unable to allocate bios\n");
+ dev_err(adev->dev, "Unable to allocate bios\n");
return false;
}
@@ -364,7 +368,7 @@ static bool amdgpu_acpi_vfct_bios(struct amdgpu_device *adev)
return false;
tbl_size = hdr->length;
if (tbl_size < sizeof(UEFI_ACPI_VFCT)) {
- DRM_ERROR("ACPI VFCT table present but broken (too short #1)\n");
+ dev_info(adev->dev, "ACPI VFCT table present but broken (too short #1),skipping\n");
return false;
}
@@ -377,13 +381,13 @@ static bool amdgpu_acpi_vfct_bios(struct amdgpu_device *adev)
offset += sizeof(VFCT_IMAGE_HEADER);
if (offset > tbl_size) {
- DRM_ERROR("ACPI VFCT image header truncated\n");
+ dev_info(adev->dev, "ACPI VFCT image header truncated,skipping\n");
return false;
}
offset += vhdr->ImageLength;
if (offset > tbl_size) {
- DRM_ERROR("ACPI VFCT image truncated\n");
+ dev_info(adev->dev, "ACPI VFCT image truncated,skipping\n");
return false;
}
@@ -406,7 +410,7 @@ static bool amdgpu_acpi_vfct_bios(struct amdgpu_device *adev)
}
}
- DRM_ERROR("ACPI VFCT table present but broken (too short #2)\n");
+ dev_info(adev->dev, "ACPI VFCT table present but broken (too short #2),skipping\n");
return false;
}
#else
@@ -453,7 +457,7 @@ bool amdgpu_get_bios(struct amdgpu_device *adev)
goto success;
}
- DRM_ERROR("Unable to locate a BIOS ROM\n");
+ dev_err(adev->dev, "Unable to locate a BIOS ROM\n");
return false;
success:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
index 3e240b952e79..b5c766998045 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c
@@ -117,7 +117,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
if (cs->in.num_chunks == 0)
return 0;
- chunk_array = kmalloc_array(cs->in.num_chunks, sizeof(uint64_t), GFP_KERNEL);
+ chunk_array = kvmalloc_array(cs->in.num_chunks, sizeof(uint64_t), GFP_KERNEL);
if (!chunk_array)
return -ENOMEM;
@@ -144,7 +144,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
}
p->nchunks = cs->in.num_chunks;
- p->chunks = kmalloc_array(p->nchunks, sizeof(struct amdgpu_cs_chunk),
+ p->chunks = kvmalloc_array(p->nchunks, sizeof(struct amdgpu_cs_chunk),
GFP_KERNEL);
if (!p->chunks) {
ret = -ENOMEM;
@@ -238,7 +238,7 @@ static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, union drm_amdgpu_cs
if (p->uf_entry.tv.bo)
p->job->uf_addr = uf_offset;
- kfree(chunk_array);
+ kvfree(chunk_array);
/* Use this opportunity to fill in task info for the vm */
amdgpu_vm_set_task_info(vm);
@@ -250,11 +250,11 @@ free_all_kdata:
free_partial_kdata:
for (; i >= 0; i--)
kvfree(p->chunks[i].kdata);
- kfree(p->chunks);
+ kvfree(p->chunks);
p->chunks = NULL;
p->nchunks = 0;
free_chunk:
- kfree(chunk_array);
+ kvfree(chunk_array);
return ret;
}
@@ -559,7 +559,7 @@ static int amdgpu_cs_parser_bos(struct amdgpu_cs_parser *p,
sizeof(struct page *),
GFP_KERNEL | __GFP_ZERO);
if (!e->user_pages) {
- DRM_ERROR("calloc failure\n");
+ DRM_ERROR("kvmalloc_array failure\n");
return -ENOMEM;
}
@@ -706,7 +706,7 @@ static void amdgpu_cs_parser_fini(struct amdgpu_cs_parser *parser, int error,
for (i = 0; i < parser->nchunks; i++)
kvfree(parser->chunks[i].kdata);
- kfree(parser->chunks);
+ kvfree(parser->chunks);
if (parser->job)
amdgpu_job_free(parser->job);
if (parser->uf_entry.tv.bo) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
index 43059ead733b..bcaf271b39bf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c
@@ -28,7 +28,6 @@
#include <linux/uaccess.h>
#include <linux/pm_runtime.h>
#include <linux/poll.h>
-#include <drm/drm_debugfs.h>
#include "amdgpu.h"
#include "amdgpu_pm.h"
@@ -38,45 +37,6 @@
#include "amdgpu_securedisplay.h"
#include "amdgpu_fw_attestation.h"
-/**
- * amdgpu_debugfs_add_files - Add simple debugfs entries
- *
- * @adev: Device to attach debugfs entries to
- * @files: Array of function callbacks that respond to reads
- * @nfiles: Number of callbacks to register
- *
- */
-int amdgpu_debugfs_add_files(struct amdgpu_device *adev,
- const struct drm_info_list *files,
- unsigned nfiles)
-{
- unsigned i;
-
- for (i = 0; i < adev->debugfs_count; i++) {
- if (adev->debugfs[i].files == files) {
- /* Already registered */
- return 0;
- }
- }
-
- i = adev->debugfs_count + 1;
- if (i > AMDGPU_DEBUGFS_MAX_COMPONENTS) {
- DRM_ERROR("Reached maximum number of debugfs components.\n");
- DRM_ERROR("Report so we increase "
- "AMDGPU_DEBUGFS_MAX_COMPONENTS.\n");
- return -EINVAL;
- }
- adev->debugfs[adev->debugfs_count].files = files;
- adev->debugfs[adev->debugfs_count].num_files = nfiles;
- adev->debugfs_count = i;
-#if defined(CONFIG_DEBUG_FS)
- drm_debugfs_create_files(files, nfiles,
- adev_to_drm(adev)->primary->debugfs_root,
- adev_to_drm(adev)->primary);
-#endif
- return 0;
-}
-
int amdgpu_debugfs_wait_dump(struct amdgpu_device *adev)
{
#if defined(CONFIG_DEBUG_FS)
@@ -1228,22 +1188,20 @@ int amdgpu_debugfs_regs_init(struct amdgpu_device *adev)
adev, debugfs_regs[i]);
if (!i && !IS_ERR_OR_NULL(ent))
i_size_write(ent->d_inode, adev->rmmio_size);
- adev->debugfs_regs[i] = ent;
}
return 0;
}
-static int amdgpu_debugfs_test_ib(struct seq_file *m, void *data)
+static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct drm_device *dev = adev_to_drm(adev);
int r = 0, i;
r = pm_runtime_get_sync(dev->dev);
if (r < 0) {
- pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
+ pm_runtime_put_autosuspend(dev->dev);
return r;
}
@@ -1285,30 +1243,19 @@ static int amdgpu_debugfs_test_ib(struct seq_file *m, void *data)
return 0;
}
-static int amdgpu_debugfs_get_vbios_dump(struct seq_file *m, void *data)
+static int amdgpu_debugfs_evict_vram(void *data, u64 *val)
{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
-
- seq_write(m, adev->bios, adev->bios_size);
- return 0;
-}
-
-static int amdgpu_debugfs_evict_vram(struct seq_file *m, void *data)
-{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)data;
+ struct drm_device *dev = adev_to_drm(adev);
int r;
r = pm_runtime_get_sync(dev->dev);
if (r < 0) {
- pm_runtime_put_autosuspend(adev_to_drm(adev)->dev);
+ pm_runtime_put_autosuspend(dev->dev);
return r;
}
- seq_printf(m, "(%d)\n", amdgpu_bo_evict_vram(adev));
+ *val = amdgpu_bo_evict_vram(adev);
pm_runtime_mark_last_busy(dev->dev);
pm_runtime_put_autosuspend(dev->dev);
@@ -1316,11 +1263,11 @@ static int amdgpu_debugfs_evict_vram(struct seq_file *m, void *data)
return 0;
}
-static int amdgpu_debugfs_evict_gtt(struct seq_file *m, void *data)
+
+static int amdgpu_debugfs_evict_gtt(void *data, u64 *val)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)data;
+ struct drm_device *dev = adev_to_drm(adev);
struct ttm_resource_manager *man;
int r;
@@ -1331,8 +1278,7 @@ static int amdgpu_debugfs_evict_gtt(struct seq_file *m, void *data)
}
man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
- r = ttm_resource_manager_evict_all(&adev->mman.bdev, man);
- seq_printf(m, "(%d)\n", r);
+ *val = ttm_resource_manager_evict_all(&adev->mman.bdev, man);
pm_runtime_mark_last_busy(dev->dev);
pm_runtime_put_autosuspend(dev->dev);
@@ -1340,10 +1286,11 @@ static int amdgpu_debugfs_evict_gtt(struct seq_file *m, void *data)
return 0;
}
-static int amdgpu_debugfs_vm_info(struct seq_file *m, void *data)
+
+static int amdgpu_debugfs_vm_info_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct drm_device *dev = adev_to_drm(adev);
struct drm_file *file;
int r;
@@ -1369,13 +1316,12 @@ static int amdgpu_debugfs_vm_info(struct seq_file *m, void *data)
return r;
}
-static const struct drm_info_list amdgpu_debugfs_list[] = {
- {"amdgpu_vbios", amdgpu_debugfs_get_vbios_dump},
- {"amdgpu_test_ib", &amdgpu_debugfs_test_ib},
- {"amdgpu_evict_vram", &amdgpu_debugfs_evict_vram},
- {"amdgpu_evict_gtt", &amdgpu_debugfs_evict_gtt},
- {"amdgpu_vm_info", &amdgpu_debugfs_vm_info},
-};
+DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_test_ib);
+DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_vm_info);
+DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_evict_vram_fops, amdgpu_debugfs_evict_vram,
+ NULL, "%lld\n");
+DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_evict_gtt_fops, amdgpu_debugfs_evict_gtt,
+ NULL, "%lld\n");
static void amdgpu_ib_preempt_fences_swap(struct amdgpu_ring *ring,
struct dma_fence **fences)
@@ -1586,71 +1532,50 @@ static int amdgpu_debugfs_sclk_set(void *data, u64 val)
return 0;
}
-DEFINE_SIMPLE_ATTRIBUTE(fops_ib_preempt, NULL,
+DEFINE_DEBUGFS_ATTRIBUTE(fops_ib_preempt, NULL,
amdgpu_debugfs_ib_preempt, "%llu\n");
-DEFINE_SIMPLE_ATTRIBUTE(fops_sclk_set, NULL,
+DEFINE_DEBUGFS_ATTRIBUTE(fops_sclk_set, NULL,
amdgpu_debugfs_sclk_set, "%llu\n");
int amdgpu_debugfs_init(struct amdgpu_device *adev)
{
+ struct dentry *root = adev_to_drm(adev)->primary->debugfs_root;
+ struct dentry *ent;
int r, i;
- adev->debugfs_preempt =
- debugfs_create_file("amdgpu_preempt_ib", 0600,
- adev_to_drm(adev)->primary->debugfs_root, adev,
- &fops_ib_preempt);
- if (!(adev->debugfs_preempt)) {
+
+
+ ent = debugfs_create_file("amdgpu_preempt_ib", 0600, root, adev,
+ &fops_ib_preempt);
+ if (!ent) {
DRM_ERROR("unable to create amdgpu_preempt_ib debugsfs file\n");
return -EIO;
}
- adev->smu.debugfs_sclk =
- debugfs_create_file("amdgpu_force_sclk", 0200,
- adev_to_drm(adev)->primary->debugfs_root, adev,
- &fops_sclk_set);
- if (!(adev->smu.debugfs_sclk)) {
+ ent = debugfs_create_file("amdgpu_force_sclk", 0200, root, adev,
+ &fops_sclk_set);
+ if (!ent) {
DRM_ERROR("unable to create amdgpu_set_sclk debugsfs file\n");
return -EIO;
}
/* Register debugfs entries for amdgpu_ttm */
- r = amdgpu_ttm_debugfs_init(adev);
- if (r) {
- DRM_ERROR("Failed to init debugfs\n");
- return r;
- }
-
- r = amdgpu_debugfs_pm_init(adev);
- if (r) {
- DRM_ERROR("Failed to register debugfs file for dpm!\n");
- return r;
- }
-
- if (amdgpu_debugfs_sa_init(adev)) {
- dev_err(adev->dev, "failed to register debugfs file for SA\n");
- }
-
- if (amdgpu_debugfs_fence_init(adev))
- dev_err(adev->dev, "fence debugfs file creation failed\n");
-
- r = amdgpu_debugfs_gem_init(adev);
- if (r)
- DRM_ERROR("registering gem debugfs failed (%d).\n", r);
+ amdgpu_ttm_debugfs_init(adev);
+ amdgpu_debugfs_pm_init(adev);
+ amdgpu_debugfs_sa_init(adev);
+ amdgpu_debugfs_fence_init(adev);
+ amdgpu_debugfs_gem_init(adev);
r = amdgpu_debugfs_regs_init(adev);
if (r)
DRM_ERROR("registering register debugfs failed (%d).\n", r);
- r = amdgpu_debugfs_firmware_init(adev);
- if (r)
- DRM_ERROR("registering firmware debugfs failed (%d).\n", r);
+ amdgpu_debugfs_firmware_init(adev);
#if defined(CONFIG_DRM_AMD_DC)
- if (amdgpu_device_has_dc_support(adev)) {
- if (dtn_debugfs_init(adev))
- DRM_ERROR("amdgpu: failed initialize dtn debugfs support.\n");
- }
+ if (amdgpu_device_has_dc_support(adev))
+ dtn_debugfs_init(adev);
#endif
for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
@@ -1665,17 +1590,26 @@ int amdgpu_debugfs_init(struct amdgpu_device *adev)
}
amdgpu_ras_debugfs_create_all(adev);
-
amdgpu_debugfs_autodump_init(adev);
-
amdgpu_rap_debugfs_init(adev);
-
amdgpu_securedisplay_debugfs_init(adev);
-
amdgpu_fw_attestation_debugfs_init(adev);
- return amdgpu_debugfs_add_files(adev, amdgpu_debugfs_list,
- ARRAY_SIZE(amdgpu_debugfs_list));
+ debugfs_create_file("amdgpu_evict_vram", 0444, root, adev,
+ &amdgpu_evict_vram_fops);
+ debugfs_create_file("amdgpu_evict_gtt", 0444, root, adev,
+ &amdgpu_evict_gtt_fops);
+ debugfs_create_file("amdgpu_test_ib", 0444, root, adev,
+ &amdgpu_debugfs_test_ib_fops);
+ debugfs_create_file("amdgpu_vm_info", 0444, root, adev,
+ &amdgpu_debugfs_vm_info_fops);
+
+ adev->debugfs_vbios_blob.data = adev->bios;
+ adev->debugfs_vbios_blob.size = adev->bios_size;
+ debugfs_create_blob("amdgpu_vbios", 0444, root,
+ &adev->debugfs_vbios_blob);
+
+ return 0;
}
#else
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.h
index 2803884d338d..141a8474e24f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.h
@@ -26,11 +26,6 @@
/*
* Debugfs
*/
-struct amdgpu_debugfs {
- const struct drm_info_list *files;
- unsigned num_files;
-};
-
struct amdgpu_autodump {
struct completion dumping;
struct wait_queue_head gpu_hang;
@@ -39,10 +34,7 @@ struct amdgpu_autodump {
int amdgpu_debugfs_regs_init(struct amdgpu_device *adev);
int amdgpu_debugfs_init(struct amdgpu_device *adev);
void amdgpu_debugfs_fini(struct amdgpu_device *adev);
-int amdgpu_debugfs_add_files(struct amdgpu_device *adev,
- const struct drm_info_list *files,
- unsigned nfiles);
-int amdgpu_debugfs_fence_init(struct amdgpu_device *adev);
-int amdgpu_debugfs_firmware_init(struct amdgpu_device *adev);
-int amdgpu_debugfs_gem_init(struct amdgpu_device *adev);
+void amdgpu_debugfs_fence_init(struct amdgpu_device *adev);
+void amdgpu_debugfs_firmware_init(struct amdgpu_device *adev);
+void amdgpu_debugfs_gem_init(struct amdgpu_device *adev);
int amdgpu_debugfs_wait_dump(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index 6447cd6ca5a8..8b2a37bf2adf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -65,6 +65,7 @@
#include "amdgpu_ras.h"
#include "amdgpu_pmu.h"
#include "amdgpu_fru_eeprom.h"
+#include "amdgpu_reset.h"
#include <linux/suspend.h>
#include <drm/task_barrier.h>
@@ -110,6 +111,7 @@ const char *amdgpu_asic_name[] = {
"RAVEN",
"ARCTURUS",
"RENOIR",
+ "ALDEBARAN",
"NAVI10",
"NAVI14",
"NAVI12",
@@ -136,7 +138,7 @@ static ssize_t amdgpu_device_get_pcie_replay_count(struct device *dev,
struct amdgpu_device *adev = drm_to_adev(ddev);
uint64_t cnt = amdgpu_asic_get_pcie_replay_count(adev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", cnt);
+ return sysfs_emit(buf, "%llu\n", cnt);
}
static DEVICE_ATTR(pcie_replay_count, S_IRUGO,
@@ -160,7 +162,7 @@ static ssize_t amdgpu_device_get_product_name(struct device *dev,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%s\n", adev->product_name);
+ return sysfs_emit(buf, "%s\n", adev->product_name);
}
static DEVICE_ATTR(product_name, S_IRUGO,
@@ -182,7 +184,7 @@ static ssize_t amdgpu_device_get_product_number(struct device *dev,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%s\n", adev->product_number);
+ return sysfs_emit(buf, "%s\n", adev->product_number);
}
static DEVICE_ATTR(product_number, S_IRUGO,
@@ -204,25 +206,25 @@ static ssize_t amdgpu_device_get_serial_number(struct device *dev,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%s\n", adev->serial);
+ return sysfs_emit(buf, "%s\n", adev->serial);
}
static DEVICE_ATTR(serial_number, S_IRUGO,
amdgpu_device_get_serial_number, NULL);
/**
- * amdgpu_device_supports_atpx - Is the device a dGPU with HG/PX power control
+ * amdgpu_device_supports_px - Is the device a dGPU with ATPX power control
*
* @dev: drm_device pointer
*
- * Returns true if the device is a dGPU with HG/PX power control,
+ * Returns true if the device is a dGPU with ATPX power control,
* otherwise return false.
*/
-bool amdgpu_device_supports_atpx(struct drm_device *dev)
+bool amdgpu_device_supports_px(struct drm_device *dev)
{
struct amdgpu_device *adev = drm_to_adev(dev);
- if (adev->flags & AMD_IS_PX)
+ if ((adev->flags & AMD_IS_PX) && !amdgpu_is_atpx_hybrid())
return true;
return false;
}
@@ -232,14 +234,15 @@ bool amdgpu_device_supports_atpx(struct drm_device *dev)
*
* @dev: drm_device pointer
*
- * Returns true if the device is a dGPU with HG/PX power control,
+ * Returns true if the device is a dGPU with ACPI power control,
* otherwise return false.
*/
bool amdgpu_device_supports_boco(struct drm_device *dev)
{
struct amdgpu_device *adev = drm_to_adev(dev);
- if (adev->has_pr3)
+ if (adev->has_pr3 ||
+ ((adev->flags & AMD_IS_PX) && amdgpu_is_atpx_hybrid()))
return true;
return false;
}
@@ -325,6 +328,35 @@ void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
/*
* register access helper functions.
*/
+
+/* Check if hw access should be skipped because of hotplug or device error */
+bool amdgpu_device_skip_hw_access(struct amdgpu_device *adev)
+{
+ if (adev->in_pci_err_recovery)
+ return true;
+
+#ifdef CONFIG_LOCKDEP
+ /*
+ * This is a bit complicated to understand, so worth a comment. What we assert
+ * here is that the GPU reset is not running on another thread in parallel.
+ *
+ * For this we trylock the read side of the reset semaphore, if that succeeds
+ * we know that the reset is not running in paralell.
+ *
+ * If the trylock fails we assert that we are either already holding the read
+ * side of the lock or are the reset thread itself and hold the write side of
+ * the lock.
+ */
+ if (in_task()) {
+ if (down_read_trylock(&adev->reset_sem))
+ up_read(&adev->reset_sem);
+ else
+ lockdep_assert_held(&adev->reset_sem);
+ }
+#endif
+ return false;
+}
+
/**
* amdgpu_device_rreg - read a memory mapped IO or indirect register
*
@@ -339,7 +371,7 @@ uint32_t amdgpu_device_rreg(struct amdgpu_device *adev,
{
uint32_t ret;
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if ((reg * 4) < adev->rmmio_size) {
@@ -376,7 +408,7 @@ uint32_t amdgpu_device_rreg(struct amdgpu_device *adev,
*/
uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if (offset < adev->rmmio_size)
@@ -401,7 +433,7 @@ uint8_t amdgpu_mm_rreg8(struct amdgpu_device *adev, uint32_t offset)
*/
void amdgpu_mm_wreg8(struct amdgpu_device *adev, uint32_t offset, uint8_t value)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (offset < adev->rmmio_size)
@@ -424,7 +456,7 @@ void amdgpu_device_wreg(struct amdgpu_device *adev,
uint32_t reg, uint32_t v,
uint32_t acc_flags)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if ((reg * 4) < adev->rmmio_size) {
@@ -451,63 +483,20 @@ void amdgpu_device_wreg(struct amdgpu_device *adev,
void amdgpu_mm_wreg_mmio_rlc(struct amdgpu_device *adev,
uint32_t reg, uint32_t v)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (amdgpu_sriov_fullaccess(adev) &&
adev->gfx.rlc.funcs &&
adev->gfx.rlc.funcs->is_rlcg_access_range) {
if (adev->gfx.rlc.funcs->is_rlcg_access_range(adev, reg))
- return adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, v);
+ return adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, v, 0);
} else {
writel(v, ((void __iomem *)adev->rmmio) + (reg * 4));
}
}
/**
- * amdgpu_io_rreg - read an IO register
- *
- * @adev: amdgpu_device pointer
- * @reg: dword aligned register offset
- *
- * Returns the 32 bit value from the offset specified.
- */
-u32 amdgpu_io_rreg(struct amdgpu_device *adev, u32 reg)
-{
- if (adev->in_pci_err_recovery)
- return 0;
-
- if ((reg * 4) < adev->rio_mem_size)
- return ioread32(adev->rio_mem + (reg * 4));
- else {
- iowrite32((reg * 4), adev->rio_mem + (mmMM_INDEX * 4));
- return ioread32(adev->rio_mem + (mmMM_DATA * 4));
- }
-}
-
-/**
- * amdgpu_io_wreg - write to an IO register
- *
- * @adev: amdgpu_device pointer
- * @reg: dword aligned register offset
- * @v: 32 bit value to write to the register
- *
- * Writes the value specified to the offset specified.
- */
-void amdgpu_io_wreg(struct amdgpu_device *adev, u32 reg, u32 v)
-{
- if (adev->in_pci_err_recovery)
- return;
-
- if ((reg * 4) < adev->rio_mem_size)
- iowrite32(v, adev->rio_mem + (reg * 4));
- else {
- iowrite32((reg * 4), adev->rio_mem + (mmMM_INDEX * 4));
- iowrite32(v, adev->rio_mem + (mmMM_DATA * 4));
- }
-}
-
-/**
* amdgpu_mm_rdoorbell - read a doorbell dword
*
* @adev: amdgpu_device pointer
@@ -518,7 +507,7 @@ void amdgpu_io_wreg(struct amdgpu_device *adev, u32 reg, u32 v)
*/
u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if (index < adev->doorbell.num_doorbells) {
@@ -541,7 +530,7 @@ u32 amdgpu_mm_rdoorbell(struct amdgpu_device *adev, u32 index)
*/
void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (index < adev->doorbell.num_doorbells) {
@@ -562,7 +551,7 @@ void amdgpu_mm_wdoorbell(struct amdgpu_device *adev, u32 index, u32 v)
*/
u64 amdgpu_mm_rdoorbell64(struct amdgpu_device *adev, u32 index)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
if (index < adev->doorbell.num_doorbells) {
@@ -585,7 +574,7 @@ u64 amdgpu_mm_rdoorbell64(struct amdgpu_device *adev, u32 index)
*/
void amdgpu_mm_wdoorbell64(struct amdgpu_device *adev, u32 index, u64 v)
{
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
if (index < adev->doorbell.num_doorbells) {
@@ -1223,6 +1212,10 @@ bool amdgpu_device_need_post(struct amdgpu_device *adev)
}
}
+ /* Don't post if we need to reset whole hive on init */
+ if (adev->gmc.xgmi.pending_reset)
+ return false;
+
if (adev->has_hw_reset) {
adev->has_hw_reset = false;
return true;
@@ -1429,7 +1422,7 @@ static void amdgpu_switcheroo_set_state(struct pci_dev *pdev,
struct drm_device *dev = pci_get_drvdata(pdev);
int r;
- if (amdgpu_device_supports_atpx(dev) && state == VGA_SWITCHEROO_OFF)
+ if (amdgpu_device_supports_px(dev) && state == VGA_SWITCHEROO_OFF)
return;
if (state == VGA_SWITCHEROO_ON) {
@@ -1690,6 +1683,19 @@ int amdgpu_device_ip_block_add(struct amdgpu_device *adev,
if (!ip_block_version)
return -EINVAL;
+ switch (ip_block_version->type) {
+ case AMD_IP_BLOCK_TYPE_VCN:
+ if (adev->harvest_ip_mask & AMD_HARVEST_IP_VCN_MASK)
+ return 0;
+ break;
+ case AMD_IP_BLOCK_TYPE_JPEG:
+ if (adev->harvest_ip_mask & AMD_HARVEST_IP_JPEG_MASK)
+ return 0;
+ break;
+ default:
+ break;
+ }
+
DRM_INFO("add ip block number %d <%s>\n", adev->num_ip_blocks,
ip_block_version->funcs->name);
@@ -1810,6 +1816,7 @@ static int amdgpu_device_parse_gpu_info_fw(struct amdgpu_device *adev)
case CHIP_CARRIZO:
case CHIP_STONEY:
case CHIP_VEGA20:
+ case CHIP_ALDEBARAN:
case CHIP_SIENNA_CICHLID:
case CHIP_NAVY_FLOUNDER:
case CHIP_DIMGREY_CAVEFISH:
@@ -2010,6 +2017,7 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
case CHIP_RAVEN:
case CHIP_ARCTURUS:
case CHIP_RENOIR:
+ case CHIP_ALDEBARAN:
if (adev->flags & AMD_IS_APU)
adev->family = AMDGPU_FAMILY_RV;
else
@@ -2045,6 +2053,8 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
adev->pm.pp_feature = amdgpu_pp_feature_mask;
if (amdgpu_sriov_vf(adev) || sched_policy == KFD_SCHED_POLICY_NO_HWS)
adev->pm.pp_feature &= ~PP_GFXOFF_MASK;
+ if (amdgpu_sriov_vf(adev) && adev->asic_type == CHIP_SIENNA_CICHLID)
+ adev->pm.pp_feature &= ~PP_OVERDRIVE_MASK;
for (i = 0; i < adev->num_ip_blocks; i++) {
if ((amdgpu_ip_block_mask & (1 << i)) == 0) {
@@ -2083,6 +2093,11 @@ static int amdgpu_device_ip_early_init(struct amdgpu_device *adev)
amdgpu_vf_error_put(adev, AMDGIM_ERROR_VF_ATOMBIOS_INIT_FAIL, 0, 0);
return r;
}
+
+ /*get pf2vf msg info at it's earliest time*/
+ if (amdgpu_sriov_vf(adev))
+ amdgpu_virt_init_data_exchange(adev);
+
}
}
@@ -2149,6 +2164,9 @@ static int amdgpu_device_fw_loading(struct amdgpu_device *adev)
if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_PSP)
continue;
+ if (!adev->ip_blocks[i].status.sw)
+ continue;
+
/* no need to do the fw loading again if already done*/
if (adev->ip_blocks[i].status.hw == true)
break;
@@ -2289,7 +2307,10 @@ static int amdgpu_device_ip_init(struct amdgpu_device *adev)
if (adev->gmc.xgmi.num_physical_nodes > 1)
amdgpu_xgmi_add_device(adev);
- amdgpu_amdkfd_device_init(adev);
+
+ /* Don't init kfd if whole hive need to be reset during init */
+ if (!adev->gmc.xgmi.pending_reset)
+ amdgpu_amdkfd_device_init(adev);
amdgpu_fru_get_product_info(adev);
@@ -2359,8 +2380,8 @@ static bool amdgpu_device_check_vram_lost(struct amdgpu_device *adev)
* Returns 0 on success, negative error code on failure.
*/
-static int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
- enum amd_clockgating_state state)
+int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
+ enum amd_clockgating_state state)
{
int i, j, r;
@@ -2371,6 +2392,10 @@ static int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
i = state == AMD_CG_STATE_GATE ? j : adev->num_ip_blocks - j - 1;
if (!adev->ip_blocks[i].status.late_initialized)
continue;
+ /* skip CG for GFX on S0ix */
+ if (adev->in_s0ix &&
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX)
+ continue;
/* skip CG for VCE/UVD, it's handled specially */
if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_UVD &&
adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_VCE &&
@@ -2391,7 +2416,8 @@ static int amdgpu_device_set_cg_state(struct amdgpu_device *adev,
return 0;
}
-static int amdgpu_device_set_pg_state(struct amdgpu_device *adev, enum amd_powergating_state state)
+int amdgpu_device_set_pg_state(struct amdgpu_device *adev,
+ enum amd_powergating_state state)
{
int i, j, r;
@@ -2402,6 +2428,10 @@ static int amdgpu_device_set_pg_state(struct amdgpu_device *adev, enum amd_power
i = state == AMD_PG_STATE_GATE ? j : adev->num_ip_blocks - j - 1;
if (!adev->ip_blocks[i].status.late_initialized)
continue;
+ /* skip PG for GFX on S0ix */
+ if (adev->in_s0ix &&
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX)
+ continue;
/* skip CG for VCE/UVD, it's handled specially */
if (adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_UVD &&
adev->ip_blocks[i].version->type != AMD_IP_BLOCK_TYPE_VCE &&
@@ -2498,6 +2528,11 @@ static int amdgpu_device_ip_late_init(struct amdgpu_device *adev)
if (r)
DRM_ERROR("enable mgpu fan boost failed (%d).\n", r);
+ /* For XGMI + passthrough configuration on arcturus, enable light SBR */
+ if (adev->asic_type == CHIP_ARCTURUS &&
+ amdgpu_passthrough(adev) &&
+ adev->gmc.xgmi.num_physical_nodes > 1)
+ smu_set_light_sbr(&adev->smu, true);
if (adev->gmc.xgmi.num_physical_nodes > 1) {
mutex_lock(&mgpu_info.mutex);
@@ -2678,11 +2713,8 @@ static int amdgpu_device_ip_suspend_phase1(struct amdgpu_device *adev)
{
int i, r;
- if (adev->in_poweroff_reboot_com ||
- !amdgpu_acpi_is_s0ix_supported(adev) || amdgpu_in_reset(adev)) {
- amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
- amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
- }
+ amdgpu_device_set_pg_state(adev, AMD_PG_STATE_UNGATE);
+ amdgpu_device_set_cg_state(adev, AMD_CG_STATE_UNGATE);
for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
if (!adev->ip_blocks[i].status.valid)
@@ -2722,6 +2754,9 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev)
{
int i, r;
+ if (adev->in_s0ix)
+ amdgpu_gfx_state_change_set(adev, sGpuChangeState_D3Entry);
+
for (i = adev->num_ip_blocks - 1; i >= 0; i--) {
if (!adev->ip_blocks[i].status.valid)
continue;
@@ -2734,6 +2769,27 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev)
adev->ip_blocks[i].status.hw = false;
continue;
}
+
+ /* skip unnecessary suspend if we do not initialize them yet */
+ if (adev->gmc.xgmi.pending_reset &&
+ !(adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SMC ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_COMMON ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_IH)) {
+ adev->ip_blocks[i].status.hw = false;
+ continue;
+ }
+
+ /* skip suspend of gfx and psp for S0ix
+ * gfx is in gfxoff state, so on resume it will exit gfxoff just
+ * like at runtime. PSP is also part of the always on hardware
+ * so no need to suspend it.
+ */
+ if (adev->in_s0ix &&
+ (adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_PSP ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GFX))
+ continue;
+
/* XXX handle errors */
r = adev->ip_blocks[i].version->funcs->suspend(adev);
/* XXX handle errors */
@@ -2753,7 +2809,6 @@ static int amdgpu_device_ip_suspend_phase2(struct amdgpu_device *adev)
}
}
}
- adev->ip_blocks[i].status.hw = false;
}
return 0;
@@ -2774,8 +2829,10 @@ int amdgpu_device_ip_suspend(struct amdgpu_device *adev)
{
int r;
- if (amdgpu_sriov_vf(adev))
+ if (amdgpu_sriov_vf(adev)) {
+ amdgpu_virt_fini_data_exchange(adev);
amdgpu_virt_request_full_gpu(adev, false);
+ }
r = amdgpu_device_ip_suspend_phase1(adev);
if (r)
@@ -3067,7 +3124,6 @@ bool amdgpu_device_has_dc_support(struct amdgpu_device *adev)
return amdgpu_device_asic_has_dc_support(adev->asic_type);
}
-
static void amdgpu_device_xgmi_reset_func(struct work_struct *__work)
{
struct amdgpu_device *adev =
@@ -3098,8 +3154,9 @@ static void amdgpu_device_xgmi_reset_func(struct work_struct *__work)
if (adev->asic_reset_res)
goto fail;
- if (adev->mmhub.funcs && adev->mmhub.funcs->reset_ras_error_count)
- adev->mmhub.funcs->reset_ras_error_count(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->reset_ras_error_count)
+ adev->mmhub.ras_funcs->reset_ras_error_count(adev);
} else {
task_barrier_full(&hive->tb);
@@ -3209,7 +3266,7 @@ int amdgpu_device_init(struct amdgpu_device *adev,
struct drm_device *ddev = adev_to_drm(adev);
struct pci_dev *pdev = adev->pdev;
int r, i;
- bool atpx = false;
+ bool px = false;
u32 max_MBps;
adev->shutdown = false;
@@ -3231,6 +3288,7 @@ int amdgpu_device_init(struct amdgpu_device *adev,
adev->vm_manager.vm_pte_funcs = NULL;
adev->vm_manager.vm_pte_num_scheds = 0;
adev->gmc.gmc_funcs = NULL;
+ adev->harvest_ip_mask = 0x0;
adev->fence_context = dma_fence_context_alloc(AMDGPU_MAX_RINGS);
bitmap_zero(adev->gfx.pipe_reserve_bitmap, AMDGPU_MAX_COMPUTE_QUEUES);
@@ -3257,7 +3315,6 @@ int amdgpu_device_init(struct amdgpu_device *adev,
/* mutex initialization are all done here so we
* can recall function without having locking issues */
- atomic_set(&adev->irq.ih.lock, 0);
mutex_init(&adev->firmware.mutex);
mutex_init(&adev->pm.mutex);
mutex_init(&adev->gfx.gpu_clock_mutex);
@@ -3290,6 +3347,8 @@ int amdgpu_device_init(struct amdgpu_device *adev,
INIT_LIST_HEAD(&adev->shadow_list);
mutex_init(&adev->shadow_list_lock);
+ INIT_LIST_HEAD(&adev->reset_list);
+
INIT_DELAYED_WORK(&adev->delayed_init_work,
amdgpu_device_delayed_init_work_handler);
INIT_DELAYED_WORK(&adev->gfx.gfx_off_delay_work,
@@ -3328,17 +3387,6 @@ int amdgpu_device_init(struct amdgpu_device *adev,
DRM_INFO("register mmio base: 0x%08X\n", (uint32_t)adev->rmmio_base);
DRM_INFO("register mmio size: %u\n", (unsigned)adev->rmmio_size);
- /* io port mapping */
- for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
- if (pci_resource_flags(adev->pdev, i) & IORESOURCE_IO) {
- adev->rio_mem_size = pci_resource_len(adev->pdev, i);
- adev->rio_mem = pci_iomap(adev->pdev, i, adev->rio_mem_size);
- break;
- }
- }
- if (adev->rio_mem == NULL)
- DRM_INFO("PCI I/O BAR is not found.\n");
-
/* enable PCIE atomic ops */
r = pci_enable_atomic_ops_to_root(adev->pdev,
PCI_EXP_DEVCAP2_ATOMIC_COMP32 |
@@ -3375,29 +3423,14 @@ int amdgpu_device_init(struct amdgpu_device *adev,
/* doorbell bar mapping and doorbell index init*/
amdgpu_device_doorbell_init(adev);
- /* if we have > 1 VGA cards, then disable the amdgpu VGA resources */
- /* this will fail for cards that aren't VGA class devices, just
- * ignore it */
- if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA)
- vga_client_register(adev->pdev, adev, NULL, amdgpu_device_vga_set_decode);
-
- if (amdgpu_device_supports_atpx(ddev))
- atpx = true;
- if (amdgpu_has_atpx() &&
- (amdgpu_is_atpx_hybrid() ||
- amdgpu_has_atpx_dgpu_power_cntl()) &&
- !pci_is_thunderbolt_attached(adev->pdev))
- vga_switcheroo_register_client(adev->pdev,
- &amdgpu_switcheroo_ops, atpx);
- if (atpx)
- vga_switcheroo_init_domain_pm_ops(adev->dev, &adev->vga_pm_domain);
-
if (amdgpu_emu_mode == 1) {
/* post the asic on emulation mode */
emu_soc_asic_init(adev);
goto fence_driver_init;
}
+ amdgpu_reset_init(adev);
+
/* detect if we are with an SRIOV vbios */
amdgpu_device_detect_sriov_bios(adev);
@@ -3405,10 +3438,28 @@ int amdgpu_device_init(struct amdgpu_device *adev,
* E.g., driver was not cleanly unloaded previously, etc.
*/
if (!amdgpu_sriov_vf(adev) && amdgpu_asic_need_reset_on_init(adev)) {
- r = amdgpu_asic_reset(adev);
- if (r) {
- dev_err(adev->dev, "asic reset on init failed\n");
- goto failed;
+ if (adev->gmc.xgmi.num_physical_nodes) {
+ dev_info(adev->dev, "Pending hive reset.\n");
+ adev->gmc.xgmi.pending_reset = true;
+ /* Only need to init necessary block for SMU to handle the reset */
+ for (i = 0; i < adev->num_ip_blocks; i++) {
+ if (!adev->ip_blocks[i].status.valid)
+ continue;
+ if (!(adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_GMC ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_COMMON ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_IH ||
+ adev->ip_blocks[i].version->type == AMD_IP_BLOCK_TYPE_SMC)) {
+ DRM_DEBUG("IP %s disabled for hw_init.\n",
+ adev->ip_blocks[i].version->funcs->name);
+ adev->ip_blocks[i].status.hw = true;
+ }
+ }
+ } else {
+ r = amdgpu_asic_reset(adev);
+ if (r) {
+ dev_err(adev->dev, "asic reset on init failed\n");
+ goto failed;
+ }
}
}
@@ -3474,11 +3525,11 @@ fence_driver_init:
adev->virt.caps &= ~AMDGPU_SRIOV_CAPS_RUNTIME;
adev->virt.ops = NULL;
r = -EAGAIN;
- goto failed;
+ goto release_ras_con;
}
dev_err(adev->dev, "amdgpu_device_ip_init failed\n");
amdgpu_vf_error_put(adev, AMDGIM_ERROR_VF_AMDGPU_INIT_FAIL, 0, 0);
- goto failed;
+ goto release_ras_con;
}
dev_info(adev->dev,
@@ -3539,19 +3590,19 @@ fence_driver_init:
/* enable clockgating, etc. after ib tests, etc. since some blocks require
* explicit gating rather than handling it automatically.
*/
- r = amdgpu_device_ip_late_init(adev);
- if (r) {
- dev_err(adev->dev, "amdgpu_device_ip_late_init failed\n");
- amdgpu_vf_error_put(adev, AMDGIM_ERROR_VF_AMDGPU_LATE_INIT_FAIL, 0, r);
- goto failed;
+ if (!adev->gmc.xgmi.pending_reset) {
+ r = amdgpu_device_ip_late_init(adev);
+ if (r) {
+ dev_err(adev->dev, "amdgpu_device_ip_late_init failed\n");
+ amdgpu_vf_error_put(adev, AMDGIM_ERROR_VF_AMDGPU_LATE_INIT_FAIL, 0, r);
+ goto release_ras_con;
+ }
+ /* must succeed. */
+ amdgpu_ras_resume(adev);
+ queue_delayed_work(system_wq, &adev->delayed_init_work,
+ msecs_to_jiffies(AMDGPU_RESUME_MS));
}
- /* must succeed. */
- amdgpu_ras_resume(adev);
-
- queue_delayed_work(system_wq, &adev->delayed_init_work,
- msecs_to_jiffies(AMDGPU_RESUME_MS));
-
if (amdgpu_sriov_vf(adev))
flush_delayed_work(&adev->delayed_init_work);
@@ -3568,12 +3619,30 @@ fence_driver_init:
if (amdgpu_device_cache_pci_state(adev->pdev))
pci_restore_state(pdev);
+ /* if we have > 1 VGA cards, then disable the amdgpu VGA resources */
+ /* this will fail for cards that aren't VGA class devices, just
+ * ignore it */
+ if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA)
+ vga_client_register(adev->pdev, adev, NULL, amdgpu_device_vga_set_decode);
+
+ if (amdgpu_device_supports_px(ddev)) {
+ px = true;
+ vga_switcheroo_register_client(adev->pdev,
+ &amdgpu_switcheroo_ops, px);
+ vga_switcheroo_init_domain_pm_ops(adev->dev, &adev->vga_pm_domain);
+ }
+
+ if (adev->gmc.xgmi.pending_reset)
+ queue_delayed_work(system_wq, &mgpu_info.delayed_reset_work,
+ msecs_to_jiffies(AMDGPU_RESUME_MS));
+
return 0;
+release_ras_con:
+ amdgpu_release_ras_context(adev);
+
failed:
amdgpu_vf_error_trans_all(adev);
- if (atpx)
- vga_switcheroo_fini_domain_pm_ops(adev->dev);
failed_unmap:
iounmap(adev->rmmio);
@@ -3594,6 +3663,7 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
{
dev_info(adev->dev, "amdgpu: finishing device.\n");
flush_delayed_work(&adev->delayed_init_work);
+ ttm_bo_lock_delayed_workqueue(&adev->mman.bdev);
adev->shutdown = true;
kfree(adev->pci_state);
@@ -3622,6 +3692,9 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
release_firmware(adev->firmware.gpu_info_fw);
adev->firmware.gpu_info_fw = NULL;
adev->accel_working = false;
+
+ amdgpu_reset_fini(adev);
+
/* free i2c buses */
if (!amdgpu_device_has_dc_support(adev))
amdgpu_i2c_fini(adev);
@@ -3631,18 +3704,12 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
kfree(adev->bios);
adev->bios = NULL;
- if (amdgpu_has_atpx() &&
- (amdgpu_is_atpx_hybrid() ||
- amdgpu_has_atpx_dgpu_power_cntl()) &&
- !pci_is_thunderbolt_attached(adev->pdev))
+ if (amdgpu_device_supports_px(adev_to_drm(adev))) {
vga_switcheroo_unregister_client(adev->pdev);
- if (amdgpu_device_supports_atpx(adev_to_drm(adev)))
vga_switcheroo_fini_domain_pm_ops(adev->dev);
+ }
if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA)
vga_client_register(adev->pdev, NULL, NULL, NULL);
- if (adev->rio_mem)
- pci_iounmap(adev->pdev, adev->rio_mem);
- adev->rio_mem = NULL;
iounmap(adev->rmmio);
adev->rmmio = NULL;
amdgpu_device_doorbell_fini(adev);
@@ -3673,14 +3740,9 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
*/
int amdgpu_device_suspend(struct drm_device *dev, bool fbcon)
{
- struct amdgpu_device *adev;
- struct drm_crtc *crtc;
- struct drm_connector *connector;
- struct drm_connector_list_iter iter;
+ struct amdgpu_device *adev = drm_to_adev(dev);
int r;
- adev = drm_to_adev(dev);
-
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
@@ -3692,61 +3754,19 @@ int amdgpu_device_suspend(struct drm_device *dev, bool fbcon)
cancel_delayed_work_sync(&adev->delayed_init_work);
- if (!amdgpu_device_has_dc_support(adev)) {
- /* turn off display hw */
- drm_modeset_lock_all(dev);
- drm_connector_list_iter_begin(dev, &iter);
- drm_for_each_connector_iter(connector, &iter)
- drm_helper_connector_dpms(connector,
- DRM_MODE_DPMS_OFF);
- drm_connector_list_iter_end(&iter);
- drm_modeset_unlock_all(dev);
- /* unpin the front buffers and cursors */
- list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
- struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
- struct drm_framebuffer *fb = crtc->primary->fb;
- struct amdgpu_bo *robj;
-
- if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
- struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
- r = amdgpu_bo_reserve(aobj, true);
- if (r == 0) {
- amdgpu_bo_unpin(aobj);
- amdgpu_bo_unreserve(aobj);
- }
- }
-
- if (fb == NULL || fb->obj[0] == NULL) {
- continue;
- }
- robj = gem_to_amdgpu_bo(fb->obj[0]);
- /* don't unpin kernel fb objects */
- if (!amdgpu_fbdev_robj_is_fb(adev, robj)) {
- r = amdgpu_bo_reserve(robj, true);
- if (r == 0) {
- amdgpu_bo_unpin(robj);
- amdgpu_bo_unreserve(robj);
- }
- }
- }
- }
-
amdgpu_ras_suspend(adev);
r = amdgpu_device_ip_suspend_phase1(adev);
- amdgpu_amdkfd_suspend(adev, adev->in_runpm);
+ if (!adev->in_s0ix)
+ amdgpu_amdkfd_suspend(adev, adev->in_runpm);
/* evict vram memory */
amdgpu_bo_evict_vram(adev);
amdgpu_fence_driver_suspend(adev);
- if (adev->in_poweroff_reboot_com ||
- !amdgpu_acpi_is_s0ix_supported(adev) || amdgpu_in_reset(adev))
- r = amdgpu_device_ip_suspend_phase2(adev);
- else
- amdgpu_gfx_state_change_set(adev, sGpuChangeState_D3Entry);
+ r = amdgpu_device_ip_suspend_phase2(adev);
/* evict remaining vram memory
* This second call to evict vram is to evict the gart page table
* using the CPU.
@@ -3768,16 +3788,13 @@ int amdgpu_device_suspend(struct drm_device *dev, bool fbcon)
*/
int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
{
- struct drm_connector *connector;
- struct drm_connector_list_iter iter;
struct amdgpu_device *adev = drm_to_adev(dev);
- struct drm_crtc *crtc;
int r = 0;
if (dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
- if (amdgpu_acpi_is_s0ix_supported(adev))
+ if (adev->in_s0ix)
amdgpu_gfx_state_change_set(adev, sGpuChangeState_D0Entry);
/* post card */
@@ -3802,50 +3819,17 @@ int amdgpu_device_resume(struct drm_device *dev, bool fbcon)
queue_delayed_work(system_wq, &adev->delayed_init_work,
msecs_to_jiffies(AMDGPU_RESUME_MS));
- if (!amdgpu_device_has_dc_support(adev)) {
- /* pin cursors */
- list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
- struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
-
- if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
- struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
- r = amdgpu_bo_reserve(aobj, true);
- if (r == 0) {
- r = amdgpu_bo_pin(aobj, AMDGPU_GEM_DOMAIN_VRAM);
- if (r != 0)
- dev_err(adev->dev, "Failed to pin cursor BO (%d)\n", r);
- amdgpu_crtc->cursor_addr = amdgpu_bo_gpu_offset(aobj);
- amdgpu_bo_unreserve(aobj);
- }
- }
- }
+ if (!adev->in_s0ix) {
+ r = amdgpu_amdkfd_resume(adev, adev->in_runpm);
+ if (r)
+ return r;
}
- r = amdgpu_amdkfd_resume(adev, adev->in_runpm);
- if (r)
- return r;
/* Make sure IB tests flushed */
flush_delayed_work(&adev->delayed_init_work);
- /* blat the mode back in */
- if (fbcon) {
- if (!amdgpu_device_has_dc_support(adev)) {
- /* pre DCE11 */
- drm_helper_resume_force_mode(dev);
-
- /* turn on display hw */
- drm_modeset_lock_all(dev);
-
- drm_connector_list_iter_begin(dev, &iter);
- drm_for_each_connector_iter(connector, &iter)
- drm_helper_connector_dpms(connector,
- DRM_MODE_DPMS_ON);
- drm_connector_list_iter_end(&iter);
-
- drm_modeset_unlock_all(dev);
- }
+ if (fbcon)
amdgpu_fbdev_set_suspend(adev, 0);
- }
drm_kms_helper_poll_enable(dev);
@@ -4143,11 +4127,11 @@ static int amdgpu_device_reset_sriov(struct amdgpu_device *adev,
amdgpu_amdkfd_post_reset(adev);
error:
- amdgpu_virt_release_full_gpu(adev, true);
if (!r && adev->virt.gim_feature & AMDGIM_FEATURE_GIM_FLR_VRAMLOST) {
amdgpu_inc_vram_lost(adev);
r = amdgpu_device_recover_vram(adev);
}
+ amdgpu_virt_release_full_gpu(adev, true);
return r;
}
@@ -4224,6 +4208,8 @@ bool amdgpu_device_should_recover_gpu(struct amdgpu_device *adev)
case CHIP_SIENNA_CICHLID:
case CHIP_NAVY_FLOUNDER:
case CHIP_DIMGREY_CAVEFISH:
+ case CHIP_VANGOGH:
+ case CHIP_ALDEBARAN:
break;
default:
goto disabled;
@@ -4237,15 +4223,60 @@ disabled:
return false;
}
+int amdgpu_device_mode1_reset(struct amdgpu_device *adev)
+{
+ u32 i;
+ int ret = 0;
+
+ amdgpu_atombios_scratch_regs_engine_hung(adev, true);
+
+ dev_info(adev->dev, "GPU mode1 reset\n");
+
+ /* disable BM */
+ pci_clear_master(adev->pdev);
+
+ amdgpu_device_cache_pci_state(adev->pdev);
-static int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
- struct amdgpu_job *job,
- bool *need_full_reset_arg)
+ if (amdgpu_dpm_is_mode1_reset_supported(adev)) {
+ dev_info(adev->dev, "GPU smu mode1 reset\n");
+ ret = amdgpu_dpm_mode1_reset(adev);
+ } else {
+ dev_info(adev->dev, "GPU psp mode1 reset\n");
+ ret = psp_gpu_reset(adev);
+ }
+
+ if (ret)
+ dev_err(adev->dev, "GPU mode1 reset failed\n");
+
+ amdgpu_device_load_pci_state(adev->pdev);
+
+ /* wait for asic to come out of reset */
+ for (i = 0; i < adev->usec_timeout; i++) {
+ u32 memsize = adev->nbio.funcs->get_memsize(adev);
+
+ if (memsize != 0xffffffff)
+ break;
+ udelay(1);
+ }
+
+ amdgpu_atombios_scratch_regs_engine_hung(adev, false);
+ return ret;
+}
+
+int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context)
{
int i, r = 0;
- bool need_full_reset = *need_full_reset_arg;
+ struct amdgpu_job *job = NULL;
+ bool need_full_reset =
+ test_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+
+ if (reset_context->reset_req_dev == adev)
+ job = reset_context->job;
- amdgpu_debugfs_wait_dump(adev);
+ /* no need to dump if device is not in good state during probe period */
+ if (!adev->gmc.xgmi.pending_reset)
+ amdgpu_debugfs_wait_dump(adev);
if (amdgpu_sriov_vf(adev)) {
/* stop the data exchange thread */
@@ -4266,6 +4297,13 @@ static int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
if(job)
drm_sched_increase_karma(&job->base);
+ r = amdgpu_reset_prepare_hwcontext(adev, reset_context);
+ /* If reset handler not implemented, continue; otherwise return */
+ if (r == -ENOSYS)
+ r = 0;
+ else
+ return r;
+
/* Don't suspend on bare metal if we are not going to HW reset the ASIC */
if (!amdgpu_sriov_vf(adev)) {
@@ -4284,30 +4322,47 @@ static int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,
if (need_full_reset)
r = amdgpu_device_ip_suspend(adev);
-
- *need_full_reset_arg = need_full_reset;
+ if (need_full_reset)
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+ else
+ clear_bit(AMDGPU_NEED_FULL_RESET,
+ &reset_context->flags);
}
return r;
}
-static int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
- struct list_head *device_list_handle,
- bool *need_full_reset_arg,
- bool skip_hw_reset)
+int amdgpu_do_asic_reset(struct list_head *device_list_handle,
+ struct amdgpu_reset_context *reset_context)
{
struct amdgpu_device *tmp_adev = NULL;
- bool need_full_reset = *need_full_reset_arg, vram_lost = false;
+ bool need_full_reset, skip_hw_reset, vram_lost = false;
int r = 0;
+ /* Try reset handler method first */
+ tmp_adev = list_first_entry(device_list_handle, struct amdgpu_device,
+ reset_list);
+ r = amdgpu_reset_perform_reset(tmp_adev, reset_context);
+ /* If reset handler not implemented, continue; otherwise return */
+ if (r == -ENOSYS)
+ r = 0;
+ else
+ return r;
+
+ /* Reset handler not implemented, use the default method */
+ need_full_reset =
+ test_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+ skip_hw_reset = test_bit(AMDGPU_SKIP_HW_RESET, &reset_context->flags);
+
/*
- * ASIC reset has to be done on all HGMI hive nodes ASAP
+ * ASIC reset has to be done on all XGMI hive nodes ASAP
* to allow proper links negotiation in FW (within 1 sec)
*/
if (!skip_hw_reset && need_full_reset) {
- list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) {
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
/* For XGMI run all resets in parallel to speed up the process */
if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
+ tmp_adev->gmc.xgmi.pending_reset = false;
if (!queue_work(system_unbound_wq, &tmp_adev->xgmi_reset_work))
r = -EALREADY;
} else
@@ -4322,8 +4377,7 @@ static int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
/* For XGMI wait for all resets to complete before proceed */
if (!r) {
- list_for_each_entry(tmp_adev, device_list_handle,
- gmc.xgmi.head) {
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
if (tmp_adev->gmc.xgmi.num_physical_nodes > 1) {
flush_work(&tmp_adev->xgmi_reset_work);
r = tmp_adev->asic_reset_res;
@@ -4335,22 +4389,22 @@ static int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
}
if (!r && amdgpu_ras_intr_triggered()) {
- list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) {
- if (tmp_adev->mmhub.funcs &&
- tmp_adev->mmhub.funcs->reset_ras_error_count)
- tmp_adev->mmhub.funcs->reset_ras_error_count(tmp_adev);
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
+ if (tmp_adev->mmhub.ras_funcs &&
+ tmp_adev->mmhub.ras_funcs->reset_ras_error_count)
+ tmp_adev->mmhub.ras_funcs->reset_ras_error_count(tmp_adev);
}
amdgpu_ras_intr_cleared();
}
- list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) {
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
if (need_full_reset) {
/* post card */
- if (amdgpu_device_asic_init(tmp_adev))
+ r = amdgpu_device_asic_init(tmp_adev);
+ if (r) {
dev_warn(tmp_adev->dev, "asic atom init failed!");
-
- if (!r) {
+ } else {
dev_info(tmp_adev->dev, "GPU reset succeeded, trying to resume\n");
r = amdgpu_device_ip_resume_phase1(tmp_adev);
if (r)
@@ -4383,6 +4437,10 @@ static int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
*/
amdgpu_register_gpu_instance(tmp_adev);
+ if (!reset_context->hive &&
+ tmp_adev->gmc.xgmi.num_physical_nodes > 1)
+ amdgpu_xgmi_add_device(tmp_adev);
+
r = amdgpu_device_ip_late_init(tmp_adev);
if (r)
goto out;
@@ -4399,7 +4457,7 @@ static int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
* bad_page_threshold value to fix this once
* probing driver again.
*/
- if (!amdgpu_ras_check_err_threshold(tmp_adev)) {
+ if (!amdgpu_ras_eeprom_check_err_threshold(tmp_adev)) {
/* must succeed. */
amdgpu_ras_resume(tmp_adev);
} else {
@@ -4408,8 +4466,10 @@ static int amdgpu_do_asic_reset(struct amdgpu_hive_info *hive,
}
/* Update PSP FW topology after reset */
- if (hive && tmp_adev->gmc.xgmi.num_physical_nodes > 1)
- r = amdgpu_xgmi_update_topology(hive, tmp_adev);
+ if (reset_context->hive &&
+ tmp_adev->gmc.xgmi.num_physical_nodes > 1)
+ r = amdgpu_xgmi_update_topology(
+ reset_context->hive, tmp_adev);
}
}
@@ -4433,7 +4493,10 @@ out:
}
end:
- *need_full_reset_arg = need_full_reset;
+ if (need_full_reset)
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
+ else
+ clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context->flags);
return r;
}
@@ -4449,7 +4512,6 @@ static bool amdgpu_device_lock_adev(struct amdgpu_device *adev,
down_write(&adev->reset_sem);
}
- atomic_inc(&adev->gpu_reset_counter);
switch (amdgpu_asic_reset_method(adev)) {
case AMD_RESET_METHOD_MODE1:
adev->mp1_state = PP_MP1_STATE_SHUTDOWN;
@@ -4571,6 +4633,74 @@ static int amdgpu_device_suspend_display_audio(struct amdgpu_device *adev)
return 0;
}
+void amdgpu_device_recheck_guilty_jobs(
+ struct amdgpu_device *adev, struct list_head *device_list_handle,
+ struct amdgpu_reset_context *reset_context)
+{
+ int i, r = 0;
+
+ for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
+ struct amdgpu_ring *ring = adev->rings[i];
+ int ret = 0;
+ struct drm_sched_job *s_job;
+
+ if (!ring || !ring->sched.thread)
+ continue;
+
+ s_job = list_first_entry_or_null(&ring->sched.pending_list,
+ struct drm_sched_job, list);
+ if (s_job == NULL)
+ continue;
+
+ /* clear job's guilty and depend the folowing step to decide the real one */
+ drm_sched_reset_karma(s_job);
+ drm_sched_resubmit_jobs_ext(&ring->sched, 1);
+
+ ret = dma_fence_wait_timeout(s_job->s_fence->parent, false, ring->sched.timeout);
+ if (ret == 0) { /* timeout */
+ DRM_ERROR("Found the real bad job! ring:%s, job_id:%llx\n",
+ ring->sched.name, s_job->id);
+
+ /* set guilty */
+ drm_sched_increase_karma(s_job);
+retry:
+ /* do hw reset */
+ if (amdgpu_sriov_vf(adev)) {
+ amdgpu_virt_fini_data_exchange(adev);
+ r = amdgpu_device_reset_sriov(adev, false);
+ if (r)
+ adev->asic_reset_res = r;
+ } else {
+ clear_bit(AMDGPU_SKIP_HW_RESET,
+ &reset_context->flags);
+ r = amdgpu_do_asic_reset(device_list_handle,
+ reset_context);
+ if (r && r == -EAGAIN)
+ goto retry;
+ }
+
+ /*
+ * add reset counter so that the following
+ * resubmitted job could flush vmid
+ */
+ atomic_inc(&adev->gpu_reset_counter);
+ continue;
+ }
+
+ /* got the hw fence, signal finished fence */
+ atomic_dec(ring->sched.score);
+ dma_fence_get(&s_job->s_fence->finished);
+ dma_fence_signal(&s_job->s_fence->finished);
+ dma_fence_put(&s_job->s_fence->finished);
+
+ /* remove node from list and free the job */
+ spin_lock(&ring->sched.job_list_lock);
+ list_del_init(&s_job->list);
+ spin_unlock(&ring->sched.job_list_lock);
+ ring->sched.ops->free_job(s_job);
+ }
+}
+
/**
* amdgpu_device_gpu_recover - reset the asic and recover scheduler
*
@@ -4586,13 +4716,16 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
struct amdgpu_job *job)
{
struct list_head device_list, *device_list_handle = NULL;
- bool need_full_reset = false;
bool job_signaled = false;
struct amdgpu_hive_info *hive = NULL;
struct amdgpu_device *tmp_adev = NULL;
int i, r = 0;
bool need_emergency_restart = false;
bool audio_suspended = false;
+ int tmp_vram_lost_counter;
+ struct amdgpu_reset_context reset_context;
+
+ memset(&reset_context, 0, sizeof(reset_context));
/*
* Special case: RAS triggered and full reset isn't supported
@@ -4633,6 +4766,12 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
mutex_lock(&hive->hive_lock);
}
+ reset_context.method = AMD_RESET_METHOD_NONE;
+ reset_context.reset_req_dev = adev;
+ reset_context.job = job;
+ reset_context.hive = hive;
+ clear_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+
/*
* lock the device before we try to operate the linked list
* if didn't get the device lock, don't touch the linked list since
@@ -4656,16 +4795,18 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
*/
INIT_LIST_HEAD(&device_list);
if (adev->gmc.xgmi.num_physical_nodes > 1) {
- if (!list_is_first(&adev->gmc.xgmi.head, &hive->device_list))
- list_rotate_to_front(&adev->gmc.xgmi.head, &hive->device_list);
- device_list_handle = &hive->device_list;
+ list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head)
+ list_add_tail(&tmp_adev->reset_list, &device_list);
+ if (!list_is_first(&adev->reset_list, &device_list))
+ list_rotate_to_front(&adev->reset_list, &device_list);
+ device_list_handle = &device_list;
} else {
- list_add_tail(&adev->gmc.xgmi.head, &device_list);
+ list_add_tail(&adev->reset_list, &device_list);
device_list_handle = &device_list;
}
/* block all schedulers and reset given job's ring */
- list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) {
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
/*
* Try to put the audio codec into suspend state
* before gpu reset started.
@@ -4710,6 +4851,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
if (need_emergency_restart)
amdgpu_job_stop_all_jobs_on_sched(&ring->sched);
}
+ atomic_inc(&tmp_adev->gpu_reset_counter);
}
if (need_emergency_restart)
@@ -4729,10 +4871,8 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,
}
retry: /* Rest of adevs pre asic reset from XGMI hive. */
- list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) {
- r = amdgpu_device_pre_asic_reset(tmp_adev,
- (tmp_adev == adev) ? job : NULL,
- &need_full_reset);
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
+ r = amdgpu_device_pre_asic_reset(tmp_adev, &reset_context);
/*TODO Should we stop ?*/
if (r) {
dev_err(tmp_adev->dev, "GPU pre asic reset failed with err, %d for drm dev, %s ",
@@ -4741,6 +4881,7 @@ retry: /* Rest of adevs pre asic reset from XGMI hive. */
}
}
+ tmp_vram_lost_counter = atomic_read(&((adev)->vram_lost_counter));
/* Actual ASIC resets if needed.*/
/* TODO Implement XGMI hive reset logic for SRIOV */
if (amdgpu_sriov_vf(adev)) {
@@ -4748,7 +4889,7 @@ retry: /* Rest of adevs pre asic reset from XGMI hive. */
if (r)
adev->asic_reset_res = r;
} else {
- r = amdgpu_do_asic_reset(hive, device_list_handle, &need_full_reset, false);
+ r = amdgpu_do_asic_reset(device_list_handle, &reset_context);
if (r && r == -EAGAIN)
goto retry;
}
@@ -4756,7 +4897,19 @@ retry: /* Rest of adevs pre asic reset from XGMI hive. */
skip_hw_reset:
/* Post ASIC reset for all devs .*/
- list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) {
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
+
+ /*
+ * Sometimes a later bad compute job can block a good gfx job as gfx
+ * and compute ring share internal GC HW mutually. We add an additional
+ * guilty jobs recheck step to find the real guilty job, it synchronously
+ * submits and pends for the first job being signaled. If it gets timeout,
+ * we identify it as a real guilty job.
+ */
+ if (amdgpu_gpu_recovery == 2 &&
+ !(tmp_vram_lost_counter < atomic_read(&adev->vram_lost_counter)))
+ amdgpu_device_recheck_guilty_jobs(
+ tmp_adev, device_list_handle, &reset_context);
for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
struct amdgpu_ring *ring = tmp_adev->rings[i];
@@ -4787,10 +4940,17 @@ skip_hw_reset:
}
skip_sched_resume:
- list_for_each_entry(tmp_adev, device_list_handle, gmc.xgmi.head) {
- /*unlock kfd: SRIOV would do it separately */
+ list_for_each_entry(tmp_adev, device_list_handle, reset_list) {
+ /* unlock kfd: SRIOV would do it separately */
if (!need_emergency_restart && !amdgpu_sriov_vf(tmp_adev))
amdgpu_amdkfd_post_reset(tmp_adev);
+
+ /* kfd_post_reset will do nothing if kfd device is not initialized,
+ * need to bring up kfd here if it's not be initialized before
+ */
+ if (!adev->kfd.init_complete)
+ amdgpu_amdkfd_device_init(adev);
+
if (audio_suspended)
amdgpu_device_resume_display_audio(tmp_adev);
amdgpu_device_unlock_adev(tmp_adev);
@@ -5052,6 +5212,7 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_sta
drm_sched_stop(&ring->sched, NULL);
}
+ atomic_inc(&adev->gpu_reset_counter);
return PCI_ERS_RESULT_NEED_RESET;
case pci_channel_io_perm_failure:
/* Permanent error, prepare for device removal */
@@ -5093,14 +5254,16 @@ pci_ers_result_t amdgpu_pci_slot_reset(struct pci_dev *pdev)
struct drm_device *dev = pci_get_drvdata(pdev);
struct amdgpu_device *adev = drm_to_adev(dev);
int r, i;
- bool need_full_reset = true;
+ struct amdgpu_reset_context reset_context;
u32 memsize;
struct list_head device_list;
DRM_INFO("PCI error: slot reset callback!!\n");
+ memset(&reset_context, 0, sizeof(reset_context));
+
INIT_LIST_HEAD(&device_list);
- list_add_tail(&adev->gmc.xgmi.head, &device_list);
+ list_add_tail(&adev->reset_list, &device_list);
/* wait for asic to come out of reset */
msleep(500);
@@ -5121,13 +5284,18 @@ pci_ers_result_t amdgpu_pci_slot_reset(struct pci_dev *pdev)
goto out;
}
+ reset_context.method = AMD_RESET_METHOD_NONE;
+ reset_context.reset_req_dev = adev;
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+ set_bit(AMDGPU_SKIP_HW_RESET, &reset_context.flags);
+
adev->in_pci_err_recovery = true;
- r = amdgpu_device_pre_asic_reset(adev, NULL, &need_full_reset);
+ r = amdgpu_device_pre_asic_reset(adev, &reset_context);
adev->in_pci_err_recovery = false;
if (r)
goto out;
- r = amdgpu_do_asic_reset(NULL, &device_list, &need_full_reset, true);
+ r = amdgpu_do_asic_reset(&device_list, &reset_context);
out:
if (!r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
index b2dbcb4df020..e1b6f5891759 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
@@ -373,6 +373,34 @@ int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id,
return -EINVAL;
}
+void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev)
+{
+ struct binary_header *bhdr;
+ struct harvest_table *harvest_info;
+ int i;
+
+ bhdr = (struct binary_header *)adev->mman.discovery_bin;
+ harvest_info = (struct harvest_table *)(adev->mman.discovery_bin +
+ le16_to_cpu(bhdr->table_list[HARVEST_INFO].offset));
+
+ for (i = 0; i < 32; i++) {
+ if (le32_to_cpu(harvest_info->list[i].hw_id) == 0)
+ break;
+
+ switch (le32_to_cpu(harvest_info->list[i].hw_id)) {
+ case VCN_HWID:
+ adev->harvest_ip_mask |= AMD_HARVEST_IP_VCN_MASK;
+ adev->harvest_ip_mask |= AMD_HARVEST_IP_JPEG_MASK;
+ break;
+ case DMU_HWID:
+ adev->harvest_ip_mask |= AMD_HARVEST_IP_DMU_MASK;
+ break;
+ default:
+ break;
+ }
+ }
+}
+
int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev)
{
struct binary_header *bhdr;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h
index 8f6183801cb3..1b1ae21b1037 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.h
@@ -29,6 +29,7 @@
void amdgpu_discovery_fini(struct amdgpu_device *adev);
int amdgpu_discovery_reg_base_init(struct amdgpu_device *adev);
+void amdgpu_discovery_harvest_ip(struct amdgpu_device *adev);
int amdgpu_discovery_get_ip_version(struct amdgpu_device *adev, int hw_id,
int *major, int *minor, int *revision);
int amdgpu_discovery_get_gfx_info(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
index 48cb33e5b382..8a1fb8b6606e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.c
@@ -837,6 +837,174 @@ static int convert_tiling_flags_to_modifier(struct amdgpu_framebuffer *afb)
return 0;
}
+static void get_block_dimensions(unsigned int block_log2, unsigned int cpp,
+ unsigned int *width, unsigned int *height)
+{
+ unsigned int cpp_log2 = ilog2(cpp);
+ unsigned int pixel_log2 = block_log2 - cpp_log2;
+ unsigned int width_log2 = (pixel_log2 + 1) / 2;
+ unsigned int height_log2 = pixel_log2 - width_log2;
+
+ *width = 1 << width_log2;
+ *height = 1 << height_log2;
+}
+
+static unsigned int get_dcc_block_size(uint64_t modifier, bool rb_aligned,
+ bool pipe_aligned)
+{
+ unsigned int ver = AMD_FMT_MOD_GET(TILE_VERSION, modifier);
+
+ switch (ver) {
+ case AMD_FMT_MOD_TILE_VER_GFX9: {
+ /*
+ * TODO: for pipe aligned we may need to check the alignment of the
+ * total size of the surface, which may need to be bigger than the
+ * natural alignment due to some HW workarounds
+ */
+ return max(10 + (rb_aligned ? (int)AMD_FMT_MOD_GET(RB, modifier) : 0), 12);
+ }
+ case AMD_FMT_MOD_TILE_VER_GFX10:
+ case AMD_FMT_MOD_TILE_VER_GFX10_RBPLUS: {
+ int pipes_log2 = AMD_FMT_MOD_GET(PIPE_XOR_BITS, modifier);
+
+ if (ver == AMD_FMT_MOD_TILE_VER_GFX10_RBPLUS && pipes_log2 > 1 &&
+ AMD_FMT_MOD_GET(PACKERS, modifier) == pipes_log2)
+ ++pipes_log2;
+
+ return max(8 + (pipe_aligned ? pipes_log2 : 0), 12);
+ }
+ default:
+ return 0;
+ }
+}
+
+static int amdgpu_display_verify_plane(struct amdgpu_framebuffer *rfb, int plane,
+ const struct drm_format_info *format,
+ unsigned int block_width, unsigned int block_height,
+ unsigned int block_size_log2)
+{
+ unsigned int width = rfb->base.width /
+ ((plane && plane < format->num_planes) ? format->hsub : 1);
+ unsigned int height = rfb->base.height /
+ ((plane && plane < format->num_planes) ? format->vsub : 1);
+ unsigned int cpp = plane < format->num_planes ? format->cpp[plane] : 1;
+ unsigned int block_pitch = block_width * cpp;
+ unsigned int min_pitch = ALIGN(width * cpp, block_pitch);
+ unsigned int block_size = 1 << block_size_log2;
+ uint64_t size;
+
+ if (rfb->base.pitches[plane] % block_pitch) {
+ drm_dbg_kms(rfb->base.dev,
+ "pitch %d for plane %d is not a multiple of block pitch %d\n",
+ rfb->base.pitches[plane], plane, block_pitch);
+ return -EINVAL;
+ }
+ if (rfb->base.pitches[plane] < min_pitch) {
+ drm_dbg_kms(rfb->base.dev,
+ "pitch %d for plane %d is less than minimum pitch %d\n",
+ rfb->base.pitches[plane], plane, min_pitch);
+ return -EINVAL;
+ }
+
+ /* Force at least natural alignment. */
+ if (rfb->base.offsets[plane] % block_size) {
+ drm_dbg_kms(rfb->base.dev,
+ "offset 0x%x for plane %d is not a multiple of block pitch 0x%x\n",
+ rfb->base.offsets[plane], plane, block_size);
+ return -EINVAL;
+ }
+
+ size = rfb->base.offsets[plane] +
+ (uint64_t)rfb->base.pitches[plane] / block_pitch *
+ block_size * DIV_ROUND_UP(height, block_height);
+
+ if (rfb->base.obj[0]->size < size) {
+ drm_dbg_kms(rfb->base.dev,
+ "BO size 0x%zx is less than 0x%llx required for plane %d\n",
+ rfb->base.obj[0]->size, size, plane);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+
+static int amdgpu_display_verify_sizes(struct amdgpu_framebuffer *rfb)
+{
+ const struct drm_format_info *format_info = drm_format_info(rfb->base.format->format);
+ uint64_t modifier = rfb->base.modifier;
+ int ret;
+ unsigned int i, block_width, block_height, block_size_log2;
+
+ if (!rfb->base.dev->mode_config.allow_fb_modifiers)
+ return 0;
+
+ for (i = 0; i < format_info->num_planes; ++i) {
+ if (modifier == DRM_FORMAT_MOD_LINEAR) {
+ block_width = 256 / format_info->cpp[i];
+ block_height = 1;
+ block_size_log2 = 8;
+ } else {
+ int swizzle = AMD_FMT_MOD_GET(TILE, modifier);
+
+ switch ((swizzle & ~3) + 1) {
+ case DC_SW_256B_S:
+ block_size_log2 = 8;
+ break;
+ case DC_SW_4KB_S:
+ case DC_SW_4KB_S_X:
+ block_size_log2 = 12;
+ break;
+ case DC_SW_64KB_S:
+ case DC_SW_64KB_S_T:
+ case DC_SW_64KB_S_X:
+ block_size_log2 = 16;
+ break;
+ default:
+ drm_dbg_kms(rfb->base.dev,
+ "Swizzle mode with unknown block size: %d\n", swizzle);
+ return -EINVAL;
+ }
+
+ get_block_dimensions(block_size_log2, format_info->cpp[i],
+ &block_width, &block_height);
+ }
+
+ ret = amdgpu_display_verify_plane(rfb, i, format_info,
+ block_width, block_height, block_size_log2);
+ if (ret)
+ return ret;
+ }
+
+ if (AMD_FMT_MOD_GET(DCC, modifier)) {
+ if (AMD_FMT_MOD_GET(DCC_RETILE, modifier)) {
+ block_size_log2 = get_dcc_block_size(modifier, false, false);
+ get_block_dimensions(block_size_log2 + 8, format_info->cpp[0],
+ &block_width, &block_height);
+ ret = amdgpu_display_verify_plane(rfb, i, format_info,
+ block_width, block_height,
+ block_size_log2);
+ if (ret)
+ return ret;
+
+ ++i;
+ block_size_log2 = get_dcc_block_size(modifier, true, true);
+ } else {
+ bool pipe_aligned = AMD_FMT_MOD_GET(DCC_PIPE_ALIGN, modifier);
+
+ block_size_log2 = get_dcc_block_size(modifier, true, pipe_aligned);
+ }
+ get_block_dimensions(block_size_log2 + 8, format_info->cpp[0],
+ &block_width, &block_height);
+ ret = amdgpu_display_verify_plane(rfb, i, format_info,
+ block_width, block_height, block_size_log2);
+ if (ret)
+ return ret;
+ }
+
+ return 0;
+}
+
static int amdgpu_display_get_fb_info(const struct amdgpu_framebuffer *amdgpu_fb,
uint64_t *tiling_flags, bool *tmz_surface)
{
@@ -870,17 +1038,73 @@ static int amdgpu_display_get_fb_info(const struct amdgpu_framebuffer *amdgpu_fb
return r;
}
+int amdgpu_display_gem_fb_init(struct drm_device *dev,
+ struct amdgpu_framebuffer *rfb,
+ const struct drm_mode_fb_cmd2 *mode_cmd,
+ struct drm_gem_object *obj)
+{
+ int ret;
+
+ rfb->base.obj[0] = obj;
+ drm_helper_mode_fill_fb_struct(dev, &rfb->base, mode_cmd);
+ ret = drm_framebuffer_init(dev, &rfb->base, &amdgpu_fb_funcs);
+ if (ret)
+ goto err;
+
+ ret = amdgpu_display_framebuffer_init(dev, rfb, mode_cmd, obj);
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ drm_err(dev, "Failed to init gem fb: %d\n", ret);
+ rfb->base.obj[0] = NULL;
+ return ret;
+}
+
+int amdgpu_display_gem_fb_verify_and_init(
+ struct drm_device *dev, struct amdgpu_framebuffer *rfb,
+ struct drm_file *file_priv, const struct drm_mode_fb_cmd2 *mode_cmd,
+ struct drm_gem_object *obj)
+{
+ int ret;
+
+ rfb->base.obj[0] = obj;
+ drm_helper_mode_fill_fb_struct(dev, &rfb->base, mode_cmd);
+ ret = drm_framebuffer_init(dev, &rfb->base, &amdgpu_fb_funcs);
+ if (ret)
+ goto err;
+ /* Verify that the modifier is supported. */
+ if (!drm_any_plane_has_format(dev, mode_cmd->pixel_format,
+ mode_cmd->modifier[0])) {
+ struct drm_format_name_buf format_name;
+ drm_dbg_kms(dev,
+ "unsupported pixel format %s / modifier 0x%llx\n",
+ drm_get_format_name(mode_cmd->pixel_format,
+ &format_name),
+ mode_cmd->modifier[0]);
+
+ ret = -EINVAL;
+ goto err;
+ }
+
+ ret = amdgpu_display_framebuffer_init(dev, rfb, mode_cmd, obj);
+ if (ret)
+ goto err;
+
+ return 0;
+err:
+ drm_err(dev, "Failed to verify and init gem fb: %d\n", ret);
+ rfb->base.obj[0] = NULL;
+ return ret;
+}
+
int amdgpu_display_framebuffer_init(struct drm_device *dev,
struct amdgpu_framebuffer *rfb,
const struct drm_mode_fb_cmd2 *mode_cmd,
struct drm_gem_object *obj)
{
int ret, i;
- rfb->base.obj[0] = obj;
- drm_helper_mode_fill_fb_struct(dev, &rfb->base, mode_cmd);
- ret = drm_framebuffer_init(dev, &rfb->base, &amdgpu_fb_funcs);
- if (ret)
- goto fail;
/*
* This needs to happen before modifier conversion as that might change
@@ -891,13 +1115,13 @@ int amdgpu_display_framebuffer_init(struct drm_device *dev,
drm_dbg_kms(dev, "Plane 0 and %d have different BOs: %u vs. %u\n",
i, mode_cmd->handles[0], mode_cmd->handles[i]);
ret = -EINVAL;
- goto fail;
+ return ret;
}
}
ret = amdgpu_display_get_fb_info(rfb, &rfb->tiling_flags, &rfb->tmz_surface);
if (ret)
- goto fail;
+ return ret;
if (dev->mode_config.allow_fb_modifiers &&
!(rfb->base.flags & DRM_MODE_FB_MODIFIERS)) {
@@ -905,20 +1129,20 @@ int amdgpu_display_framebuffer_init(struct drm_device *dev,
if (ret) {
drm_dbg_kms(dev, "Failed to convert tiling flags 0x%llX to a modifier",
rfb->tiling_flags);
- goto fail;
+ return ret;
}
}
- for (i = 1; i < rfb->base.format->num_planes; ++i) {
+ ret = amdgpu_display_verify_sizes(rfb);
+ if (ret)
+ return ret;
+
+ for (i = 0; i < rfb->base.format->num_planes; ++i) {
+ drm_gem_object_get(rfb->base.obj[0]);
rfb->base.obj[i] = rfb->base.obj[0];
- drm_gem_object_get(rfb->base.obj[i]);
}
return 0;
-
-fail:
- rfb->base.obj[0] = NULL;
- return ret;
}
struct drm_framebuffer *
@@ -944,6 +1168,7 @@ amdgpu_display_user_framebuffer_create(struct drm_device *dev,
domains = amdgpu_display_supported_domains(drm_to_adev(dev), bo->flags);
if (obj->import_attach && !(domains & AMDGPU_GEM_DOMAIN_GTT)) {
drm_dbg_kms(dev, "Cannot create framebuffer from imported dma_buf\n");
+ drm_gem_object_put(obj);
return ERR_PTR(-EINVAL);
}
@@ -953,13 +1178,15 @@ amdgpu_display_user_framebuffer_create(struct drm_device *dev,
return ERR_PTR(-ENOMEM);
}
- ret = amdgpu_display_framebuffer_init(dev, amdgpu_fb, mode_cmd, obj);
+ ret = amdgpu_display_gem_fb_verify_and_init(dev, amdgpu_fb, file_priv,
+ mode_cmd, obj);
if (ret) {
kfree(amdgpu_fb);
drm_gem_object_put(obj);
return ERR_PTR(ret);
}
+ drm_gem_object_put(obj);
return &amdgpu_fb->base;
}
@@ -1310,3 +1537,92 @@ bool amdgpu_crtc_get_scanout_position(struct drm_crtc *crtc,
return amdgpu_display_get_crtc_scanoutpos(dev, pipe, 0, vpos, hpos,
stime, etime, mode);
}
+
+int amdgpu_display_suspend_helper(struct amdgpu_device *adev)
+{
+ struct drm_device *dev = adev_to_drm(adev);
+ struct drm_crtc *crtc;
+ struct drm_connector *connector;
+ struct drm_connector_list_iter iter;
+ int r;
+
+ /* turn off display hw */
+ drm_modeset_lock_all(dev);
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter)
+ drm_helper_connector_dpms(connector,
+ DRM_MODE_DPMS_OFF);
+ drm_connector_list_iter_end(&iter);
+ drm_modeset_unlock_all(dev);
+ /* unpin the front buffers and cursors */
+ list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
+ struct drm_framebuffer *fb = crtc->primary->fb;
+ struct amdgpu_bo *robj;
+
+ if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
+ struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
+ r = amdgpu_bo_reserve(aobj, true);
+ if (r == 0) {
+ amdgpu_bo_unpin(aobj);
+ amdgpu_bo_unreserve(aobj);
+ }
+ }
+
+ if (fb == NULL || fb->obj[0] == NULL) {
+ continue;
+ }
+ robj = gem_to_amdgpu_bo(fb->obj[0]);
+ /* don't unpin kernel fb objects */
+ if (!amdgpu_fbdev_robj_is_fb(adev, robj)) {
+ r = amdgpu_bo_reserve(robj, true);
+ if (r == 0) {
+ amdgpu_bo_unpin(robj);
+ amdgpu_bo_unreserve(robj);
+ }
+ }
+ }
+ return 0;
+}
+
+int amdgpu_display_resume_helper(struct amdgpu_device *adev)
+{
+ struct drm_device *dev = adev_to_drm(adev);
+ struct drm_connector *connector;
+ struct drm_connector_list_iter iter;
+ struct drm_crtc *crtc;
+ int r;
+
+ /* pin cursors */
+ list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
+ struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
+
+ if (amdgpu_crtc->cursor_bo && !adev->enable_virtual_display) {
+ struct amdgpu_bo *aobj = gem_to_amdgpu_bo(amdgpu_crtc->cursor_bo);
+ r = amdgpu_bo_reserve(aobj, true);
+ if (r == 0) {
+ r = amdgpu_bo_pin(aobj, AMDGPU_GEM_DOMAIN_VRAM);
+ if (r != 0)
+ dev_err(adev->dev, "Failed to pin cursor BO (%d)\n", r);
+ amdgpu_crtc->cursor_addr = amdgpu_bo_gpu_offset(aobj);
+ amdgpu_bo_unreserve(aobj);
+ }
+ }
+ }
+
+ drm_helper_resume_force_mode(dev);
+
+ /* turn on display hw */
+ drm_modeset_lock_all(dev);
+
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter)
+ drm_helper_connector_dpms(connector,
+ DRM_MODE_DPMS_ON);
+ drm_connector_list_iter_end(&iter);
+
+ drm_modeset_unlock_all(dev);
+
+ return 0;
+}
+
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h
index dc7b7d116549..7b6d83e2b13c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_display.h
@@ -47,4 +47,7 @@ amdgpu_display_user_framebuffer_create(struct drm_device *dev,
const struct drm_format_info *
amdgpu_lookup_format_info(u32 format, uint64_t modifier);
+int amdgpu_display_suspend_helper(struct amdgpu_device *adev);
+int amdgpu_display_resume_helper(struct amdgpu_device *adev);
+
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
index 47e0b48dc26f..baa980a477d9 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dma_buf.c
@@ -291,8 +291,8 @@ static struct sg_table *amdgpu_dma_buf_map(struct dma_buf_attachment *attach,
break;
case TTM_PL_VRAM:
- r = amdgpu_vram_mgr_alloc_sgt(adev, &bo->tbo.mem, attach->dev,
- dir, &sgt);
+ r = amdgpu_vram_mgr_alloc_sgt(adev, &bo->tbo.mem, 0,
+ bo->tbo.base.size, attach->dev, dir, &sgt);
if (r)
return ERR_PTR(r);
break;
@@ -321,17 +321,12 @@ static void amdgpu_dma_buf_unmap(struct dma_buf_attachment *attach,
struct sg_table *sgt,
enum dma_data_direction dir)
{
- struct dma_buf *dma_buf = attach->dmabuf;
- struct drm_gem_object *obj = dma_buf->priv;
- struct amdgpu_bo *bo = gem_to_amdgpu_bo(obj);
- struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
-
if (sgt->sgl->page_link) {
dma_unmap_sgtable(attach->dev, sgt, dir, 0);
sg_free_table(sgt);
kfree(sgt);
} else {
- amdgpu_vram_mgr_free_sgt(adev, attach->dev, dir, sgt);
+ amdgpu_vram_mgr_free_sgt(attach->dev, dir, sgt);
}
}
@@ -434,22 +429,22 @@ amdgpu_dma_buf_create_obj(struct drm_device *dev, struct dma_buf *dma_buf)
{
struct dma_resv *resv = dma_buf->resv;
struct amdgpu_device *adev = drm_to_adev(dev);
- struct amdgpu_bo *bo;
- struct amdgpu_bo_param bp;
struct drm_gem_object *gobj;
+ struct amdgpu_bo *bo;
+ uint64_t flags = 0;
int ret;
- memset(&bp, 0, sizeof(bp));
- bp.size = dma_buf->size;
- bp.byte_align = PAGE_SIZE;
- bp.domain = AMDGPU_GEM_DOMAIN_CPU;
- bp.flags = 0;
- bp.type = ttm_bo_type_sg;
- bp.resv = resv;
dma_resv_lock(resv, NULL);
+
+ if (dma_buf->ops == &amdgpu_dmabuf_ops) {
+ struct amdgpu_bo *other = gem_to_amdgpu_bo(dma_buf->priv);
+
+ flags |= other->flags & AMDGPU_GEM_CREATE_CPU_GTT_USWC;
+ }
+
ret = amdgpu_gem_object_create(adev, dma_buf->size, PAGE_SIZE,
- AMDGPU_GEM_DOMAIN_CPU,
- 0, ttm_bo_type_sg, resv, &gobj);
+ AMDGPU_GEM_DOMAIN_CPU, flags,
+ ttm_bo_type_sg, resv, &gobj);
if (ret)
goto error;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
index 4575192d9b08..f93883db2b46 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
@@ -36,6 +36,7 @@
#include <linux/vga_switcheroo.h>
#include <drm/drm_probe_helper.h>
#include <linux/mmu_notifier.h>
+#include <linux/suspend.h>
#include "amdgpu.h"
#include "amdgpu_irq.h"
@@ -45,6 +46,8 @@
#include "amdgpu_amdkfd.h"
#include "amdgpu_ras.h"
+#include "amdgpu_xgmi.h"
+#include "amdgpu_reset.h"
/*
* KMS wrapper.
@@ -90,9 +93,10 @@
* - 3.38.0 - Add AMDGPU_IB_FLAG_EMIT_MEM_SYNC
* - 3.39.0 - DMABUF implicit sync does a full pipeline sync
* - 3.40.0 - Add AMDGPU_IDS_FLAGS_TMZ
+ * - 3.41.0 - Add video codec query
*/
#define KMS_DRIVER_MAJOR 3
-#define KMS_DRIVER_MINOR 40
+#define KMS_DRIVER_MINOR 41
#define KMS_DRIVER_PATCHLEVEL 0
int amdgpu_vram_limit;
@@ -145,6 +149,7 @@ int amdgpu_compute_multipipe = -1;
int amdgpu_gpu_recovery = -1; /* auto */
int amdgpu_emu_mode;
uint amdgpu_smu_memory_pool_size;
+int amdgpu_smu_pptable_id = -1;
/*
* FBC (bit 0) disabled by default
* MULTI_MON_PP_MCLK_SWITCH (bit 1) enabled by default
@@ -162,16 +167,26 @@ int amdgpu_discovery = -1;
int amdgpu_mes;
int amdgpu_noretry = -1;
int amdgpu_force_asic_type = -1;
-int amdgpu_tmz;
+int amdgpu_tmz = -1; /* auto */
+uint amdgpu_freesync_vid_mode;
int amdgpu_reset_method = -1; /* auto */
int amdgpu_num_kcq = -1;
+static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work);
+
struct amdgpu_mgpu_info mgpu_info = {
.mutex = __MUTEX_INITIALIZER(mgpu_info.mutex),
+ .delayed_reset_work = __DELAYED_WORK_INITIALIZER(
+ mgpu_info.delayed_reset_work,
+ amdgpu_drv_delayed_reset_work_handler, 0),
};
int amdgpu_ras_enable = -1;
uint amdgpu_ras_mask = 0xffffffff;
int amdgpu_bad_page_threshold = -1;
+struct amdgpu_watchdog_timer amdgpu_watchdog_timer = {
+ .timeout_fatal_disable = false,
+ .period = 0x0, /* default to 0x0 (timeout disable) */
+};
/**
* DOC: vramlimit (int)
@@ -502,7 +517,7 @@ module_param_named(compute_multipipe, amdgpu_compute_multipipe, int, 0444);
* DOC: gpu_recovery (int)
* Set to enable GPU recovery mechanism (1 = enable, 0 = disable). The default is -1 (auto, disabled except SRIOV).
*/
-MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism, (1 = enable, 0 = disable, -1 = auto)");
+MODULE_PARM_DESC(gpu_recovery, "Enable GPU recovery mechanism, (2 = advanced tdr mode, 1 = enable, 0 = disable, -1 = auto)");
module_param_named(gpu_recovery, amdgpu_gpu_recovery, int, 0444);
/**
@@ -528,6 +543,20 @@ MODULE_PARM_DESC(ras_mask, "Mask of RAS features to enable (default 0xffffffff),
module_param_named(ras_mask, amdgpu_ras_mask, uint, 0444);
/**
+ * DOC: timeout_fatal_disable (bool)
+ * Disable Watchdog timeout fatal error event
+ */
+MODULE_PARM_DESC(timeout_fatal_disable, "disable watchdog timeout fatal error (false = default)");
+module_param_named(timeout_fatal_disable, amdgpu_watchdog_timer.timeout_fatal_disable, bool, 0644);
+
+/**
+ * DOC: timeout_period (uint)
+ * Modify the watchdog timeout max_cycles as (1 << period)
+ */
+MODULE_PARM_DESC(timeout_period, "watchdog timeout period (0 = timeout disabled, 1 ~ 0x23 = timeout maxcycles = (1 << period)");
+module_param_named(timeout_period, amdgpu_watchdog_timer.period, uint, 0644);
+
+/**
* DOC: si_support (int)
* Set SI support driver. This parameter works after set config CONFIG_DRM_AMDGPU_SI. For SI asic, when radeon driver is enabled,
* set value 0 to use radeon driver, while set value 1 to use amdgpu driver. The default is using radeon driver when it available,
@@ -748,6 +777,13 @@ bool no_system_mem_limit;
module_param(no_system_mem_limit, bool, 0644);
MODULE_PARM_DESC(no_system_mem_limit, "disable system memory limit (false = default)");
+/**
+ * DOC: no_queue_eviction_on_vm_fault (int)
+ * If set, process queues will not be evicted on gpuvm fault. This is to keep the wavefront context for debugging (0 = queue eviction, 1 = no queue eviction). The default is 0 (queue eviction).
+ */
+int amdgpu_no_queue_eviction_on_vm_fault = 0;
+MODULE_PARM_DESC(no_queue_eviction_on_vm_fault, "No queue eviction on VM fault (0 = queue eviction, 1 = no queue eviction)");
+module_param_named(no_queue_eviction_on_vm_fault, amdgpu_no_queue_eviction_on_vm_fault, int, 0444);
#endif
/**
@@ -781,6 +817,10 @@ uint amdgpu_dm_abm_level;
MODULE_PARM_DESC(abmlevel, "ABM level (0 = off (default), 1-4 = backlight reduction level) ");
module_param_named(abmlevel, amdgpu_dm_abm_level, uint, 0444);
+int amdgpu_backlight = -1;
+MODULE_PARM_DESC(backlight, "Backlight control (0 = pwm, 1 = aux, -1 auto (default))");
+module_param_named(backlight, amdgpu_backlight, bint, 0444);
+
/**
* DOC: tmz (int)
* Trusted Memory Zone (TMZ) is a method to protect data being written
@@ -788,10 +828,21 @@ module_param_named(abmlevel, amdgpu_dm_abm_level, uint, 0444);
*
* The default value: 0 (off). TODO: change to auto till it is completed.
*/
-MODULE_PARM_DESC(tmz, "Enable TMZ feature (-1 = auto, 0 = off (default), 1 = on)");
+MODULE_PARM_DESC(tmz, "Enable TMZ feature (-1 = auto (default), 0 = off, 1 = on)");
module_param_named(tmz, amdgpu_tmz, int, 0444);
/**
+ * DOC: freesync_video (uint)
+ * Enabled the optimization to adjust front porch timing to achieve seamless mode change experience
+ * when setting a freesync supported mode for which full modeset is not needed.
+ * The default value: 0 (off).
+ */
+MODULE_PARM_DESC(
+ freesync_video,
+ "Enable freesync modesetting optimization feature (0 = off (default), 1 = on)");
+module_param_named(freesync_video, amdgpu_freesync_vid_mode, uint, 0444);
+
+/**
* DOC: reset_method (int)
* GPU reset method (-1 = auto (default), 0 = legacy, 1 = mode0, 2 = mode1, 3 = mode2, 4 = baco, 5 = pci)
*/
@@ -811,6 +862,15 @@ module_param_named(bad_page_threshold, amdgpu_bad_page_threshold, int, 0444);
MODULE_PARM_DESC(num_kcq, "number of kernel compute queue user want to setup (8 if set to greater than 8 or less than 0, only affect gfx 8+)");
module_param_named(num_kcq, amdgpu_num_kcq, int, 0444);
+/**
+ * DOC: smu_pptable_id (int)
+ * Used to override pptable id. id = 0 use VBIOS pptable.
+ * id > 0 use the soft pptable with specicfied id.
+ */
+MODULE_PARM_DESC(smu_pptable_id,
+ "specify pptable id to be used (-1 = auto(default) value, 0 = use pptable from vbios, > 0 = soft pptable id)");
+module_param_named(smu_pptable_id, amdgpu_smu_pptable_id, int, 0444);
+
static const struct pci_device_id pciidlist[] = {
#ifdef CONFIG_DRM_AMDGPU_SI
{0x1002, 0x6780, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_TAHITI},
@@ -1103,6 +1163,7 @@ static const struct pci_device_id pciidlist[] = {
{0x1002, 0x73A3, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
{0x1002, 0x73AB, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
{0x1002, 0x73AE, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
+ {0x1002, 0x73AF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
{0x1002, 0x73BF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_SIENNA_CICHLID},
/* Van Gogh */
@@ -1120,6 +1181,11 @@ static const struct pci_device_id pciidlist[] = {
{0x1002, 0x73E2, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_DIMGREY_CAVEFISH},
{0x1002, 0x73FF, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_DIMGREY_CAVEFISH},
+ /* Aldebaran */
+ {0x1002, 0x7408, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT},
+ {0x1002, 0x740C, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT},
+ {0x1002, 0x740F, PCI_ANY_ID, PCI_ANY_ID, 0, 0, CHIP_ALDEBARAN|AMD_EXP_HW_SUPPORT},
+
{0, 0, 0}
};
@@ -1270,24 +1336,127 @@ amdgpu_pci_shutdown(struct pci_dev *pdev)
*/
if (!amdgpu_passthrough(adev))
adev->mp1_state = PP_MP1_STATE_UNLOAD;
- adev->in_poweroff_reboot_com = true;
amdgpu_device_ip_suspend(adev);
- adev->in_poweroff_reboot_com = false;
adev->mp1_state = PP_MP1_STATE_NONE;
}
+/**
+ * amdgpu_drv_delayed_reset_work_handler - work handler for reset
+ *
+ * @work: work_struct.
+ */
+static void amdgpu_drv_delayed_reset_work_handler(struct work_struct *work)
+{
+ struct list_head device_list;
+ struct amdgpu_device *adev;
+ int i, r;
+ struct amdgpu_reset_context reset_context;
+
+ memset(&reset_context, 0, sizeof(reset_context));
+
+ mutex_lock(&mgpu_info.mutex);
+ if (mgpu_info.pending_reset == true) {
+ mutex_unlock(&mgpu_info.mutex);
+ return;
+ }
+ mgpu_info.pending_reset = true;
+ mutex_unlock(&mgpu_info.mutex);
+
+ /* Use a common context, just need to make sure full reset is done */
+ reset_context.method = AMD_RESET_METHOD_NONE;
+ set_bit(AMDGPU_NEED_FULL_RESET, &reset_context.flags);
+
+ for (i = 0; i < mgpu_info.num_dgpu; i++) {
+ adev = mgpu_info.gpu_ins[i].adev;
+ reset_context.reset_req_dev = adev;
+ r = amdgpu_device_pre_asic_reset(adev, &reset_context);
+ if (r) {
+ dev_err(adev->dev, "GPU pre asic reset failed with err, %d for drm dev, %s ",
+ r, adev_to_drm(adev)->unique);
+ }
+ if (!queue_work(system_unbound_wq, &adev->xgmi_reset_work))
+ r = -EALREADY;
+ }
+ for (i = 0; i < mgpu_info.num_dgpu; i++) {
+ adev = mgpu_info.gpu_ins[i].adev;
+ flush_work(&adev->xgmi_reset_work);
+ adev->gmc.xgmi.pending_reset = false;
+ }
+
+ /* reset function will rebuild the xgmi hive info , clear it now */
+ for (i = 0; i < mgpu_info.num_dgpu; i++)
+ amdgpu_xgmi_remove_device(mgpu_info.gpu_ins[i].adev);
+
+ INIT_LIST_HEAD(&device_list);
+
+ for (i = 0; i < mgpu_info.num_dgpu; i++)
+ list_add_tail(&mgpu_info.gpu_ins[i].adev->reset_list, &device_list);
+
+ /* unregister the GPU first, reset function will add them back */
+ list_for_each_entry(adev, &device_list, reset_list)
+ amdgpu_unregister_gpu_instance(adev);
+
+ /* Use a common context, just need to make sure full reset is done */
+ set_bit(AMDGPU_SKIP_HW_RESET, &reset_context.flags);
+ r = amdgpu_do_asic_reset(&device_list, &reset_context);
+
+ if (r) {
+ DRM_ERROR("reinit gpus failure");
+ return;
+ }
+ for (i = 0; i < mgpu_info.num_dgpu; i++) {
+ adev = mgpu_info.gpu_ins[i].adev;
+ if (!adev->kfd.init_complete)
+ amdgpu_amdkfd_device_init(adev);
+ amdgpu_ttm_set_buffer_funcs_status(adev, true);
+ }
+ return;
+}
+
+static int amdgpu_pmops_prepare(struct device *dev)
+{
+ struct drm_device *drm_dev = dev_get_drvdata(dev);
+
+ /* Return a positive number here so
+ * DPM_FLAG_SMART_SUSPEND works properly
+ */
+ if (amdgpu_device_supports_boco(drm_dev))
+ return pm_runtime_suspended(dev) &&
+ pm_suspend_via_firmware();
+
+ return 0;
+}
+
+static void amdgpu_pmops_complete(struct device *dev)
+{
+ /* nothing to do */
+}
+
static int amdgpu_pmops_suspend(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
+ struct amdgpu_device *adev = drm_to_adev(drm_dev);
+ int r;
- return amdgpu_device_suspend(drm_dev, true);
+ if (amdgpu_acpi_is_s0ix_supported(adev))
+ adev->in_s0ix = true;
+ adev->in_s3 = true;
+ r = amdgpu_device_suspend(drm_dev, true);
+ adev->in_s3 = false;
+
+ return r;
}
static int amdgpu_pmops_resume(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
+ struct amdgpu_device *adev = drm_to_adev(drm_dev);
+ int r;
- return amdgpu_device_resume(drm_dev, true);
+ r = amdgpu_device_resume(drm_dev, true);
+ if (amdgpu_acpi_is_s0ix_supported(adev))
+ adev->in_s0ix = false;
+ return r;
}
static int amdgpu_pmops_freeze(struct device *dev)
@@ -1296,9 +1465,9 @@ static int amdgpu_pmops_freeze(struct device *dev)
struct amdgpu_device *adev = drm_to_adev(drm_dev);
int r;
- adev->in_hibernate = true;
+ adev->in_s4 = true;
r = amdgpu_device_suspend(drm_dev, true);
- adev->in_hibernate = false;
+ adev->in_s4 = false;
if (r)
return r;
return amdgpu_asic_reset(adev);
@@ -1314,13 +1483,8 @@ static int amdgpu_pmops_thaw(struct device *dev)
static int amdgpu_pmops_poweroff(struct device *dev)
{
struct drm_device *drm_dev = dev_get_drvdata(dev);
- struct amdgpu_device *adev = drm_to_adev(drm_dev);
- int r;
- adev->in_poweroff_reboot_com = true;
- r = amdgpu_device_suspend(drm_dev, true);
- adev->in_poweroff_reboot_com = false;
- return r;
+ return amdgpu_device_suspend(drm_dev, true);
}
static int amdgpu_pmops_restore(struct device *dev)
@@ -1353,7 +1517,7 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev)
}
adev->in_runpm = true;
- if (amdgpu_device_supports_atpx(drm_dev))
+ if (amdgpu_device_supports_px(drm_dev))
drm_dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
ret = amdgpu_device_suspend(drm_dev, false);
@@ -1362,16 +1526,14 @@ static int amdgpu_pmops_runtime_suspend(struct device *dev)
return ret;
}
- if (amdgpu_device_supports_atpx(drm_dev)) {
+ if (amdgpu_device_supports_px(drm_dev)) {
/* Only need to handle PCI state in the driver for ATPX
* PCI core handles it for _PR3.
*/
- if (!amdgpu_is_atpx_hybrid()) {
- amdgpu_device_cache_pci_state(pdev);
- pci_disable_device(pdev);
- pci_ignore_hotplug(pdev);
- pci_set_power_state(pdev, PCI_D3cold);
- }
+ amdgpu_device_cache_pci_state(pdev);
+ pci_disable_device(pdev);
+ pci_ignore_hotplug(pdev);
+ pci_set_power_state(pdev, PCI_D3cold);
drm_dev->switch_power_state = DRM_SWITCH_POWER_DYNAMIC_OFF;
} else if (amdgpu_device_supports_baco(drm_dev)) {
amdgpu_device_baco_enter(drm_dev);
@@ -1390,19 +1552,17 @@ static int amdgpu_pmops_runtime_resume(struct device *dev)
if (!adev->runpm)
return -EINVAL;
- if (amdgpu_device_supports_atpx(drm_dev)) {
+ if (amdgpu_device_supports_px(drm_dev)) {
drm_dev->switch_power_state = DRM_SWITCH_POWER_CHANGING;
/* Only need to handle PCI state in the driver for ATPX
* PCI core handles it for _PR3.
*/
- if (!amdgpu_is_atpx_hybrid()) {
- pci_set_power_state(pdev, PCI_D0);
- amdgpu_device_load_pci_state(pdev);
- ret = pci_enable_device(pdev);
- if (ret)
- return ret;
- }
+ pci_set_power_state(pdev, PCI_D0);
+ amdgpu_device_load_pci_state(pdev);
+ ret = pci_enable_device(pdev);
+ if (ret)
+ return ret;
pci_set_master(pdev);
} else if (amdgpu_device_supports_boco(drm_dev)) {
/* Only need to handle PCI state in the driver for ATPX
@@ -1413,7 +1573,10 @@ static int amdgpu_pmops_runtime_resume(struct device *dev)
amdgpu_device_baco_exit(drm_dev);
}
ret = amdgpu_device_resume(drm_dev, false);
- if (amdgpu_device_supports_atpx(drm_dev))
+ if (ret)
+ return ret;
+
+ if (amdgpu_device_supports_px(drm_dev))
drm_dev->switch_power_state = DRM_SWITCH_POWER_ON;
adev->in_runpm = false;
return 0;
@@ -1494,6 +1657,8 @@ out:
}
static const struct dev_pm_ops amdgpu_pm_ops = {
+ .prepare = amdgpu_pmops_prepare,
+ .complete = amdgpu_pmops_complete,
.suspend = amdgpu_pmops_suspend,
.resume = amdgpu_pmops_resume,
.freeze = amdgpu_pmops_freeze,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c
index 51cd49c6f38f..4f10c4529840 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fb.c
@@ -146,7 +146,7 @@ static int amdgpufb_create_pinned_object(struct amdgpu_fbdev *rfbdev,
size = mode_cmd->pitches[0] * height;
aligned_size = ALIGN(size, PAGE_SIZE);
ret = amdgpu_gem_object_create(adev, aligned_size, 0, domain, flags,
- ttm_bo_type_kernel, NULL, &gobj);
+ ttm_bo_type_device, NULL, &gobj);
if (ret) {
pr_err("failed to allocate framebuffer (%d)\n", aligned_size);
return -ENOMEM;
@@ -205,7 +205,6 @@ static int amdgpufb_create(struct drm_fb_helper *helper,
struct drm_gem_object *gobj = NULL;
struct amdgpu_bo *abo = NULL;
int ret;
- unsigned long tmp;
memset(&mode_cmd, 0, sizeof(mode_cmd));
mode_cmd.width = sizes->surface_width;
@@ -232,8 +231,8 @@ static int amdgpufb_create(struct drm_fb_helper *helper,
goto out;
}
- ret = amdgpu_display_framebuffer_init(adev_to_drm(adev), &rfbdev->rfb,
- &mode_cmd, gobj);
+ ret = amdgpu_display_gem_fb_init(adev_to_drm(adev), &rfbdev->rfb,
+ &mode_cmd, gobj);
if (ret) {
DRM_ERROR("failed to initialize framebuffer %d\n", ret);
goto out;
@@ -246,8 +245,7 @@ static int amdgpufb_create(struct drm_fb_helper *helper,
info->fbops = &amdgpufb_ops;
- tmp = amdgpu_bo_gpu_offset(abo) - adev->gmc.vram_start;
- info->fix.smem_start = adev->gmc.aper_base + tmp;
+ info->fix.smem_start = amdgpu_gmc_vram_cpu_pa(adev, abo);
info->fix.smem_len = amdgpu_bo_size(abo);
info->screen_base = amdgpu_bo_kptr(abo);
info->screen_size = amdgpu_bo_size(abo);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
index d56f4023ebb3..47ea46859618 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
@@ -36,8 +36,6 @@
#include <linux/firmware.h>
#include <linux/pm_runtime.h>
-#include <drm/drm_debugfs.h>
-
#include "amdgpu.h"
#include "amdgpu_trace.h"
@@ -441,7 +439,8 @@ int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring,
* Helper function for amdgpu_fence_driver_init().
*/
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
- unsigned num_hw_submission)
+ unsigned num_hw_submission,
+ atomic_t *sched_score)
{
struct amdgpu_device *adev = ring->adev;
long timeout;
@@ -469,30 +468,31 @@ int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
return -ENOMEM;
/* No need to setup the GPU scheduler for rings that don't need it */
- if (!ring->no_scheduler) {
- switch (ring->funcs->type) {
- case AMDGPU_RING_TYPE_GFX:
- timeout = adev->gfx_timeout;
- break;
- case AMDGPU_RING_TYPE_COMPUTE:
- timeout = adev->compute_timeout;
- break;
- case AMDGPU_RING_TYPE_SDMA:
- timeout = adev->sdma_timeout;
- break;
- default:
- timeout = adev->video_timeout;
- break;
- }
+ if (ring->no_scheduler)
+ return 0;
- r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
- num_hw_submission, amdgpu_job_hang_limit,
- timeout, ring->name);
- if (r) {
- DRM_ERROR("Failed to create scheduler on ring %s.\n",
- ring->name);
- return r;
- }
+ switch (ring->funcs->type) {
+ case AMDGPU_RING_TYPE_GFX:
+ timeout = adev->gfx_timeout;
+ break;
+ case AMDGPU_RING_TYPE_COMPUTE:
+ timeout = adev->compute_timeout;
+ break;
+ case AMDGPU_RING_TYPE_SDMA:
+ timeout = adev->sdma_timeout;
+ break;
+ default:
+ timeout = adev->video_timeout;
+ break;
+ }
+
+ r = drm_sched_init(&ring->sched, &amdgpu_sched_ops,
+ num_hw_submission, amdgpu_job_hang_limit,
+ timeout, sched_score, ring->name);
+ if (r) {
+ DRM_ERROR("Failed to create scheduler on ring %s.\n",
+ ring->name);
+ return r;
}
return 0;
@@ -533,6 +533,8 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
if (!ring || !ring->fence_drv.initialized)
continue;
+ if (!ring->no_scheduler)
+ drm_sched_fini(&ring->sched);
r = amdgpu_fence_wait_empty(ring);
if (r) {
/* no need to trigger GPU reset as we are unloading */
@@ -541,8 +543,7 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev)
if (ring->fence_drv.irq_src)
amdgpu_irq_put(adev, ring->fence_drv.irq_src,
ring->fence_drv.irq_type);
- if (!ring->no_scheduler)
- drm_sched_fini(&ring->sched);
+
del_timer_sync(&ring->fence_drv.fallback_timer);
for (j = 0; j <= ring->fence_drv.num_fences_mask; ++j)
dma_fence_put(ring->fence_drv.fences[j]);
@@ -697,11 +698,9 @@ static const struct dma_fence_ops amdgpu_fence_ops = {
* Fence debugfs
*/
#if defined(CONFIG_DEBUG_FS)
-static int amdgpu_debugfs_fence_info(struct seq_file *m, void *data)
+static int amdgpu_debugfs_fence_info_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
int i;
for (i = 0; i < AMDGPU_MAX_RINGS; ++i) {
@@ -746,11 +745,10 @@ static int amdgpu_debugfs_fence_info(struct seq_file *m, void *data)
*
* Manually trigger a gpu reset at the next fence wait.
*/
-static int amdgpu_debugfs_gpu_recover(struct seq_file *m, void *data)
+static int gpu_recover_get(void *data, u64 *val)
{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)data;
+ struct drm_device *dev = adev_to_drm(adev);
int r;
r = pm_runtime_get_sync(dev->dev);
@@ -759,8 +757,7 @@ static int amdgpu_debugfs_gpu_recover(struct seq_file *m, void *data)
return 0;
}
- seq_printf(m, "gpu recover\n");
- amdgpu_device_gpu_recover(adev, NULL);
+ *val = amdgpu_device_gpu_recover(adev, NULL);
pm_runtime_mark_last_busy(dev->dev);
pm_runtime_put_autosuspend(dev->dev);
@@ -768,26 +765,24 @@ static int amdgpu_debugfs_gpu_recover(struct seq_file *m, void *data)
return 0;
}
-static const struct drm_info_list amdgpu_debugfs_fence_list[] = {
- {"amdgpu_fence_info", &amdgpu_debugfs_fence_info, 0, NULL},
- {"amdgpu_gpu_recover", &amdgpu_debugfs_gpu_recover, 0, NULL}
-};
+DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_fence_info);
+DEFINE_DEBUGFS_ATTRIBUTE(amdgpu_debugfs_gpu_recover_fops, gpu_recover_get, NULL,
+ "%lld\n");
-static const struct drm_info_list amdgpu_debugfs_fence_list_sriov[] = {
- {"amdgpu_fence_info", &amdgpu_debugfs_fence_info, 0, NULL},
-};
#endif
-int amdgpu_debugfs_fence_init(struct amdgpu_device *adev)
+void amdgpu_debugfs_fence_init(struct amdgpu_device *adev)
{
#if defined(CONFIG_DEBUG_FS)
- if (amdgpu_sriov_vf(adev))
- return amdgpu_debugfs_add_files(adev, amdgpu_debugfs_fence_list_sriov,
- ARRAY_SIZE(amdgpu_debugfs_fence_list_sriov));
- return amdgpu_debugfs_add_files(adev, amdgpu_debugfs_fence_list,
- ARRAY_SIZE(amdgpu_debugfs_fence_list));
-#else
- return 0;
+ struct drm_minor *minor = adev_to_drm(adev)->primary;
+ struct dentry *root = minor->debugfs_root;
+
+ debugfs_create_file("amdgpu_fence_info", 0444, root, adev,
+ &amdgpu_debugfs_fence_info_fops);
+
+ if (!amdgpu_sriov_vf(adev))
+ debugfs_create_file("amdgpu_gpu_recover", 0444, root, adev,
+ &amdgpu_debugfs_gpu_recover_fops);
#endif
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
index 0db933026722..c5a9a4fb10d2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.c
@@ -71,7 +71,7 @@
*/
static int amdgpu_gart_dummy_page_init(struct amdgpu_device *adev)
{
- struct page *dummy_page = ttm_bo_glob.dummy_read_page;
+ struct page *dummy_page = ttm_glob.dummy_read_page;
if (adev->dummy_page_addr)
return 0;
@@ -126,6 +126,8 @@ int amdgpu_gart_table_vram_alloc(struct amdgpu_device *adev)
AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
r = amdgpu_bo_create(adev, &bp, &adev->gart.bo);
if (r) {
return r;
@@ -202,6 +204,7 @@ void amdgpu_gart_table_vram_free(struct amdgpu_device *adev)
return;
}
amdgpu_bo_unref(&adev->gart.bo);
+ adev->gart.ptr = NULL;
}
/*
@@ -236,9 +239,6 @@ int amdgpu_gart_unbind(struct amdgpu_device *adev, uint64_t offset,
t = offset / AMDGPU_GPU_PAGE_SIZE;
p = t / AMDGPU_GPU_PAGES_IN_CPU_PAGE;
for (i = 0; i < pages; i++, p++) {
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
- adev->gart.pages[p] = NULL;
-#endif
page_base = adev->dummy_page_addr;
if (!adev->gart.ptr)
continue;
@@ -312,9 +312,6 @@ int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,
int pages, struct page **pagelist, dma_addr_t *dma_addr,
uint64_t flags)
{
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
- unsigned t,p;
-#endif
int r, i;
if (!adev->gart.ready) {
@@ -322,13 +319,6 @@ int amdgpu_gart_bind(struct amdgpu_device *adev, uint64_t offset,
return -EINVAL;
}
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
- t = offset / AMDGPU_GPU_PAGE_SIZE;
- p = t / AMDGPU_GPU_PAGES_IN_CPU_PAGE;
- for (i = 0; i < pages; i++, p++)
- adev->gart.pages[p] = pagelist ? pagelist[i] : NULL;
-#endif
-
if (!adev->gart.ptr)
return 0;
@@ -373,14 +363,6 @@ int amdgpu_gart_init(struct amdgpu_device *adev)
DRM_INFO("GART: num cpu pages %u, num gpu pages %u\n",
adev->gart.num_cpu_pages, adev->gart.num_gpu_pages);
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
- /* Allocate pages table */
- adev->gart.pages = vzalloc(array_size(sizeof(void *),
- adev->gart.num_cpu_pages));
- if (adev->gart.pages == NULL)
- return -ENOMEM;
-#endif
-
return 0;
}
@@ -393,9 +375,5 @@ int amdgpu_gart_init(struct amdgpu_device *adev)
*/
void amdgpu_gart_fini(struct amdgpu_device *adev)
{
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
- vfree(adev->gart.pages);
- adev->gart.pages = NULL;
-#endif
amdgpu_gart_dummy_page_fini(adev);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
index afa2e2877d87..a25fe97b0196 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gart.h
@@ -46,9 +46,6 @@ struct amdgpu_gart {
unsigned num_gpu_pages;
unsigned num_cpu_pages;
unsigned table_size;
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
- struct page **pages;
-#endif
bool ready;
/* Asic default pte flags */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
index b443907afcea..311bcdc59eda 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gem.c
@@ -32,7 +32,6 @@
#include <linux/dma-buf.h>
#include <drm/amdgpu_drm.h>
-#include <drm/drm_debugfs.h>
#include <drm/drm_gem_ttm_helper.h>
#include "amdgpu.h"
@@ -59,6 +58,7 @@ int amdgpu_gem_object_create(struct amdgpu_device *adev, unsigned long size,
struct drm_gem_object **obj)
{
struct amdgpu_bo *bo;
+ struct amdgpu_bo_user *ubo;
struct amdgpu_bo_param bp;
int r;
@@ -72,10 +72,13 @@ int amdgpu_gem_object_create(struct amdgpu_device *adev, unsigned long size,
bp.preferred_domain = initial_domain;
bp.flags = flags;
bp.domain = initial_domain;
- r = amdgpu_bo_create(adev, &bp, &bo);
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
+ r = amdgpu_bo_create_user(adev, &bp, &ubo);
if (r)
return r;
+ bo = &ubo->bo;
*obj = &bo->tbo.base;
(*obj)->funcs = &amdgpu_gem_object_funcs;
@@ -855,10 +858,10 @@ int amdgpu_mode_dumb_create(struct drm_file *file_priv,
}
#if defined(CONFIG_DEBUG_FS)
-static int amdgpu_debugfs_gem_info(struct seq_file *m, void *data)
+static int amdgpu_debugfs_gem_info_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct drm_device *dev = adev_to_drm(adev);
struct drm_file *file;
int r;
@@ -896,16 +899,17 @@ static int amdgpu_debugfs_gem_info(struct seq_file *m, void *data)
return 0;
}
-static const struct drm_info_list amdgpu_debugfs_gem_list[] = {
- {"amdgpu_gem_info", &amdgpu_debugfs_gem_info, 0, NULL},
-};
+DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_gem_info);
+
#endif
-int amdgpu_debugfs_gem_init(struct amdgpu_device *adev)
+void amdgpu_debugfs_gem_init(struct amdgpu_device *adev)
{
#if defined(CONFIG_DEBUG_FS)
- return amdgpu_debugfs_add_files(adev, amdgpu_debugfs_gem_list,
- ARRAY_SIZE(amdgpu_debugfs_gem_list));
+ struct drm_minor *minor = adev_to_drm(adev)->primary;
+ struct dentry *root = minor->debugfs_root;
+
+ debugfs_create_file("amdgpu_gem_info", 0444, root, adev,
+ &amdgpu_debugfs_gem_info_fops);
#endif
- return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
index 8e0a6c62322e..95d4f43a03df 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.c
@@ -310,9 +310,8 @@ int amdgpu_gfx_kiq_init_ring(struct amdgpu_device *adev,
ring->eop_gpu_addr = kiq->eop_gpu_addr;
ring->no_scheduler = true;
sprintf(ring->name, "kiq_%d.%d.%d", ring->me, ring->pipe, ring->queue);
- r = amdgpu_ring_init(adev, ring, 1024,
- irq, AMDGPU_CP_KIQ_IRQ_DRIVER0,
- AMDGPU_RING_PRIO_DEFAULT);
+ r = amdgpu_ring_init(adev, ring, 1024, irq, AMDGPU_CP_KIQ_IRQ_DRIVER0,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
dev_warn(adev->dev, "(%d) failed to init kiq ring\n", r);
@@ -463,20 +462,25 @@ int amdgpu_gfx_disable_kcq(struct amdgpu_device *adev)
{
struct amdgpu_kiq *kiq = &adev->gfx.kiq;
struct amdgpu_ring *kiq_ring = &kiq->ring;
- int i;
+ int i, r;
if (!kiq->pmf || !kiq->pmf->kiq_unmap_queues)
return -EINVAL;
+ spin_lock(&adev->gfx.kiq.ring_lock);
if (amdgpu_ring_alloc(kiq_ring, kiq->pmf->unmap_queues_size *
- adev->gfx.num_compute_rings))
+ adev->gfx.num_compute_rings)) {
+ spin_unlock(&adev->gfx.kiq.ring_lock);
return -ENOMEM;
+ }
for (i = 0; i < adev->gfx.num_compute_rings; i++)
kiq->pmf->kiq_unmap_queues(kiq_ring, &adev->gfx.compute_ring[i],
RESET_QUEUES, 0, 0);
+ r = amdgpu_ring_test_helper(kiq_ring);
+ spin_unlock(&adev->gfx.kiq.ring_lock);
- return amdgpu_ring_test_helper(kiq_ring);
+ return r;
}
int amdgpu_queue_mask_bit_to_set_resource_bit(struct amdgpu_device *adev,
@@ -519,12 +523,13 @@ int amdgpu_gfx_enable_kcq(struct amdgpu_device *adev)
DRM_INFO("kiq ring mec %d pipe %d q %d\n", kiq_ring->me, kiq_ring->pipe,
kiq_ring->queue);
-
+ spin_lock(&adev->gfx.kiq.ring_lock);
r = amdgpu_ring_alloc(kiq_ring, kiq->pmf->map_queues_size *
adev->gfx.num_compute_rings +
kiq->pmf->set_resources_size);
if (r) {
DRM_ERROR("Failed to lock KIQ (%d).\n", r);
+ spin_unlock(&adev->gfx.kiq.ring_lock);
return r;
}
@@ -533,6 +538,7 @@ int amdgpu_gfx_enable_kcq(struct amdgpu_device *adev)
kiq->pmf->kiq_map_queues(kiq_ring, &adev->gfx.compute_ring[i]);
r = amdgpu_ring_test_helper(kiq_ring);
+ spin_unlock(&adev->gfx.kiq.ring_lock);
if (r)
DRM_ERROR("KCQ enable failed\n");
@@ -601,6 +607,7 @@ int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev)
struct ras_ih_if ih_info = {
.cb = amdgpu_gfx_process_ras_data_cb,
};
+ struct ras_query_if info = { 0 };
if (!adev->gfx.ras_if) {
adev->gfx.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
@@ -612,13 +619,19 @@ int amdgpu_gfx_ras_late_init(struct amdgpu_device *adev)
strcpy(adev->gfx.ras_if->name, "gfx");
}
fs_info.head = ih_info.head = *adev->gfx.ras_if;
-
r = amdgpu_ras_late_init(adev, adev->gfx.ras_if,
&fs_info, &ih_info);
if (r)
goto free;
if (amdgpu_ras_is_supported(adev, adev->gfx.ras_if->block)) {
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ info.head = *adev->gfx.ras_if;
+ amdgpu_ras_query_error_status(adev, &info);
+ } else {
+ amdgpu_ras_reset_error_status(adev, AMDGPU_RAS_BLOCK__GFX);
+ }
+
r = amdgpu_irq_get(adev, &adev->gfx.cp_ecc_error_irq, 0);
if (r)
goto late_fini;
@@ -664,8 +677,9 @@ int amdgpu_gfx_process_ras_data_cb(struct amdgpu_device *adev,
*/
if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX)) {
kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
- if (adev->gfx.funcs->query_ras_error_count)
- adev->gfx.funcs->query_ras_error_count(adev, err_data);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_count)
+ adev->gfx.ras_funcs->query_ras_error_count(adev, err_data);
amdgpu_ras_reset_gpu(adev);
}
return AMDGPU_RAS_SUCCESS;
@@ -698,7 +712,7 @@ uint32_t amdgpu_kiq_rreg(struct amdgpu_device *adev, uint32_t reg)
struct amdgpu_kiq *kiq = &adev->gfx.kiq;
struct amdgpu_ring *ring = &kiq->ring;
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return 0;
BUG_ON(!ring->funcs->emit_rreg);
@@ -765,7 +779,7 @@ void amdgpu_kiq_wreg(struct amdgpu_device *adev, uint32_t reg, uint32_t v)
BUG_ON(!ring->funcs->emit_wreg);
- if (adev->in_pci_err_recovery)
+ if (amdgpu_device_skip_hw_access(adev))
return;
spin_lock_irqsave(&kiq->ring_lock, flags);
@@ -829,14 +843,10 @@ int amdgpu_gfx_get_num_kcq(struct amdgpu_device *adev)
void amdgpu_gfx_state_change_set(struct amdgpu_device *adev, enum gfx_change_state state)
{
- if (is_support_sw_smu(adev)) {
- smu_gfx_state_change_set(&adev->smu, state);
- } else {
- mutex_lock(&adev->pm.mutex);
- if (adev->powerplay.pp_funcs &&
- adev->powerplay.pp_funcs->gfx_state_change_set)
- ((adev)->powerplay.pp_funcs->gfx_state_change_set(
- (adev)->powerplay.pp_handle, state));
- mutex_unlock(&adev->pm.mutex);
- }
+ mutex_lock(&adev->pm.mutex);
+ if (adev->powerplay.pp_funcs &&
+ adev->powerplay.pp_funcs->gfx_state_change_set)
+ ((adev)->powerplay.pp_funcs->gfx_state_change_set(
+ (adev)->powerplay.pp_handle, state));
+ mutex_unlock(&adev->pm.mutex);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
index 72dbcd2bc6a6..d43fe2ed8116 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gfx.h
@@ -30,6 +30,7 @@
#include "clearstate_defs.h"
#include "amdgpu_ring.h"
#include "amdgpu_rlc.h"
+#include "soc15.h"
/* GFX current status */
#define AMDGPU_GFX_NORMAL_MODE 0x00000000L
@@ -204,6 +205,19 @@ struct amdgpu_cu_info {
uint32_t bitmap[4][4];
};
+struct amdgpu_gfx_ras_funcs {
+ int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
+ int (*ras_error_inject)(struct amdgpu_device *adev,
+ void *inject_if);
+ int (*query_ras_error_count)(struct amdgpu_device *adev,
+ void *ras_error_status);
+ void (*reset_ras_error_count)(struct amdgpu_device *adev);
+ void (*query_ras_error_status)(struct amdgpu_device *adev);
+ void (*reset_ras_error_status)(struct amdgpu_device *adev);
+ void (*enable_watchdog_timer)(struct amdgpu_device *adev);
+};
+
struct amdgpu_gfx_funcs {
/* get the gpu clock counter */
uint64_t (*get_gpu_clock_counter)(struct amdgpu_device *adev);
@@ -219,11 +233,7 @@ struct amdgpu_gfx_funcs {
uint32_t *dst);
void (*select_me_pipe_q)(struct amdgpu_device *adev, u32 me, u32 pipe,
u32 queue, u32 vmid);
- int (*ras_error_inject)(struct amdgpu_device *adev, void *inject_if);
- int (*query_ras_error_count) (struct amdgpu_device *adev, void *ras_error_status);
- void (*reset_ras_error_count) (struct amdgpu_device *adev);
void (*init_spm_golden)(struct amdgpu_device *adev);
- void (*query_ras_error_status) (struct amdgpu_device *adev);
void (*update_perfmon_mgcg)(struct amdgpu_device *adev, bool enable);
};
@@ -327,7 +337,8 @@ struct amdgpu_gfx {
DECLARE_BITMAP (pipe_reserve_bitmap, AMDGPU_MAX_COMPUTE_QUEUES);
/*ras */
- struct ras_common_if *ras_if;
+ struct ras_common_if *ras_if;
+ const struct amdgpu_gfx_ras_funcs *ras_funcs;
};
#define amdgpu_gfx_get_gpu_clock_counter(adev) (adev)->gfx.funcs->get_gpu_clock_counter((adev))
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
index fe1a39ffda72..c39ed9eb0987 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.c
@@ -32,6 +32,59 @@
#include "amdgpu_xgmi.h"
/**
+ * amdgpu_gmc_pdb0_alloc - allocate vram for pdb0
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * Allocate video memory for pdb0 and map it for CPU access
+ * Returns 0 for success, error for failure.
+ */
+int amdgpu_gmc_pdb0_alloc(struct amdgpu_device *adev)
+{
+ int r;
+ struct amdgpu_bo_param bp;
+ u64 vram_size = adev->gmc.xgmi.node_segment_size * adev->gmc.xgmi.num_physical_nodes;
+ uint32_t pde0_page_shift = adev->gmc.vmid0_page_table_block_size + 21;
+ uint32_t npdes = (vram_size + (1ULL << pde0_page_shift) -1) >> pde0_page_shift;
+
+ memset(&bp, 0, sizeof(bp));
+ bp.size = PAGE_ALIGN((npdes + 1) * 8);
+ bp.byte_align = PAGE_SIZE;
+ bp.domain = AMDGPU_GEM_DOMAIN_VRAM;
+ bp.flags = AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED |
+ AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
+ bp.type = ttm_bo_type_kernel;
+ bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
+
+ r = amdgpu_bo_create(adev, &bp, &adev->gmc.pdb0_bo);
+ if (r)
+ return r;
+
+ r = amdgpu_bo_reserve(adev->gmc.pdb0_bo, false);
+ if (unlikely(r != 0))
+ goto bo_reserve_failure;
+
+ r = amdgpu_bo_pin(adev->gmc.pdb0_bo, AMDGPU_GEM_DOMAIN_VRAM);
+ if (r)
+ goto bo_pin_failure;
+ r = amdgpu_bo_kmap(adev->gmc.pdb0_bo, &adev->gmc.ptr_pdb0);
+ if (r)
+ goto bo_kmap_failure;
+
+ amdgpu_bo_unreserve(adev->gmc.pdb0_bo);
+ return 0;
+
+bo_kmap_failure:
+ amdgpu_bo_unpin(adev->gmc.pdb0_bo);
+bo_pin_failure:
+ amdgpu_bo_unreserve(adev->gmc.pdb0_bo);
+bo_reserve_failure:
+ amdgpu_bo_unref(&adev->gmc.pdb0_bo);
+ return r;
+}
+
+/**
* amdgpu_gmc_get_pde_for_bo - get the PDE for a BO
*
* @bo: the BO to get the PDE for
@@ -158,6 +211,39 @@ void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
mc->vram_end, mc->real_vram_size >> 20);
}
+/** amdgpu_gmc_sysvm_location - place vram and gart in sysvm aperture
+ *
+ * @adev: amdgpu device structure holding all necessary information
+ * @mc: memory controller structure holding memory information
+ *
+ * This function is only used if use GART for FB translation. In such
+ * case, we use sysvm aperture (vmid0 page tables) for both vram
+ * and gart (aka system memory) access.
+ *
+ * GPUVM (and our organization of vmid0 page tables) require sysvm
+ * aperture to be placed at a location aligned with 8 times of native
+ * page size. For example, if vm_context0_cntl.page_table_block_size
+ * is 12, then native page size is 8G (2M*2^12), sysvm should start
+ * with a 64G aligned address. For simplicity, we just put sysvm at
+ * address 0. So vram start at address 0 and gart is right after vram.
+ */
+void amdgpu_gmc_sysvm_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
+{
+ u64 hive_vram_start = 0;
+ u64 hive_vram_end = mc->xgmi.node_segment_size * mc->xgmi.num_physical_nodes - 1;
+ mc->vram_start = mc->xgmi.node_segment_size * mc->xgmi.physical_node_id;
+ mc->vram_end = mc->vram_start + mc->xgmi.node_segment_size - 1;
+ mc->gart_start = hive_vram_end + 1;
+ mc->gart_end = mc->gart_start + mc->gart_size - 1;
+ mc->fb_start = hive_vram_start;
+ mc->fb_end = hive_vram_end;
+ dev_info(adev->dev, "VRAM: %lluM 0x%016llX - 0x%016llX (%lluM used)\n",
+ mc->mc_vram_size >> 20, mc->vram_start,
+ mc->vram_end, mc->real_vram_size >> 20);
+ dev_info(adev->dev, "GART: %lluM 0x%016llX - 0x%016llX\n",
+ mc->gart_size >> 20, mc->gart_start, mc->gart_end);
+}
+
/**
* amdgpu_gmc_gart_location - try to find GART location
*
@@ -165,7 +251,6 @@ void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
* @mc: memory controller structure holding memory information
*
* Function will place try to place GART before or after VRAM.
- *
* If GART size is bigger than space left then we ajust GART size.
* Thus function will never fails.
*/
@@ -176,8 +261,6 @@ void amdgpu_gmc_gart_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc)
/*To avoid the hole, limit the max mc address to AMDGPU_GMC_HOLE_START*/
u64 max_mc_address = min(adev->gmc.mc_mask, AMDGPU_GMC_HOLE_START - 1);
- mc->gart_size += adev->pm.smu_prv_buffer_size;
-
/* VCE doesn't like it when BOs cross a 4GB segment, so align
* the GART base on a 4GB boundary as well.
*/
@@ -308,26 +391,46 @@ int amdgpu_gmc_ras_late_init(struct amdgpu_device *adev)
{
int r;
- if (adev->umc.funcs && adev->umc.funcs->ras_late_init) {
- r = adev->umc.funcs->ras_late_init(adev);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ras_late_init) {
+ r = adev->umc.ras_funcs->ras_late_init(adev);
if (r)
return r;
}
- if (adev->mmhub.funcs && adev->mmhub.funcs->ras_late_init) {
- r = adev->mmhub.funcs->ras_late_init(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->ras_late_init) {
+ r = adev->mmhub.ras_funcs->ras_late_init(adev);
if (r)
return r;
}
- return amdgpu_xgmi_ras_late_init(adev);
+ if (!adev->gmc.xgmi.connected_to_cpu)
+ adev->gmc.xgmi.ras_funcs = &xgmi_ras_funcs;
+
+ if (adev->gmc.xgmi.ras_funcs &&
+ adev->gmc.xgmi.ras_funcs->ras_late_init) {
+ r = adev->gmc.xgmi.ras_funcs->ras_late_init(adev);
+ if (r)
+ return r;
+ }
+
+ return 0;
}
void amdgpu_gmc_ras_fini(struct amdgpu_device *adev)
{
- amdgpu_umc_ras_fini(adev);
- amdgpu_mmhub_ras_fini(adev);
- amdgpu_xgmi_ras_fini(adev);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->ras_fini)
+ adev->umc.ras_funcs->ras_fini(adev);
+
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->ras_fini)
+ amdgpu_mmhub_ras_fini(adev);
+
+ if (adev->gmc.xgmi.ras_funcs &&
+ adev->gmc.xgmi.ras_funcs->ras_fini)
+ adev->gmc.xgmi.ras_funcs->ras_fini(adev);
}
/*
@@ -385,6 +488,16 @@ void amdgpu_gmc_tmz_set(struct amdgpu_device *adev)
switch (adev->asic_type) {
case CHIP_RAVEN:
case CHIP_RENOIR:
+ if (amdgpu_tmz == 0) {
+ adev->gmc.tmz_enabled = false;
+ dev_info(adev->dev,
+ "Trusted Memory Zone (TMZ) feature disabled (cmd line)\n");
+ } else {
+ adev->gmc.tmz_enabled = true;
+ dev_info(adev->dev,
+ "Trusted Memory Zone (TMZ) feature enabled\n");
+ }
+ break;
case CHIP_NAVI10:
case CHIP_NAVI14:
case CHIP_NAVI12:
@@ -423,6 +536,8 @@ void amdgpu_gmc_noretry_set(struct amdgpu_device *adev)
switch (adev->asic_type) {
case CHIP_VEGA10:
case CHIP_VEGA20:
+ case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
/*
* noretry = 0 will cause kfd page fault tests fail
* for some ASICs, so set default to 1 for these ASICs.
@@ -518,3 +633,90 @@ void amdgpu_gmc_get_vbios_allocations(struct amdgpu_device *adev)
adev->mman.stolen_extended_size = 0;
}
}
+
+/**
+ * amdgpu_gmc_init_pdb0 - initialize PDB0
+ *
+ * @adev: amdgpu_device pointer
+ *
+ * This function is only used when GART page table is used
+ * for FB address translatioin. In such a case, we construct
+ * a 2-level system VM page table: PDB0->PTB, to cover both
+ * VRAM of the hive and system memory.
+ *
+ * PDB0 is static, initialized once on driver initialization.
+ * The first n entries of PDB0 are used as PTE by setting
+ * P bit to 1, pointing to VRAM. The n+1'th entry points
+ * to a big PTB covering system memory.
+ *
+ */
+void amdgpu_gmc_init_pdb0(struct amdgpu_device *adev)
+{
+ int i;
+ uint64_t flags = adev->gart.gart_pte_flags; //TODO it is UC. explore NC/RW?
+ /* Each PDE0 (used as PTE) covers (2^vmid0_page_table_block_size)*2M
+ */
+ u64 vram_size = adev->gmc.xgmi.node_segment_size * adev->gmc.xgmi.num_physical_nodes;
+ u64 pde0_page_size = (1ULL<<adev->gmc.vmid0_page_table_block_size)<<21;
+ u64 vram_addr = adev->vm_manager.vram_base_offset -
+ adev->gmc.xgmi.physical_node_id * adev->gmc.xgmi.node_segment_size;
+ u64 vram_end = vram_addr + vram_size;
+ u64 gart_ptb_gpu_pa = amdgpu_gmc_vram_pa(adev, adev->gart.bo);
+
+ flags |= AMDGPU_PTE_VALID | AMDGPU_PTE_READABLE;
+ flags |= AMDGPU_PTE_WRITEABLE;
+ flags |= AMDGPU_PTE_SNOOPED;
+ flags |= AMDGPU_PTE_FRAG((adev->gmc.vmid0_page_table_block_size + 9*1));
+ flags |= AMDGPU_PDE_PTE;
+
+ /* The first n PDE0 entries are used as PTE,
+ * pointing to vram
+ */
+ for (i = 0; vram_addr < vram_end; i++, vram_addr += pde0_page_size)
+ amdgpu_gmc_set_pte_pde(adev, adev->gmc.ptr_pdb0, i, vram_addr, flags);
+
+ /* The n+1'th PDE0 entry points to a huge
+ * PTB who has more than 512 entries each
+ * pointing to a 4K system page
+ */
+ flags = AMDGPU_PTE_VALID;
+ flags |= AMDGPU_PDE_BFS(0) | AMDGPU_PTE_SNOOPED;
+ /* Requires gart_ptb_gpu_pa to be 4K aligned */
+ amdgpu_gmc_set_pte_pde(adev, adev->gmc.ptr_pdb0, i, gart_ptb_gpu_pa, flags);
+}
+
+/**
+ * amdgpu_gmc_vram_mc2pa - calculate vram buffer's physical address from MC
+ * address
+ *
+ * @adev: amdgpu_device pointer
+ * @mc_addr: MC address of buffer
+ */
+uint64_t amdgpu_gmc_vram_mc2pa(struct amdgpu_device *adev, uint64_t mc_addr)
+{
+ return mc_addr - adev->gmc.vram_start + adev->vm_manager.vram_base_offset;
+}
+
+/**
+ * amdgpu_gmc_vram_pa - calculate vram buffer object's physical address from
+ * GPU's view
+ *
+ * @adev: amdgpu_device pointer
+ * @bo: amdgpu buffer object
+ */
+uint64_t amdgpu_gmc_vram_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo)
+{
+ return amdgpu_gmc_vram_mc2pa(adev, amdgpu_bo_gpu_offset(bo));
+}
+
+/**
+ * amdgpu_gmc_vram_cpu_pa - calculate vram buffer object's physical address
+ * from CPU's view
+ *
+ * @adev: amdgpu_device pointer
+ * @bo: amdgpu buffer object
+ */
+uint64_t amdgpu_gmc_vram_cpu_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo)
+{
+ return amdgpu_bo_gpu_offset(bo) - adev->gmc.vram_start + adev->gmc.aper_base;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
index aa0c83776ce0..9d11c02a3938 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gmc.h
@@ -135,6 +135,14 @@ struct amdgpu_gmc_funcs {
unsigned int (*get_vbios_fb_size)(struct amdgpu_device *adev);
};
+struct amdgpu_xgmi_ras_funcs {
+ int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
+ int (*query_ras_error_count)(struct amdgpu_device *adev,
+ void *ras_error_status);
+ void (*reset_ras_error_count)(struct amdgpu_device *adev);
+};
+
struct amdgpu_xgmi {
/* from psp */
u64 node_id;
@@ -149,6 +157,9 @@ struct amdgpu_xgmi {
struct list_head head;
bool supported;
struct ras_common_if *ras_if;
+ bool connected_to_cpu;
+ bool pending_reset;
+ const struct amdgpu_xgmi_ras_funcs *ras_funcs;
};
struct amdgpu_gmc {
@@ -189,10 +200,13 @@ struct amdgpu_gmc {
u64 gart_end;
/* Frame buffer aperture of this GPU device. Different from
* fb_start (see below), this only covers the local GPU device.
- * Driver get fb_start from MC_VM_FB_LOCATION_BASE (set by vbios)
- * and calculate vram_start of this local device by adding an
- * offset inside the XGMI hive.
- * Under VMID0, logical address == MC address
+ * If driver uses FB aperture to access FB, driver get fb_start from
+ * MC_VM_FB_LOCATION_BASE (set by vbios) and calculate vram_start
+ * of this local device by adding an offset inside the XGMI hive.
+ * If driver uses GART table for VMID0 FB access, driver finds a hole in
+ * VMID0's virtual address space to place the SYSVM aperture inside
+ * which the first part is vram and the second part is gart (covering
+ * system ram).
*/
u64 vram_start;
u64 vram_end;
@@ -240,6 +254,12 @@ struct amdgpu_gmc {
struct amdgpu_xgmi xgmi;
struct amdgpu_irq_src ecc_irq;
int noretry;
+
+ uint32_t vmid0_page_table_block_size;
+ uint32_t vmid0_page_table_depth;
+ struct amdgpu_bo *pdb0_bo;
+ /* CPU kmapped address of pdb0*/
+ void *ptr_pdb0;
};
#define amdgpu_gmc_flush_gpu_tlb(adev, vmid, vmhub, type) ((adev)->gmc.gmc_funcs->flush_gpu_tlb((adev), (vmid), (vmhub), (type)))
@@ -281,6 +301,7 @@ static inline uint64_t amdgpu_gmc_sign_extend(uint64_t addr)
return addr;
}
+int amdgpu_gmc_pdb0_alloc(struct amdgpu_device *adev);
void amdgpu_gmc_get_pde_for_bo(struct amdgpu_bo *bo, int level,
uint64_t *addr, uint64_t *flags);
int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr,
@@ -288,6 +309,7 @@ int amdgpu_gmc_set_pte_pde(struct amdgpu_device *adev, void *cpu_pt_addr,
uint64_t flags);
uint64_t amdgpu_gmc_pd_addr(struct amdgpu_bo *bo);
uint64_t amdgpu_gmc_agp_addr(struct ttm_buffer_object *bo);
+void amdgpu_gmc_sysvm_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc);
void amdgpu_gmc_vram_location(struct amdgpu_device *adev, struct amdgpu_gmc *mc,
u64 base);
void amdgpu_gmc_gart_location(struct amdgpu_device *adev,
@@ -309,4 +331,8 @@ amdgpu_gmc_set_vm_fault_masks(struct amdgpu_device *adev, int hub_type,
void amdgpu_gmc_get_vbios_allocations(struct amdgpu_device *adev);
+void amdgpu_gmc_init_pdb0(struct amdgpu_device *adev);
+uint64_t amdgpu_gmc_vram_mc2pa(struct amdgpu_device *adev, uint64_t mc_addr);
+uint64_t amdgpu_gmc_vram_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo);
+uint64_t amdgpu_gmc_vram_cpu_pa(struct amdgpu_device *adev, struct amdgpu_bo *bo);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
index 8980329cded0..540c01052b21 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_gtt_mgr.c
@@ -49,8 +49,7 @@ static ssize_t amdgpu_mem_info_gtt_total_show(struct device *dev,
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- man->size * PAGE_SIZE);
+ return sysfs_emit(buf, "%llu\n", man->size * PAGE_SIZE);
}
/**
@@ -68,8 +67,7 @@ static ssize_t amdgpu_mem_info_gtt_used_show(struct device *dev,
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_TT);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- amdgpu_gtt_mgr_usage(man));
+ return sysfs_emit(buf, "%llu\n", amdgpu_gtt_mgr_usage(man));
}
static DEVICE_ATTR(mem_info_gtt_total, S_IRUGO,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
index 7645223ea0ef..a2fe2dac32c1 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ib.c
@@ -30,7 +30,6 @@
#include <linux/slab.h>
#include <drm/amdgpu_drm.h>
-#include <drm/drm_debugfs.h>
#include "amdgpu.h"
#include "atom.h"
@@ -77,6 +76,8 @@ int amdgpu_ib_get(struct amdgpu_device *adev, struct amdgpu_vm *vm,
}
ib->ptr = amdgpu_sa_bo_cpu_addr(ib->sa_bo);
+ /* flush the cache before commit the IB */
+ ib->flags = AMDGPU_IB_FLAG_EMIT_MEM_SYNC;
if (!vm)
ib->gpu_addr = amdgpu_sa_bo_gpu_addr(ib->sa_bo);
@@ -453,11 +454,9 @@ int amdgpu_ib_ring_tests(struct amdgpu_device *adev)
*/
#if defined(CONFIG_DEBUG_FS)
-static int amdgpu_debugfs_sa_info(struct seq_file *m, void *data)
+static int amdgpu_debugfs_sa_info_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
seq_printf(m, "--------------------- DELAYED --------------------- \n");
amdgpu_sa_bo_dump_debug_info(&adev->ib_pools[AMDGPU_IB_POOL_DELAYED],
@@ -471,18 +470,18 @@ static int amdgpu_debugfs_sa_info(struct seq_file *m, void *data)
return 0;
}
-static const struct drm_info_list amdgpu_debugfs_sa_list[] = {
- {"amdgpu_sa_info", &amdgpu_debugfs_sa_info, 0, NULL},
-};
+DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_sa_info);
#endif
-int amdgpu_debugfs_sa_init(struct amdgpu_device *adev)
+void amdgpu_debugfs_sa_init(struct amdgpu_device *adev)
{
#if defined(CONFIG_DEBUG_FS)
- return amdgpu_debugfs_add_files(adev, amdgpu_debugfs_sa_list,
- ARRAY_SIZE(amdgpu_debugfs_sa_list));
-#else
- return 0;
+ struct drm_minor *minor = adev_to_drm(adev)->primary;
+ struct dentry *root = minor->debugfs_root;
+
+ debugfs_create_file("amdgpu_sa_info", 0444, root, adev,
+ &amdgpu_debugfs_sa_info_fops);
+
#endif
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
index 94b069630db3..b4971e90b98c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ids.c
@@ -215,7 +215,11 @@ static int amdgpu_vmid_grab_idle(struct amdgpu_vm *vm,
/* Check if we have an idle VMID */
i = 0;
list_for_each_entry((*idle), &id_mgr->ids_lru, list) {
- fences[i] = amdgpu_sync_peek_fence(&(*idle)->active, ring);
+ /* Don't use per engine and per process VMID at the same time */
+ struct amdgpu_ring *r = adev->vm_manager.concurrent_flush ?
+ NULL : ring;
+
+ fences[i] = amdgpu_sync_peek_fence(&(*idle)->active, r);
if (!fences[i])
break;
++i;
@@ -281,7 +285,7 @@ static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm,
if (updates && (*id)->flushed_updates &&
updates->context == (*id)->flushed_updates->context &&
!dma_fence_is_later(updates, (*id)->flushed_updates))
- updates = NULL;
+ updates = NULL;
if ((*id)->owner != vm->immediate.fence_context ||
job->vm_pd_addr != (*id)->pd_gpu_addr ||
@@ -290,6 +294,10 @@ static int amdgpu_vmid_grab_reserved(struct amdgpu_vm *vm,
!dma_fence_is_signaled((*id)->last_flush))) {
struct dma_fence *tmp;
+ /* Don't use per engine and per process VMID at the same time */
+ if (adev->vm_manager.concurrent_flush)
+ ring = NULL;
+
/* to prevent one context starved by another context */
(*id)->pd_gpu_addr = 0;
tmp = amdgpu_sync_peek_fence(&(*id)->active, ring);
@@ -365,12 +373,7 @@ static int amdgpu_vmid_grab_used(struct amdgpu_vm *vm,
if (updates && (!flushed || dma_fence_is_later(updates, flushed)))
needs_flush = true;
- /* Concurrent flushes are only possible starting with Vega10 and
- * are broken on Navi10 and Navi14.
- */
- if (needs_flush && (adev->asic_type < CHIP_VEGA10 ||
- adev->asic_type == CHIP_NAVI10 ||
- adev->asic_type == CHIP_NAVI14))
+ if (needs_flush && !adev->vm_manager.concurrent_flush)
continue;
/* Good, we can use this VMID. Remember this submission as
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c
index dc852af4f3b7..faaa6aa2faaf 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.c
@@ -99,6 +99,8 @@ int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih,
ih->rptr_addr = adev->wb.gpu_addr + rptr_offs * 4;
ih->rptr_cpu = &adev->wb.wb[rptr_offs];
}
+
+ init_waitqueue_head(&ih->wait_process);
return 0;
}
@@ -160,6 +162,52 @@ void amdgpu_ih_ring_write(struct amdgpu_ih_ring *ih, const uint32_t *iv,
}
}
+/* Waiter helper that checks current rptr matches or passes checkpoint wptr */
+static bool amdgpu_ih_has_checkpoint_processed(struct amdgpu_device *adev,
+ struct amdgpu_ih_ring *ih,
+ uint32_t checkpoint_wptr,
+ uint32_t *prev_rptr)
+{
+ uint32_t cur_rptr = ih->rptr | (*prev_rptr & ~ih->ptr_mask);
+
+ /* rptr has wrapped. */
+ if (cur_rptr < *prev_rptr)
+ cur_rptr += ih->ptr_mask + 1;
+ *prev_rptr = cur_rptr;
+
+ return cur_rptr >= checkpoint_wptr;
+}
+
+/**
+ * amdgpu_ih_wait_on_checkpoint_process - wait to process IVs up to checkpoint
+ *
+ * @adev: amdgpu_device pointer
+ * @ih: ih ring to process
+ *
+ * Used to ensure ring has processed IVs up to the checkpoint write pointer.
+ */
+int amdgpu_ih_wait_on_checkpoint_process(struct amdgpu_device *adev,
+ struct amdgpu_ih_ring *ih)
+{
+ uint32_t checkpoint_wptr, rptr;
+
+ if (!ih->enabled || adev->shutdown)
+ return -ENODEV;
+
+ checkpoint_wptr = amdgpu_ih_get_wptr(adev, ih);
+ /* Order wptr with rptr. */
+ rmb();
+ rptr = READ_ONCE(ih->rptr);
+
+ /* wptr has wrapped. */
+ if (rptr > checkpoint_wptr)
+ checkpoint_wptr += ih->ptr_mask + 1;
+
+ return wait_event_interruptible(ih->wait_process,
+ amdgpu_ih_has_checkpoint_processed(adev, ih,
+ checkpoint_wptr, &rptr));
+}
+
/**
* amdgpu_ih_process - interrupt handler
*
@@ -180,10 +228,6 @@ int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih)
wptr = amdgpu_ih_get_wptr(adev, ih);
restart_ih:
- /* is somebody else already processing irqs? */
- if (atomic_xchg(&ih->lock, 1))
- return IRQ_NONE;
-
DRM_DEBUG("%s: rptr %d, wptr %d\n", __func__, ih->rptr, wptr);
/* Order reading of wptr vs. reading of IH ring data */
@@ -195,7 +239,7 @@ restart_ih:
}
amdgpu_ih_set_rptr(adev, ih);
- atomic_set(&ih->lock, 0);
+ wake_up_all(&ih->wait_process);
/* make sure wptr hasn't changed while processing */
wptr = amdgpu_ih_get_wptr(adev, ih);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
index 6ed4a85fc7c3..0649b59830a5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ih.h
@@ -64,8 +64,10 @@ struct amdgpu_ih_ring {
bool enabled;
unsigned rptr;
- atomic_t lock;
struct amdgpu_ih_regs ih_regs;
+
+ /* For waiting on IH processing at checkpoint. */
+ wait_queue_head_t wait_process;
};
/* provided by the ih block */
@@ -87,6 +89,8 @@ int amdgpu_ih_ring_init(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih,
void amdgpu_ih_ring_fini(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih);
void amdgpu_ih_ring_write(struct amdgpu_ih_ring *ih, const uint32_t *iv,
unsigned int num_dw);
+int amdgpu_ih_wait_on_checkpoint_process(struct amdgpu_device *adev,
+ struct amdgpu_ih_ring *ih);
int amdgpu_ih_process(struct amdgpu_device *adev, struct amdgpu_ih_ring *ih);
void amdgpu_ih_decode_iv_helper(struct amdgpu_device *adev,
struct amdgpu_ih_ring *ih,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
index afbbec82a289..90f50561b43a 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c
@@ -65,6 +65,41 @@
#define AMDGPU_WAIT_IDLE_TIMEOUT 200
+const char *soc15_ih_clientid_name[] = {
+ "IH",
+ "SDMA2 or ACP",
+ "ATHUB",
+ "BIF",
+ "SDMA3 or DCE",
+ "SDMA4 or ISP",
+ "VMC1 or PCIE0",
+ "RLC",
+ "SDMA0",
+ "SDMA1",
+ "SE0SH",
+ "SE1SH",
+ "SE2SH",
+ "SE3SH",
+ "VCN1 or UVD1",
+ "THM",
+ "VCN or UVD",
+ "SDMA5 or VCE0",
+ "VMC",
+ "SDMA6 or XDMA",
+ "GRBM_CP",
+ "ATS",
+ "ROM_SMUIO",
+ "DF",
+ "SDMA7 or VCE1",
+ "PWR",
+ "reserved",
+ "UTCL2",
+ "EA",
+ "UTCL2LOG",
+ "MP0",
+ "MP1"
+};
+
/**
* amdgpu_hotplug_work_func - work handler for display hotplug event
*
@@ -164,13 +199,13 @@ irqreturn_t amdgpu_irq_handler(int irq, void *arg)
* ack the interrupt if it is there
*/
if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__PCIE_BIF)) {
- if (adev->nbio.funcs &&
- adev->nbio.funcs->handle_ras_controller_intr_no_bifring)
- adev->nbio.funcs->handle_ras_controller_intr_no_bifring(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring)
+ adev->nbio.ras_funcs->handle_ras_controller_intr_no_bifring(adev);
- if (adev->nbio.funcs &&
- adev->nbio.funcs->handle_ras_err_event_athub_intr_no_bifring)
- adev->nbio.funcs->handle_ras_err_event_athub_intr_no_bifring(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring)
+ adev->nbio.ras_funcs->handle_ras_err_event_athub_intr_no_bifring(adev);
}
return ret;
@@ -347,11 +382,6 @@ void amdgpu_irq_fini(struct amdgpu_device *adev)
kfree(src->enabled_types);
src->enabled_types = NULL;
- if (src->data) {
- kfree(src->data);
- kfree(src);
- adev->irq.client[i].sources[j] = NULL;
- }
}
kfree(adev->irq.client[i].sources);
adev->irq.client[i].sources = NULL;
@@ -535,7 +565,7 @@ void amdgpu_irq_gpu_reset_resume_helper(struct amdgpu_device *adev)
for (j = 0; j < AMDGPU_MAX_IRQ_SRC_ID; ++j) {
struct amdgpu_irq_src *src = adev->irq.client[i].sources[j];
- if (!src)
+ if (!src || !src->funcs || !src->funcs->set)
continue;
for (k = 0; k < src->num_types; k++)
amdgpu_irq_update(adev, src, k);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
index ac527e5deae6..cf6116648322 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h
@@ -62,7 +62,6 @@ struct amdgpu_irq_src {
unsigned num_types;
atomic_t *enabled_types;
const struct amdgpu_irq_src_funcs *funcs;
- void *data;
};
struct amdgpu_irq_client {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
index ff48101bab55..759b34799221 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_job.c
@@ -28,7 +28,7 @@
#include "amdgpu.h"
#include "amdgpu_trace.h"
-static void amdgpu_job_timedout(struct drm_sched_job *s_job)
+static enum drm_gpu_sched_stat amdgpu_job_timedout(struct drm_sched_job *s_job)
{
struct amdgpu_ring *ring = to_amdgpu_ring(s_job->sched);
struct amdgpu_job *job = to_amdgpu_job(s_job);
@@ -41,7 +41,7 @@ static void amdgpu_job_timedout(struct drm_sched_job *s_job)
amdgpu_ring_soft_recovery(ring, job->vmid, s_job->s_fence->parent)) {
DRM_ERROR("ring %s timeout, but soft recovered\n",
s_job->sched->name);
- return;
+ return DRM_GPU_SCHED_STAT_NOMINAL;
}
amdgpu_vm_get_task_info(ring->adev, job->pasid, &ti);
@@ -53,10 +53,12 @@ static void amdgpu_job_timedout(struct drm_sched_job *s_job)
if (amdgpu_device_should_recover_gpu(ring->adev)) {
amdgpu_device_gpu_recover(ring->adev, job);
+ return DRM_GPU_SCHED_STAT_NOMINAL;
} else {
drm_sched_suspend_timeout(&ring->sched);
if (amdgpu_sriov_vf(adev))
adev->virt.tdr_debug = true;
+ return DRM_GPU_SCHED_STAT_NOMINAL;
}
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index 64beb3399604..39ee88d29cca 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -27,7 +27,6 @@
*/
#include "amdgpu.h"
-#include <drm/drm_debugfs.h>
#include <drm/amdgpu_drm.h>
#include "amdgpu_uvd.h"
#include "amdgpu_vce.h"
@@ -160,7 +159,7 @@ int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags)
goto out;
}
- if (amdgpu_device_supports_atpx(dev) &&
+ if (amdgpu_device_supports_px(dev) &&
(amdgpu_runtime_pm != 0)) { /* enable runpm by default for atpx */
adev->runpm = true;
dev_info(adev->dev, "Using ATPX for runtime pm\n");
@@ -201,9 +200,13 @@ int amdgpu_driver_load_kms(struct amdgpu_device *adev, unsigned long flags)
if (adev->runpm) {
/* only need to skip on ATPX */
- if (amdgpu_device_supports_atpx(dev) &&
- !amdgpu_is_atpx_hybrid())
+ if (amdgpu_device_supports_px(dev))
dev_pm_set_driver_flags(dev->dev, DPM_FLAG_NO_DIRECT_COMPLETE);
+ /* we want direct complete for BOCO */
+ if (amdgpu_device_supports_boco(dev))
+ dev_pm_set_driver_flags(dev->dev, DPM_FLAG_SMART_PREPARE |
+ DPM_FLAG_SMART_SUSPEND |
+ DPM_FLAG_MAY_SKIP_RESUME);
pm_runtime_use_autosuspend(dev->dev);
pm_runtime_set_autosuspend_delay(dev->dev, 5000);
pm_runtime_allow(dev->dev);
@@ -287,22 +290,30 @@ static int amdgpu_firmware_info(struct drm_amdgpu_info_firmware *fw_info,
break;
case AMDGPU_INFO_FW_TA:
switch (query_fw->index) {
- case 0:
+ case TA_FW_TYPE_PSP_XGMI:
fw_info->ver = adev->psp.ta_fw_version;
fw_info->feature = adev->psp.ta_xgmi_ucode_version;
break;
- case 1:
+ case TA_FW_TYPE_PSP_RAS:
fw_info->ver = adev->psp.ta_fw_version;
fw_info->feature = adev->psp.ta_ras_ucode_version;
break;
- case 2:
+ case TA_FW_TYPE_PSP_HDCP:
fw_info->ver = adev->psp.ta_fw_version;
fw_info->feature = adev->psp.ta_hdcp_ucode_version;
break;
- case 3:
+ case TA_FW_TYPE_PSP_DTM:
fw_info->ver = adev->psp.ta_fw_version;
fw_info->feature = adev->psp.ta_dtm_ucode_version;
break;
+ case TA_FW_TYPE_PSP_RAP:
+ fw_info->ver = adev->psp.ta_fw_version;
+ fw_info->feature = adev->psp.ta_rap_ucode_version;
+ break;
+ case TA_FW_TYPE_PSP_SECUREDISPLAY:
+ fw_info->ver = adev->psp.ta_fw_version;
+ fw_info->feature = adev->psp.ta_securedisplay_ucode_version;
+ break;
default:
return -EINVAL;
}
@@ -778,9 +789,9 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
dev_info->high_va_offset = AMDGPU_GMC_HOLE_END;
dev_info->high_va_max = AMDGPU_GMC_HOLE_END | vm_size;
}
- dev_info->virtual_address_alignment = max((int)PAGE_SIZE, AMDGPU_GPU_PAGE_SIZE);
+ dev_info->virtual_address_alignment = max_t(u32, PAGE_SIZE, AMDGPU_GPU_PAGE_SIZE);
dev_info->pte_fragment_size = (1 << adev->vm_manager.fragment_size) * AMDGPU_GPU_PAGE_SIZE;
- dev_info->gart_page_size = AMDGPU_GPU_PAGE_SIZE;
+ dev_info->gart_page_size = max_t(u32, PAGE_SIZE, AMDGPU_GPU_PAGE_SIZE);
dev_info->cu_active_number = adev->gfx.cu_info.number;
dev_info->cu_ao_mask = adev->gfx.cu_info.ao_cu_mask;
dev_info->ce_ram_size = adev->gfx.ce_ram_size;
@@ -981,6 +992,63 @@ int amdgpu_info_ioctl(struct drm_device *dev, void *data, struct drm_file *filp)
min_t(u64, size, sizeof(ras_mask))) ?
-EFAULT : 0;
}
+ case AMDGPU_INFO_VIDEO_CAPS: {
+ const struct amdgpu_video_codecs *codecs;
+ struct drm_amdgpu_info_video_caps *caps;
+ int r;
+
+ switch (info->video_cap.type) {
+ case AMDGPU_INFO_VIDEO_CAPS_DECODE:
+ r = amdgpu_asic_query_video_codecs(adev, false, &codecs);
+ if (r)
+ return -EINVAL;
+ break;
+ case AMDGPU_INFO_VIDEO_CAPS_ENCODE:
+ r = amdgpu_asic_query_video_codecs(adev, true, &codecs);
+ if (r)
+ return -EINVAL;
+ break;
+ default:
+ DRM_DEBUG_KMS("Invalid request %d\n",
+ info->video_cap.type);
+ return -EINVAL;
+ }
+
+ caps = kzalloc(sizeof(*caps), GFP_KERNEL);
+ if (!caps)
+ return -ENOMEM;
+
+ for (i = 0; i < codecs->codec_count; i++) {
+ int idx = codecs->codec_array[i].codec_type;
+
+ switch (idx) {
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2:
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4:
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1:
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC:
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC:
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG:
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9:
+ case AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_AV1:
+ caps->codec_info[idx].valid = 1;
+ caps->codec_info[idx].max_width =
+ codecs->codec_array[i].max_width;
+ caps->codec_info[idx].max_height =
+ codecs->codec_array[i].max_height;
+ caps->codec_info[idx].max_pixels_per_frame =
+ codecs->codec_array[i].max_pixels_per_frame;
+ caps->codec_info[idx].max_level =
+ codecs->codec_array[i].max_level;
+ break;
+ default:
+ break;
+ }
+ }
+ r = copy_to_user(out, caps,
+ min((size_t)size, sizeof(*caps))) ? -EFAULT : 0;
+ kfree(caps);
+ return r;
+ }
default:
DRM_DEBUG_KMS("Invalid request %d\n", info->query);
return -EINVAL;
@@ -1262,16 +1330,25 @@ void amdgpu_disable_vblank_kms(struct drm_crtc *crtc)
*/
#if defined(CONFIG_DEBUG_FS)
-static int amdgpu_debugfs_firmware_info(struct seq_file *m, void *data)
+static int amdgpu_debugfs_firmware_info_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *) m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
struct drm_amdgpu_info_firmware fw_info;
struct drm_amdgpu_query_fw query_fw;
struct atom_context *ctx = adev->mode_info.atom_context;
int ret, i;
+ static const char *ta_fw_name[TA_FW_TYPE_MAX_INDEX] = {
+#define TA_FW_NAME(type) [TA_FW_TYPE_PSP_##type] = #type
+ TA_FW_NAME(XGMI),
+ TA_FW_NAME(RAS),
+ TA_FW_NAME(HDCP),
+ TA_FW_NAME(DTM),
+ TA_FW_NAME(RAP),
+ TA_FW_NAME(SECUREDISPLAY),
+#undef TA_FW_NAME
+ };
+
/* VCE */
query_fw.fw_type = AMDGPU_INFO_FW_VCE;
ret = amdgpu_firmware_info(&fw_info, &query_fw, adev);
@@ -1389,31 +1466,14 @@ static int amdgpu_debugfs_firmware_info(struct seq_file *m, void *data)
fw_info.feature, fw_info.ver);
query_fw.fw_type = AMDGPU_INFO_FW_TA;
- for (i = 0; i < 4; i++) {
+ for (i = TA_FW_TYPE_PSP_XGMI; i < TA_FW_TYPE_MAX_INDEX; i++) {
query_fw.index = i;
ret = amdgpu_firmware_info(&fw_info, &query_fw, adev);
if (ret)
continue;
- switch (query_fw.index) {
- case 0:
- seq_printf(m, "TA %s feature version: 0x%08x, firmware version: 0x%08x\n",
- "RAS", fw_info.feature, fw_info.ver);
- break;
- case 1:
- seq_printf(m, "TA %s feature version: 0x%08x, firmware version: 0x%08x\n",
- "XGMI", fw_info.feature, fw_info.ver);
- break;
- case 2:
- seq_printf(m, "TA %s feature version: 0x%08x, firmware version: 0x%08x\n",
- "HDCP", fw_info.feature, fw_info.ver);
- break;
- case 3:
- seq_printf(m, "TA %s feature version: 0x%08x, firmware version: 0x%08x\n",
- "DTM", fw_info.feature, fw_info.ver);
- break;
- default:
- return -EINVAL;
- }
+
+ seq_printf(m, "TA %s feature version: 0x%08x, firmware version: 0x%08x\n",
+ ta_fw_name[i], fw_info.feature, fw_info.ver);
}
/* SMC */
@@ -1472,17 +1532,18 @@ static int amdgpu_debugfs_firmware_info(struct seq_file *m, void *data)
return 0;
}
-static const struct drm_info_list amdgpu_firmware_info_list[] = {
- {"amdgpu_firmware_info", amdgpu_debugfs_firmware_info, 0, NULL},
-};
+DEFINE_SHOW_ATTRIBUTE(amdgpu_debugfs_firmware_info);
+
#endif
-int amdgpu_debugfs_firmware_init(struct amdgpu_device *adev)
+void amdgpu_debugfs_firmware_init(struct amdgpu_device *adev)
{
#if defined(CONFIG_DEBUG_FS)
- return amdgpu_debugfs_add_files(adev, amdgpu_firmware_info_list,
- ARRAY_SIZE(amdgpu_firmware_info_list));
-#else
- return 0;
+ struct drm_minor *minor = adev_to_drm(adev)->primary;
+ struct dentry *root = minor->debugfs_root;
+
+ debugfs_create_file("amdgpu_firmware_info", 0444, root,
+ adev, &amdgpu_debugfs_firmware_info_fops);
+
#endif
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.h
index 1ae9bdae7311..11aa29933c1f 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mmhub.h
@@ -21,12 +21,16 @@
#ifndef __AMDGPU_MMHUB_H__
#define __AMDGPU_MMHUB_H__
-struct amdgpu_mmhub_funcs {
- void (*ras_init)(struct amdgpu_device *adev);
+struct amdgpu_mmhub_ras_funcs {
int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
void (*query_ras_error_count)(struct amdgpu_device *adev,
- void *ras_error_status);
+ void *ras_error_status);
+ void (*query_ras_error_status)(struct amdgpu_device *adev);
void (*reset_ras_error_count)(struct amdgpu_device *adev);
+};
+
+struct amdgpu_mmhub_funcs {
u64 (*get_fb_location)(struct amdgpu_device *adev);
void (*init)(struct amdgpu_device *adev);
int (*gart_enable)(struct amdgpu_device *adev);
@@ -40,12 +44,12 @@ struct amdgpu_mmhub_funcs {
uint64_t page_table_base);
void (*update_power_gating)(struct amdgpu_device *adev,
bool enable);
- void (*query_ras_error_status)(struct amdgpu_device *adev);
};
struct amdgpu_mmhub {
struct ras_common_if *ras_if;
const struct amdgpu_mmhub_funcs *funcs;
+ const struct amdgpu_mmhub_ras_funcs *ras_funcs;
};
int amdgpu_mmhub_ras_late_init(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
index 319cb19e1b99..cb0b581bbce7 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_mode.h
@@ -602,6 +602,14 @@ int amdgpu_display_get_crtc_scanoutpos(struct drm_device *dev,
int *hpos, ktime_t *stime, ktime_t *etime,
const struct drm_display_mode *mode);
+int amdgpu_display_gem_fb_init(struct drm_device *dev,
+ struct amdgpu_framebuffer *rfb,
+ const struct drm_mode_fb_cmd2 *mode_cmd,
+ struct drm_gem_object *obj);
+int amdgpu_display_gem_fb_verify_and_init(
+ struct drm_device *dev, struct amdgpu_framebuffer *rfb,
+ struct drm_file *file_priv, const struct drm_mode_fb_cmd2 *mode_cmd,
+ struct drm_gem_object *obj);
int amdgpu_display_framebuffer_init(struct drm_device *dev,
struct amdgpu_framebuffer *rfb,
const struct drm_mode_fb_cmd2 *mode_cmd,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.h
index 7c11bce4514b..25ee53545837 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_nbio.h
@@ -47,6 +47,17 @@ struct nbio_hdp_flush_reg {
u32 ref_and_mask_sdma7;
};
+struct amdgpu_nbio_ras_funcs {
+ void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev);
+ void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev);
+ int (*init_ras_controller_interrupt)(struct amdgpu_device *adev);
+ int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev);
+ void (*query_ras_error_count)(struct amdgpu_device *adev,
+ void *ras_error_status);
+ int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
+};
+
struct amdgpu_nbio_funcs {
const struct nbio_hdp_flush_reg *hdp_flush_reg;
u32 (*get_hdp_flush_req_offset)(struct amdgpu_device *adev);
@@ -79,13 +90,6 @@ struct amdgpu_nbio_funcs {
void (*ih_control)(struct amdgpu_device *adev);
void (*init_registers)(struct amdgpu_device *adev);
void (*remap_hdp_registers)(struct amdgpu_device *adev);
- void (*handle_ras_controller_intr_no_bifring)(struct amdgpu_device *adev);
- void (*handle_ras_err_event_athub_intr_no_bifring)(struct amdgpu_device *adev);
- int (*init_ras_controller_interrupt)(struct amdgpu_device *adev);
- int (*init_ras_err_event_athub_interrupt)(struct amdgpu_device *adev);
- void (*query_ras_error_count)(struct amdgpu_device *adev,
- void *ras_error_status);
- int (*ras_late_init)(struct amdgpu_device *adev);
void (*enable_aspm)(struct amdgpu_device *adev,
bool enable);
void (*program_aspm)(struct amdgpu_device *adev);
@@ -97,6 +101,7 @@ struct amdgpu_nbio {
struct amdgpu_irq_src ras_err_event_athub_irq;
struct ras_common_if *ras_if;
const struct amdgpu_nbio_funcs *funcs;
+ const struct amdgpu_nbio_ras_funcs *ras_funcs;
};
int amdgpu_nbio_ras_late_init(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 4b29b8205442..1345f7eba011 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -77,6 +77,7 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(tbo->bdev);
struct amdgpu_bo *bo = ttm_to_amdgpu_bo(tbo);
+ struct amdgpu_bo_user *ubo;
if (bo->tbo.pin_count > 0)
amdgpu_bo_subtract_pin_size(bo);
@@ -94,7 +95,11 @@ static void amdgpu_bo_destroy(struct ttm_buffer_object *tbo)
}
amdgpu_bo_unref(&bo->parent);
- kfree(bo->metadata);
+ if (bo->tbo.type == ttm_bo_type_device) {
+ ubo = to_amdgpu_bo_user(bo);
+ kfree(ubo->metadata);
+ }
+
kfree(bo);
}
@@ -248,6 +253,7 @@ int amdgpu_bo_create_reserved(struct amdgpu_device *adev,
bp.flags |= AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
if (!*bo_ptr) {
r = amdgpu_bo_create(adev, &bp, bo_ptr);
@@ -523,7 +529,6 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
};
struct amdgpu_bo *bo;
unsigned long page_align, size = bp->size;
- size_t acc_size;
int r;
/* Note that GDS/GWS/OA allocates 1 page per byte/resource. */
@@ -544,12 +549,10 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
if (!amdgpu_bo_validate_size(adev, size, bp->domain))
return -ENOMEM;
- *bo_ptr = NULL;
+ BUG_ON(bp->bo_ptr_size < sizeof(struct amdgpu_bo));
- acc_size = ttm_bo_dma_acc_size(&adev->mman.bdev, size,
- sizeof(struct amdgpu_bo));
-
- bo = kzalloc(sizeof(struct amdgpu_bo), GFP_KERNEL);
+ *bo_ptr = NULL;
+ bo = kzalloc(bp->bo_ptr_size, GFP_KERNEL);
if (bo == NULL)
return -ENOMEM;
drm_gem_private_object_init(adev_to_drm(adev), &bo->tbo.base, size);
@@ -577,8 +580,8 @@ static int amdgpu_bo_do_create(struct amdgpu_device *adev,
bo->tbo.priority = 1;
r = ttm_bo_init_reserved(&adev->mman.bdev, &bo->tbo, size, bp->type,
- &bo->placement, page_align, &ctx, acc_size,
- NULL, bp->resv, &amdgpu_bo_destroy);
+ &bo->placement, page_align, &ctx, NULL,
+ bp->resv, &amdgpu_bo_destroy);
if (unlikely(r != 0))
return r;
@@ -639,6 +642,7 @@ static int amdgpu_bo_create_shadow(struct amdgpu_device *adev,
AMDGPU_GEM_CREATE_SHADOW;
bp.type = ttm_bo_type_kernel;
bp.resv = bo->tbo.base.resv;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
r = amdgpu_bo_do_create(adev, &bp, &bo->shadow);
if (!r) {
@@ -673,6 +677,7 @@ int amdgpu_bo_create(struct amdgpu_device *adev,
int r;
bp->flags = bp->flags & ~AMDGPU_GEM_CREATE_SHADOW;
+
r = amdgpu_bo_do_create(adev, bp, bo_ptr);
if (r)
return r;
@@ -695,6 +700,34 @@ int amdgpu_bo_create(struct amdgpu_device *adev,
}
/**
+ * amdgpu_bo_create_user - create an &amdgpu_bo_user buffer object
+ * @adev: amdgpu device object
+ * @bp: parameters to be used for the buffer object
+ * @ubo_ptr: pointer to the buffer object pointer
+ *
+ * Create a BO to be used by user application;
+ *
+ * Returns:
+ * 0 for success or a negative error code on failure.
+ */
+
+int amdgpu_bo_create_user(struct amdgpu_device *adev,
+ struct amdgpu_bo_param *bp,
+ struct amdgpu_bo_user **ubo_ptr)
+{
+ struct amdgpu_bo *bo_ptr;
+ int r;
+
+ bp->flags = bp->flags & ~AMDGPU_GEM_CREATE_SHADOW;
+ bp->bo_ptr_size = sizeof(struct amdgpu_bo_user);
+ r = amdgpu_bo_do_create(adev, bp, &bo_ptr);
+ if (r)
+ return r;
+
+ *ubo_ptr = to_amdgpu_bo_user(bo_ptr);
+ return r;
+}
+/**
* amdgpu_bo_validate - validate an &amdgpu_bo buffer object
* @bo: pointer to the buffer object
*
@@ -1028,13 +1061,10 @@ int amdgpu_bo_evict_vram(struct amdgpu_device *adev)
{
struct ttm_resource_manager *man;
- /* late 2.6.33 fix IGP hibernate - we need pm ops to do this correct */
-#ifndef CONFIG_HIBERNATION
- if (adev->flags & AMD_IS_APU) {
- /* Useless to evict on IGP chips */
+ if (adev->in_s3 && (adev->flags & AMD_IS_APU)) {
+ /* No need to evict vram on APUs for suspend to ram */
return 0;
}
-#endif
man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
return ttm_resource_manager_evict_all(&adev->mman.bdev, man);
@@ -1065,13 +1095,17 @@ static const char *amdgpu_vram_names[] = {
*/
int amdgpu_bo_init(struct amdgpu_device *adev)
{
- /* reserve PAT memory space to WC for VRAM */
- arch_io_reserve_memtype_wc(adev->gmc.aper_base,
- adev->gmc.aper_size);
+ /* On A+A platform, VRAM can be mapped as WB */
+ if (!adev->gmc.xgmi.connected_to_cpu) {
+ /* reserve PAT memory space to WC for VRAM */
+ arch_io_reserve_memtype_wc(adev->gmc.aper_base,
+ adev->gmc.aper_size);
+
+ /* Add an MTRR for the VRAM */
+ adev->gmc.vram_mtrr = arch_phys_wc_add(adev->gmc.aper_base,
+ adev->gmc.aper_size);
+ }
- /* Add an MTRR for the VRAM */
- adev->gmc.vram_mtrr = arch_phys_wc_add(adev->gmc.aper_base,
- adev->gmc.aper_size);
DRM_INFO("Detected VRAM RAM=%lluM, BAR=%lluM\n",
adev->gmc.mc_vram_size >> 20,
(unsigned long long)adev->gmc.aper_size >> 20);
@@ -1089,27 +1123,10 @@ int amdgpu_bo_init(struct amdgpu_device *adev)
void amdgpu_bo_fini(struct amdgpu_device *adev)
{
amdgpu_ttm_fini(adev);
- arch_phys_wc_del(adev->gmc.vram_mtrr);
- arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size);
-}
-
-/**
- * amdgpu_bo_fbdev_mmap - mmap fbdev memory
- * @bo: &amdgpu_bo buffer object
- * @vma: vma as input from the fbdev mmap method
- *
- * Calls ttm_fbdev_mmap() to mmap fbdev memory if it is backed by a bo.
- *
- * Returns:
- * 0 for success or a negative error code on failure.
- */
-int amdgpu_bo_fbdev_mmap(struct amdgpu_bo *bo,
- struct vm_area_struct *vma)
-{
- if (vma->vm_pgoff != 0)
- return -EACCES;
-
- return ttm_bo_mmap_obj(vma, &bo->tbo);
+ if (!adev->gmc.xgmi.connected_to_cpu) {
+ arch_phys_wc_del(adev->gmc.vram_mtrr);
+ arch_io_free_memtype_wc(adev->gmc.aper_base, adev->gmc.aper_size);
+ }
}
/**
@@ -1126,12 +1143,15 @@ int amdgpu_bo_fbdev_mmap(struct amdgpu_bo *bo,
int amdgpu_bo_set_tiling_flags(struct amdgpu_bo *bo, u64 tiling_flags)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+ struct amdgpu_bo_user *ubo;
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
if (adev->family <= AMDGPU_FAMILY_CZ &&
AMDGPU_TILING_GET(tiling_flags, TILE_SPLIT) > 6)
return -EINVAL;
- bo->tiling_flags = tiling_flags;
+ ubo = to_amdgpu_bo_user(bo);
+ ubo->tiling_flags = tiling_flags;
return 0;
}
@@ -1145,10 +1165,14 @@ int amdgpu_bo_set_tiling_flags(struct amdgpu_bo *bo, u64 tiling_flags)
*/
void amdgpu_bo_get_tiling_flags(struct amdgpu_bo *bo, u64 *tiling_flags)
{
+ struct amdgpu_bo_user *ubo;
+
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
dma_resv_assert_held(bo->tbo.base.resv);
+ ubo = to_amdgpu_bo_user(bo);
if (tiling_flags)
- *tiling_flags = bo->tiling_flags;
+ *tiling_flags = ubo->tiling_flags;
}
/**
@@ -1167,13 +1191,16 @@ void amdgpu_bo_get_tiling_flags(struct amdgpu_bo *bo, u64 *tiling_flags)
int amdgpu_bo_set_metadata (struct amdgpu_bo *bo, void *metadata,
uint32_t metadata_size, uint64_t flags)
{
+ struct amdgpu_bo_user *ubo;
void *buffer;
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
+ ubo = to_amdgpu_bo_user(bo);
if (!metadata_size) {
- if (bo->metadata_size) {
- kfree(bo->metadata);
- bo->metadata = NULL;
- bo->metadata_size = 0;
+ if (ubo->metadata_size) {
+ kfree(ubo->metadata);
+ ubo->metadata = NULL;
+ ubo->metadata_size = 0;
}
return 0;
}
@@ -1185,10 +1212,10 @@ int amdgpu_bo_set_metadata (struct amdgpu_bo *bo, void *metadata,
if (buffer == NULL)
return -ENOMEM;
- kfree(bo->metadata);
- bo->metadata_flags = flags;
- bo->metadata = buffer;
- bo->metadata_size = metadata_size;
+ kfree(ubo->metadata);
+ ubo->metadata_flags = flags;
+ ubo->metadata = buffer;
+ ubo->metadata_size = metadata_size;
return 0;
}
@@ -1212,21 +1239,25 @@ int amdgpu_bo_get_metadata(struct amdgpu_bo *bo, void *buffer,
size_t buffer_size, uint32_t *metadata_size,
uint64_t *flags)
{
+ struct amdgpu_bo_user *ubo;
+
if (!buffer && !metadata_size)
return -EINVAL;
+ BUG_ON(bo->tbo.type == ttm_bo_type_kernel);
+ ubo = to_amdgpu_bo_user(bo);
if (buffer) {
- if (buffer_size < bo->metadata_size)
+ if (buffer_size < ubo->metadata_size)
return -EINVAL;
- if (bo->metadata_size)
- memcpy(buffer, bo->metadata, bo->metadata_size);
+ if (ubo->metadata_size)
+ memcpy(buffer, ubo->metadata, ubo->metadata_size);
}
if (metadata_size)
- *metadata_size = bo->metadata_size;
+ *metadata_size = ubo->metadata_size;
if (flags)
- *flags = bo->metadata_flags;
+ *flags = ubo->metadata_flags;
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index 9ac37569823f..2d1fefbe1e99 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -37,9 +37,12 @@
#define AMDGPU_BO_INVALID_OFFSET LONG_MAX
#define AMDGPU_BO_MAX_PLACEMENTS 3
+#define to_amdgpu_bo_user(abo) container_of((abo), struct amdgpu_bo_user, bo)
+
struct amdgpu_bo_param {
unsigned long size;
int byte_align;
+ u32 bo_ptr_size;
u32 domain;
u32 preferred_domain;
u64 flags;
@@ -89,10 +92,6 @@ struct amdgpu_bo {
struct ttm_buffer_object tbo;
struct ttm_bo_kmap_obj kmap;
u64 flags;
- u64 tiling_flags;
- u64 metadata_flags;
- void *metadata;
- u32 metadata_size;
unsigned prime_shared_count;
/* per VM structure for page tables and with virtual addresses */
struct amdgpu_vm_bo_base *vm_bo;
@@ -100,7 +99,6 @@ struct amdgpu_bo {
struct amdgpu_bo *parent;
struct amdgpu_bo *shadow;
- struct amdgpu_mn *mn;
#ifdef CONFIG_MMU_NOTIFIER
@@ -112,6 +110,15 @@ struct amdgpu_bo {
struct kgd_mem *kfd_bo;
};
+struct amdgpu_bo_user {
+ struct amdgpu_bo bo;
+ u64 tiling_flags;
+ u64 metadata_flags;
+ void *metadata;
+ u32 metadata_size;
+
+};
+
static inline struct amdgpu_bo *ttm_to_amdgpu_bo(struct ttm_buffer_object *tbo)
{
return container_of(tbo, struct amdgpu_bo, tbo);
@@ -255,6 +262,9 @@ int amdgpu_bo_create_kernel(struct amdgpu_device *adev,
int amdgpu_bo_create_kernel_at(struct amdgpu_device *adev,
uint64_t offset, uint64_t size, uint32_t domain,
struct amdgpu_bo **bo_ptr, void **cpu_addr);
+int amdgpu_bo_create_user(struct amdgpu_device *adev,
+ struct amdgpu_bo_param *bp,
+ struct amdgpu_bo_user **ubo_ptr);
void amdgpu_bo_free_kernel(struct amdgpu_bo **bo, u64 *gpu_addr,
void **cpu_addr);
int amdgpu_bo_kmap(struct amdgpu_bo *bo, void **ptr);
@@ -269,8 +279,6 @@ void amdgpu_bo_unpin(struct amdgpu_bo *bo);
int amdgpu_bo_evict_vram(struct amdgpu_device *adev);
int amdgpu_bo_init(struct amdgpu_device *adev);
void amdgpu_bo_fini(struct amdgpu_device *adev);
-int amdgpu_bo_fbdev_mmap(struct amdgpu_bo *bo,
- struct vm_area_struct *vma);
int amdgpu_bo_set_tiling_flags(struct amdgpu_bo *bo, u64 tiling_flags);
void amdgpu_bo_get_tiling_flags(struct amdgpu_bo *bo, u64 *tiling_flags);
int amdgpu_bo_set_metadata (struct amdgpu_bo *bo, void *metadata,
@@ -329,7 +337,7 @@ void amdgpu_sa_bo_dump_debug_info(struct amdgpu_sa_manager *sa_manager,
struct seq_file *m);
u64 amdgpu_bo_print_info(int id, struct amdgpu_bo *bo, struct seq_file *m);
#endif
-int amdgpu_debugfs_sa_init(struct amdgpu_device *adev);
+void amdgpu_debugfs_sa_init(struct amdgpu_device *adev);
bool amdgpu_bo_support_uswc(u64 bo_flags);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c
index 19c0a3655228..82e9ecf84352 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_pmu.c
@@ -519,8 +519,10 @@ static int init_pmu_entry_by_type_and_add(struct amdgpu_pmu_entry *pmu_entry,
pmu_entry->pmu.attr_groups = kmemdup(attr_groups, sizeof(attr_groups),
GFP_KERNEL);
- if (!pmu_entry->pmu.attr_groups)
+ if (!pmu_entry->pmu.attr_groups) {
+ ret = -ENOMEM;
goto err_attr_group;
+ }
snprintf(pmu_name, PMU_NAME_SIZE, "%s_%d", pmu_entry->pmu_file_prefix,
adev_to_drm(pmu_entry->adev)->primary->index);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
index 839917eb7bc3..a09483beb968 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c
@@ -34,6 +34,7 @@
#include "psp_v10_0.h"
#include "psp_v11_0.h"
#include "psp_v12_0.h"
+#include "psp_v13_0.h"
#include "amdgpu_ras.h"
#include "amdgpu_securedisplay.h"
@@ -56,7 +57,7 @@ static int psp_load_smu_fw(struct psp_context *psp);
* - Load XGMI/RAS/HDCP/DTM TA if any
*
* This new sequence is required for
- * - Arcturus
+ * - Arcturus and onwards
* - Navi12 and onwards
*/
static void psp_check_pmfw_centralized_cstate_management(struct psp_context *psp)
@@ -71,7 +72,7 @@ static void psp_check_pmfw_centralized_cstate_management(struct psp_context *psp
if (adev->flags & AMD_IS_APU)
return;
- if ((adev->asic_type == CHIP_ARCTURUS) ||
+ if ((adev->asic_type >= CHIP_ARCTURUS) ||
(adev->asic_type >= CHIP_NAVI12))
psp->pmfw_centralized_cstate_management = true;
}
@@ -109,6 +110,9 @@ static int psp_early_init(void *handle)
case CHIP_RENOIR:
psp_v12_0_set_psp_funcs(psp);
break;
+ case CHIP_ALDEBARAN:
+ psp_v13_0_set_psp_funcs(psp);
+ break;
default:
return -EINVAL;
}
@@ -324,8 +328,12 @@ psp_cmd_submit_buf(struct psp_context *psp,
static void psp_prep_tmr_cmd_buf(struct psp_context *psp,
struct psp_gfx_cmd_resp *cmd,
- uint64_t tmr_mc, uint32_t size)
+ uint64_t tmr_mc, struct amdgpu_bo *tmr_bo)
{
+ struct amdgpu_device *adev = psp->adev;
+ uint32_t size = amdgpu_bo_size(tmr_bo);
+ uint64_t tmr_pa = amdgpu_gmc_vram_pa(adev, tmr_bo);
+
if (amdgpu_sriov_vf(psp->adev))
cmd->cmd_id = GFX_CMD_ID_SETUP_VMR;
else
@@ -333,6 +341,9 @@ static void psp_prep_tmr_cmd_buf(struct psp_context *psp,
cmd->cmd.cmd_setup_tmr.buf_phy_addr_lo = lower_32_bits(tmr_mc);
cmd->cmd.cmd_setup_tmr.buf_phy_addr_hi = upper_32_bits(tmr_mc);
cmd->cmd.cmd_setup_tmr.buf_size = size;
+ cmd->cmd.cmd_setup_tmr.bitfield.virt_phy_addr = 1;
+ cmd->cmd.cmd_setup_tmr.system_phy_addr_lo = lower_32_bits(tmr_pa);
+ cmd->cmd.cmd_setup_tmr.system_phy_addr_hi = upper_32_bits(tmr_pa);
}
static void psp_prep_load_toc_cmd_buf(struct psp_gfx_cmd_resp *cmd,
@@ -383,7 +394,7 @@ static int psp_tmr_init(struct psp_context *psp)
* Note: this memory need be reserved till the driver
* uninitializes.
*/
- tmr_size = PSP_TMR_SIZE;
+ tmr_size = PSP_TMR_SIZE(psp->adev);
/* For ASICs support RLC autoload, psp will parse the toc
* and calculate the total size of TMR needed */
@@ -399,7 +410,7 @@ static int psp_tmr_init(struct psp_context *psp)
}
pptr = amdgpu_sriov_vf(psp->adev) ? &tmr_buf : NULL;
- ret = amdgpu_bo_create_kernel(psp->adev, tmr_size, PSP_TMR_SIZE,
+ ret = amdgpu_bo_create_kernel(psp->adev, tmr_size, PSP_TMR_SIZE(psp->adev),
AMDGPU_GEM_DOMAIN_VRAM,
&psp->tmr_bo, &psp->tmr_mc_addr, pptr);
@@ -452,8 +463,7 @@ static int psp_tmr_load(struct psp_context *psp)
if (!cmd)
return -ENOMEM;
- psp_prep_tmr_cmd_buf(psp, cmd, psp->tmr_mc_addr,
- amdgpu_bo_size(psp->tmr_bo));
+ psp_prep_tmr_cmd_buf(psp, cmd, psp->tmr_mc_addr, psp->tmr_bo);
DRM_INFO("reserve 0x%lx from 0x%llx for PSP TMR\n",
amdgpu_bo_size(psp->tmr_bo), psp->tmr_mc_addr);
@@ -542,6 +552,46 @@ int psp_get_fw_attestation_records_addr(struct psp_context *psp,
return ret;
}
+static int psp_boot_config_set(struct amdgpu_device *adev)
+{
+ struct psp_context *psp = &adev->psp;
+ struct psp_gfx_cmd_resp *cmd = psp->cmd;
+
+ if (adev->asic_type != CHIP_SIENNA_CICHLID || amdgpu_sriov_vf(adev))
+ return 0;
+
+ memset(cmd, 0, sizeof(struct psp_gfx_cmd_resp));
+
+ cmd->cmd_id = GFX_CMD_ID_BOOT_CFG;
+ cmd->cmd.boot_cfg.sub_cmd = BOOTCFG_CMD_SET;
+ cmd->cmd.boot_cfg.boot_config = BOOT_CONFIG_GECC;
+ cmd->cmd.boot_cfg.boot_config_valid = BOOT_CONFIG_GECC;
+
+ return psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr);
+}
+
+static int psp_rl_load(struct amdgpu_device *adev)
+{
+ struct psp_context *psp = &adev->psp;
+ struct psp_gfx_cmd_resp *cmd = psp->cmd;
+
+ if (psp->rl_bin_size == 0)
+ return 0;
+
+ memset(psp->fw_pri_buf, 0, PSP_1_MEG);
+ memcpy(psp->fw_pri_buf, psp->rl_start_addr, psp->rl_bin_size);
+
+ memset(cmd, 0, sizeof(struct psp_gfx_cmd_resp));
+
+ cmd->cmd_id = GFX_CMD_ID_LOAD_IP_FW;
+ cmd->cmd.cmd_load_ip_fw.fw_phy_addr_lo = lower_32_bits(psp->fw_pri_mc_addr);
+ cmd->cmd.cmd_load_ip_fw.fw_phy_addr_hi = upper_32_bits(psp->fw_pri_mc_addr);
+ cmd->cmd.cmd_load_ip_fw.fw_size = psp->rl_bin_size;
+ cmd->cmd.cmd_load_ip_fw.fw_type = GFX_FW_TYPE_REG_LIST;
+
+ return psp_cmd_submit_buf(psp, NULL, cmd, psp->fence_buf_mc_addr);
+}
+
static void psp_prep_asd_load_cmd_buf(struct psp_gfx_cmd_resp *cmd,
uint64_t asd_mc, uint32_t size)
{
@@ -755,8 +805,9 @@ static int psp_xgmi_unload(struct psp_context *psp)
struct psp_gfx_cmd_resp *cmd;
struct amdgpu_device *adev = psp->adev;
- /* XGMI TA unload currently is not supported on Arcturus */
- if (adev->asic_type == CHIP_ARCTURUS)
+ /* XGMI TA unload currently is not supported on Arcturus/Aldebaran A+A */
+ if (adev->asic_type == CHIP_ARCTURUS ||
+ (adev->asic_type == CHIP_ALDEBARAN && adev->gmc.xgmi.connected_to_cpu))
return 0;
/*
@@ -1561,6 +1612,7 @@ static int psp_rap_unload(struct psp_context *psp)
static int psp_rap_initialize(struct psp_context *psp)
{
int ret;
+ enum ta_rap_status status = TA_RAP_STATUS__SUCCESS;
/*
* TODO: bypass the initialize in sriov for now
@@ -1584,8 +1636,8 @@ static int psp_rap_initialize(struct psp_context *psp)
if (ret)
return ret;
- ret = psp_rap_invoke(psp, TA_CMD_RAP__INITIALIZE);
- if (ret != TA_RAP_STATUS__SUCCESS) {
+ ret = psp_rap_invoke(psp, TA_CMD_RAP__INITIALIZE, &status);
+ if (ret || status != TA_RAP_STATUS__SUCCESS) {
psp_rap_unload(psp);
amdgpu_bo_free_kernel(&psp->rap_context.rap_shared_bo,
@@ -1594,8 +1646,10 @@ static int psp_rap_initialize(struct psp_context *psp)
psp->rap_context.rap_initialized = false;
- dev_warn(psp->adev->dev, "RAP TA initialize fail.\n");
- return -EINVAL;
+ dev_warn(psp->adev->dev, "RAP TA initialize fail (%d) status %d.\n",
+ ret, status);
+
+ return ret;
}
return 0;
@@ -1620,13 +1674,13 @@ static int psp_rap_terminate(struct psp_context *psp)
return ret;
}
-int psp_rap_invoke(struct psp_context *psp, uint32_t ta_cmd_id)
+int psp_rap_invoke(struct psp_context *psp, uint32_t ta_cmd_id, enum ta_rap_status *status)
{
struct ta_rap_shared_memory *rap_cmd;
- int ret;
+ int ret = 0;
if (!psp->rap_context.rap_initialized)
- return -EINVAL;
+ return 0;
if (ta_cmd_id != TA_CMD_RAP__INITIALIZE &&
ta_cmd_id != TA_CMD_RAP__VALIDATE_L0)
@@ -1642,14 +1696,16 @@ int psp_rap_invoke(struct psp_context *psp, uint32_t ta_cmd_id)
rap_cmd->validation_method_id = METHOD_A;
ret = psp_ta_invoke(psp, rap_cmd->cmd_id, psp->rap_context.session_id);
- if (ret) {
- mutex_unlock(&psp->rap_context.mutex);
- return ret;
- }
+ if (ret)
+ goto out_unlock;
+
+ if (status)
+ *status = rap_cmd->rap_status;
+out_unlock:
mutex_unlock(&psp->rap_context.mutex);
- return rap_cmd->rap_status;
+ return ret;
}
// RAP end
@@ -1870,6 +1926,11 @@ static int psp_hw_start(struct psp_context *psp)
return ret;
}
+ ret = psp_boot_config_set(adev);
+ if (ret) {
+ DRM_WARN("PSP set boot config@\n");
+ }
+
ret = psp_tmr_init(psp);
if (ret) {
DRM_ERROR("PSP tmr init failed!\n");
@@ -2104,9 +2165,13 @@ static int psp_load_smu_fw(struct psp_context *psp)
if (!ucode->fw || amdgpu_sriov_vf(psp->adev))
return 0;
-
- if (amdgpu_in_reset(adev) && ras && ras->supported &&
- adev->asic_type == CHIP_ARCTURUS) {
+ if ((amdgpu_in_reset(adev) &&
+ ras && ras->supported &&
+ (adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_VEGA20)) ||
+ (adev->in_runpm &&
+ adev->asic_type >= CHIP_NAVI10 &&
+ adev->asic_type <= CHIP_NAVI12)) {
ret = amdgpu_dpm_set_mp1_state(adev, PP_MP1_STATE_UNLOAD);
if (ret) {
DRM_WARN("Failed to set MP1 state prepare for reload\n");
@@ -2159,6 +2224,22 @@ static bool fw_load_skip_check(struct psp_context *psp,
return false;
}
+int psp_load_fw_list(struct psp_context *psp,
+ struct amdgpu_firmware_info **ucode_list, int ucode_count)
+{
+ int ret = 0, i;
+ struct amdgpu_firmware_info *ucode;
+
+ for (i = 0; i < ucode_count; ++i) {
+ ucode = ucode_list[i];
+ psp_print_fw_hdr(psp, ucode);
+ ret = psp_execute_np_fw_load(psp, ucode);
+ if (ret)
+ return ret;
+ }
+ return ret;
+}
+
static int psp_np_fw_load(struct psp_context *psp)
{
int i, ret;
@@ -2276,6 +2357,12 @@ skip_memalloc:
return ret;
}
+ ret = psp_rl_load(adev);
+ if (ret) {
+ DRM_ERROR("PSP load RL failed!\n");
+ return ret;
+ }
+
if (psp->adev->psp.ta_fw) {
ret = psp_ras_initialize(psp);
if (ret)
@@ -2751,6 +2838,9 @@ int psp_init_sos_microcode(struct psp_context *psp,
adev->psp.spl_bin_size = le32_to_cpu(sos_hdr_v1_3->spl_size_bytes);
adev->psp.spl_start_addr = (uint8_t *)adev->psp.sys_start_addr +
le32_to_cpu(sos_hdr_v1_3->spl_offset_bytes);
+ adev->psp.rl_bin_size = le32_to_cpu(sos_hdr_v1_3->rl_size_bytes);
+ adev->psp.rl_start_addr = (uint8_t *)adev->psp.sys_start_addr +
+ le32_to_cpu(sos_hdr_v1_3->rl_offset_bytes);
}
break;
default:
@@ -2916,7 +3006,7 @@ static ssize_t psp_usbc_pd_fw_sysfs_read(struct device *dev,
return ret;
}
- return snprintf(buf, PAGE_SIZE, "%x\n", fw_ver);
+ return sysfs_emit(buf, "%x\n", fw_ver);
}
static ssize_t psp_usbc_pd_fw_sysfs_write(struct device *dev,
@@ -3052,3 +3142,11 @@ const struct amdgpu_ip_block_version psp_v12_0_ip_block =
.rev = 0,
.funcs = &psp_ip_funcs,
};
+
+const struct amdgpu_ip_block_version psp_v13_0_ip_block = {
+ .type = AMD_IP_BLOCK_TYPE_PSP,
+ .major = 13,
+ .minor = 0,
+ .rev = 0,
+ .funcs = &psp_ip_funcs,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
index cb50ba445f8c..46a5328e00e0 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_psp.h
@@ -37,7 +37,7 @@
#define PSP_XGMI_SHARED_MEM_SIZE 0x4000
#define PSP_RAS_SHARED_MEM_SIZE 0x4000
#define PSP_1_MEG 0x100000
-#define PSP_TMR_SIZE 0x400000
+#define PSP_TMR_SIZE(adev) ((adev)->asic_type == CHIP_ALDEBARAN ? 0x800000 : 0x400000)
#define PSP_HDCP_SHARED_MEM_SIZE 0x4000
#define PSP_DTM_SHARED_MEM_SIZE 0x4000
#define PSP_RAP_SHARED_MEM_SIZE 0x4000
@@ -248,11 +248,13 @@ struct psp_context
uint32_t toc_bin_size;
uint32_t kdb_bin_size;
uint32_t spl_bin_size;
+ uint32_t rl_bin_size;
uint8_t *sys_start_addr;
uint8_t *sos_start_addr;
uint8_t *toc_start_addr;
uint8_t *kdb_start_addr;
uint8_t *spl_start_addr;
+ uint8_t *rl_start_addr;
/* tmr buffer */
struct amdgpu_bo *tmr_bo;
@@ -365,11 +367,13 @@ struct amdgpu_psp_funcs {
extern const struct amd_ip_funcs psp_ip_funcs;
extern const struct amdgpu_ip_block_version psp_v3_1_ip_block;
-extern int psp_wait_for(struct psp_context *psp, uint32_t reg_index,
- uint32_t field_val, uint32_t mask, bool check_changed);
-
extern const struct amdgpu_ip_block_version psp_v10_0_ip_block;
+extern const struct amdgpu_ip_block_version psp_v11_0_ip_block;
extern const struct amdgpu_ip_block_version psp_v12_0_ip_block;
+extern const struct amdgpu_ip_block_version psp_v13_0_ip_block;
+
+extern int psp_wait_for(struct psp_context *psp, uint32_t reg_index,
+ uint32_t field_val, uint32_t mask, bool check_changed);
int psp_gpu_reset(struct amdgpu_device *adev);
int psp_update_vcn_sram(struct amdgpu_device *adev, int inst_idx,
@@ -395,12 +399,11 @@ int psp_ras_trigger_error(struct psp_context *psp,
int psp_hdcp_invoke(struct psp_context *psp, uint32_t ta_cmd_id);
int psp_dtm_invoke(struct psp_context *psp, uint32_t ta_cmd_id);
-int psp_rap_invoke(struct psp_context *psp, uint32_t ta_cmd_id);
+int psp_rap_invoke(struct psp_context *psp, uint32_t ta_cmd_id, enum ta_rap_status *status);
int psp_securedisplay_invoke(struct psp_context *psp, uint32_t ta_cmd_id);
int psp_rlc_autoload_start(struct psp_context *psp);
-extern const struct amdgpu_ip_block_version psp_v11_0_ip_block;
int psp_reg_program(struct psp_context *psp, enum psp_reg_prog_id reg,
uint32_t value);
int psp_ring_cmd_submit(struct psp_context *psp,
@@ -417,4 +420,7 @@ int psp_init_ta_microcode(struct psp_context *psp,
const char *chip_name);
int psp_get_fw_attestation_records_addr(struct psp_context *psp,
uint64_t *output_ptr);
+
+int psp_load_fw_list(struct psp_context *psp,
+ struct amdgpu_firmware_info **ucode_list, int ucode_count);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_rap.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_rap.c
index 8da5356c36f1..51909bf8798c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_rap.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_rap.c
@@ -48,6 +48,7 @@ static ssize_t amdgpu_rap_debugfs_write(struct file *f, const char __user *buf,
struct ta_rap_cmd_output_data *rap_cmd_output;
struct drm_device *dev = adev_to_drm(adev);
uint32_t op;
+ enum ta_rap_status status;
int ret;
if (*pos || size != 2)
@@ -70,9 +71,8 @@ static ssize_t amdgpu_rap_debugfs_write(struct file *f, const char __user *buf,
switch (op) {
case 2:
- ret = psp_rap_invoke(&adev->psp, op);
-
- if (ret == TA_RAP_STATUS__SUCCESS) {
+ ret = psp_rap_invoke(&adev->psp, op, &status);
+ if (!ret && status == TA_RAP_STATUS__SUCCESS) {
dev_info(adev->dev, "RAP L0 validate test success.\n");
} else {
rap_shared_mem = (struct ta_rap_shared_memory *)
@@ -97,6 +97,7 @@ static ssize_t amdgpu_rap_debugfs_write(struct file *f, const char __user *buf,
default:
dev_info(adev->dev, "Unsupported op id: %d, ", op);
dev_info(adev->dev, "Only support op 2(L0 validate test).\n");
+ break;
}
amdgpu_gfx_off_ctrl(adev, true);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
index 1fb2a91ad30a..b0d2fc9454ca 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c
@@ -99,6 +99,49 @@ static bool amdgpu_ras_get_error_query_ready(struct amdgpu_device *adev)
return false;
}
+static int amdgpu_reserve_page_direct(struct amdgpu_device *adev, uint64_t address)
+{
+ struct ras_err_data err_data = {0, 0, 0, NULL};
+ struct eeprom_table_record err_rec;
+
+ if ((address >= adev->gmc.mc_vram_size) ||
+ (address >= RAS_UMC_INJECT_ADDR_LIMIT)) {
+ dev_warn(adev->dev,
+ "RAS WARN: input address 0x%llx is invalid.\n",
+ address);
+ return -EINVAL;
+ }
+
+ if (amdgpu_ras_check_bad_page(adev, address)) {
+ dev_warn(adev->dev,
+ "RAS WARN: 0x%llx has already been marked as bad page!\n",
+ address);
+ return 0;
+ }
+
+ memset(&err_rec, 0x0, sizeof(struct eeprom_table_record));
+
+ err_rec.address = address;
+ err_rec.retired_page = address >> AMDGPU_GPU_PAGE_SHIFT;
+ err_rec.ts = (uint64_t)ktime_get_real_seconds();
+ err_rec.err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
+
+ err_data.err_addr = &err_rec;
+ err_data.err_addr_cnt = 1;
+
+ if (amdgpu_bad_page_threshold != 0) {
+ amdgpu_ras_add_bad_pages(adev, err_data.err_addr,
+ err_data.err_addr_cnt);
+ amdgpu_ras_save_bad_pages(adev);
+ }
+
+ dev_warn(adev->dev, "WARNING: THIS IS ONLY FOR TEST PURPOSES AND WILL CORRUPT RAS EEPROM\n");
+ dev_warn(adev->dev, "Clear EEPROM:\n");
+ dev_warn(adev->dev, " echo 1 > /sys/kernel/debug/dri/0/ras/ras_eeprom_reset\n");
+
+ return 0;
+}
+
static ssize_t amdgpu_ras_debugfs_read(struct file *f, char __user *buf,
size_t size, loff_t *pos)
{
@@ -109,7 +152,7 @@ static ssize_t amdgpu_ras_debugfs_read(struct file *f, char __user *buf,
ssize_t s;
char val[128];
- if (amdgpu_ras_error_query(obj->adev, &info))
+ if (amdgpu_ras_query_error_status(obj->adev, &info))
return -EINVAL;
s = snprintf(val, sizeof(val), "%s: %lu\n%s: %lu\n",
@@ -178,11 +221,24 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f,
op = 1;
else if (sscanf(str, "inject %32s %8s", block_name, err) == 2)
op = 2;
+ else if (strstr(str, "retire_page") != NULL)
+ op = 3;
else if (str[0] && str[1] && str[2] && str[3])
/* ascii string, but commands are not matched. */
return -EINVAL;
if (op != -1) {
+ if (op == 3) {
+ if (sscanf(str, "%*s 0x%llx", &address) != 1 &&
+ sscanf(str, "%*s %llu", &address) != 1)
+ return -EINVAL;
+
+ data->op = op;
+ data->inject.address = address;
+
+ return 0;
+ }
+
if (amdgpu_ras_find_block_id_by_name(block_name, &block_id))
return -EINVAL;
@@ -198,11 +254,11 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f,
data->op = op;
if (op == 2) {
- if (sscanf(str, "%*s %*s %*s %u %llu %llu",
- &sub_block, &address, &value) != 3)
- if (sscanf(str, "%*s %*s %*s 0x%x 0x%llx 0x%llx",
- &sub_block, &address, &value) != 3)
- return -EINVAL;
+ if (sscanf(str, "%*s %*s %*s 0x%x 0x%llx 0x%llx",
+ &sub_block, &address, &value) != 3 &&
+ sscanf(str, "%*s %*s %*s %u %llu %llu",
+ &sub_block, &address, &value) != 3)
+ return -EINVAL;
data->head.sub_block_index = sub_block;
data->inject.address = address;
data->inject.value = value;
@@ -221,7 +277,7 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f,
/**
* DOC: AMDGPU RAS debugfs control interface
*
- * It accepts struct ras_debug_if who has two members.
+ * The control interface accepts struct ras_debug_if which has two members.
*
* First member: ras_debug_if::head or ras_debug_if::inject.
*
@@ -246,32 +302,33 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f,
*
* How to use the interface?
*
- * Programs
+ * In a program
*
- * Copy the struct ras_debug_if in your codes and initialize it.
- * Write the struct to the control node.
+ * Copy the struct ras_debug_if in your code and initialize it.
+ * Write the struct to the control interface.
*
- * Shells
+ * From shell
*
* .. code-block:: bash
*
- * echo op block [error [sub_block address value]] > .../ras/ras_ctrl
+ * echo "disable <block>" > /sys/kernel/debug/dri/<N>/ras/ras_ctrl
+ * echo "enable <block> <error>" > /sys/kernel/debug/dri/<N>/ras/ras_ctrl
+ * echo "inject <block> <error> <sub-block> <address> <value> > /sys/kernel/debug/dri/<N>/ras/ras_ctrl
*
- * Parameters:
+ * Where N, is the card which you want to affect.
*
- * op: disable, enable, inject
- * disable: only block is needed
- * enable: block and error are needed
- * inject: error, address, value are needed
- * block: umc, sdma, gfx, .........
+ * "disable" requires only the block.
+ * "enable" requires the block and error type.
+ * "inject" requires the block, error type, address, and value.
+ * The block is one of: umc, sdma, gfx, etc.
* see ras_block_string[] for details
- * error: ue, ce
- * ue: multi_uncorrectable
- * ce: single_correctable
- * sub_block:
- * sub block index, pass 0 if there is no sub block
+ * The error type is one of: ue, ce, where,
+ * ue is multi-uncorrectable
+ * ce is single-correctable
+ * The sub-block is a the sub-block index, pass 0 if there is no sub-block.
+ * The address and value are hexadecimal numbers, leading 0x is optional.
*
- * here are some examples for bash commands:
+ * For instance,
*
* .. code-block:: bash
*
@@ -279,17 +336,17 @@ static int amdgpu_ras_debugfs_ctrl_parse_data(struct file *f,
* echo inject umc ce 0 0 0 > /sys/kernel/debug/dri/0/ras/ras_ctrl
* echo disable umc > /sys/kernel/debug/dri/0/ras/ras_ctrl
*
- * How to check the result?
+ * How to check the result of the operation?
*
- * For disable/enable, please check ras features at
+ * To check disable/enable, see "ras" features at,
* /sys/class/drm/card[0/1/2...]/device/ras/features
*
- * For inject, please check corresponding err count at
- * /sys/class/drm/card[0/1/2...]/device/ras/[gfx/sdma/...]_err_count
+ * To check inject, see the corresponding error count at,
+ * /sys/class/drm/card[0/1/2...]/device/ras/[gfx|sdma|umc|...]_err_count
*
* .. note::
* Operations are only allowed on blocks which are supported.
- * Please check ras mask at /sys/module/amdgpu/parameters/ras_mask
+ * Check the "ras" mask at /sys/module/amdgpu/parameters/ras_mask
* to see which blocks support RAS on a particular asic.
*
*/
@@ -310,6 +367,14 @@ static ssize_t amdgpu_ras_debugfs_ctrl_write(struct file *f, const char __user *
if (ret)
return -EINVAL;
+ if (data.op == 3) {
+ ret = amdgpu_reserve_page_direct(adev, data.inject.address);
+ if (!ret)
+ return size;
+ else
+ return ret;
+ }
+
if (!amdgpu_ras_is_supported(adev, data.head.block))
return -EINVAL;
@@ -431,15 +496,19 @@ static ssize_t amdgpu_ras_sysfs_read(struct device *dev,
};
if (!amdgpu_ras_get_error_query_ready(obj->adev))
- return snprintf(buf, PAGE_SIZE,
- "Query currently inaccessible\n");
+ return sysfs_emit(buf, "Query currently inaccessible\n");
- if (amdgpu_ras_error_query(obj->adev, &info))
+ if (amdgpu_ras_query_error_status(obj->adev, &info))
return -EINVAL;
- return snprintf(buf, PAGE_SIZE, "%s: %lu\n%s: %lu\n",
- "ue", info.ue_count,
- "ce", info.ce_count);
+
+ if (obj->adev->asic_type == CHIP_ALDEBARAN) {
+ if (amdgpu_ras_reset_error_status(obj->adev, info.head.block))
+ DRM_WARN("Failed to reset error counter and error status");
+ }
+
+ return sysfs_emit(buf, "%s: %lu\n%s: %lu\n", "ue", info.ue_count,
+ "ce", info.ce_count);
}
/* obj begin */
@@ -449,11 +518,10 @@ static ssize_t amdgpu_ras_sysfs_read(struct device *dev,
static inline void put_obj(struct ras_manager *obj)
{
- if (obj && --obj->use == 0)
+ if (obj && (--obj->use == 0))
list_del(&obj->node);
- if (obj && obj->use < 0) {
- DRM_ERROR("RAS ERROR: Unbalance obj(%s) use\n", obj->head.name);
- }
+ if (obj && (obj->use < 0))
+ DRM_ERROR("RAS ERROR: Unbalance obj(%s) use\n", obj->head.name);
}
/* make one obj and return it. */
@@ -463,7 +531,7 @@ static struct ras_manager *amdgpu_ras_create_obj(struct amdgpu_device *adev,
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
struct ras_manager *obj;
- if (!con)
+ if (!adev->ras_features || !con)
return NULL;
if (head->block >= AMDGPU_RAS_BLOCK_COUNT)
@@ -490,7 +558,7 @@ struct ras_manager *amdgpu_ras_find_obj(struct amdgpu_device *adev,
struct ras_manager *obj;
int i;
- if (!con)
+ if (!adev->ras_features || !con)
return NULL;
if (head) {
@@ -590,7 +658,11 @@ static int __amdgpu_ras_feature_enable(struct amdgpu_device *adev,
con->features |= BIT(head->block);
} else {
if (obj && amdgpu_ras_is_feature_enabled(adev, head)) {
- con->features &= ~BIT(head->block);
+ /* skip clean gfx ras context feature for VEGA20 Gaming.
+ * will clean later
+ */
+ if (!(!adev->ras_features && con->features & BIT(AMDGPU_RAS_BLOCK__GFX)))
+ con->features &= ~BIT(head->block);
put_obj(obj);
}
}
@@ -693,6 +765,10 @@ int amdgpu_ras_feature_enable_on_boot(struct amdgpu_device *adev,
if (ret)
return ret;
+ /* gfx block ras dsiable cmd must send to ras-ta */
+ if (head->block == AMDGPU_RAS_BLOCK__GFX)
+ con->features |= BIT(head->block);
+
ret = amdgpu_ras_feature_enable(adev, head, 0);
}
} else
@@ -757,8 +833,8 @@ static int amdgpu_ras_enable_all_features(struct amdgpu_device *adev,
/* feature ctl end */
/* query/inject/cure begin */
-int amdgpu_ras_error_query(struct amdgpu_device *adev,
- struct ras_query_if *info)
+int amdgpu_ras_query_error_status(struct amdgpu_device *adev,
+ struct ras_query_if *info)
{
struct ras_manager *obj = amdgpu_ras_find_obj(adev, &info->head);
struct ras_err_data err_data = {0, 0, 0, NULL};
@@ -769,13 +845,15 @@ int amdgpu_ras_error_query(struct amdgpu_device *adev,
switch (info->head.block) {
case AMDGPU_RAS_BLOCK__UMC:
- if (adev->umc.funcs->query_ras_error_count)
- adev->umc.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_count)
+ adev->umc.ras_funcs->query_ras_error_count(adev, &err_data);
/* umc query_ras_error_address is also responsible for clearing
* error status
*/
- if (adev->umc.funcs->query_ras_error_address)
- adev->umc.funcs->query_ras_error_address(adev, &err_data);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_address)
+ adev->umc.ras_funcs->query_ras_error_address(adev, &err_data);
break;
case AMDGPU_RAS_BLOCK__SDMA:
if (adev->sdma.funcs->query_ras_error_count) {
@@ -785,19 +863,32 @@ int amdgpu_ras_error_query(struct amdgpu_device *adev,
}
break;
case AMDGPU_RAS_BLOCK__GFX:
- if (adev->gfx.funcs->query_ras_error_count)
- adev->gfx.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_count)
+ adev->gfx.ras_funcs->query_ras_error_count(adev, &err_data);
+
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_status)
+ adev->gfx.ras_funcs->query_ras_error_status(adev);
break;
case AMDGPU_RAS_BLOCK__MMHUB:
- if (adev->mmhub.funcs->query_ras_error_count)
- adev->mmhub.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->query_ras_error_count)
+ adev->mmhub.ras_funcs->query_ras_error_count(adev, &err_data);
+
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->query_ras_error_status)
+ adev->mmhub.ras_funcs->query_ras_error_status(adev);
break;
case AMDGPU_RAS_BLOCK__PCIE_BIF:
- if (adev->nbio.funcs->query_ras_error_count)
- adev->nbio.funcs->query_ras_error_count(adev, &err_data);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->query_ras_error_count)
+ adev->nbio.ras_funcs->query_ras_error_count(adev, &err_data);
break;
case AMDGPU_RAS_BLOCK__XGMI_WAFL:
- amdgpu_xgmi_query_ras_error_count(adev, &err_data);
+ if (adev->gmc.xgmi.ras_funcs &&
+ adev->gmc.xgmi.ras_funcs->query_ras_error_count)
+ adev->gmc.xgmi.ras_funcs->query_ras_error_count(adev, &err_data);
break;
default:
break;
@@ -826,6 +917,38 @@ int amdgpu_ras_error_query(struct amdgpu_device *adev,
return 0;
}
+int amdgpu_ras_reset_error_status(struct amdgpu_device *adev,
+ enum amdgpu_ras_block block)
+{
+ if (!amdgpu_ras_is_supported(adev, block))
+ return -EINVAL;
+
+ switch (block) {
+ case AMDGPU_RAS_BLOCK__GFX:
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->reset_ras_error_count)
+ adev->gfx.ras_funcs->reset_ras_error_count(adev);
+
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->reset_ras_error_status)
+ adev->gfx.ras_funcs->reset_ras_error_status(adev);
+ break;
+ case AMDGPU_RAS_BLOCK__MMHUB:
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->reset_ras_error_count)
+ adev->mmhub.ras_funcs->reset_ras_error_count(adev);
+ break;
+ case AMDGPU_RAS_BLOCK__SDMA:
+ if (adev->sdma.funcs->reset_ras_error_count)
+ adev->sdma.funcs->reset_ras_error_count(adev);
+ break;
+ default:
+ break;
+ }
+
+ return 0;
+}
+
/* Trigger XGMI/WAFL error */
static int amdgpu_ras_error_inject_xgmi(struct amdgpu_device *adev,
struct ta_ras_trigger_error_input *block_info)
@@ -878,12 +1001,14 @@ int amdgpu_ras_error_inject(struct amdgpu_device *adev,
switch (info->head.block) {
case AMDGPU_RAS_BLOCK__GFX:
- if (adev->gfx.funcs->ras_error_inject)
- ret = adev->gfx.funcs->ras_error_inject(adev, info);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->ras_error_inject)
+ ret = adev->gfx.ras_funcs->ras_error_inject(adev, info);
else
ret = -EINVAL;
break;
case AMDGPU_RAS_BLOCK__UMC:
+ case AMDGPU_RAS_BLOCK__SDMA:
case AMDGPU_RAS_BLOCK__MMHUB:
case AMDGPU_RAS_BLOCK__PCIE_BIF:
ret = psp_ras_trigger_error(&adev->psp, &block_info);
@@ -913,7 +1038,7 @@ unsigned long amdgpu_ras_query_error_count(struct amdgpu_device *adev,
struct ras_manager *obj;
struct ras_err_data data = {0, 0};
- if (!con)
+ if (!adev->ras_features || !con)
return 0;
list_for_each_entry(obj, &con->head, node) {
@@ -921,7 +1046,7 @@ unsigned long amdgpu_ras_query_error_count(struct amdgpu_device *adev,
.head = obj->head,
};
- if (amdgpu_ras_error_query(adev, &info))
+ if (amdgpu_ras_query_error_status(adev, &info))
return 0;
data.ce_count += info.ce_count;
@@ -1137,16 +1262,19 @@ static int amdgpu_ras_sysfs_remove_all(struct amdgpu_device *adev)
*
*/
/* debugfs begin */
-static void amdgpu_ras_debugfs_create_ctrl_node(struct amdgpu_device *adev)
+static struct dentry *amdgpu_ras_debugfs_create_ctrl_node(struct amdgpu_device *adev)
{
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
+ struct dentry *dir;
struct drm_minor *minor = adev_to_drm(adev)->primary;
- con->dir = debugfs_create_dir(RAS_FS_NAME, minor->debugfs_root);
- debugfs_create_file("ras_ctrl", S_IWUGO | S_IRUGO, con->dir,
- adev, &amdgpu_ras_debugfs_ctrl_ops);
- debugfs_create_file("ras_eeprom_reset", S_IWUGO | S_IRUGO, con->dir,
- adev, &amdgpu_ras_debugfs_eeprom_ops);
+ dir = debugfs_create_dir(RAS_FS_NAME, minor->debugfs_root);
+ debugfs_create_file("ras_ctrl", S_IWUGO | S_IRUGO, dir, adev,
+ &amdgpu_ras_debugfs_ctrl_ops);
+ debugfs_create_file("ras_eeprom_reset", S_IWUGO | S_IRUGO, dir, adev,
+ &amdgpu_ras_debugfs_eeprom_ops);
+ debugfs_create_u32("bad_page_cnt_threshold", 0444, dir,
+ &con->bad_page_cnt_threshold);
/*
* After one uncorrectable error happens, usually GPU recovery will
@@ -1156,24 +1284,24 @@ static void amdgpu_ras_debugfs_create_ctrl_node(struct amdgpu_device *adev)
* ERREVENT_ATHUB_INTERRUPT generated. Normal GPU recovery routine
* will never be called.
*/
- debugfs_create_bool("auto_reboot", S_IWUGO | S_IRUGO, con->dir,
- &con->reboot);
+ debugfs_create_bool("auto_reboot", S_IWUGO | S_IRUGO, dir, &con->reboot);
/*
* User could set this not to clean up hardware's error count register
* of RAS IPs during ras recovery.
*/
- debugfs_create_bool("disable_ras_err_cnt_harvest", 0644,
- con->dir, &con->disable_ras_err_cnt_harvest);
+ debugfs_create_bool("disable_ras_err_cnt_harvest", 0644, dir,
+ &con->disable_ras_err_cnt_harvest);
+ return dir;
}
static void amdgpu_ras_debugfs_create(struct amdgpu_device *adev,
- struct ras_fs_if *head)
+ struct ras_fs_if *head,
+ struct dentry *dir)
{
- struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
struct ras_manager *obj = amdgpu_ras_find_obj(adev, &head->head);
- if (!obj || obj->ent)
+ if (!obj || !dir)
return;
get_obj(obj);
@@ -1182,14 +1310,14 @@ static void amdgpu_ras_debugfs_create(struct amdgpu_device *adev,
head->debugfs_name,
sizeof(obj->fs_data.debugfs_name));
- obj->ent = debugfs_create_file(obj->fs_data.debugfs_name,
- S_IWUGO | S_IRUGO, con->dir, obj,
- &amdgpu_ras_debugfs_ops);
+ debugfs_create_file(obj->fs_data.debugfs_name, S_IWUGO | S_IRUGO, dir,
+ obj, &amdgpu_ras_debugfs_ops);
}
void amdgpu_ras_debugfs_create_all(struct amdgpu_device *adev)
{
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
+ struct dentry *dir;
struct ras_manager *obj;
struct ras_fs_if fs_info;
@@ -1200,7 +1328,7 @@ void amdgpu_ras_debugfs_create_all(struct amdgpu_device *adev)
if (!IS_ENABLED(CONFIG_DEBUG_FS) || !con)
return;
- amdgpu_ras_debugfs_create_ctrl_node(adev);
+ dir = amdgpu_ras_debugfs_create_ctrl_node(adev);
list_for_each_entry(obj, &con->head, node) {
if (amdgpu_ras_is_supported(adev, obj->head.block) &&
@@ -1208,34 +1336,11 @@ void amdgpu_ras_debugfs_create_all(struct amdgpu_device *adev)
sprintf(fs_info.debugfs_name, "%s_err_inject",
ras_block_str(obj->head.block));
fs_info.head = obj->head;
- amdgpu_ras_debugfs_create(adev, &fs_info);
+ amdgpu_ras_debugfs_create(adev, &fs_info, dir);
}
}
}
-static void amdgpu_ras_debugfs_remove(struct amdgpu_device *adev,
- struct ras_common_if *head)
-{
- struct ras_manager *obj = amdgpu_ras_find_obj(adev, head);
-
- if (!obj || !obj->ent)
- return;
-
- obj->ent = NULL;
- put_obj(obj);
-}
-
-static void amdgpu_ras_debugfs_remove_all(struct amdgpu_device *adev)
-{
- struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
- struct ras_manager *obj, *tmp;
-
- list_for_each_entry_safe(obj, tmp, &con->head, node) {
- amdgpu_ras_debugfs_remove(adev, &obj->head);
- }
-
- con->dir = NULL;
-}
/* debugfs end */
/* ras fs */
@@ -1282,8 +1387,17 @@ static int amdgpu_ras_fs_init(struct amdgpu_device *adev)
static int amdgpu_ras_fs_fini(struct amdgpu_device *adev)
{
- if (IS_ENABLED(CONFIG_DEBUG_FS))
- amdgpu_ras_debugfs_remove_all(adev);
+ struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
+ struct ras_manager *con_obj, *ip_obj, *tmp;
+
+ if (IS_ENABLED(CONFIG_DEBUG_FS)) {
+ list_for_each_entry_safe(con_obj, tmp, &con->head, node) {
+ ip_obj = amdgpu_ras_find_obj(adev, &con_obj->head);
+ if (ip_obj)
+ put_obj(ip_obj);
+ }
+ }
+
amdgpu_ras_sysfs_remove_all(adev);
return 0;
}
@@ -1447,7 +1561,7 @@ static void amdgpu_ras_log_on_err_counter(struct amdgpu_device *adev)
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
struct ras_manager *obj;
- if (!con)
+ if (!adev->ras_features || !con)
return;
list_for_each_entry(obj, &con->head, node) {
@@ -1464,7 +1578,7 @@ static void amdgpu_ras_log_on_err_counter(struct amdgpu_device *adev)
if (info.head.block == AMDGPU_RAS_BLOCK__PCIE_BIF)
continue;
- amdgpu_ras_error_query(adev, &info);
+ amdgpu_ras_query_error_status(adev, &info);
}
}
@@ -1478,12 +1592,14 @@ static void amdgpu_ras_error_status_query(struct amdgpu_device *adev,
*/
switch (info->head.block) {
case AMDGPU_RAS_BLOCK__GFX:
- if (adev->gfx.funcs->query_ras_error_status)
- adev->gfx.funcs->query_ras_error_status(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->query_ras_error_status)
+ adev->gfx.ras_funcs->query_ras_error_status(adev);
break;
case AMDGPU_RAS_BLOCK__MMHUB:
- if (adev->mmhub.funcs->query_ras_error_status)
- adev->mmhub.funcs->query_ras_error_status(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->query_ras_error_status)
+ adev->mmhub.ras_funcs->query_ras_error_status(adev);
break;
default:
break;
@@ -1495,7 +1611,7 @@ static void amdgpu_ras_query_err_status(struct amdgpu_device *adev)
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
struct ras_manager *obj;
- if (!con)
+ if (!adev->ras_features || !con)
return;
list_for_each_entry(obj, &con->head, node) {
@@ -1809,7 +1925,7 @@ int amdgpu_ras_recovery_init(struct amdgpu_device *adev)
bool exc_err_limit = false;
int ret;
- if (con)
+ if (adev->ras_features && con)
data = &con->eh_data;
else
return 0;
@@ -1828,6 +1944,12 @@ int amdgpu_ras_recovery_init(struct amdgpu_device *adev)
max_eeprom_records_len = amdgpu_ras_eeprom_get_record_max_length();
amdgpu_ras_validate_threshold(adev, max_eeprom_records_len);
+ /* Todo: During test the SMU might fail to read the eeprom through I2C
+ * when the GPU is pending on XGMI reset during probe time
+ * (Mostly after second bus reset), skip it now
+ */
+ if (adev->gmc.xgmi.pending_reset)
+ return 0;
ret = amdgpu_ras_eeprom_init(&con->eeprom_control, &exc_err_limit);
/*
* This calling fails when exc_err_limit is true or
@@ -1897,15 +2019,13 @@ int amdgpu_ras_request_reset_on_boot(struct amdgpu_device *adev,
return 0;
}
-static int amdgpu_ras_check_asic_type(struct amdgpu_device *adev)
+static bool amdgpu_ras_asic_supported(struct amdgpu_device *adev)
{
- if (adev->asic_type != CHIP_VEGA10 &&
- adev->asic_type != CHIP_VEGA20 &&
- adev->asic_type != CHIP_ARCTURUS &&
- adev->asic_type != CHIP_SIENNA_CICHLID)
- return 1;
- else
- return 0;
+ return adev->asic_type == CHIP_VEGA10 ||
+ adev->asic_type == CHIP_VEGA20 ||
+ adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN ||
+ adev->asic_type == CHIP_SIENNA_CICHLID;
}
/*
@@ -1924,22 +2044,32 @@ static void amdgpu_ras_check_supported(struct amdgpu_device *adev,
*supported = 0;
if (amdgpu_sriov_vf(adev) || !adev->is_atom_fw ||
- amdgpu_ras_check_asic_type(adev))
+ !amdgpu_ras_asic_supported(adev))
return;
- if (amdgpu_atomfirmware_mem_ecc_supported(adev)) {
- dev_info(adev->dev, "HBM ECC is active.\n");
- *hw_supported |= (1 << AMDGPU_RAS_BLOCK__UMC |
- 1 << AMDGPU_RAS_BLOCK__DF);
- } else
- dev_info(adev->dev, "HBM ECC is not presented.\n");
+ if (!adev->gmc.xgmi.connected_to_cpu) {
+ if (amdgpu_atomfirmware_mem_ecc_supported(adev)) {
+ dev_info(adev->dev, "MEM ECC is active.\n");
+ *hw_supported |= (1 << AMDGPU_RAS_BLOCK__UMC |
+ 1 << AMDGPU_RAS_BLOCK__DF);
+ } else {
+ dev_info(adev->dev, "MEM ECC is not presented.\n");
+ }
- if (amdgpu_atomfirmware_sram_ecc_supported(adev)) {
- dev_info(adev->dev, "SRAM ECC is active.\n");
- *hw_supported |= ~(1 << AMDGPU_RAS_BLOCK__UMC |
- 1 << AMDGPU_RAS_BLOCK__DF);
- } else
- dev_info(adev->dev, "SRAM ECC is not presented.\n");
+ if (amdgpu_atomfirmware_sram_ecc_supported(adev)) {
+ dev_info(adev->dev, "SRAM ECC is active.\n");
+ *hw_supported |= ~(1 << AMDGPU_RAS_BLOCK__UMC |
+ 1 << AMDGPU_RAS_BLOCK__DF);
+ } else {
+ dev_info(adev->dev, "SRAM ECC is not presented.\n");
+ }
+ } else {
+ /* driver only manages a few IP blocks RAS feature
+ * when GPU is connected cpu through XGMI */
+ *hw_supported |= (1 << AMDGPU_RAS_BLOCK__GFX |
+ 1 << AMDGPU_RAS_BLOCK__SDMA |
+ 1 << AMDGPU_RAS_BLOCK__MMHUB);
+ }
/* hw_supported needs to be aligned with RAS block mask. */
*hw_supported &= AMDGPU_RAS_BLOCK_MASK;
@@ -1970,6 +2100,15 @@ int amdgpu_ras_init(struct amdgpu_device *adev)
amdgpu_ras_check_supported(adev, &con->hw_supported,
&con->supported);
if (!con->hw_supported || (adev->asic_type == CHIP_VEGA10)) {
+ /* set gfx block ras context feature for VEGA20 Gaming
+ * send ras disable cmd to ras ta during ras late init.
+ */
+ if (!adev->ras_features && adev->asic_type == CHIP_VEGA20) {
+ con->features |= BIT(AMDGPU_RAS_BLOCK__GFX);
+
+ return 0;
+ }
+
r = 0;
goto release_con;
}
@@ -1979,14 +2118,31 @@ int amdgpu_ras_init(struct amdgpu_device *adev)
/* Might need get this flag from vbios. */
con->flags = RAS_DEFAULT_FLAGS;
- if (adev->nbio.funcs->init_ras_controller_interrupt) {
- r = adev->nbio.funcs->init_ras_controller_interrupt(adev);
+ /* initialize nbio ras function ahead of any other
+ * ras functions so hardware fatal error interrupt
+ * can be enabled as early as possible */
+ switch (adev->asic_type) {
+ case CHIP_VEGA20:
+ case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
+ if (!adev->gmc.xgmi.connected_to_cpu)
+ adev->nbio.ras_funcs = &nbio_v7_4_ras_funcs;
+ break;
+ default:
+ /* nbio ras is not available */
+ break;
+ }
+
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_controller_interrupt) {
+ r = adev->nbio.ras_funcs->init_ras_controller_interrupt(adev);
if (r)
goto release_con;
}
- if (adev->nbio.funcs->init_ras_err_event_athub_interrupt) {
- r = adev->nbio.funcs->init_ras_err_event_athub_interrupt(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt) {
+ r = adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt(adev);
if (r)
goto release_con;
}
@@ -2007,6 +2163,32 @@ release_con:
return r;
}
+static int amdgpu_persistent_edc_harvesting_supported(struct amdgpu_device *adev)
+{
+ if (adev->gmc.xgmi.connected_to_cpu)
+ return 1;
+ return 0;
+}
+
+static int amdgpu_persistent_edc_harvesting(struct amdgpu_device *adev,
+ struct ras_common_if *ras_block)
+{
+ struct ras_query_if info = {
+ .head = *ras_block,
+ };
+
+ if (!amdgpu_persistent_edc_harvesting_supported(adev))
+ return 0;
+
+ if (amdgpu_ras_query_error_status(adev, &info) != 0)
+ DRM_WARN("RAS init harvest failure");
+
+ if (amdgpu_ras_reset_error_status(adev, ras_block->block) != 0)
+ DRM_WARN("RAS init harvest reset failure");
+
+ return 0;
+}
+
/* helper function to handle common stuff in ip late init phase */
int amdgpu_ras_late_init(struct amdgpu_device *adev,
struct ras_common_if *ras_block,
@@ -2036,6 +2218,9 @@ int amdgpu_ras_late_init(struct amdgpu_device *adev,
return r;
}
+ /* check for errors on warm reset edc persisant supported ASIC */
+ amdgpu_persistent_edc_harvesting(adev, ras_block);
+
/* in resume phase, no need to create ras fs node */
if (adev->in_suspend || amdgpu_in_reset(adev))
return 0;
@@ -2083,8 +2268,12 @@ void amdgpu_ras_resume(struct amdgpu_device *adev)
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
struct ras_manager *obj, *tmp;
- if (!con)
+ if (!adev->ras_features || !con) {
+ /* clean ras context for VEGA20 Gaming after send ras disable cmd */
+ amdgpu_release_ras_context(adev);
+
return;
+ }
if (con->flags & AMDGPU_RAS_FLAG_INIT_BY_VBIOS) {
/* Set up all other IPs which are not implemented. There is a
@@ -2125,7 +2314,7 @@ void amdgpu_ras_suspend(struct amdgpu_device *adev)
{
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
- if (!con)
+ if (!adev->ras_features || !con)
return;
amdgpu_ras_disable_all_features(adev, 0);
@@ -2139,7 +2328,7 @@ int amdgpu_ras_pre_fini(struct amdgpu_device *adev)
{
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
- if (!con)
+ if (!adev->ras_features || !con)
return 0;
/* Need disable ras on all IPs here before ip [hw/sw]fini */
@@ -2152,7 +2341,7 @@ int amdgpu_ras_fini(struct amdgpu_device *adev)
{
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
- if (!con)
+ if (!adev->ras_features || !con)
return 0;
amdgpu_ras_fs_fini(adev);
@@ -2196,18 +2385,16 @@ bool amdgpu_ras_need_emergency_restart(struct amdgpu_device *adev)
return false;
}
-bool amdgpu_ras_check_err_threshold(struct amdgpu_device *adev)
+void amdgpu_release_ras_context(struct amdgpu_device *adev)
{
struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
- bool exc_err_limit = false;
- if (con && (amdgpu_bad_page_threshold != 0))
- amdgpu_ras_eeprom_check_err_threshold(&con->eeprom_control,
- &exc_err_limit);
+ if (!con)
+ return;
- /*
- * We are only interested in variable exc_err_limit,
- * as it says if GPU is in bad state or not.
- */
- return exc_err_limit;
+ if (!adev->ras_features && con->features & BIT(AMDGPU_RAS_BLOCK__GFX)) {
+ con->features &= ~BIT(AMDGPU_RAS_BLOCK__GFX);
+ amdgpu_ras_set_context(adev, NULL);
+ kfree(con);
+ }
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h
index 762f5e46c007..60df268a0c66 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras.h
@@ -318,8 +318,6 @@ struct amdgpu_ras {
uint32_t supported;
uint32_t features;
struct list_head head;
- /* debugfs */
- struct dentry *dir;
/* sysfs */
struct device_attribute features_attr;
struct bin_attribute badpages_attr;
@@ -395,8 +393,6 @@ struct ras_manager {
struct list_head node;
/* the device */
struct amdgpu_device *adev;
- /* debugfs */
- struct dentry *ent;
/* sysfs */
struct device_attribute sysfs_attr;
int attr_inuse;
@@ -495,8 +491,6 @@ void amdgpu_ras_suspend(struct amdgpu_device *adev);
unsigned long amdgpu_ras_query_error_count(struct amdgpu_device *adev,
bool is_ce);
-bool amdgpu_ras_check_err_threshold(struct amdgpu_device *adev);
-
/* error handling functions */
int amdgpu_ras_add_bad_pages(struct amdgpu_device *adev,
struct eeprom_table_record *bps, int pages);
@@ -594,9 +588,12 @@ int amdgpu_ras_sysfs_remove(struct amdgpu_device *adev,
void amdgpu_ras_debugfs_create_all(struct amdgpu_device *adev);
-int amdgpu_ras_error_query(struct amdgpu_device *adev,
+int amdgpu_ras_query_error_status(struct amdgpu_device *adev,
struct ras_query_if *info);
+int amdgpu_ras_reset_error_status(struct amdgpu_device *adev,
+ enum amdgpu_ras_block block);
+
int amdgpu_ras_error_inject(struct amdgpu_device *adev,
struct ras_inject_if *info);
@@ -629,4 +626,6 @@ void amdgpu_ras_global_ras_isr(struct amdgpu_device *adev);
void amdgpu_ras_set_error_query_ready(struct amdgpu_device *adev, bool ready);
bool amdgpu_ras_need_emergency_restart(struct amdgpu_device *adev);
+
+void amdgpu_release_ras_context(struct amdgpu_device *adev);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c
index 19d9aa76cfbf..f40c871da0c6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.c
@@ -31,6 +31,7 @@
#define EEPROM_I2C_TARGET_ADDR_ARCTURUS 0xA8
#define EEPROM_I2C_TARGET_ADDR_ARCTURUS_D342 0xA0
#define EEPROM_I2C_TARGET_ADDR_SIENNA_CICHLID 0xA0
+#define EEPROM_I2C_TARGET_ADDR_ALDEBARAN 0xA0
/*
* The 2 macros bellow represent the actual size in bytes that
@@ -64,7 +65,8 @@ static bool __is_ras_eeprom_supported(struct amdgpu_device *adev)
{
if ((adev->asic_type == CHIP_VEGA20) ||
(adev->asic_type == CHIP_ARCTURUS) ||
- (adev->asic_type == CHIP_SIENNA_CICHLID))
+ (adev->asic_type == CHIP_SIENNA_CICHLID) ||
+ (adev->asic_type == CHIP_ALDEBARAN))
return true;
return false;
@@ -106,6 +108,10 @@ static bool __get_eeprom_i2c_addr(struct amdgpu_device *adev,
*i2c_addr = EEPROM_I2C_TARGET_ADDR_SIENNA_CICHLID;
break;
+ case CHIP_ALDEBARAN:
+ *i2c_addr = EEPROM_I2C_TARGET_ADDR_ALDEBARAN;
+ break;
+
default:
return false;
}
@@ -434,47 +440,28 @@ static uint32_t __correct_eeprom_dest_address(uint32_t curr_address)
return curr_address;
}
-int amdgpu_ras_eeprom_check_err_threshold(
- struct amdgpu_ras_eeprom_control *control,
- bool *exceed_err_limit)
+bool amdgpu_ras_eeprom_check_err_threshold(struct amdgpu_device *adev)
{
- struct amdgpu_device *adev = to_amdgpu_device(control);
- unsigned char buff[EEPROM_ADDRESS_SIZE +
- EEPROM_TABLE_HEADER_SIZE] = { 0 };
- struct amdgpu_ras_eeprom_table_header *hdr = &control->tbl_hdr;
- struct i2c_msg msg = {
- .addr = control->i2c_address,
- .flags = I2C_M_RD,
- .len = EEPROM_ADDRESS_SIZE + EEPROM_TABLE_HEADER_SIZE,
- .buf = buff,
- };
- int ret;
-
- *exceed_err_limit = false;
+ struct amdgpu_ras *con = amdgpu_ras_get_context(adev);
if (!__is_ras_eeprom_supported(adev))
- return 0;
-
- /* read EEPROM table header */
- mutex_lock(&control->tbl_mutex);
- ret = i2c_transfer(&adev->pm.smu_i2c, &msg, 1);
- if (ret < 1) {
- dev_err(adev->dev, "Failed to read EEPROM table header.\n");
- goto err;
- }
+ return false;
- __decode_table_header_from_buff(hdr, &buff[2]);
+ /* skip check eeprom table for VEGA20 Gaming */
+ if (!con)
+ return false;
+ else
+ if (!(con->features & BIT(AMDGPU_RAS_BLOCK__UMC)))
+ return false;
- if (hdr->header == EEPROM_TABLE_HDR_BAD) {
+ if (con->eeprom_control.tbl_hdr.header == EEPROM_TABLE_HDR_BAD) {
dev_warn(adev->dev, "This GPU is in BAD status.");
dev_warn(adev->dev, "Please retire it or setting one bigger "
"threshold value when reloading driver.\n");
- *exceed_err_limit = true;
+ return true;
}
-err:
- mutex_unlock(&control->tbl_mutex);
- return 0;
+ return false;
}
int amdgpu_ras_eeprom_process_recods(struct amdgpu_ras_eeprom_control *control,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h
index c7a5e5c7c61e..178721170974 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ras_eeprom.h
@@ -80,9 +80,7 @@ int amdgpu_ras_eeprom_init(struct amdgpu_ras_eeprom_control *control,
bool *exceed_err_limit);
int amdgpu_ras_eeprom_reset_table(struct amdgpu_ras_eeprom_control *control);
-int amdgpu_ras_eeprom_check_err_threshold(
- struct amdgpu_ras_eeprom_control *control,
- bool *exceed_err_limit);
+bool amdgpu_ras_eeprom_check_err_threshold(struct amdgpu_device *adev);
int amdgpu_ras_eeprom_process_recods(struct amdgpu_ras_eeprom_control *control,
struct eeprom_table_record *records,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
new file mode 100644
index 000000000000..40f2adf305bc
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_res_cursor.h
@@ -0,0 +1,105 @@
+// SPDX-License-Identifier: GPL-2.0 OR MIT
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: Christian König
+ */
+
+#ifndef __AMDGPU_RES_CURSOR_H__
+#define __AMDGPU_RES_CURSOR_H__
+
+#include <drm/drm_mm.h>
+#include <drm/ttm/ttm_resource.h>
+
+/* state back for walking over vram_mgr and gtt_mgr allocations */
+struct amdgpu_res_cursor {
+ uint64_t start;
+ uint64_t size;
+ uint64_t remaining;
+ struct drm_mm_node *node;
+};
+
+/**
+ * amdgpu_res_first - initialize a amdgpu_res_cursor
+ *
+ * @res: TTM resource object to walk
+ * @start: Start of the range
+ * @size: Size of the range
+ * @cur: cursor object to initialize
+ *
+ * Start walking over the range of allocations between @start and @size.
+ */
+static inline void amdgpu_res_first(struct ttm_resource *res,
+ uint64_t start, uint64_t size,
+ struct amdgpu_res_cursor *cur)
+{
+ struct drm_mm_node *node;
+
+ if (!res || !res->mm_node) {
+ cur->start = start;
+ cur->size = size;
+ cur->remaining = size;
+ cur->node = NULL;
+ return;
+ }
+
+ BUG_ON(start + size > res->num_pages << PAGE_SHIFT);
+
+ node = res->mm_node;
+ while (start >= node->size << PAGE_SHIFT)
+ start -= node++->size << PAGE_SHIFT;
+
+ cur->start = (node->start << PAGE_SHIFT) + start;
+ cur->size = min((node->size << PAGE_SHIFT) - start, size);
+ cur->remaining = size;
+ cur->node = node;
+}
+
+/**
+ * amdgpu_res_next - advance the cursor
+ *
+ * @cur: the cursor to advance
+ * @size: number of bytes to move forward
+ *
+ * Move the cursor @size bytes forwrad, walking to the next node if necessary.
+ */
+static inline void amdgpu_res_next(struct amdgpu_res_cursor *cur, uint64_t size)
+{
+ struct drm_mm_node *node = cur->node;
+
+ BUG_ON(size > cur->remaining);
+
+ cur->remaining -= size;
+ if (!cur->remaining)
+ return;
+
+ cur->size -= size;
+ if (cur->size) {
+ cur->start += size;
+ return;
+ }
+
+ cur->node = ++node;
+ cur->start = node->start << PAGE_SHIFT;
+ cur->size = min(node->size << PAGE_SHIFT, cur->remaining);
+}
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
new file mode 100644
index 000000000000..02afd4115675
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.c
@@ -0,0 +1,98 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#include "amdgpu_reset.h"
+#include "aldebaran.h"
+
+int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_handler *handler)
+{
+ /* TODO: Check if handler exists? */
+ list_add_tail(&handler->handler_list, &reset_ctl->reset_handlers);
+ return 0;
+}
+
+int amdgpu_reset_init(struct amdgpu_device *adev)
+{
+ int ret = 0;
+
+ switch (adev->asic_type) {
+ case CHIP_ALDEBARAN:
+ ret = aldebaran_reset_init(adev);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+int amdgpu_reset_fini(struct amdgpu_device *adev)
+{
+ int ret = 0;
+
+ switch (adev->asic_type) {
+ case CHIP_ALDEBARAN:
+ ret = aldebaran_reset_fini(adev);
+ break;
+ default:
+ break;
+ }
+
+ return ret;
+}
+
+int amdgpu_reset_prepare_hwcontext(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context)
+{
+ struct amdgpu_reset_handler *reset_handler = NULL;
+
+ if (adev->reset_cntl && adev->reset_cntl->get_reset_handler)
+ reset_handler = adev->reset_cntl->get_reset_handler(
+ adev->reset_cntl, reset_context);
+ if (!reset_handler)
+ return -ENOSYS;
+
+ return reset_handler->prepare_hwcontext(adev->reset_cntl,
+ reset_context);
+}
+
+int amdgpu_reset_perform_reset(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context)
+{
+ int ret;
+ struct amdgpu_reset_handler *reset_handler = NULL;
+
+ if (adev->reset_cntl)
+ reset_handler = adev->reset_cntl->get_reset_handler(
+ adev->reset_cntl, reset_context);
+ if (!reset_handler)
+ return -ENOSYS;
+
+ ret = reset_handler->perform_reset(adev->reset_cntl, reset_context);
+ if (ret)
+ return ret;
+
+ return reset_handler->restore_hwcontext(adev->reset_cntl,
+ reset_context);
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.h
new file mode 100644
index 000000000000..e00d38d9160a
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_reset.h
@@ -0,0 +1,85 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __AMDGPU_RESET_H__
+#define __AMDGPU_RESET_H__
+
+#include "amdgpu.h"
+
+enum AMDGPU_RESET_FLAGS {
+
+ AMDGPU_NEED_FULL_RESET = 0,
+ AMDGPU_SKIP_HW_RESET = 1,
+};
+
+struct amdgpu_reset_context {
+ enum amd_reset_method method;
+ struct amdgpu_device *reset_req_dev;
+ struct amdgpu_job *job;
+ struct amdgpu_hive_info *hive;
+ unsigned long flags;
+};
+
+struct amdgpu_reset_handler {
+ enum amd_reset_method reset_method;
+ struct list_head handler_list;
+ int (*prepare_env)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*prepare_hwcontext)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*perform_reset)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*restore_hwcontext)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ int (*restore_env)(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+
+ int (*do_reset)(struct amdgpu_device *adev);
+};
+
+struct amdgpu_reset_control {
+ void *handle;
+ struct work_struct reset_work;
+ struct mutex reset_lock;
+ struct list_head reset_handlers;
+ atomic_t in_reset;
+ enum amd_reset_method active_reset;
+ struct amdgpu_reset_handler *(*get_reset_handler)(
+ struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_context *context);
+ void (*async_reset)(struct work_struct *work);
+};
+
+int amdgpu_reset_init(struct amdgpu_device *adev);
+int amdgpu_reset_fini(struct amdgpu_device *adev);
+
+int amdgpu_reset_prepare_hwcontext(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context);
+
+int amdgpu_reset_perform_reset(struct amdgpu_device *adev,
+ struct amdgpu_reset_context *reset_context);
+
+int amdgpu_reset_add_handler(struct amdgpu_reset_control *reset_ctl,
+ struct amdgpu_reset_handler *handler);
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
index b644c78475fd..688624ebe421 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.c
@@ -164,7 +164,8 @@ void amdgpu_ring_undo(struct amdgpu_ring *ring)
*/
int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
unsigned int max_dw, struct amdgpu_irq_src *irq_src,
- unsigned int irq_type, unsigned int hw_prio)
+ unsigned int irq_type, unsigned int hw_prio,
+ atomic_t *sched_score)
{
int r;
int sched_hw_submission = amdgpu_sched_hw_submission;
@@ -189,7 +190,8 @@ int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
ring->adev = adev;
ring->idx = adev->num_rings++;
adev->rings[ring->idx] = ring;
- r = amdgpu_fence_driver_init_ring(ring, sched_hw_submission);
+ r = amdgpu_fence_driver_init_ring(ring, sched_hw_submission,
+ sched_score);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
index 56acec1075ac..ca1622835296 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h
@@ -111,7 +111,8 @@ void amdgpu_fence_driver_fini(struct amdgpu_device *adev);
void amdgpu_fence_driver_force_completion(struct amdgpu_ring *ring);
int amdgpu_fence_driver_init_ring(struct amdgpu_ring *ring,
- unsigned num_hw_submission);
+ unsigned num_hw_submission,
+ atomic_t *sched_score);
int amdgpu_fence_driver_start_ring(struct amdgpu_ring *ring,
struct amdgpu_irq_src *irq_src,
unsigned irq_type);
@@ -282,7 +283,8 @@ void amdgpu_ring_commit(struct amdgpu_ring *ring);
void amdgpu_ring_undo(struct amdgpu_ring *ring);
int amdgpu_ring_init(struct amdgpu_device *adev, struct amdgpu_ring *ring,
unsigned int ring_size, struct amdgpu_irq_src *irq_src,
- unsigned int irq_type, unsigned int prio);
+ unsigned int irq_type, unsigned int prio,
+ atomic_t *sched_score);
void amdgpu_ring_fini(struct amdgpu_ring *ring);
void amdgpu_ring_emit_reg_write_reg_wait_helper(struct amdgpu_ring *ring,
uint32_t reg0, uint32_t val0,
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_rlc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_rlc.h
index aeaaae713c59..4fc2ce8ce8ab 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_rlc.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_rlc.h
@@ -127,7 +127,8 @@ struct amdgpu_rlc_funcs {
void (*reset)(struct amdgpu_device *adev);
void (*start)(struct amdgpu_device *adev);
void (*update_spm_vmid)(struct amdgpu_device *adev, unsigned vmid);
- void (*rlcg_wreg)(struct amdgpu_device *adev, u32 offset, u32 v);
+ void (*rlcg_wreg)(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag);
+ u32 (*rlcg_rreg)(struct amdgpu_device *adev, u32 offset, u32 flag);
bool (*is_rlcg_access_range)(struct amdgpu_device *adev, uint32_t reg);
};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h
index e5b8fb8e75c5..f8fb755e3aa6 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h
@@ -64,6 +64,11 @@ struct amdgpu_sdma {
struct amdgpu_irq_src trap_irq;
struct amdgpu_irq_src illegal_inst_irq;
struct amdgpu_irq_src ecc_irq;
+ struct amdgpu_irq_src vm_hole_irq;
+ struct amdgpu_irq_src doorbell_invalid_irq;
+ struct amdgpu_irq_src pool_timeout_irq;
+ struct amdgpu_irq_src srbm_write_irq;
+
int num_instances;
uint32_t srbm_soft_reset;
bool has_page_queue;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c
index 834440ab9ff7..5369c8dd0764 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_securedisplay.c
@@ -69,6 +69,9 @@ void psp_securedisplay_parse_resp_status(struct psp_context *psp,
case TA_SECUREDISPLAY_STATUS__READ_CRC_ERROR:
dev_err(psp->adev->dev, "Secure display: Failed to Read CRC");
break;
+ case TA_SECUREDISPLAY_STATUS__I2C_INIT_ERROR:
+ dev_err(psp->adev->dev, "Secure display: Failed to initialize I2C.");
+ break;
default:
dev_err(psp->adev->dev, "Secure display: Failed to parse status: %d\n", status);
}
@@ -92,9 +95,7 @@ static ssize_t amdgpu_securedisplay_debugfs_write(struct file *f, const char __u
struct drm_device *dev = adev_to_drm(adev);
uint32_t phy_id;
uint32_t op;
- int i;
char str[64];
- char i2c_output[256];
int ret;
if (*pos || size > sizeof(str) - 1)
@@ -136,11 +137,9 @@ static ssize_t amdgpu_securedisplay_debugfs_write(struct file *f, const char __u
ret = psp_securedisplay_invoke(psp, TA_SECUREDISPLAY_COMMAND__SEND_ROI_CRC);
if (!ret) {
if (securedisplay_cmd->status == TA_SECUREDISPLAY_STATUS__SUCCESS) {
- memset(i2c_output, 0, sizeof(i2c_output));
- for (i = 0; i < TA_SECUREDISPLAY_I2C_BUFFER_SIZE; i++)
- sprintf(i2c_output, "%s 0x%X", i2c_output,
- securedisplay_cmd->securedisplay_out_message.send_roi_crc.i2c_buf[i]);
- dev_info(adev->dev, "SECUREDISPLAY: I2C buffer out put is :%s\n", i2c_output);
+ dev_info(adev->dev, "SECUREDISPLAY: I2C buffer out put is: %*ph\n",
+ TA_SECUREDISPLAY_I2C_BUFFER_SIZE,
+ securedisplay_cmd->securedisplay_out_message.send_roi_crc.i2c_buf);
} else {
psp_securedisplay_parse_resp_status(psp, securedisplay_cmd->status);
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_smuio.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_smuio.h
index 03009157aec8..b860ec913ac5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_smuio.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_smuio.h
@@ -28,6 +28,8 @@ struct amdgpu_smuio_funcs {
u32 (*get_rom_data_offset)(struct amdgpu_device *adev);
void (*update_rom_clock_gating)(struct amdgpu_device *adev, bool enable);
void (*get_clock_gating_state)(struct amdgpu_device *adev, u32 *flags);
+ u32 (*get_die_id)(struct amdgpu_device *adev);
+ bool (*is_host_gpu_xgmi_supported)(struct amdgpu_device *adev);
};
struct amdgpu_smuio {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_test.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_test.c
index 7b230bcbf2c6..909d830b513e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_test.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_test.c
@@ -62,6 +62,7 @@ static void amdgpu_do_test_moves(struct amdgpu_device *adev)
bp.flags = 0;
bp.type = ttm_bo_type_kernel;
bp.resv = NULL;
+ bp.bo_ptr_size = sizeof(struct amdgpu_bo);
r = amdgpu_bo_create(adev, &bp, &vram_obj);
if (r) {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
index 9fd2157b133a..3bef0432cac2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
@@ -47,7 +47,6 @@
#include <drm/ttm/ttm_bo_driver.h>
#include <drm/ttm/ttm_placement.h>
-#include <drm/drm_debugfs.h>
#include <drm/amdgpu_drm.h>
#include "amdgpu.h"
@@ -57,14 +56,15 @@
#include "amdgpu_sdma.h"
#include "amdgpu_ras.h"
#include "amdgpu_atomfirmware.h"
+#include "amdgpu_res_cursor.h"
#include "bif/bif_4_1_d.h"
#define AMDGPU_TTM_VRAM_MAX_DW_READ (size_t)128
-static int amdgpu_ttm_backend_bind(struct ttm_bo_device *bdev,
+static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
struct ttm_tt *ttm,
struct ttm_resource *bo_mem);
-static void amdgpu_ttm_backend_unbind(struct ttm_bo_device *bdev,
+static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
struct ttm_tt *ttm);
static int amdgpu_ttm_init_on_chip(struct amdgpu_device *adev,
@@ -179,54 +179,11 @@ static int amdgpu_verify_access(struct ttm_buffer_object *bo, struct file *filp)
}
/**
- * amdgpu_mm_node_addr - Compute the GPU relative offset of a GTT buffer.
- *
- * @bo: The bo to assign the memory to.
- * @mm_node: Memory manager node for drm allocator.
- * @mem: The region where the bo resides.
- *
- */
-static uint64_t amdgpu_mm_node_addr(struct ttm_buffer_object *bo,
- struct drm_mm_node *mm_node,
- struct ttm_resource *mem)
-{
- uint64_t addr = 0;
-
- if (mm_node->start != AMDGPU_BO_INVALID_OFFSET) {
- addr = mm_node->start << PAGE_SHIFT;
- addr += amdgpu_ttm_domain_start(amdgpu_ttm_adev(bo->bdev),
- mem->mem_type);
- }
- return addr;
-}
-
-/**
- * amdgpu_find_mm_node - Helper function finds the drm_mm_node corresponding to
- * @offset. It also modifies the offset to be within the drm_mm_node returned
- *
- * @mem: The region where the bo resides.
- * @offset: The offset that drm_mm_node is used for finding.
- *
- */
-static struct drm_mm_node *amdgpu_find_mm_node(struct ttm_resource *mem,
- uint64_t *offset)
-{
- struct drm_mm_node *mm_node = mem->mm_node;
-
- while (*offset >= (mm_node->size << PAGE_SHIFT)) {
- *offset -= (mm_node->size << PAGE_SHIFT);
- ++mm_node;
- }
- return mm_node;
-}
-
-/**
* amdgpu_ttm_map_buffer - Map memory into the GART windows
* @bo: buffer object to map
* @mem: memory object to map
- * @mm_node: drm_mm node object to map
+ * @mm_cur: range to map
* @num_pages: number of pages to map
- * @offset: offset into @mm_node where to start
* @window: which GART window to use
* @ring: DMA ring to use for the copy
* @tmz: if we should setup a TMZ enabled mapping
@@ -237,10 +194,10 @@ static struct drm_mm_node *amdgpu_find_mm_node(struct ttm_resource *mem,
*/
static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,
struct ttm_resource *mem,
- struct drm_mm_node *mm_node,
- unsigned num_pages, uint64_t offset,
- unsigned window, struct amdgpu_ring *ring,
- bool tmz, uint64_t *addr)
+ struct amdgpu_res_cursor *mm_cur,
+ unsigned num_pages, unsigned window,
+ struct amdgpu_ring *ring, bool tmz,
+ uint64_t *addr)
{
struct amdgpu_device *adev = ring->adev;
struct amdgpu_job *job;
@@ -257,14 +214,15 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,
/* Map only what can't be accessed directly */
if (!tmz && mem->start != AMDGPU_BO_INVALID_OFFSET) {
- *addr = amdgpu_mm_node_addr(bo, mm_node, mem) + offset;
+ *addr = amdgpu_ttm_domain_start(adev, mem->mem_type) +
+ mm_cur->start;
return 0;
}
*addr = adev->gmc.gart_start;
*addr += (u64)window * AMDGPU_GTT_MAX_TRANSFER_SIZE *
AMDGPU_GPU_PAGE_SIZE;
- *addr += offset & ~PAGE_MASK;
+ *addr += mm_cur->start & ~PAGE_MASK;
num_dw = ALIGN(adev->mman.buffer_funcs->copy_num_dw, 8);
num_bytes = num_pages * 8;
@@ -292,17 +250,17 @@ static int amdgpu_ttm_map_buffer(struct ttm_buffer_object *bo,
cpu_addr = &job->ibs[0].ptr[num_dw];
if (mem->mem_type == TTM_PL_TT) {
- dma_addr_t *dma_address;
+ dma_addr_t *dma_addr;
- dma_address = &bo->ttm->dma_address[offset >> PAGE_SHIFT];
- r = amdgpu_gart_map(adev, 0, num_pages, dma_address, flags,
+ dma_addr = &bo->ttm->dma_address[mm_cur->start >> PAGE_SHIFT];
+ r = amdgpu_gart_map(adev, 0, num_pages, dma_addr, flags,
cpu_addr);
if (r)
goto error_free;
} else {
dma_addr_t dma_address;
- dma_address = (mm_node->start << PAGE_SHIFT) + offset;
+ dma_address = mm_cur->start;
dma_address += adev->vm_manager.vram_base_offset;
for (i = 0; i < num_pages; ++i) {
@@ -354,9 +312,8 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
const uint32_t GTT_MAX_BYTES = (AMDGPU_GTT_MAX_TRANSFER_SIZE *
AMDGPU_GPU_PAGE_SIZE);
- uint64_t src_node_size, dst_node_size, src_offset, dst_offset;
struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
- struct drm_mm_node *src_mm, *dst_mm;
+ struct amdgpu_res_cursor src_mm, dst_mm;
struct dma_fence *fence = NULL;
int r = 0;
@@ -365,29 +322,13 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
return -EINVAL;
}
- src_offset = src->offset;
- if (src->mem->mm_node) {
- src_mm = amdgpu_find_mm_node(src->mem, &src_offset);
- src_node_size = (src_mm->size << PAGE_SHIFT) - src_offset;
- } else {
- src_mm = NULL;
- src_node_size = ULLONG_MAX;
- }
-
- dst_offset = dst->offset;
- if (dst->mem->mm_node) {
- dst_mm = amdgpu_find_mm_node(dst->mem, &dst_offset);
- dst_node_size = (dst_mm->size << PAGE_SHIFT) - dst_offset;
- } else {
- dst_mm = NULL;
- dst_node_size = ULLONG_MAX;
- }
+ amdgpu_res_first(src->mem, src->offset, size, &src_mm);
+ amdgpu_res_first(dst->mem, dst->offset, size, &dst_mm);
mutex_lock(&adev->mman.gtt_window_lock);
-
- while (size) {
- uint32_t src_page_offset = src_offset & ~PAGE_MASK;
- uint32_t dst_page_offset = dst_offset & ~PAGE_MASK;
+ while (src_mm.remaining) {
+ uint32_t src_page_offset = src_mm.start & ~PAGE_MASK;
+ uint32_t dst_page_offset = dst_mm.start & ~PAGE_MASK;
struct dma_fence *next;
uint32_t cur_size;
uint64_t from, to;
@@ -396,19 +337,19 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
* begins at an offset, then adjust the size accordingly
*/
cur_size = max(src_page_offset, dst_page_offset);
- cur_size = min(min3(src_node_size, dst_node_size, size),
+ cur_size = min(min3(src_mm.size, dst_mm.size, size),
(uint64_t)(GTT_MAX_BYTES - cur_size));
/* Map src to window 0 and dst to window 1. */
- r = amdgpu_ttm_map_buffer(src->bo, src->mem, src_mm,
+ r = amdgpu_ttm_map_buffer(src->bo, src->mem, &src_mm,
PFN_UP(cur_size + src_page_offset),
- src_offset, 0, ring, tmz, &from);
+ 0, ring, tmz, &from);
if (r)
goto error;
- r = amdgpu_ttm_map_buffer(dst->bo, dst->mem, dst_mm,
+ r = amdgpu_ttm_map_buffer(dst->bo, dst->mem, &dst_mm,
PFN_UP(cur_size + dst_page_offset),
- dst_offset, 1, ring, tmz, &to);
+ 1, ring, tmz, &to);
if (r)
goto error;
@@ -420,27 +361,8 @@ int amdgpu_ttm_copy_mem_to_mem(struct amdgpu_device *adev,
dma_fence_put(fence);
fence = next;
- size -= cur_size;
- if (!size)
- break;
-
- src_node_size -= cur_size;
- if (!src_node_size) {
- ++src_mm;
- src_node_size = src_mm->size << PAGE_SHIFT;
- src_offset = 0;
- } else {
- src_offset += cur_size;
- }
-
- dst_node_size -= cur_size;
- if (!dst_node_size) {
- ++dst_mm;
- dst_node_size = dst_mm->size << PAGE_SHIFT;
- dst_offset = 0;
- } else {
- dst_offset += cur_size;
- }
+ amdgpu_res_next(&src_mm, cur_size);
+ amdgpu_res_next(&dst_mm, cur_size);
}
error:
mutex_unlock(&adev->mman.gtt_window_lock);
@@ -519,7 +441,8 @@ error:
static bool amdgpu_mem_visible(struct amdgpu_device *adev,
struct ttm_resource *mem)
{
- struct drm_mm_node *nodes = mem->mm_node;
+ uint64_t mem_size = (u64)mem->num_pages << PAGE_SHIFT;
+ struct amdgpu_res_cursor cursor;
if (mem->mem_type == TTM_PL_SYSTEM ||
mem->mem_type == TTM_PL_TT)
@@ -527,12 +450,13 @@ static bool amdgpu_mem_visible(struct amdgpu_device *adev,
if (mem->mem_type != TTM_PL_VRAM)
return false;
+ amdgpu_res_first(mem, 0, mem_size, &cursor);
+
/* ttm_resource_ioremap only supports contiguous memory */
- if (nodes->size != mem->num_pages)
+ if (cursor.size != mem_size)
return false;
- return ((nodes->start + nodes->size) << PAGE_SHIFT)
- <= adev->gmc.visible_vram_size;
+ return cursor.start + cursor.size <= adev->gmc.visible_vram_size;
}
/*
@@ -646,7 +570,7 @@ out:
*
* Called by ttm_mem_io_reserve() ultimately via ttm_bo_vm_fault()
*/
-static int amdgpu_ttm_io_mem_reserve(struct ttm_bo_device *bdev, struct ttm_resource *mem)
+static int amdgpu_ttm_io_mem_reserve(struct ttm_device *bdev, struct ttm_resource *mem)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
struct drm_mm_node *mm_node = mem->mm_node;
@@ -674,7 +598,10 @@ static int amdgpu_ttm_io_mem_reserve(struct ttm_bo_device *bdev, struct ttm_reso
mem->bus.offset += adev->gmc.aper_base;
mem->bus.is_iomem = true;
- mem->bus.caching = ttm_write_combined;
+ if (adev->gmc.xgmi.connected_to_cpu)
+ mem->bus.caching = ttm_cached;
+ else
+ mem->bus.caching = ttm_write_combined;
break;
default:
return -EINVAL;
@@ -686,12 +613,10 @@ static unsigned long amdgpu_ttm_io_mem_pfn(struct ttm_buffer_object *bo,
unsigned long page_offset)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bo->bdev);
- uint64_t offset = (page_offset << PAGE_SHIFT);
- struct drm_mm_node *mm;
+ struct amdgpu_res_cursor cursor;
- mm = amdgpu_find_mm_node(&bo->mem, &offset);
- offset += adev->gmc.aper_base;
- return mm->start + (offset >> PAGE_SHIFT);
+ amdgpu_res_first(&bo->mem, (u64)page_offset << PAGE_SHIFT, 0, &cursor);
+ return (adev->gmc.aper_base + cursor.start) >> PAGE_SHIFT;
}
/**
@@ -893,20 +818,19 @@ void amdgpu_ttm_tt_set_user_pages(struct ttm_tt *ttm, struct page **pages)
*
* Called by amdgpu_ttm_backend_bind()
**/
-static int amdgpu_ttm_tt_pin_userptr(struct ttm_bo_device *bdev,
+static int amdgpu_ttm_tt_pin_userptr(struct ttm_device *bdev,
struct ttm_tt *ttm)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
struct amdgpu_ttm_tt *gtt = (void *)ttm;
- int r;
-
int write = !(gtt->userflags & AMDGPU_GEM_USERPTR_READONLY);
enum dma_data_direction direction = write ?
DMA_BIDIRECTIONAL : DMA_TO_DEVICE;
+ int r;
/* Allocate an SG array and squash pages into it */
r = sg_alloc_table_from_pages(ttm->sg, ttm->pages, ttm->num_pages, 0,
- ttm->num_pages << PAGE_SHIFT,
+ (u64)ttm->num_pages << PAGE_SHIFT,
GFP_KERNEL);
if (r)
goto release_sg;
@@ -931,18 +855,17 @@ release_sg:
/*
* amdgpu_ttm_tt_unpin_userptr - Unpin and unmap userptr pages
*/
-static void amdgpu_ttm_tt_unpin_userptr(struct ttm_bo_device *bdev,
+static void amdgpu_ttm_tt_unpin_userptr(struct ttm_device *bdev,
struct ttm_tt *ttm)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
struct amdgpu_ttm_tt *gtt = (void *)ttm;
-
int write = !(gtt->userflags & AMDGPU_GEM_USERPTR_READONLY);
enum dma_data_direction direction = write ?
DMA_BIDIRECTIONAL : DMA_TO_DEVICE;
/* double check that we don't free the table twice */
- if (!ttm->sg->sgl)
+ if (!ttm->sg || !ttm->sg->sgl)
return;
/* unmap the pages mapped to the device */
@@ -1015,7 +938,7 @@ gart_bind_fail:
* Called by ttm_tt_bind() on behalf of ttm_bo_handle_move_mem().
* This handles binding GTT memory to the device address space.
*/
-static int amdgpu_ttm_backend_bind(struct ttm_bo_device *bdev,
+static int amdgpu_ttm_backend_bind(struct ttm_device *bdev,
struct ttm_tt *ttm,
struct ttm_resource *bo_mem)
{
@@ -1155,20 +1078,20 @@ int amdgpu_ttm_recover_gart(struct ttm_buffer_object *tbo)
* Called by ttm_tt_unbind() on behalf of ttm_bo_move_ttm() and
* ttm_tt_destroy().
*/
-static void amdgpu_ttm_backend_unbind(struct ttm_bo_device *bdev,
+static void amdgpu_ttm_backend_unbind(struct ttm_device *bdev,
struct ttm_tt *ttm)
{
struct amdgpu_device *adev = amdgpu_ttm_adev(bdev);
struct amdgpu_ttm_tt *gtt = (void *)ttm;
int r;
- if (!gtt->bound)
- return;
-
/* if the pages have userptr pinning then clear that first */
if (gtt->userptr)
amdgpu_ttm_tt_unpin_userptr(bdev, ttm);
+ if (!gtt->bound)
+ return;
+
if (gtt->offset == AMDGPU_BO_INVALID_OFFSET)
return;
@@ -1180,7 +1103,7 @@ static void amdgpu_ttm_backend_unbind(struct ttm_bo_device *bdev,
gtt->bound = false;
}
-static void amdgpu_ttm_backend_destroy(struct ttm_bo_device *bdev,
+static void amdgpu_ttm_backend_destroy(struct ttm_device *bdev,
struct ttm_tt *ttm)
{
struct amdgpu_ttm_tt *gtt = (void *)ttm;
@@ -1234,7 +1157,7 @@ static struct ttm_tt *amdgpu_ttm_tt_create(struct ttm_buffer_object *bo,
* Map the pages of a ttm_tt object to an address space visible
* to the underlying device.
*/
-static int amdgpu_ttm_tt_populate(struct ttm_bo_device *bdev,
+static int amdgpu_ttm_tt_populate(struct ttm_device *bdev,
struct ttm_tt *ttm,
struct ttm_operation_ctx *ctx)
{
@@ -1278,7 +1201,7 @@ static int amdgpu_ttm_tt_populate(struct ttm_bo_device *bdev,
* Unmaps pages of a ttm_tt object from the device address space and
* unpopulates the page array backing it.
*/
-static void amdgpu_ttm_tt_unpopulate(struct ttm_bo_device *bdev,
+static void amdgpu_ttm_tt_unpopulate(struct ttm_device *bdev,
struct ttm_tt *ttm)
{
struct amdgpu_ttm_tt *gtt = (void *)ttm;
@@ -1430,6 +1353,10 @@ uint64_t amdgpu_ttm_tt_pde_flags(struct ttm_tt *ttm, struct ttm_resource *mem)
flags |= AMDGPU_PTE_SNOOPED;
}
+ if (mem && mem->mem_type == TTM_PL_VRAM &&
+ mem->bus.caching == ttm_cached)
+ flags |= AMDGPU_PTE_SNOOPED;
+
return flags;
}
@@ -1469,7 +1396,7 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
const struct ttm_place *place)
{
unsigned long num_pages = bo->mem.num_pages;
- struct drm_mm_node *node = bo->mem.mm_node;
+ struct amdgpu_res_cursor cursor;
struct dma_resv_list *flist;
struct dma_fence *f;
int i;
@@ -1501,13 +1428,15 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
case TTM_PL_VRAM:
/* Check each drm MM node individually */
- while (num_pages) {
- if (place->fpfn < (node->start + node->size) &&
- !(place->lpfn && place->lpfn <= node->start))
+ amdgpu_res_first(&bo->mem, 0, (u64)num_pages << PAGE_SHIFT,
+ &cursor);
+ while (cursor.remaining) {
+ if (place->fpfn < PFN_DOWN(cursor.start + cursor.size)
+ && !(place->lpfn &&
+ place->lpfn <= PFN_DOWN(cursor.start)))
return true;
- num_pages -= node->size;
- ++node;
+ amdgpu_res_next(&cursor, cursor.size);
}
return false;
@@ -1531,41 +1460,36 @@ static bool amdgpu_ttm_bo_eviction_valuable(struct ttm_buffer_object *bo,
* access for debugging purposes.
*/
static int amdgpu_ttm_access_memory(struct ttm_buffer_object *bo,
- unsigned long offset,
- void *buf, int len, int write)
+ unsigned long offset, void *buf, int len,
+ int write)
{
struct amdgpu_bo *abo = ttm_to_amdgpu_bo(bo);
struct amdgpu_device *adev = amdgpu_ttm_adev(abo->tbo.bdev);
- struct drm_mm_node *nodes;
+ struct amdgpu_res_cursor cursor;
+ unsigned long flags;
uint32_t value = 0;
int ret = 0;
- uint64_t pos;
- unsigned long flags;
if (bo->mem.mem_type != TTM_PL_VRAM)
return -EIO;
- pos = offset;
- nodes = amdgpu_find_mm_node(&abo->tbo.mem, &pos);
- pos += (nodes->start << PAGE_SHIFT);
-
- while (len && pos < adev->gmc.mc_vram_size) {
- uint64_t aligned_pos = pos & ~(uint64_t)3;
- uint64_t bytes = 4 - (pos & 3);
- uint32_t shift = (pos & 3) * 8;
+ amdgpu_res_first(&bo->mem, offset, len, &cursor);
+ while (cursor.remaining) {
+ uint64_t aligned_pos = cursor.start & ~(uint64_t)3;
+ uint64_t bytes = 4 - (cursor.start & 3);
+ uint32_t shift = (cursor.start & 3) * 8;
uint32_t mask = 0xffffffff << shift;
- if (len < bytes) {
- mask &= 0xffffffff >> (bytes - len) * 8;
- bytes = len;
+ if (cursor.size < bytes) {
+ mask &= 0xffffffff >> (bytes - cursor.size) * 8;
+ bytes = cursor.size;
}
if (mask != 0xffffffff) {
spin_lock_irqsave(&adev->mmio_idx_lock, flags);
WREG32_NO_KIQ(mmMM_INDEX, ((uint32_t)aligned_pos) | 0x80000000);
WREG32_NO_KIQ(mmMM_INDEX_HI, aligned_pos >> 31);
- if (!write || mask != 0xffffffff)
- value = RREG32_NO_KIQ(mmMM_DATA);
+ value = RREG32_NO_KIQ(mmMM_DATA);
if (write) {
value &= ~mask;
value |= (*(uint32_t *)buf << shift) & mask;
@@ -1577,21 +1501,15 @@ static int amdgpu_ttm_access_memory(struct ttm_buffer_object *bo,
memcpy(buf, &value, bytes);
}
} else {
- bytes = (nodes->start + nodes->size) << PAGE_SHIFT;
- bytes = min(bytes - pos, (uint64_t)len & ~0x3ull);
-
- amdgpu_device_vram_access(adev, pos, (uint32_t *)buf,
- bytes, write);
+ bytes = cursor.size & ~0x3ULL;
+ amdgpu_device_vram_access(adev, cursor.start,
+ (uint32_t *)buf, bytes,
+ write);
}
ret += bytes;
buf = (uint8_t *)buf + bytes;
- pos += bytes;
- len -= bytes;
- if (pos >= (nodes->start + nodes->size) << PAGE_SHIFT) {
- ++nodes;
- pos = (nodes->start << PAGE_SHIFT);
- }
+ amdgpu_res_next(&cursor, bytes);
}
return ret;
@@ -1603,7 +1521,7 @@ amdgpu_bo_delete_mem_notify(struct ttm_buffer_object *bo)
amdgpu_bo_move_notify(bo, false, NULL);
}
-static struct ttm_bo_driver amdgpu_bo_driver = {
+static struct ttm_device_funcs amdgpu_bo_driver = {
.ttm_tt_create = &amdgpu_ttm_tt_create,
.ttm_tt_populate = &amdgpu_ttm_tt_populate,
.ttm_tt_unpopulate = &amdgpu_ttm_tt_unpopulate,
@@ -1696,7 +1614,7 @@ static void amdgpu_ttm_training_data_block_init(struct amdgpu_device *adev)
(adev->gmc.mc_vram_size - GDDR6_MEM_TRAINING_OFFSET);
ctx->train_data_size =
GDDR6_MEM_TRAINING_DATA_SIZE_IN_BYTES;
-
+
DRM_DEBUG("train_data_size:%llx,p2c_train_data_offset:%llx,c2p_train_data_offset:%llx.\n",
ctx->train_data_size,
ctx->p2c_train_data_offset,
@@ -1785,7 +1703,7 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
mutex_init(&adev->mman.gtt_window_lock);
/* No others user of address space so set it to 0 */
- r = ttm_bo_device_init(&adev->mman.bdev, &amdgpu_bo_driver, adev->dev,
+ r = ttm_device_init(&adev->mman.bdev, &amdgpu_bo_driver, adev->dev,
adev_to_drm(adev)->anon_inode->i_mapping,
adev_to_drm(adev)->vma_offset_manager,
adev->need_swiotlb,
@@ -1812,8 +1730,15 @@ int amdgpu_ttm_init(struct amdgpu_device *adev)
/* Change the size here instead of the init above so only lpfn is affected */
amdgpu_ttm_set_buffer_funcs_status(adev, false);
#ifdef CONFIG_64BIT
- adev->mman.aper_base_kaddr = ioremap_wc(adev->gmc.aper_base,
- adev->gmc.visible_vram_size);
+#ifdef CONFIG_X86
+ if (adev->gmc.xgmi.connected_to_cpu)
+ adev->mman.aper_base_kaddr = ioremap_cache(adev->gmc.aper_base,
+ adev->gmc.visible_vram_size);
+
+ else
+#endif
+ adev->mman.aper_base_kaddr = ioremap_wc(adev->gmc.aper_base,
+ adev->gmc.visible_vram_size);
#endif
/*
@@ -1926,7 +1851,7 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev)
ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_GDS);
ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_GWS);
ttm_range_man_fini(&adev->mman.bdev, AMDGPU_PL_OA);
- ttm_bo_device_release(&adev->mman.bdev);
+ ttm_device_fini(&adev->mman.bdev);
adev->mman.initialized = false;
DRM_INFO("amdgpu: ttm finalized\n");
}
@@ -2002,7 +1927,7 @@ unlock:
return ret;
}
-static struct vm_operations_struct amdgpu_ttm_vm_ops = {
+static const struct vm_operations_struct amdgpu_ttm_vm_ops = {
.fault = amdgpu_ttm_fault,
.open = ttm_bo_vm_open,
.close = ttm_bo_vm_close,
@@ -2053,7 +1978,8 @@ int amdgpu_copy_buffer(struct amdgpu_ring *ring, uint64_t src_offset,
return r;
if (vm_needs_flush) {
- job->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gart.bo);
+ job->vm_pd_addr = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo ?
+ adev->gmc.pdb0_bo : adev->gart.bo);
job->vm_needs_flush = true;
}
if (resv) {
@@ -2104,9 +2030,9 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo,
uint32_t max_bytes = adev->mman.buffer_funcs->fill_max_bytes;
struct amdgpu_ring *ring = adev->mman.buffer_funcs_ring;
- struct drm_mm_node *mm_node;
- unsigned long num_pages;
+ struct amdgpu_res_cursor cursor;
unsigned int num_loops, num_dw;
+ uint64_t num_bytes;
struct amdgpu_job *job;
int r;
@@ -2122,15 +2048,13 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo,
return r;
}
- num_pages = bo->tbo.mem.num_pages;
- mm_node = bo->tbo.mem.mm_node;
+ num_bytes = bo->tbo.mem.num_pages << PAGE_SHIFT;
num_loops = 0;
- while (num_pages) {
- uint64_t byte_count = mm_node->size << PAGE_SHIFT;
- num_loops += DIV_ROUND_UP_ULL(byte_count, max_bytes);
- num_pages -= mm_node->size;
- ++mm_node;
+ amdgpu_res_first(&bo->tbo.mem, 0, num_bytes, &cursor);
+ while (cursor.remaining) {
+ num_loops += DIV_ROUND_UP_ULL(cursor.size, max_bytes);
+ amdgpu_res_next(&cursor, cursor.size);
}
num_dw = num_loops * adev->mman.buffer_funcs->fill_num_dw;
@@ -2152,27 +2076,16 @@ int amdgpu_fill_buffer(struct amdgpu_bo *bo,
}
}
- num_pages = bo->tbo.mem.num_pages;
- mm_node = bo->tbo.mem.mm_node;
+ amdgpu_res_first(&bo->tbo.mem, 0, num_bytes, &cursor);
+ while (cursor.remaining) {
+ uint32_t cur_size = min_t(uint64_t, cursor.size, max_bytes);
+ uint64_t dst_addr = cursor.start;
- while (num_pages) {
- uint64_t byte_count = mm_node->size << PAGE_SHIFT;
- uint64_t dst_addr;
+ dst_addr += amdgpu_ttm_domain_start(adev, bo->tbo.mem.mem_type);
+ amdgpu_emit_fill_buffer(adev, &job->ibs[0], src_data, dst_addr,
+ cur_size);
- dst_addr = amdgpu_mm_node_addr(&bo->tbo, mm_node, &bo->tbo.mem);
- while (byte_count) {
- uint32_t cur_size_in_bytes = min_t(uint64_t, byte_count,
- max_bytes);
-
- amdgpu_emit_fill_buffer(adev, &job->ibs[0], src_data,
- dst_addr, cur_size_in_bytes);
-
- dst_addr += cur_size_in_bytes;
- byte_count -= cur_size_in_bytes;
- }
-
- num_pages -= mm_node->size;
- ++mm_node;
+ amdgpu_res_next(&cursor, cur_size);
}
amdgpu_ring_pad_ib(ring, &job->ibs[0]);
@@ -2191,36 +2104,74 @@ error_free:
#if defined(CONFIG_DEBUG_FS)
-static int amdgpu_mm_dump_table(struct seq_file *m, void *data)
+static int amdgpu_mm_vram_table_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- unsigned ttm_pl = (uintptr_t)node->info_ent->data;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
- struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, ttm_pl);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev,
+ TTM_PL_VRAM);
struct drm_printer p = drm_seq_file_printer(m);
man->func->debug(man, &p);
return 0;
}
-static int amdgpu_ttm_pool_debugfs(struct seq_file *m, void *data)
+static int amdgpu_ttm_page_pool_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
return ttm_pool_debugfs(&adev->mman.bdev.pool, m);
}
-static const struct drm_info_list amdgpu_ttm_debugfs_list[] = {
- {"amdgpu_vram_mm", amdgpu_mm_dump_table, 0, (void *)TTM_PL_VRAM},
- {"amdgpu_gtt_mm", amdgpu_mm_dump_table, 0, (void *)TTM_PL_TT},
- {"amdgpu_gds_mm", amdgpu_mm_dump_table, 0, (void *)AMDGPU_PL_GDS},
- {"amdgpu_gws_mm", amdgpu_mm_dump_table, 0, (void *)AMDGPU_PL_GWS},
- {"amdgpu_oa_mm", amdgpu_mm_dump_table, 0, (void *)AMDGPU_PL_OA},
- {"ttm_page_pool", amdgpu_ttm_pool_debugfs, 0, NULL},
-};
+static int amdgpu_mm_tt_table_show(struct seq_file *m, void *unused)
+{
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev,
+ TTM_PL_TT);
+ struct drm_printer p = drm_seq_file_printer(m);
+
+ man->func->debug(man, &p);
+ return 0;
+}
+
+static int amdgpu_mm_gds_table_show(struct seq_file *m, void *unused)
+{
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev,
+ AMDGPU_PL_GDS);
+ struct drm_printer p = drm_seq_file_printer(m);
+
+ man->func->debug(man, &p);
+ return 0;
+}
+
+static int amdgpu_mm_gws_table_show(struct seq_file *m, void *unused)
+{
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev,
+ AMDGPU_PL_GWS);
+ struct drm_printer p = drm_seq_file_printer(m);
+
+ man->func->debug(man, &p);
+ return 0;
+}
+
+static int amdgpu_mm_oa_table_show(struct seq_file *m, void *unused)
+{
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev,
+ AMDGPU_PL_OA);
+ struct drm_printer p = drm_seq_file_printer(m);
+
+ man->func->debug(man, &p);
+ return 0;
+}
+
+DEFINE_SHOW_ATTRIBUTE(amdgpu_mm_vram_table);
+DEFINE_SHOW_ATTRIBUTE(amdgpu_mm_tt_table);
+DEFINE_SHOW_ATTRIBUTE(amdgpu_mm_gds_table);
+DEFINE_SHOW_ATTRIBUTE(amdgpu_mm_gws_table);
+DEFINE_SHOW_ATTRIBUTE(amdgpu_mm_oa_table);
+DEFINE_SHOW_ATTRIBUTE(amdgpu_ttm_page_pool);
/*
* amdgpu_ttm_vram_read - Linear read access to VRAM
@@ -2308,58 +2259,6 @@ static const struct file_operations amdgpu_ttm_vram_fops = {
.llseek = default_llseek,
};
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
-
-/*
- * amdgpu_ttm_gtt_read - Linear read access to GTT memory
- */
-static ssize_t amdgpu_ttm_gtt_read(struct file *f, char __user *buf,
- size_t size, loff_t *pos)
-{
- struct amdgpu_device *adev = file_inode(f)->i_private;
- ssize_t result = 0;
- int r;
-
- while (size) {
- loff_t p = *pos / PAGE_SIZE;
- unsigned off = *pos & ~PAGE_MASK;
- size_t cur_size = min_t(size_t, size, PAGE_SIZE - off);
- struct page *page;
- void *ptr;
-
- if (p >= adev->gart.num_cpu_pages)
- return result;
-
- page = adev->gart.pages[p];
- if (page) {
- ptr = kmap(page);
- ptr += off;
-
- r = copy_to_user(buf, ptr, cur_size);
- kunmap(adev->gart.pages[p]);
- } else
- r = clear_user(buf, cur_size);
-
- if (r)
- return -EFAULT;
-
- result += cur_size;
- buf += cur_size;
- *pos += cur_size;
- size -= cur_size;
- }
-
- return result;
-}
-
-static const struct file_operations amdgpu_ttm_gtt_fops = {
- .owner = THIS_MODULE,
- .read = amdgpu_ttm_gtt_read,
- .llseek = default_llseek
-};
-
-#endif
-
/*
* amdgpu_iomem_read - Virtual read access to GPU mapped memory
*
@@ -2474,46 +2373,29 @@ static const struct file_operations amdgpu_ttm_iomem_fops = {
.llseek = default_llseek
};
-static const struct {
- char *name;
- const struct file_operations *fops;
- int domain;
-} ttm_debugfs_entries[] = {
- { "amdgpu_vram", &amdgpu_ttm_vram_fops, TTM_PL_VRAM },
-#ifdef CONFIG_DRM_AMDGPU_GART_DEBUGFS
- { "amdgpu_gtt", &amdgpu_ttm_gtt_fops, TTM_PL_TT },
#endif
- { "amdgpu_iomem", &amdgpu_ttm_iomem_fops, TTM_PL_SYSTEM },
-};
-#endif
-
-int amdgpu_ttm_debugfs_init(struct amdgpu_device *adev)
+void amdgpu_ttm_debugfs_init(struct amdgpu_device *adev)
{
#if defined(CONFIG_DEBUG_FS)
- unsigned count;
-
struct drm_minor *minor = adev_to_drm(adev)->primary;
- struct dentry *ent, *root = minor->debugfs_root;
-
- for (count = 0; count < ARRAY_SIZE(ttm_debugfs_entries); count++) {
- ent = debugfs_create_file(
- ttm_debugfs_entries[count].name,
- S_IFREG | S_IRUGO, root,
- adev,
- ttm_debugfs_entries[count].fops);
- if (IS_ERR(ent))
- return PTR_ERR(ent);
- if (ttm_debugfs_entries[count].domain == TTM_PL_VRAM)
- i_size_write(ent->d_inode, adev->gmc.mc_vram_size);
- else if (ttm_debugfs_entries[count].domain == TTM_PL_TT)
- i_size_write(ent->d_inode, adev->gmc.gart_size);
- adev->mman.debugfs_entries[count] = ent;
- }
-
- count = ARRAY_SIZE(amdgpu_ttm_debugfs_list);
- return amdgpu_debugfs_add_files(adev, amdgpu_ttm_debugfs_list, count);
-#else
- return 0;
+ struct dentry *root = minor->debugfs_root;
+
+ debugfs_create_file_size("amdgpu_vram", 0444, root, adev,
+ &amdgpu_ttm_vram_fops, adev->gmc.mc_vram_size);
+ debugfs_create_file("amdgpu_iomem", 0444, root, adev,
+ &amdgpu_ttm_iomem_fops);
+ debugfs_create_file("amdgpu_vram_mm", 0444, root, adev,
+ &amdgpu_mm_vram_table_fops);
+ debugfs_create_file("amdgpu_gtt_mm", 0444, root, adev,
+ &amdgpu_mm_tt_table_fops);
+ debugfs_create_file("amdgpu_gds_mm", 0444, root, adev,
+ &amdgpu_mm_gds_table_fops);
+ debugfs_create_file("amdgpu_gws_mm", 0444, root, adev,
+ &amdgpu_mm_gws_table_fops);
+ debugfs_create_file("amdgpu_oa_mm", 0444, root, adev,
+ &amdgpu_mm_oa_table_fops);
+ debugfs_create_file("ttm_page_pool", 0444, root, adev,
+ &amdgpu_ttm_page_pool_fops);
#endif
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
index d2987536d7cd..9e38475e0f8d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.h
@@ -60,14 +60,10 @@ struct amdgpu_gtt_mgr {
};
struct amdgpu_mman {
- struct ttm_bo_device bdev;
+ struct ttm_device bdev;
bool initialized;
void __iomem *aper_base_kaddr;
-#if defined(CONFIG_DEBUG_FS)
- struct dentry *debugfs_entries[8];
-#endif
-
/* buffer handling */
const struct amdgpu_buffer_funcs *buffer_funcs;
struct amdgpu_ring *buffer_funcs_ring;
@@ -116,11 +112,11 @@ int amdgpu_gtt_mgr_recover(struct ttm_resource_manager *man);
u64 amdgpu_vram_mgr_bo_visible_size(struct amdgpu_bo *bo);
int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev,
struct ttm_resource *mem,
+ u64 offset, u64 size,
struct device *dev,
enum dma_data_direction dir,
struct sg_table **sgt);
-void amdgpu_vram_mgr_free_sgt(struct amdgpu_device *adev,
- struct device *dev,
+void amdgpu_vram_mgr_free_sgt(struct device *dev,
enum dma_data_direction dir,
struct sg_table *sgt);
uint64_t amdgpu_vram_mgr_usage(struct ttm_resource_manager *man);
@@ -186,6 +182,6 @@ uint64_t amdgpu_ttm_tt_pde_flags(struct ttm_tt *ttm, struct ttm_resource *mem);
uint64_t amdgpu_ttm_tt_pte_flags(struct amdgpu_device *adev, struct ttm_tt *ttm,
struct ttm_resource *mem);
-int amdgpu_ttm_debugfs_init(struct amdgpu_device *adev);
+void amdgpu_ttm_debugfs_init(struct amdgpu_device *adev);
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c
index 1beb08af347f..9733224117e3 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c
@@ -402,6 +402,7 @@ amdgpu_ucode_get_load_type(struct amdgpu_device *adev, int load_type)
case CHIP_NAVY_FLOUNDER:
case CHIP_VANGOGH:
case CHIP_DIMGREY_CAVEFISH:
+ case CHIP_ALDEBARAN:
if (!load_type)
return AMDGPU_FW_LOAD_DIRECT;
else
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h
index 46449e70348b..2c42874f7784 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.h
@@ -105,6 +105,9 @@ struct psp_firmware_header_v1_3 {
uint32_t spl_header_version;
uint32_t spl_offset_bytes;
uint32_t spl_size_bytes;
+ uint32_t rl_header_version;
+ uint32_t rl_offset_bytes;
+ uint32_t rl_size_bytes;
};
/* version_major=1, version_minor=0 */
@@ -136,6 +139,7 @@ enum ta_fw_type {
TA_FW_TYPE_PSP_DTM,
TA_FW_TYPE_PSP_RAP,
TA_FW_TYPE_PSP_SECUREDISPLAY,
+ TA_FW_TYPE_MAX_INDEX,
};
struct ta_fw_bin_desc {
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
index a2975c8092a9..ea6f99be070b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.c
@@ -60,8 +60,9 @@ int amdgpu_umc_ras_late_init(struct amdgpu_device *adev)
}
/* ras init of specific umc version */
- if (adev->umc.funcs && adev->umc.funcs->err_cnt_init)
- adev->umc.funcs->err_cnt_init(adev);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->err_cnt_init)
+ adev->umc.ras_funcs->err_cnt_init(adev);
return 0;
@@ -95,12 +96,12 @@ int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
kgd2kfd_set_sram_ecc_flag(adev->kfd.dev);
- if (adev->umc.funcs &&
- adev->umc.funcs->query_ras_error_count)
- adev->umc.funcs->query_ras_error_count(adev, ras_error_status);
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_count)
+ adev->umc.ras_funcs->query_ras_error_count(adev, ras_error_status);
- if (adev->umc.funcs &&
- adev->umc.funcs->query_ras_error_address &&
+ if (adev->umc.ras_funcs &&
+ adev->umc.ras_funcs->query_ras_error_address &&
adev->umc.max_ras_err_cnt_per_query) {
err_data->err_addr =
kcalloc(adev->umc.max_ras_err_cnt_per_query,
@@ -116,7 +117,7 @@ int amdgpu_umc_process_ras_data_cb(struct amdgpu_device *adev,
/* umc query_ras_error_address is also responsible for clearing
* error status
*/
- adev->umc.funcs->query_ras_error_address(adev, ras_error_status);
+ adev->umc.ras_funcs->query_ras_error_address(adev, ras_error_status);
}
/* only uncorrectable error needs gpu reset */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
index 183814493658..bbcccf53080d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_umc.h
@@ -35,13 +35,17 @@
#define LOOP_UMC_CH_INST(ch_inst) for ((ch_inst) = 0; (ch_inst) < adev->umc.channel_inst_num; (ch_inst)++)
#define LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) LOOP_UMC_INST((umc_inst)) LOOP_UMC_CH_INST((ch_inst))
-struct amdgpu_umc_funcs {
+struct amdgpu_umc_ras_funcs {
void (*err_cnt_init)(struct amdgpu_device *adev);
int (*ras_late_init)(struct amdgpu_device *adev);
+ void (*ras_fini)(struct amdgpu_device *adev);
void (*query_ras_error_count)(struct amdgpu_device *adev,
- void *ras_error_status);
+ void *ras_error_status);
void (*query_ras_error_address)(struct amdgpu_device *adev,
void *ras_error_status);
+};
+
+struct amdgpu_umc_funcs {
void (*init_registers)(struct amdgpu_device *adev);
};
@@ -59,6 +63,7 @@ struct amdgpu_umc {
struct ras_common_if *ras_if;
const struct amdgpu_umc_funcs *funcs;
+ const struct amdgpu_umc_ras_funcs *ras_funcs;
};
int amdgpu_umc_ras_late_init(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
index e2ed4689118a..c6dbc0801604 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_uvd.c
@@ -259,7 +259,7 @@ int amdgpu_uvd_sw_init(struct amdgpu_device *adev)
if ((adev->asic_type == CHIP_POLARIS10 ||
adev->asic_type == CHIP_POLARIS11) &&
(adev->uvd.fw_version < FW_1_66_16))
- DRM_ERROR("POLARIS10/11 UVD firmware version %hu.%hu is too old.\n",
+ DRM_ERROR("POLARIS10/11 UVD firmware version %u.%u is too old.\n",
version_major, version_minor);
} else {
unsigned int enc_major, enc_minor, dec_minor;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
index 99b82f3c2617..201645963ba5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
@@ -47,6 +47,7 @@
#define FIRMWARE_NAVY_FLOUNDER "amdgpu/navy_flounder_vcn.bin"
#define FIRMWARE_VANGOGH "amdgpu/vangogh_vcn.bin"
#define FIRMWARE_DIMGREY_CAVEFISH "amdgpu/dimgrey_cavefish_vcn.bin"
+#define FIRMWARE_ALDEBARAN "amdgpu/aldebaran_vcn.bin"
MODULE_FIRMWARE(FIRMWARE_RAVEN);
MODULE_FIRMWARE(FIRMWARE_PICASSO);
@@ -54,6 +55,7 @@ MODULE_FIRMWARE(FIRMWARE_RAVEN2);
MODULE_FIRMWARE(FIRMWARE_ARCTURUS);
MODULE_FIRMWARE(FIRMWARE_RENOIR);
MODULE_FIRMWARE(FIRMWARE_GREEN_SARDINE);
+MODULE_FIRMWARE(FIRMWARE_ALDEBARAN);
MODULE_FIRMWARE(FIRMWARE_NAVI10);
MODULE_FIRMWARE(FIRMWARE_NAVI14);
MODULE_FIRMWARE(FIRMWARE_NAVI12);
@@ -104,6 +106,12 @@ int amdgpu_vcn_sw_init(struct amdgpu_device *adev)
(adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG))
adev->vcn.indirect_sram = true;
break;
+ case CHIP_ALDEBARAN:
+ fw_name = FIRMWARE_ALDEBARAN;
+ if ((adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) &&
+ (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG))
+ adev->vcn.indirect_sram = true;
+ break;
case CHIP_NAVI10:
fw_name = FIRMWARE_NAVI10;
if ((adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) &&
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
index 13aa417f6be7..bc76cab67697 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.h
@@ -155,6 +155,7 @@
} \
} while (0)
+#define AMDGPU_VCN_FW_SHARED_FLAG_0_RB (1 << 6)
#define AMDGPU_VCN_MULTI_QUEUE_FLAG (1 << 8)
#define AMDGPU_VCN_SW_RING_FLAG (1 << 9)
@@ -211,6 +212,7 @@ struct amdgpu_vcn_inst {
void *saved_bo;
struct amdgpu_ring ring_dec;
struct amdgpu_ring ring_enc[AMDGPU_VCN_MAX_ENC_RINGS];
+ atomic_t sched_score;
struct amdgpu_irq_src irq;
struct amdgpu_vcn_reg external;
struct amdgpu_bo *dpg_sram_bo;
@@ -243,6 +245,12 @@ struct amdgpu_vcn {
int inst_idx, struct dpg_pause_state *new_state);
};
+struct amdgpu_fw_shared_rb_ptrs_struct {
+ /* to WA DPG R/W ptr issues.*/
+ uint32_t rptr;
+ uint32_t wptr;
+};
+
struct amdgpu_fw_shared_multi_queue {
uint8_t decode_queue_mode;
uint8_t encode_generalpurpose_queue_mode;
@@ -258,10 +266,12 @@ struct amdgpu_fw_shared_sw_ring {
struct amdgpu_fw_shared {
uint32_t present_flag_0;
- uint8_t pad[53];
+ uint8_t pad[44];
+ struct amdgpu_fw_shared_rb_ptrs_struct rb;
+ uint8_t pad1[1];
struct amdgpu_fw_shared_multi_queue multi_queue;
struct amdgpu_fw_shared_sw_ring sw_ring;
-} __attribute__((__packed__));
+};
struct amdgpu_vcn_decode_buffer {
uint32_t valid_buf_flag;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
index 5da04d45b637..0c9c5255aa42 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.c
@@ -466,6 +466,8 @@ static int amdgpu_virt_read_pf2vf_data(struct amdgpu_device *adev)
((struct amd_sriov_msg_pf2vf_info *)pf2vf_info)->vf2pf_update_interval_ms;
adev->virt.gim_feature =
((struct amd_sriov_msg_pf2vf_info *)pf2vf_info)->feature_flags.all;
+ adev->virt.reg_access =
+ ((struct amd_sriov_msg_pf2vf_info *)pf2vf_info)->reg_access_flags.all;
break;
default:
@@ -617,6 +619,14 @@ void amdgpu_virt_init_data_exchange(struct amdgpu_device *adev)
if (adev->virt.ras_init_done)
amdgpu_virt_add_bad_page(adev, bp_block_offset, bp_block_size);
}
+ } else if (adev->bios != NULL) {
+ adev->virt.fw_reserve.p_pf2vf =
+ (struct amd_sriov_msg_pf2vf_info_header *)
+ (adev->bios + (AMD_SRIOV_MSG_PF2VF_OFFSET_KB << 10));
+
+ amdgpu_virt_read_pf2vf_data(adev);
+
+ return;
}
if (adev->virt.vf2pf_update_interval_ms != 0) {
@@ -640,6 +650,7 @@ void amdgpu_detect_virtualization(struct amdgpu_device *adev)
case CHIP_NAVI12:
case CHIP_SIENNA_CICHLID:
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
reg = RREG32(mmRCC_IOV_FUNC_IDENTIFIER);
break;
default: /* other chip doesn't support SRIOV */
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
index 8dd624c20f89..383d4bdc3fb5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_virt.h
@@ -104,6 +104,17 @@ enum AMDGIM_FEATURE_FLAG {
AMDGIM_FEATURE_GIM_MM_BW_MGR = 0x8,
/* PP ONE VF MODE in GIM */
AMDGIM_FEATURE_PP_ONE_VF = (1 << 4),
+ /* Indirect Reg Access enabled */
+ AMDGIM_FEATURE_INDIRECT_REG_ACCESS = (1 << 5),
+};
+
+enum AMDGIM_REG_ACCESS_FLAG {
+ /* Use PSP to program IH_RB_CNTL */
+ AMDGIM_FEATURE_IH_REG_PSP_EN = (1 << 0),
+ /* Use RLC to program MMHUB regs */
+ AMDGIM_FEATURE_MMHUB_REG_RLC_EN = (1 << 1),
+ /* Use RLC to program GC regs */
+ AMDGIM_FEATURE_GC_REG_RLC_EN = (1 << 2),
};
struct amdgim_pf2vf_info_v1 {
@@ -217,6 +228,7 @@ struct amdgpu_virt {
bool tdr_debug;
struct amdgpu_virt_ras_err_handler_data *virt_eh_data;
bool ras_init_done;
+ uint32_t reg_access;
/* vf2pf message */
struct delayed_work vf2pf_work;
@@ -238,6 +250,22 @@ struct amdgpu_virt {
#define amdgpu_sriov_fullaccess(adev) \
(amdgpu_sriov_vf((adev)) && !amdgpu_sriov_runtime((adev)))
+#define amdgpu_sriov_reg_indirect_en(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.gim_feature & (AMDGIM_FEATURE_INDIRECT_REG_ACCESS)))
+
+#define amdgpu_sriov_reg_indirect_ih(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.reg_access & (AMDGIM_FEATURE_IH_REG_PSP_EN)))
+
+#define amdgpu_sriov_reg_indirect_mmhub(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.reg_access & (AMDGIM_FEATURE_MMHUB_REG_RLC_EN)))
+
+#define amdgpu_sriov_reg_indirect_gc(adev) \
+(amdgpu_sriov_vf((adev)) && \
+ ((adev)->virt.reg_access & (AMDGIM_FEATURE_GC_REG_RLC_EN)))
+
#define amdgpu_passthrough(adev) \
((adev)->virt.caps & AMDGPU_PASSTHROUGH_MODE)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index ad91c0c3c423..9acee4a5b2ba 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -92,13 +92,13 @@ struct amdgpu_prt_cb {
static inline void amdgpu_vm_eviction_lock(struct amdgpu_vm *vm)
{
mutex_lock(&vm->eviction_lock);
- vm->saved_flags = memalloc_nofs_save();
+ vm->saved_flags = memalloc_noreclaim_save();
}
static inline int amdgpu_vm_eviction_trylock(struct amdgpu_vm *vm)
{
if (mutex_trylock(&vm->eviction_lock)) {
- vm->saved_flags = memalloc_nofs_save();
+ vm->saved_flags = memalloc_noreclaim_save();
return 1;
}
return 0;
@@ -106,7 +106,7 @@ static inline int amdgpu_vm_eviction_trylock(struct amdgpu_vm *vm)
static inline void amdgpu_vm_eviction_unlock(struct amdgpu_vm *vm)
{
- memalloc_nofs_restore(vm->saved_flags);
+ memalloc_noreclaim_restore(vm->saved_flags);
mutex_unlock(&vm->eviction_lock);
}
@@ -638,15 +638,15 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev,
struct amdgpu_vm_bo_base *bo_base;
if (vm->bulk_moveable) {
- spin_lock(&ttm_bo_glob.lru_lock);
+ spin_lock(&adev->mman.bdev.lru_lock);
ttm_bo_bulk_move_lru_tail(&vm->lru_bulk_move);
- spin_unlock(&ttm_bo_glob.lru_lock);
+ spin_unlock(&adev->mman.bdev.lru_lock);
return;
}
memset(&vm->lru_bulk_move, 0, sizeof(vm->lru_bulk_move));
- spin_lock(&ttm_bo_glob.lru_lock);
+ spin_lock(&adev->mman.bdev.lru_lock);
list_for_each_entry(bo_base, &vm->idle, vm_status) {
struct amdgpu_bo *bo = bo_base->bo;
@@ -660,7 +660,7 @@ void amdgpu_vm_move_to_lru_tail(struct amdgpu_device *adev,
&bo->shadow->tbo.mem,
&vm->lru_bulk_move);
}
- spin_unlock(&ttm_bo_glob.lru_lock);
+ spin_unlock(&adev->mman.bdev.lru_lock);
vm->bulk_moveable = true;
}
@@ -869,6 +869,7 @@ static void amdgpu_vm_bo_param(struct amdgpu_device *adev, struct amdgpu_vm *vm,
bp->domain = amdgpu_bo_get_preferred_pin_domain(adev, bp->domain);
bp->flags = AMDGPU_GEM_CREATE_VRAM_CONTIGUOUS |
AMDGPU_GEM_CREATE_CPU_GTT_USWC;
+ bp->bo_ptr_size = sizeof(struct amdgpu_bo);
if (vm->use_cpu_for_update)
bp->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
else if (!vm->root.base.bo || vm->root.base.bo->shadow)
@@ -2197,8 +2198,8 @@ int amdgpu_vm_bo_map(struct amdgpu_device *adev,
uint64_t eaddr;
/* validate the parameters */
- if (saddr & AMDGPU_GPU_PAGE_MASK || offset & AMDGPU_GPU_PAGE_MASK ||
- size == 0 || size & AMDGPU_GPU_PAGE_MASK)
+ if (saddr & ~PAGE_MASK || offset & ~PAGE_MASK ||
+ size == 0 || size & ~PAGE_MASK)
return -EINVAL;
/* make sure object fit at this offset */
@@ -2263,8 +2264,8 @@ int amdgpu_vm_bo_replace_map(struct amdgpu_device *adev,
int r;
/* validate the parameters */
- if (saddr & AMDGPU_GPU_PAGE_MASK || offset & AMDGPU_GPU_PAGE_MASK ||
- size == 0 || size & AMDGPU_GPU_PAGE_MASK)
+ if (saddr & ~PAGE_MASK || offset & ~PAGE_MASK ||
+ size == 0 || size & ~PAGE_MASK)
return -EINVAL;
/* make sure object fit at this offset */
@@ -2409,7 +2410,7 @@ int amdgpu_vm_bo_clear_mappings(struct amdgpu_device *adev,
after->start = eaddr + 1;
after->last = tmp->last;
after->offset = tmp->offset;
- after->offset += after->start - tmp->start;
+ after->offset += (after->start - tmp->start) << PAGE_SHIFT;
after->flags = tmp->flags;
after->bo_va = tmp->bo_va;
list_add(&after->list, &tmp->bo_va->invalids);
@@ -3147,6 +3148,12 @@ void amdgpu_vm_manager_init(struct amdgpu_device *adev)
{
unsigned i;
+ /* Concurrent flushes are only possible starting with Vega10 and
+ * are broken on Navi10 and Navi14.
+ */
+ adev->vm_manager.concurrent_flush = !(adev->asic_type < CHIP_VEGA10 ||
+ adev->asic_type == CHIP_NAVI10 ||
+ adev->asic_type == CHIP_NAVI14);
amdgpu_vmid_mgr_init(adev);
adev->vm_manager.fence_context =
@@ -3300,7 +3307,7 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
struct amdgpu_bo *root;
uint64_t value, flags;
struct amdgpu_vm *vm;
- long r;
+ int r;
spin_lock(&adev->vm_manager.pasid_lock);
vm = idr_find(&adev->vm_manager.pasid_idr, pasid);
@@ -3349,6 +3356,12 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
value = 0;
}
+ r = dma_resv_reserve_shared(root->tbo.base.resv, 1);
+ if (r) {
+ pr_debug("failed %d to reserve fence slot\n", r);
+ goto error_unlock;
+ }
+
r = amdgpu_vm_bo_update_mapping(adev, adev, vm, true, false, NULL, addr,
addr, flags, value, NULL, NULL,
NULL);
@@ -3360,7 +3373,7 @@ bool amdgpu_vm_handle_fault(struct amdgpu_device *adev, u32 pasid,
error_unlock:
amdgpu_bo_unreserve(root);
if (r < 0)
- DRM_ERROR("Can't handle page fault (%ld)\n", r);
+ DRM_ERROR("Can't handle page fault (%d)\n", r);
error_unref:
amdgpu_bo_unref(&root);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
index 976a12e5a8b9..4e140288159c 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h
@@ -331,6 +331,7 @@ struct amdgpu_vm_manager {
/* Handling of VMIDs */
struct amdgpu_vmid_mgr id_mgr[AMDGPU_MAX_VMHUBS];
unsigned int first_kfd_vmid;
+ bool concurrent_flush;
/* Handling of VM fences */
u64 fence_context;
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
index c89b66bb70e2..bce105e2973e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vram_mgr.c
@@ -25,6 +25,7 @@
#include <linux/dma-mapping.h>
#include "amdgpu.h"
#include "amdgpu_vm.h"
+#include "amdgpu_res_cursor.h"
#include "amdgpu_atomfirmware.h"
#include "atom.h"
@@ -52,7 +53,7 @@ static ssize_t amdgpu_mem_info_vram_total_show(struct device *dev,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", adev->gmc.real_vram_size);
+ return sysfs_emit(buf, "%llu\n", adev->gmc.real_vram_size);
}
/**
@@ -69,7 +70,7 @@ static ssize_t amdgpu_mem_info_vis_vram_total_show(struct device *dev,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", adev->gmc.visible_vram_size);
+ return sysfs_emit(buf, "%llu\n", adev->gmc.visible_vram_size);
}
/**
@@ -87,8 +88,7 @@ static ssize_t amdgpu_mem_info_vram_used_show(struct device *dev,
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- amdgpu_vram_mgr_usage(man));
+ return sysfs_emit(buf, "%llu\n", amdgpu_vram_mgr_usage(man));
}
/**
@@ -106,8 +106,7 @@ static ssize_t amdgpu_mem_info_vis_vram_used_show(struct device *dev,
struct amdgpu_device *adev = drm_to_adev(ddev);
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
- return snprintf(buf, PAGE_SIZE, "%llu\n",
- amdgpu_vram_mgr_vis_usage(man));
+ return sysfs_emit(buf, "%llu\n", amdgpu_vram_mgr_vis_usage(man));
}
static ssize_t amdgpu_mem_info_vram_vendor(struct device *dev,
@@ -119,27 +118,27 @@ static ssize_t amdgpu_mem_info_vram_vendor(struct device *dev,
switch (adev->gmc.vram_vendor) {
case SAMSUNG:
- return snprintf(buf, PAGE_SIZE, "samsung\n");
+ return sysfs_emit(buf, "samsung\n");
case INFINEON:
- return snprintf(buf, PAGE_SIZE, "infineon\n");
+ return sysfs_emit(buf, "infineon\n");
case ELPIDA:
- return snprintf(buf, PAGE_SIZE, "elpida\n");
+ return sysfs_emit(buf, "elpida\n");
case ETRON:
- return snprintf(buf, PAGE_SIZE, "etron\n");
+ return sysfs_emit(buf, "etron\n");
case NANYA:
- return snprintf(buf, PAGE_SIZE, "nanya\n");
+ return sysfs_emit(buf, "nanya\n");
case HYNIX:
- return snprintf(buf, PAGE_SIZE, "hynix\n");
+ return sysfs_emit(buf, "hynix\n");
case MOSEL:
- return snprintf(buf, PAGE_SIZE, "mosel\n");
+ return sysfs_emit(buf, "mosel\n");
case WINBOND:
- return snprintf(buf, PAGE_SIZE, "winbond\n");
+ return sysfs_emit(buf, "winbond\n");
case ESMT:
- return snprintf(buf, PAGE_SIZE, "esmt\n");
+ return sysfs_emit(buf, "esmt\n");
case MICRON:
- return snprintf(buf, PAGE_SIZE, "micron\n");
+ return sysfs_emit(buf, "micron\n");
default:
- return snprintf(buf, PAGE_SIZE, "unknown\n");
+ return sysfs_emit(buf, "unknown\n");
}
}
@@ -567,6 +566,8 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man,
*
* @adev: amdgpu device pointer
* @mem: TTM memory object
+ * @offset: byte offset from the base of VRAM BO
+ * @length: number of bytes to export in sg_table
* @dev: the other device
* @dir: dma direction
* @sgt: resulting sg table
@@ -575,39 +576,47 @@ static void amdgpu_vram_mgr_del(struct ttm_resource_manager *man,
*/
int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev,
struct ttm_resource *mem,
+ u64 offset, u64 length,
struct device *dev,
enum dma_data_direction dir,
struct sg_table **sgt)
{
- struct drm_mm_node *node;
+ struct amdgpu_res_cursor cursor;
struct scatterlist *sg;
int num_entries = 0;
- unsigned int pages;
int i, r;
*sgt = kmalloc(sizeof(**sgt), GFP_KERNEL);
if (!*sgt)
return -ENOMEM;
- for (pages = mem->num_pages, node = mem->mm_node;
- pages; pages -= node->size, ++node)
- ++num_entries;
+ /* Determine the number of DRM_MM nodes to export */
+ amdgpu_res_first(mem, offset, length, &cursor);
+ while (cursor.remaining) {
+ num_entries++;
+ amdgpu_res_next(&cursor, cursor.size);
+ }
r = sg_alloc_table(*sgt, num_entries, GFP_KERNEL);
if (r)
goto error_free;
+ /* Initialize scatterlist nodes of sg_table */
for_each_sgtable_sg((*sgt), sg, i)
sg->length = 0;
- node = mem->mm_node;
+ /*
+ * Walk down DRM_MM nodes to populate scatterlist nodes
+ * @note: Use iterator api to get first the DRM_MM node
+ * and the number of bytes from it. Access the following
+ * DRM_MM node(s) if more buffer needs to exported
+ */
+ amdgpu_res_first(mem, offset, length, &cursor);
for_each_sgtable_sg((*sgt), sg, i) {
- phys_addr_t phys = (node->start << PAGE_SHIFT) +
- adev->gmc.aper_base;
- size_t size = node->size << PAGE_SHIFT;
+ phys_addr_t phys = cursor.start + adev->gmc.aper_base;
+ size_t size = cursor.size;
dma_addr_t addr;
- ++node;
addr = dma_map_resource(dev, phys, size, dir,
DMA_ATTR_SKIP_CPU_SYNC);
r = dma_mapping_error(dev, addr);
@@ -617,7 +626,10 @@ int amdgpu_vram_mgr_alloc_sgt(struct amdgpu_device *adev,
sg_set_page(sg, NULL, size, 0);
sg_dma_address(sg) = addr;
sg_dma_len(sg) = size;
+
+ amdgpu_res_next(&cursor, cursor.size);
}
+
return 0;
error_unmap:
@@ -639,15 +651,13 @@ error_free:
/**
* amdgpu_vram_mgr_free_sgt - allocate and fill a sg table
*
- * @adev: amdgpu device pointer
* @dev: device pointer
* @dir: data direction of resource to unmap
* @sgt: sg table to free
*
* Free a previously allocate sg table.
*/
-void amdgpu_vram_mgr_free_sgt(struct amdgpu_device *adev,
- struct device *dev,
+void amdgpu_vram_mgr_free_sgt(struct device *dev,
enum dma_data_direction dir,
struct sg_table *sgt)
{
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
index 659b385b27b5..8567d5d77346 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.c
@@ -24,7 +24,6 @@
#include <linux/list.h>
#include "amdgpu.h"
#include "amdgpu_xgmi.h"
-#include "amdgpu_smu.h"
#include "amdgpu_ras.h"
#include "soc15.h"
#include "df/df_3_6_offset.h"
@@ -217,7 +216,7 @@ static ssize_t amdgpu_xgmi_show_device_id(struct device *dev,
struct drm_device *ddev = dev_get_drvdata(dev);
struct amdgpu_device *adev = drm_to_adev(ddev);
- return snprintf(buf, PAGE_SIZE, "%llu\n", adev->gmc.xgmi.node_id);
+ return sysfs_emit(buf, "%llu\n", adev->gmc.xgmi.node_id);
}
@@ -246,7 +245,7 @@ static ssize_t amdgpu_xgmi_show_error(struct device *dev,
adev->df.funcs->set_fica(adev, ficaa_pie_status_in, 0, 0);
- return snprintf(buf, PAGE_SIZE, "%u\n", error_count);
+ return sysfs_emit(buf, "%u\n", error_count);
}
@@ -468,15 +467,22 @@ int amdgpu_xgmi_update_topology(struct amdgpu_hive_info *hive, struct amdgpu_dev
}
+/*
+ * NOTE psp_xgmi_node_info.num_hops layout is as follows:
+ * num_hops[7:6] = link type (0 = xGMI2, 1 = xGMI3, 2/3 = reserved)
+ * num_hops[5:3] = reserved
+ * num_hops[2:0] = number of hops
+ */
int amdgpu_xgmi_get_hops_count(struct amdgpu_device *adev,
struct amdgpu_device *peer_adev)
{
struct psp_xgmi_topology_info *top = &adev->psp.xgmi_context.top_info;
+ uint8_t num_hops_mask = 0x7;
int i;
for (i = 0 ; i < top->num_nodes; ++i)
if (top->nodes[i].node_id == peer_adev->gmc.xgmi.node_id)
- return top->nodes[i].num_hops;
+ return top->nodes[i].num_hops & num_hops_mask;
return -EINVAL;
}
@@ -492,7 +498,8 @@ int amdgpu_xgmi_add_device(struct amdgpu_device *adev)
if (!adev->gmc.xgmi.supported)
return 0;
- if (amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_PSP)) {
+ if (!adev->gmc.xgmi.pending_reset &&
+ amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_PSP)) {
ret = psp_xgmi_initialize(&adev->psp);
if (ret) {
dev_err(adev->dev,
@@ -538,7 +545,8 @@ int amdgpu_xgmi_add_device(struct amdgpu_device *adev)
task_barrier_add_task(&hive->tb);
- if (amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_PSP)) {
+ if (!adev->gmc.xgmi.pending_reset &&
+ amdgpu_device_ip_get_ip_block(adev, AMD_IP_BLOCK_TYPE_PSP)) {
list_for_each_entry(tmp_adev, &hive->device_list, gmc.xgmi.head) {
/* update node list for other device in the hive */
if (tmp_adev != adev) {
@@ -567,7 +575,7 @@ int amdgpu_xgmi_add_device(struct amdgpu_device *adev)
}
}
- if (!ret)
+ if (!ret && !adev->gmc.xgmi.pending_reset)
ret = amdgpu_xgmi_sysfs_add_dev_info(adev, hive);
exit_unlock:
@@ -620,7 +628,7 @@ int amdgpu_xgmi_remove_device(struct amdgpu_device *adev)
return psp_xgmi_terminate(&adev->psp);
}
-int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev)
+static int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev)
{
int r;
struct ras_ih_if ih_info = {
@@ -634,7 +642,7 @@ int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev)
adev->gmc.xgmi.num_physical_nodes == 0)
return 0;
- amdgpu_xgmi_reset_ras_error_count(adev);
+ adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev);
if (!adev->gmc.xgmi.ras_if) {
adev->gmc.xgmi.ras_if = kmalloc(sizeof(struct ras_common_if), GFP_KERNEL);
@@ -656,7 +664,7 @@ int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev)
return r;
}
-void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev)
+static void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev)
{
if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__XGMI_WAFL) &&
adev->gmc.xgmi.ras_if) {
@@ -683,7 +691,7 @@ static void pcs_clear_status(struct amdgpu_device *adev, uint32_t pcs_status_reg
WREG32_PCIE(pcs_status_reg, 0);
}
-void amdgpu_xgmi_reset_ras_error_count(struct amdgpu_device *adev)
+static void amdgpu_xgmi_reset_ras_error_count(struct amdgpu_device *adev)
{
uint32_t i;
@@ -743,8 +751,8 @@ static int amdgpu_xgmi_query_pcs_error_status(struct amdgpu_device *adev,
return 0;
}
-int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status)
+static int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
{
struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
int i;
@@ -793,10 +801,17 @@ int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
break;
}
- amdgpu_xgmi_reset_ras_error_count(adev);
+ adev->gmc.xgmi.ras_funcs->reset_ras_error_count(adev);
err_data->ue_count += ue_cnt;
err_data->ce_count += ce_cnt;
return 0;
}
+
+const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs = {
+ .ras_late_init = amdgpu_xgmi_ras_late_init,
+ .ras_fini = amdgpu_xgmi_ras_fini,
+ .query_ras_error_count = amdgpu_xgmi_query_ras_error_count,
+ .reset_ras_error_count = amdgpu_xgmi_reset_ras_error_count,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.h
index 148560d63554..12969c0830d5 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_xgmi.h
@@ -50,6 +50,7 @@ struct amdgpu_pcs_ras_field {
uint32_t pcs_err_shift;
};
+extern const struct amdgpu_xgmi_ras_funcs xgmi_ras_funcs;
struct amdgpu_hive_info *amdgpu_get_xgmi_hive(struct amdgpu_device *adev);
void amdgpu_put_xgmi_hive(struct amdgpu_hive_info *hive);
int amdgpu_xgmi_update_topology(struct amdgpu_hive_info *hive, struct amdgpu_device *adev);
@@ -58,14 +59,8 @@ int amdgpu_xgmi_remove_device(struct amdgpu_device *adev);
int amdgpu_xgmi_set_pstate(struct amdgpu_device *adev, int pstate);
int amdgpu_xgmi_get_hops_count(struct amdgpu_device *adev,
struct amdgpu_device *peer_adev);
-int amdgpu_xgmi_ras_late_init(struct amdgpu_device *adev);
-void amdgpu_xgmi_ras_fini(struct amdgpu_device *adev);
uint64_t amdgpu_xgmi_get_relative_phy_addr(struct amdgpu_device *adev,
uint64_t addr);
-int amdgpu_xgmi_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status);
-void amdgpu_xgmi_reset_ras_error_count(struct amdgpu_device *adev);
-
static inline bool amdgpu_xgmi_same_hive(struct amdgpu_device *adev,
struct amdgpu_device *bo_adev)
{
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h b/drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h
index 5355827ed0ae..1a8f6d4baab2 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgv_sriovmsg.h
@@ -90,11 +90,22 @@ union amd_sriov_msg_feature_flags {
uint32_t host_flr_vramlost : 1;
uint32_t mm_bw_management : 1;
uint32_t pp_one_vf_mode : 1;
- uint32_t reserved : 27;
+ uint32_t reg_indirect_acc : 1;
+ uint32_t reserved : 26;
} flags;
uint32_t all;
};
+union amd_sriov_reg_access_flags {
+ struct {
+ uint32_t vf_reg_access_ih : 1;
+ uint32_t vf_reg_access_mmhub : 1;
+ uint32_t vf_reg_access_gc : 1;
+ uint32_t reserved : 29;
+ } flags;
+ uint32_t all;
+};
+
union amd_sriov_msg_os_info {
struct {
uint32_t windows : 1;
@@ -149,8 +160,10 @@ struct amd_sriov_msg_pf2vf_info {
/* identification in ROCm SMI */
uint64_t uuid;
uint32_t fcn_idx;
+ /* flags which indicate the register access method VF should use */
+ union amd_sriov_reg_access_flags reg_access_flags;
/* reserved */
- uint32_t reserved[256-26];
+ uint32_t reserved[256-27];
};
struct amd_sriov_msg_vf2pf_info_header {
diff --git a/drivers/gpu/drm/amd/amdgpu/athub_v2_1.c b/drivers/gpu/drm/amd/amdgpu/athub_v2_1.c
index 7b1b18350bf9..2ac4988ea0ff 100644
--- a/drivers/gpu/drm/amd/amdgpu/athub_v2_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/athub_v2_1.c
@@ -74,10 +74,8 @@ int athub_v2_1_set_clockgating(struct amdgpu_device *adev,
case CHIP_SIENNA_CICHLID:
case CHIP_NAVY_FLOUNDER:
case CHIP_DIMGREY_CAVEFISH:
- athub_v2_1_update_medium_grain_clock_gating(adev,
- state == AMD_CG_STATE_GATE ? true : false);
- athub_v2_1_update_medium_grain_light_sleep(adev,
- state == AMD_CG_STATE_GATE ? true : false);
+ athub_v2_1_update_medium_grain_clock_gating(adev, state == AMD_CG_STATE_GATE);
+ athub_v2_1_update_medium_grain_light_sleep(adev, state == AMD_CG_STATE_GATE);
break;
default:
break;
diff --git a/drivers/gpu/drm/amd/amdgpu/atom.c b/drivers/gpu/drm/amd/amdgpu/atom.c
index 515890f4f5a0..3dcb8b32f48b 100644
--- a/drivers/gpu/drm/amd/amdgpu/atom.c
+++ b/drivers/gpu/drm/amd/amdgpu/atom.c
@@ -114,11 +114,11 @@ static uint32_t atom_iio_execute(struct atom_context *ctx, int base,
base++;
break;
case ATOM_IIO_READ:
- temp = ctx->card->ioreg_read(ctx->card, CU16(base + 1));
+ temp = ctx->card->reg_read(ctx->card, CU16(base + 1));
base += 3;
break;
case ATOM_IIO_WRITE:
- ctx->card->ioreg_write(ctx->card, CU16(base + 1), temp);
+ ctx->card->reg_write(ctx->card, CU16(base + 1), temp);
base += 3;
break;
case ATOM_IIO_CLEAR:
diff --git a/drivers/gpu/drm/amd/amdgpu/atom.h b/drivers/gpu/drm/amd/amdgpu/atom.h
index 4205bbe5d8d7..d279759cab47 100644
--- a/drivers/gpu/drm/amd/amdgpu/atom.h
+++ b/drivers/gpu/drm/amd/amdgpu/atom.h
@@ -116,8 +116,6 @@ struct card_info {
struct drm_device *dev;
void (* reg_write)(struct card_info *, uint32_t, uint32_t); /* filled by driver */
uint32_t (* reg_read)(struct card_info *, uint32_t); /* filled by driver */
- void (* ioreg_write)(struct card_info *, uint32_t, uint32_t); /* filled by driver */
- uint32_t (* ioreg_read)(struct card_info *, uint32_t); /* filled by driver */
void (* mc_write)(struct card_info *, uint32_t, uint32_t); /* filled by driver */
uint32_t (* mc_read)(struct card_info *, uint32_t); /* filled by driver */
void (* pll_write)(struct card_info *, uint32_t, uint32_t); /* filled by driver */
diff --git a/drivers/gpu/drm/amd/amdgpu/cik.c b/drivers/gpu/drm/amd/amdgpu/cik.c
index 4d6832cc7fb0..c0fcc41ee574 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik.c
@@ -26,6 +26,8 @@
#include <linux/module.h>
#include <linux/pci.h>
+#include <drm/amdgpu_drm.h>
+
#include "amdgpu.h"
#include "amdgpu_atombios.h"
#include "amdgpu_ih.h"
@@ -70,6 +72,80 @@
#include "amdgpu_amdkfd.h"
#include "dce_virtual.h"
+static const struct amdgpu_video_codec_info cik_video_codecs_encode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs cik_video_codecs_encode =
+{
+ .codec_count = ARRAY_SIZE(cik_video_codecs_encode_array),
+ .codec_array = cik_video_codecs_encode_array,
+};
+
+static const struct amdgpu_video_codec_info cik_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 41,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 4,
+ },
+};
+
+static const struct amdgpu_video_codecs cik_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(cik_video_codecs_decode_array),
+ .codec_array = cik_video_codecs_decode_array,
+};
+
+static int cik_query_video_codecs(struct amdgpu_device *adev, bool encode,
+ const struct amdgpu_video_codecs **codecs)
+{
+ switch (adev->asic_type) {
+ case CHIP_BONAIRE:
+ case CHIP_HAWAII:
+ case CHIP_KAVERI:
+ case CHIP_KABINI:
+ case CHIP_MULLINS:
+ if (encode)
+ *codecs = &cik_video_codecs_encode;
+ else
+ *codecs = &cik_video_codecs_decode;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
/*
* Indirect registers accessor
*/
@@ -1933,6 +2009,7 @@ static const struct amdgpu_asic_funcs cik_asic_funcs =
.get_pcie_replay_count = &cik_get_pcie_replay_count,
.supports_baco = &cik_asic_supports_baco,
.pre_asic_init = &cik_pre_asic_init,
+ .query_video_codecs = &cik_query_video_codecs,
};
static int cik_common_early_init(void *handle)
diff --git a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
index 43b978144b79..c4bb8eed246d 100644
--- a/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
+++ b/drivers/gpu/drm/amd/amdgpu/cik_sdma.c
@@ -984,10 +984,9 @@ static int cik_sdma_sw_init(void *handle)
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
&adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
index 7944781e1086..d1570a462a51 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v10_0.c
@@ -1862,7 +1862,6 @@ static int dce_v10_0_crtc_do_set_base(struct drm_crtc *crtc,
u32 tmp, viewport_w, viewport_h;
int r;
bool bypass_lut = false;
- struct drm_format_name_buf format_name;
/* no fb bound */
if (!atomic && !crtc->primary->fb) {
@@ -1981,8 +1980,8 @@ static int dce_v10_0_crtc_do_set_base(struct drm_crtc *crtc,
#endif
break;
default:
- DRM_ERROR("Unsupported screen format %s\n",
- drm_get_format_name(target_fb->format->format, &format_name));
+ DRM_ERROR("Unsupported screen format %p4cc\n",
+ &target_fb->format->format);
return -EINVAL;
}
@@ -2897,6 +2896,11 @@ static int dce_v10_0_hw_fini(void *handle)
static int dce_v10_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
@@ -2921,8 +2925,10 @@ static int dce_v10_0_resume(void *handle)
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v10_0_is_idle(void *handle)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
index 1b6ff0470011..18a7b3bd633b 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v11_0.c
@@ -1904,7 +1904,6 @@ static int dce_v11_0_crtc_do_set_base(struct drm_crtc *crtc,
u32 tmp, viewport_w, viewport_h;
int r;
bool bypass_lut = false;
- struct drm_format_name_buf format_name;
/* no fb bound */
if (!atomic && !crtc->primary->fb) {
@@ -2023,8 +2022,8 @@ static int dce_v11_0_crtc_do_set_base(struct drm_crtc *crtc,
#endif
break;
default:
- DRM_ERROR("Unsupported screen format %s\n",
- drm_get_format_name(target_fb->format->format, &format_name));
+ DRM_ERROR("Unsupported screen format %p4cc\n",
+ &target_fb->format->format);
return -EINVAL;
}
@@ -3027,6 +3026,11 @@ static int dce_v11_0_hw_fini(void *handle)
static int dce_v11_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
@@ -3051,8 +3055,10 @@ static int dce_v11_0_resume(void *handle)
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v11_0_is_idle(void *handle)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
index 83a88385b762..dbcb09cf83e6 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v6_0.c
@@ -1820,7 +1820,6 @@ static int dce_v6_0_crtc_do_set_base(struct drm_crtc *crtc,
u32 viewport_w, viewport_h;
int r;
bool bypass_lut = false;
- struct drm_format_name_buf format_name;
/* no fb bound */
if (!atomic && !crtc->primary->fb) {
@@ -1929,8 +1928,8 @@ static int dce_v6_0_crtc_do_set_base(struct drm_crtc *crtc,
#endif
break;
default:
- DRM_ERROR("Unsupported screen format %s\n",
- drm_get_format_name(target_fb->format->format, &format_name));
+ DRM_ERROR("Unsupported screen format %p4cc\n",
+ &target_fb->format->format);
return -EINVAL;
}
@@ -2770,7 +2769,11 @@ static int dce_v6_0_hw_fini(void *handle)
static int dce_v6_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
@@ -2794,8 +2797,10 @@ static int dce_v6_0_resume(void *handle)
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v6_0_is_idle(void *handle)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
index 224b30214427..b200b9e722d9 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_v8_0.c
@@ -1791,7 +1791,6 @@ static int dce_v8_0_crtc_do_set_base(struct drm_crtc *crtc,
u32 viewport_w, viewport_h;
int r;
bool bypass_lut = false;
- struct drm_format_name_buf format_name;
/* no fb bound */
if (!atomic && !crtc->primary->fb) {
@@ -1902,8 +1901,8 @@ static int dce_v8_0_crtc_do_set_base(struct drm_crtc *crtc,
#endif
break;
default:
- DRM_ERROR("Unsupported screen format %s\n",
- drm_get_format_name(target_fb->format->format, &format_name));
+ DRM_ERROR("Unsupported screen format %p4cc\n",
+ &target_fb->format->format);
return -EINVAL;
}
@@ -2796,6 +2795,11 @@ static int dce_v8_0_hw_fini(void *handle)
static int dce_v8_0_suspend(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
adev->mode_info.bl_level =
amdgpu_atombios_encoder_get_backlight_level_from_reg(adev);
@@ -2820,8 +2824,10 @@ static int dce_v8_0_resume(void *handle)
amdgpu_display_backlight_set_level(adev, adev->mode_info.bl_encoder,
bl_level);
}
+ if (ret)
+ return ret;
- return ret;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_v8_0_is_idle(void *handle)
diff --git a/drivers/gpu/drm/amd/amdgpu/dce_virtual.c b/drivers/gpu/drm/amd/amdgpu/dce_virtual.c
index 9810af712cc0..5c11144da051 100644
--- a/drivers/gpu/drm/amd/amdgpu/dce_virtual.c
+++ b/drivers/gpu/drm/amd/amdgpu/dce_virtual.c
@@ -39,6 +39,7 @@
#include "dce_v11_0.h"
#include "dce_virtual.h"
#include "ivsrcid/ivsrcid_vislands30.h"
+#include "amdgpu_display.h"
#define DCE_VIRTUAL_VBLANK_PERIOD 16666666
@@ -491,12 +492,24 @@ static int dce_virtual_hw_fini(void *handle)
static int dce_virtual_suspend(void *handle)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = amdgpu_display_suspend_helper(adev);
+ if (r)
+ return r;
return dce_virtual_hw_fini(handle);
}
static int dce_virtual_resume(void *handle)
{
- return dce_virtual_hw_init(handle);
+ struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ int r;
+
+ r = dce_virtual_hw_init(handle);
+ if (r)
+ return r;
+ return amdgpu_display_resume_helper(adev);
}
static bool dce_virtual_is_idle(void *handle)
diff --git a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
index 6b4b30a8dce5..0d8459d63bac 100644
--- a/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
+++ b/drivers/gpu/drm/amd/amdgpu/df_v3_6.c
@@ -205,7 +205,7 @@ static ssize_t df_v3_6_get_df_cntr_avail(struct device *dev,
count++;
}
- return snprintf(buf, PAGE_SIZE, "%i\n", count);
+ return sysfs_emit(buf, "%i\n", count);
}
/* device attr for available perfmon counters */
@@ -568,6 +568,8 @@ static int df_v3_6_pmc_stop(struct amdgpu_device *adev, uint64_t config,
if (ret)
return ret;
+ df_v3_6_perfmon_wreg(adev, lo_base_addr, lo_val,
+ hi_base_addr, hi_val);
if (is_remove) {
df_v3_6_reset_perfmon_cntr(adev, config, counter_idx);
diff --git a/drivers/gpu/drm/amd/amdgpu/dimgrey_cavefish_reg_init.c b/drivers/gpu/drm/amd/amdgpu/dimgrey_cavefish_reg_init.c
index e9f177e9e3cf..e9f177e9e3cf 100755..100644
--- a/drivers/gpu/drm/amd/amdgpu/dimgrey_cavefish_reg_init.c
+++ b/drivers/gpu/drm/amd/amdgpu/dimgrey_cavefish_reg_init.c
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
index 45d1172b7bff..2408ed4c7d84 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c
@@ -29,7 +29,6 @@
#include "amdgpu.h"
#include "amdgpu_gfx.h"
#include "amdgpu_psp.h"
-#include "amdgpu_smu.h"
#include "nv.h"
#include "nvd.h"
@@ -174,6 +173,11 @@
#define mmGC_THROTTLE_CTRL_Sienna_Cichlid 0x2030
#define mmGC_THROTTLE_CTRL_Sienna_Cichlid_BASE_IDX 0
+#define GFX_RLCG_GC_WRITE_OLD (0x8 << 28)
+#define GFX_RLCG_GC_WRITE (0x0 << 28)
+#define GFX_RLCG_GC_READ (0x1 << 28)
+#define GFX_RLCG_MMHUB_WRITE (0x2 << 28)
+
MODULE_FIRMWARE("amdgpu/navi10_ce.bin");
MODULE_FIRMWARE("amdgpu/navi10_pfp.bin");
MODULE_FIRMWARE("amdgpu/navi10_me.bin");
@@ -1419,38 +1423,127 @@ static const struct soc15_reg_golden golden_settings_gc_10_1_2[] =
SOC15_REG_GOLDEN_VALUE(GC, 0, mmUTCL1_CTRL, 0xffffffff, 0x00800000)
};
-static void gfx_v10_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 v)
+static bool gfx_v10_is_rlcg_rw(struct amdgpu_device *adev, u32 offset, uint32_t *flag, bool write)
+{
+ /* always programed by rlcg, only for gc */
+ if (offset == SOC15_REG_OFFSET(GC, 0, mmRLC_CSIB_ADDR_HI) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmRLC_CSIB_ADDR_LO) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmRLC_CSIB_LENGTH) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_CNTL) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmGRBM_GFX_INDEX) ||
+ offset == SOC15_REG_OFFSET(GC, 0, mmCP_ME_CNTL)) {
+ if (!amdgpu_sriov_reg_indirect_gc(adev))
+ *flag = GFX_RLCG_GC_WRITE_OLD;
+ else
+ *flag = write ? GFX_RLCG_GC_WRITE : GFX_RLCG_GC_READ;
+
+ return true;
+ }
+
+ /* currently support gc read/write, mmhub write */
+ if (offset >= SOC15_REG_OFFSET(GC, 0, mmSDMA0_DEC_START) &&
+ offset <= SOC15_REG_OFFSET(GC, 0, mmRLC_GTS_OFFSET_MSB)) {
+ if (amdgpu_sriov_reg_indirect_gc(adev))
+ *flag = write ? GFX_RLCG_GC_WRITE : GFX_RLCG_GC_READ;
+ else
+ return false;
+ } else {
+ if (amdgpu_sriov_reg_indirect_mmhub(adev))
+ *flag = GFX_RLCG_MMHUB_WRITE;
+ else
+ return false;
+ }
+
+ return true;
+}
+
+static u32 gfx_v10_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, uint32_t flag)
{
static void *scratch_reg0;
static void *scratch_reg1;
+ static void *scratch_reg2;
+ static void *scratch_reg3;
static void *spare_int;
+ static uint32_t grbm_cntl;
+ static uint32_t grbm_idx;
uint32_t i = 0;
uint32_t retries = 50000;
+ u32 ret = 0;
+
+ scratch_reg0 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0) * 4;
+ scratch_reg1 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1) * 4;
+ scratch_reg2 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG2) * 4;
+ scratch_reg3 = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3) * 4;
+ spare_int = adev->rmmio +
+ (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT) * 4;
+
+ grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL;
+ grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX;
+
+ if (offset == grbm_cntl || offset == grbm_idx) {
+ if (offset == grbm_cntl)
+ writel(v, scratch_reg2);
+ else if (offset == grbm_idx)
+ writel(v, scratch_reg3);
+
+ writel(v, ((void __iomem *)adev->rmmio) + (offset * 4));
+ } else {
+ writel(v, scratch_reg0);
+ writel(offset | flag, scratch_reg1);
+ writel(1, spare_int);
+ for (i = 0; i < retries; i++) {
+ u32 tmp;
+
+ tmp = readl(scratch_reg1);
+ if (!(tmp & flag))
+ break;
- scratch_reg0 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0)*4;
- scratch_reg1 = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1)*4;
- spare_int = adev->rmmio + (adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT)*4;
+ udelay(10);
+ }
- if (amdgpu_sriov_runtime(adev)) {
- pr_err("shouldn't call rlcg write register during runtime\n");
- return;
+ if (i >= retries)
+ pr_err("timeout: rlcg program reg:0x%05x failed !\n", offset);
}
- writel(v, scratch_reg0);
- writel(offset | 0x80000000, scratch_reg1);
- writel(1, spare_int);
- for (i = 0; i < retries; i++) {
- u32 tmp;
+ ret = readl(scratch_reg0);
- tmp = readl(scratch_reg1);
- if (!(tmp & 0x80000000))
- break;
+ return ret;
+}
+
+static void gfx_v10_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 value, u32 flag)
+{
+ uint32_t rlcg_flag;
- udelay(10);
+ if (amdgpu_sriov_fullaccess(adev) &&
+ gfx_v10_is_rlcg_rw(adev, offset, &rlcg_flag, 1)) {
+ gfx_v10_rlcg_rw(adev, offset, value, rlcg_flag);
+
+ return;
}
+ if (flag & AMDGPU_REGS_NO_KIQ)
+ WREG32_NO_KIQ(offset, value);
+ else
+ WREG32(offset, value);
+}
- if (i >= retries)
- pr_err("timeout: rlcg program reg:0x%05x failed !\n", offset);
+static u32 gfx_v10_rlcg_rreg(struct amdgpu_device *adev, u32 offset, u32 flag)
+{
+ uint32_t rlcg_flag;
+
+ if (amdgpu_sriov_fullaccess(adev) &&
+ gfx_v10_is_rlcg_rw(adev, offset, &rlcg_flag, 0))
+ return gfx_v10_rlcg_rw(adev, offset, 0, rlcg_flag);
+
+ if (flag & AMDGPU_REGS_NO_KIQ)
+ return RREG32_NO_KIQ(offset);
+ else
+ return RREG32(offset);
+
+ return 0;
}
static const struct soc15_reg_golden golden_settings_gc_10_1_nv14[] =
@@ -3280,7 +3373,7 @@ static const struct soc15_reg_golden golden_settings_gc_10_3_4[] =
SOC15_REG_GOLDEN_VALUE(GC, 0, mmCPF_GCR_CNTL, 0x0007ffff, 0x0000c000),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG3, 0x00000280, 0x00000280),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmDB_DEBUG4, 0x07800000, 0x00800000),
- SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL, 0x00001d00, 0x00000500),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, mmGCR_GENERAL_CNTL_Sienna_Cichlid, 0x00001d00, 0x00000500),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmGE_PC_CNTL, 0x003c0000, 0x00280400),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2A_ADDR_MATCH_MASK, 0xffffffff, 0xffffffcf),
SOC15_REG_GOLDEN_VALUE(GC, 0, mmGL2C_ADDR_MATCH_MASK, 0xffffffff, 0xffffffcf),
@@ -4459,9 +4552,8 @@ static int gfx_v10_0_gfx_ring_init(struct amdgpu_device *adev, int ring_id,
sprintf(ring->name, "gfx_%d.%d.%d", ring->me, ring->pipe, ring->queue);
irq_type = AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP + ring->pipe;
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type,
- AMDGPU_RING_PRIO_DEFAULT);
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
return 0;
@@ -4495,8 +4587,8 @@ static int gfx_v10_0_compute_ring_init(struct amdgpu_device *adev, int ring_id,
hw_prio = amdgpu_gfx_is_high_priority_compute_queue(adev, ring) ?
AMDGPU_GFX_PIPE_PRIO_HIGH : AMDGPU_GFX_PIPE_PRIO_NORMAL;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type, hw_prio);
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ hw_prio, NULL);
if (r)
return r;
@@ -7172,16 +7264,10 @@ static int gfx_v10_0_hw_init(void *handle)
* loaded firstly, so in direct type, it has to load smc ucode
* here before rlc.
*/
- if (adev->smu.ppt_funcs != NULL && !(adev->flags & AMD_IS_APU)) {
- r = smu_load_microcode(&adev->smu);
+ if (!(adev->flags & AMD_IS_APU)) {
+ r = amdgpu_pm_load_smu_firmware(adev, NULL);
if (r)
return r;
-
- r = smu_check_fw_status(&adev->smu);
- if (r) {
- pr_err("SMC firmware status is not correct\n");
- return r;
- }
}
gfx_v10_0_disable_gpa_mode(adev);
}
@@ -7892,6 +7978,7 @@ static const struct amdgpu_rlc_funcs gfx_v10_0_rlc_funcs_sriov = {
.start = gfx_v10_0_rlc_start,
.update_spm_vmid = gfx_v10_0_update_spm_vmid,
.rlcg_wreg = gfx_v10_rlcg_wreg,
+ .rlcg_rreg = gfx_v10_rlcg_rreg,
.is_rlcg_access_range = gfx_v10_0_is_rlcg_access_range,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
index ca74638dec9b..3a8d52a54873 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
@@ -3114,7 +3114,7 @@ static int gfx_v6_0_sw_init(void *handle)
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -3137,7 +3137,7 @@ static int gfx_v6_0_sw_init(void *handle)
irq_type = AMDGPU_CP_IRQ_COMPUTE_MEC1_PIPE0_EOP + ring->pipe;
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq, irq_type,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
index a368724c3dfc..c35fdd2ef2d4 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v7_0.c
@@ -1877,7 +1877,7 @@ static void gfx_v7_0_init_compute_vmid(struct amdgpu_device *adev)
mutex_unlock(&adev->srbm_mutex);
/* Initialize all compute VMIDs to have no GDS, GWS, or OA
- acccess. These should be enabled by FW for target VMIDs. */
+ access. These should be enabled by FW for target VMIDs. */
for (i = adev->vm_manager.first_kfd_vmid; i < AMDGPU_NUM_VMID; i++) {
WREG32(amdgpu_gds_reg_offset[i].mem_base, 0);
WREG32(amdgpu_gds_reg_offset[i].mem_size, 0);
@@ -2058,7 +2058,7 @@ static void gfx_v7_0_constants_init(struct amdgpu_device *adev)
* @adev: amdgpu_device pointer
*
* Set up the number and offset of the CP scratch registers.
- * NOTE: use of CP scratch registers is a legacy inferface and
+ * NOTE: use of CP scratch registers is a legacy interface and
* is not used by default on newer asics (r6xx+). On newer asics,
* memory buffers are used for fences rather than scratch regs.
*/
@@ -2172,7 +2172,7 @@ static void gfx_v7_0_ring_emit_vgt_flush(struct amdgpu_ring *ring)
* @seq: sequence number
* @flags: fence related flags
*
- * Emits a fence sequnce number on the gfx ring and flushes
+ * Emits a fence sequence number on the gfx ring and flushes
* GPU caches.
*/
static void gfx_v7_0_ring_emit_fence_gfx(struct amdgpu_ring *ring, u64 addr,
@@ -2215,7 +2215,7 @@ static void gfx_v7_0_ring_emit_fence_gfx(struct amdgpu_ring *ring, u64 addr,
* @seq: sequence number
* @flags: fence related flags
*
- * Emits a fence sequnce number on the compute ring and flushes
+ * Emits a fence sequence number on the compute ring and flushes
* GPU caches.
*/
static void gfx_v7_0_ring_emit_fence_compute(struct amdgpu_ring *ring,
@@ -2245,14 +2245,14 @@ static void gfx_v7_0_ring_emit_fence_compute(struct amdgpu_ring *ring,
* gfx_v7_0_ring_emit_ib - emit an IB (Indirect Buffer) on the ring
*
* @ring: amdgpu_ring structure holding ring information
- * @job: job to retrive vmid from
+ * @job: job to retrieve vmid from
* @ib: amdgpu indirect buffer object
* @flags: options (AMDGPU_HAVE_CTX_SWITCH)
*
* Emits an DE (drawing engine) or CE (constant engine) IB
* on the gfx ring. IBs are usually generated by userspace
* acceleration drivers and submitted to the kernel for
- * sheduling on the ring. This function schedules the IB
+ * scheduling on the ring. This function schedules the IB
* on the gfx ring for execution by the GPU.
*/
static void gfx_v7_0_ring_emit_ib_gfx(struct amdgpu_ring *ring,
@@ -2402,7 +2402,7 @@ err1:
/*
* CP.
- * On CIK, gfx and compute now have independant command processors.
+ * On CIK, gfx and compute now have independent command processors.
*
* GFX
* Gfx consists of a single ring and can process both gfx jobs and
@@ -2630,7 +2630,7 @@ static int gfx_v7_0_cp_gfx_resume(struct amdgpu_device *adev)
ring->wptr = 0;
WREG32(mmCP_RB0_WPTR, lower_32_bits(ring->wptr));
- /* set the wb address wether it's enabled or not */
+ /* set the wb address whether it's enabled or not */
rptr_addr = adev->wb.gpu_addr + (ring->rptr_offs * 4);
WREG32(mmCP_RB0_RPTR_ADDR, lower_32_bits(rptr_addr));
WREG32(mmCP_RB0_RPTR_ADDR_HI, upper_32_bits(rptr_addr) & 0xFF);
@@ -2985,7 +2985,7 @@ static void gfx_v7_0_mqd_init(struct amdgpu_device *adev,
mqd->cp_hqd_pq_wptr_poll_addr_lo = wb_gpu_addr & 0xfffffffc;
mqd->cp_hqd_pq_wptr_poll_addr_hi = upper_32_bits(wb_gpu_addr) & 0xffff;
- /* set the wb address wether it's enabled or not */
+ /* set the wb address whether it's enabled or not */
wb_gpu_addr = adev->wb.gpu_addr + (ring->rptr_offs * 4);
mqd->cp_hqd_pq_rptr_report_addr_lo = wb_gpu_addr & 0xfffffffc;
mqd->cp_hqd_pq_rptr_report_addr_hi =
@@ -3198,7 +3198,7 @@ static int gfx_v7_0_cp_resume(struct amdgpu_device *adev)
/**
* gfx_v7_0_ring_emit_vm_flush - cik vm flush using the CP
*
- * @ring: the ring to emmit the commands to
+ * @ring: the ring to emit the commands to
*
* Sync the command pipeline with the PFP. E.g. wait for everything
* to be completed.
@@ -3220,7 +3220,7 @@ static void gfx_v7_0_ring_emit_pipeline_sync(struct amdgpu_ring *ring)
amdgpu_ring_write(ring, 4); /* poll interval */
if (usepfp) {
- /* synce CE with ME to prevent CE fetch CEIB before context switch done */
+ /* sync CE with ME to prevent CE fetch CEIB before context switch done */
amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
amdgpu_ring_write(ring, 0);
amdgpu_ring_write(ring, PACKET3(PACKET3_SWITCH_BUFFER, 0));
@@ -4438,7 +4438,7 @@ static int gfx_v7_0_compute_ring_init(struct amdgpu_device *adev, int ring_id,
/* type-2 packets are deprecated on MEC, use type-3 instead */
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq, irq_type,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
@@ -4512,7 +4512,7 @@ static int gfx_v7_0_sw_init(void *handle)
r = amdgpu_ring_init(adev, ring, 1024,
&adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
index 84d2eaa38101..c26e06059466 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v8_0.c
@@ -1927,8 +1927,8 @@ static int gfx_v8_0_compute_ring_init(struct amdgpu_device *adev, int ring_id,
hw_prio = amdgpu_gfx_is_high_priority_compute_queue(adev, ring) ?
AMDGPU_GFX_PIPE_PRIO_HIGH : AMDGPU_RING_PRIO_DEFAULT;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type, hw_prio);
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ hw_prio, NULL);
if (r)
return r;
@@ -2033,7 +2033,7 @@ static int gfx_v8_0_sw_init(void *handle)
r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -6718,7 +6718,8 @@ static int gfx_v8_0_cp_ecc_error_irq(struct amdgpu_device *adev,
return 0;
}
-static void gfx_v8_0_parse_sq_irq(struct amdgpu_device *adev, unsigned ih_data)
+static void gfx_v8_0_parse_sq_irq(struct amdgpu_device *adev, unsigned ih_data,
+ bool from_wq)
{
u32 enc, se_id, sh_id, cu_id;
char type[20];
@@ -6756,7 +6757,7 @@ static void gfx_v8_0_parse_sq_irq(struct amdgpu_device *adev, unsigned ih_data)
* or from BH in which case we can access SQ_EDC_INFO
* instance
*/
- if (in_task()) {
+ if (from_wq) {
mutex_lock(&adev->grbm_idx_mutex);
gfx_v8_0_select_se_sh(adev, se_id, sh_id, cu_id);
@@ -6794,7 +6795,7 @@ static void gfx_v8_0_sq_irq_work_func(struct work_struct *work)
struct amdgpu_device *adev = container_of(work, struct amdgpu_device, gfx.sq_work.work);
struct sq_work *sq_work = container_of(work, struct sq_work, work);
- gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data);
+ gfx_v8_0_parse_sq_irq(adev, sq_work->ih_data, true);
}
static int gfx_v8_0_sq_irq(struct amdgpu_device *adev,
@@ -6809,7 +6810,7 @@ static int gfx_v8_0_sq_irq(struct amdgpu_device *adev,
* just print whatever info is possible directly from the ISR.
*/
if (work_pending(&adev->gfx.sq_work.work)) {
- gfx_v8_0_parse_sq_irq(adev, ih_data);
+ gfx_v8_0_parse_sq_irq(adev, ih_data, false);
} else {
adev->gfx.sq_work.ih_data = ih_data;
schedule_work(&adev->gfx.sq_work.work);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
index 65db88bb6cbc..a078a38c2cee 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c
@@ -49,6 +49,7 @@
#include "gfx_v9_4.h"
#include "gfx_v9_0.h"
+#include "gfx_v9_4_2.h"
#include "asic_reg/pwr/pwr_10_0_offset.h"
#include "asic_reg/pwr/pwr_10_0_sh_mask.h"
@@ -107,14 +108,12 @@ MODULE_FIRMWARE("amdgpu/raven2_rlc.bin");
MODULE_FIRMWARE("amdgpu/raven_kicker_rlc.bin");
MODULE_FIRMWARE("amdgpu/arcturus_mec.bin");
-MODULE_FIRMWARE("amdgpu/arcturus_mec2.bin");
MODULE_FIRMWARE("amdgpu/arcturus_rlc.bin");
MODULE_FIRMWARE("amdgpu/renoir_ce.bin");
MODULE_FIRMWARE("amdgpu/renoir_pfp.bin");
MODULE_FIRMWARE("amdgpu/renoir_me.bin");
MODULE_FIRMWARE("amdgpu/renoir_mec.bin");
-MODULE_FIRMWARE("amdgpu/renoir_mec2.bin");
MODULE_FIRMWARE("amdgpu/renoir_rlc.bin");
MODULE_FIRMWARE("amdgpu/green_sardine_ce.bin");
@@ -124,6 +123,10 @@ MODULE_FIRMWARE("amdgpu/green_sardine_mec.bin");
MODULE_FIRMWARE("amdgpu/green_sardine_mec2.bin");
MODULE_FIRMWARE("amdgpu/green_sardine_rlc.bin");
+MODULE_FIRMWARE("amdgpu/aldebaran_mec.bin");
+MODULE_FIRMWARE("amdgpu/aldebaran_mec2.bin");
+MODULE_FIRMWARE("amdgpu/aldebaran_rlc.bin");
+
#define mmTCP_CHAN_STEER_0_ARCT 0x0b03
#define mmTCP_CHAN_STEER_0_ARCT_BASE_IDX 0
#define mmTCP_CHAN_STEER_1_ARCT 0x0b04
@@ -731,7 +734,7 @@ static const u32 GFX_RLC_SRM_INDEX_CNTL_DATA_OFFSETS[] =
mmRLC_SRM_INDEX_CNTL_DATA_7 - mmRLC_SRM_INDEX_CNTL_DATA_0,
};
-static void gfx_v9_0_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 v)
+static void gfx_v9_0_rlcg_rw(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag)
{
static void *scratch_reg0;
static void *scratch_reg1;
@@ -784,6 +787,20 @@ static void gfx_v9_0_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 v)
}
+static void gfx_v9_0_rlcg_wreg(struct amdgpu_device *adev, u32 offset, u32 v, u32 flag)
+{
+ if (amdgpu_sriov_fullaccess(adev)) {
+ gfx_v9_0_rlcg_rw(adev, offset, v, flag);
+
+ return;
+ }
+
+ if (flag & AMDGPU_REGS_NO_KIQ)
+ WREG32_NO_KIQ(offset, v);
+ else
+ WREG32(offset, v);
+}
+
#define VEGA10_GB_ADDR_CONFIG_GOLDEN 0x2a114042
#define VEGA12_GB_ADDR_CONFIG_GOLDEN 0x24104041
#define RAVEN_GB_ADDR_CONFIG_GOLDEN 0x24000042
@@ -981,11 +998,16 @@ static void gfx_v9_0_init_golden_registers(struct amdgpu_device *adev)
golden_settings_gc_9_1_rn,
ARRAY_SIZE(golden_settings_gc_9_1_rn));
return; /* for renoir, don't need common goldensetting */
+ case CHIP_ALDEBARAN:
+ gfx_v9_4_2_init_golden_registers(adev,
+ adev->smuio.funcs->get_die_id(adev));
+ break;
default:
break;
}
- if (adev->asic_type != CHIP_ARCTURUS)
+ if ((adev->asic_type != CHIP_ARCTURUS) &&
+ (adev->asic_type != CHIP_ALDEBARAN))
soc15_program_register_sequence(adev, golden_settings_gc_9_x_common,
(const u32)ARRAY_SIZE(golden_settings_gc_9_x_common));
}
@@ -1517,6 +1539,16 @@ out:
return err;
}
+static bool gfx_v9_0_load_mec2_fw_bin_support(struct amdgpu_device *adev)
+{
+ if (adev->asic_type == CHIP_ALDEBARAN ||
+ adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_RENOIR)
+ return false;
+
+ return true;
+}
+
static int gfx_v9_0_init_cp_compute_microcode(struct amdgpu_device *adev,
const char *chip_name)
{
@@ -1538,21 +1570,26 @@ static int gfx_v9_0_init_cp_compute_microcode(struct amdgpu_device *adev,
adev->gfx.mec_feature_version = le32_to_cpu(cp_hdr->ucode_feature_version);
- snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_mec2.bin", chip_name);
- err = request_firmware(&adev->gfx.mec2_fw, fw_name, adev->dev);
- if (!err) {
- err = amdgpu_ucode_validate(adev->gfx.mec2_fw);
- if (err)
- goto out;
- cp_hdr = (const struct gfx_firmware_header_v1_0 *)
- adev->gfx.mec2_fw->data;
- adev->gfx.mec2_fw_version =
- le32_to_cpu(cp_hdr->header.ucode_version);
- adev->gfx.mec2_feature_version =
- le32_to_cpu(cp_hdr->ucode_feature_version);
+ if (gfx_v9_0_load_mec2_fw_bin_support(adev)) {
+ snprintf(fw_name, sizeof(fw_name), "amdgpu/%s_mec2.bin", chip_name);
+ err = request_firmware(&adev->gfx.mec2_fw, fw_name, adev->dev);
+ if (!err) {
+ err = amdgpu_ucode_validate(adev->gfx.mec2_fw);
+ if (err)
+ goto out;
+ cp_hdr = (const struct gfx_firmware_header_v1_0 *)
+ adev->gfx.mec2_fw->data;
+ adev->gfx.mec2_fw_version =
+ le32_to_cpu(cp_hdr->header.ucode_version);
+ adev->gfx.mec2_feature_version =
+ le32_to_cpu(cp_hdr->ucode_feature_version);
+ } else {
+ err = 0;
+ adev->gfx.mec2_fw = NULL;
+ }
} else {
- err = 0;
- adev->gfx.mec2_fw = NULL;
+ adev->gfx.mec2_fw_version = adev->gfx.mec_fw_version;
+ adev->gfx.mec2_feature_version = adev->gfx.mec_feature_version;
}
if (adev->firmware.load_type == AMDGPU_FW_LOAD_PSP) {
@@ -1581,8 +1618,7 @@ static int gfx_v9_0_init_cp_compute_microcode(struct amdgpu_device *adev,
/* TODO: Determine if MEC2 JT FW loading can be removed
for all GFX V9 asic and above */
- if (adev->asic_type != CHIP_ARCTURUS &&
- adev->asic_type != CHIP_RENOIR) {
+ if (gfx_v9_0_load_mec2_fw_bin_support(adev)) {
info = &adev->firmware.ucode[AMDGPU_UCODE_ID_CP_MEC2_JT];
info->ucode_id = AMDGPU_UCODE_ID_CP_MEC2_JT;
info->fw = adev->gfx.mec2_fw;
@@ -1642,6 +1678,9 @@ static int gfx_v9_0_init_microcode(struct amdgpu_device *adev)
else
chip_name = "green_sardine";
break;
+ case CHIP_ALDEBARAN:
+ chip_name = "aldebaran";
+ break;
default:
BUG();
}
@@ -1882,7 +1921,10 @@ static void gfx_v9_0_enable_lbpw(struct amdgpu_device *adev, bool enable)
static int gfx_v9_0_cp_jump_table_num(struct amdgpu_device *adev)
{
- return 5;
+ if (gfx_v9_0_load_mec2_fw_bin_support(adev))
+ return 5;
+ else
+ return 4;
}
static int gfx_v9_0_rlc_init(struct amdgpu_device *adev)
@@ -2064,30 +2106,22 @@ static void gfx_v9_0_select_me_pipe_q(struct amdgpu_device *adev,
}
static const struct amdgpu_gfx_funcs gfx_v9_0_gfx_funcs = {
- .get_gpu_clock_counter = &gfx_v9_0_get_gpu_clock_counter,
- .select_se_sh = &gfx_v9_0_select_se_sh,
- .read_wave_data = &gfx_v9_0_read_wave_data,
- .read_wave_sgprs = &gfx_v9_0_read_wave_sgprs,
- .read_wave_vgprs = &gfx_v9_0_read_wave_vgprs,
- .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
+ .get_gpu_clock_counter = &gfx_v9_0_get_gpu_clock_counter,
+ .select_se_sh = &gfx_v9_0_select_se_sh,
+ .read_wave_data = &gfx_v9_0_read_wave_data,
+ .read_wave_sgprs = &gfx_v9_0_read_wave_sgprs,
+ .read_wave_vgprs = &gfx_v9_0_read_wave_vgprs,
+ .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
+};
+
+static const struct amdgpu_gfx_ras_funcs gfx_v9_0_ras_funcs = {
+ .ras_late_init = amdgpu_gfx_ras_late_init,
+ .ras_fini = amdgpu_gfx_ras_fini,
.ras_error_inject = &gfx_v9_0_ras_error_inject,
.query_ras_error_count = &gfx_v9_0_query_ras_error_count,
.reset_ras_error_count = &gfx_v9_0_reset_ras_error_count,
};
-static const struct amdgpu_gfx_funcs gfx_v9_4_gfx_funcs = {
- .get_gpu_clock_counter = &gfx_v9_0_get_gpu_clock_counter,
- .select_se_sh = &gfx_v9_0_select_se_sh,
- .read_wave_data = &gfx_v9_0_read_wave_data,
- .read_wave_sgprs = &gfx_v9_0_read_wave_sgprs,
- .read_wave_vgprs = &gfx_v9_0_read_wave_vgprs,
- .select_me_pipe_q = &gfx_v9_0_select_me_pipe_q,
- .ras_error_inject = &gfx_v9_4_ras_error_inject,
- .query_ras_error_count = &gfx_v9_4_query_ras_error_count,
- .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count,
- .query_ras_error_status = &gfx_v9_4_query_ras_error_status,
-};
-
static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)
{
u32 gb_addr_config;
@@ -2114,6 +2148,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)
DRM_INFO("fix gfx.config for vega12\n");
break;
case CHIP_VEGA20:
+ adev->gfx.ras_funcs = &gfx_v9_0_ras_funcs;
adev->gfx.config.max_hw_contexts = 8;
adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
@@ -2139,7 +2174,7 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)
gb_addr_config = RAVEN_GB_ADDR_CONFIG_GOLDEN;
break;
case CHIP_ARCTURUS:
- adev->gfx.funcs = &gfx_v9_4_gfx_funcs;
+ adev->gfx.ras_funcs = &gfx_v9_4_ras_funcs;
adev->gfx.config.max_hw_contexts = 8;
adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
@@ -2159,6 +2194,21 @@ static int gfx_v9_0_gpu_early_init(struct amdgpu_device *adev)
gb_addr_config &= ~0xf3e777ff;
gb_addr_config |= 0x22010042;
break;
+ case CHIP_ALDEBARAN:
+ adev->gfx.ras_funcs = &gfx_v9_4_2_ras_funcs;
+ adev->gfx.config.max_hw_contexts = 8;
+ adev->gfx.config.sc_prim_fifo_size_frontend = 0x20;
+ adev->gfx.config.sc_prim_fifo_size_backend = 0x100;
+ adev->gfx.config.sc_hiz_tile_fifo_size = 0x30;
+ adev->gfx.config.sc_earlyz_tile_fifo_size = 0x4C0;
+ gb_addr_config = RREG32_SOC15(GC, 0, mmGB_ADDR_CONFIG);
+ gb_addr_config &= ~0xf3e777ff;
+ gb_addr_config |= 0x22014042;
+ /* check vbios table if gpu info is not available */
+ err = amdgpu_atomfirmware_get_gfx_info(adev);
+ if (err)
+ return err;
+ break;
default:
BUG();
break;
@@ -2231,8 +2281,8 @@ static int gfx_v9_0_compute_ring_init(struct amdgpu_device *adev, int ring_id,
hw_prio = amdgpu_gfx_is_high_priority_compute_queue(adev, ring) ?
AMDGPU_GFX_PIPE_PRIO_HIGH : AMDGPU_GFX_PIPE_PRIO_NORMAL;
/* type-2 packets are deprecated on MEC, use type-3 instead */
- return amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq, irq_type, hw_prio);
+ return amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq, irq_type,
+ hw_prio, NULL);
}
static int gfx_v9_0_sw_init(void *handle)
@@ -2249,6 +2299,7 @@ static int gfx_v9_0_sw_init(void *handle)
case CHIP_RAVEN:
case CHIP_ARCTURUS:
case CHIP_RENOIR:
+ case CHIP_ALDEBARAN:
adev->gfx.mec.num_mec = 2;
break;
default:
@@ -2320,10 +2371,9 @@ static int gfx_v9_0_sw_init(void *handle)
sprintf(ring->name, "gfx_%d", i);
ring->use_doorbell = true;
ring->doorbell_index = adev->doorbell_index.gfx_ring0 << 1;
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->gfx.eop_irq,
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->gfx.eop_irq,
AMDGPU_CP_IRQ_GFX_ME0_PIPE0_EOP,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -2378,7 +2428,9 @@ static int gfx_v9_0_sw_fini(void *handle)
int i;
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_gfx_ras_fini(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->ras_fini)
+ adev->gfx.ras_funcs->ras_fini(adev);
for (i = 0; i < adev->gfx.num_gfx_rings; i++)
amdgpu_ring_fini(&adev->gfx.gfx_ring[i]);
@@ -2634,17 +2686,15 @@ static void gfx_v9_0_enable_gui_idle_interrupt(struct amdgpu_device *adev,
{
u32 tmp;
- /* don't toggle interrupts that are only applicable
- * to me0 pipe0 on AISCs that have me0 removed */
- if (!adev->gfx.num_gfx_rings)
- return;
+ /* These interrupts should be enabled to drive DS clock */
tmp= RREG32_SOC15(GC, 0, mmCP_INT_CNTL_RING0);
tmp = REG_SET_FIELD(tmp, CP_INT_CNTL_RING0, CNTX_BUSY_INT_ENABLE, enable ? 1 : 0);
tmp = REG_SET_FIELD(tmp, CP_INT_CNTL_RING0, CNTX_EMPTY_INT_ENABLE, enable ? 1 : 0);
tmp = REG_SET_FIELD(tmp, CP_INT_CNTL_RING0, CMP_BUSY_INT_ENABLE, enable ? 1 : 0);
- tmp = REG_SET_FIELD(tmp, CP_INT_CNTL_RING0, GFX_IDLE_INT_ENABLE, enable ? 1 : 0);
+ if(adev->gfx.num_gfx_rings)
+ tmp = REG_SET_FIELD(tmp, CP_INT_CNTL_RING0, GFX_IDLE_INT_ENABLE, enable ? 1 : 0);
WREG32_SOC15(GC, 0, mmCP_INT_CNTL_RING0, tmp);
}
@@ -3696,11 +3746,18 @@ static int gfx_v9_0_kiq_init_queue(struct amdgpu_ring *ring)
struct amdgpu_device *adev = ring->adev;
struct v9_mqd *mqd = ring->mqd_ptr;
int mqd_idx = AMDGPU_MAX_COMPUTE_RINGS;
+ struct v9_mqd *tmp_mqd;
gfx_v9_0_kiq_setting(ring);
- if (amdgpu_in_reset(adev)) { /* for GPU_RESET case */
- /* reset MQD to a clean status */
+ /* GPU could be in bad state during probe, driver trigger the reset
+ * after load the SMU, in this case , the mqd is not be initialized.
+ * driver need to re-init the mqd.
+ * check mqd->cp_hqd_pq_control since this value should not be 0
+ */
+ tmp_mqd = (struct v9_mqd *)adev->gfx.mec.mqd_backup[mqd_idx];
+ if (amdgpu_in_reset(adev) && tmp_mqd->cp_hqd_pq_control){
+ /* for GPU_RESET case , reset MQD to a clean status */
if (adev->gfx.mec.mqd_backup[mqd_idx])
memcpy(mqd, adev->gfx.mec.mqd_backup[mqd_idx], sizeof(struct v9_mqd_allocation));
@@ -3736,8 +3793,15 @@ static int gfx_v9_0_kcq_init_queue(struct amdgpu_ring *ring)
struct amdgpu_device *adev = ring->adev;
struct v9_mqd *mqd = ring->mqd_ptr;
int mqd_idx = ring - &adev->gfx.compute_ring[0];
+ struct v9_mqd *tmp_mqd;
- if (!amdgpu_in_reset(adev) && !adev->in_suspend) {
+ /* Same as above kiq init, driver need to re-init the mqd if mqd->cp_hqd_pq_control
+ * is not be initialized before
+ */
+ tmp_mqd = (struct v9_mqd *)adev->gfx.mec.mqd_backup[mqd_idx];
+
+ if (!tmp_mqd->cp_hqd_pq_control ||
+ (!amdgpu_in_reset(adev) && !adev->in_suspend)) {
memset((void *)mqd, 0, sizeof(struct v9_mqd_allocation));
((struct v9_mqd_allocation *)mqd)->dynamic_cu_mask = 0xFFFFFFFF;
((struct v9_mqd_allocation *)mqd)->dynamic_rb_mask = 0xFFFFFFFF;
@@ -3913,6 +3977,9 @@ static int gfx_v9_0_hw_init(void *handle)
if (r)
return r;
+ if (adev->asic_type == CHIP_ALDEBARAN)
+ gfx_v9_4_2_set_power_brake_sequence(adev);
+
return r;
}
@@ -3954,8 +4021,14 @@ static int gfx_v9_0_hw_fini(void *handle)
}
gfx_v9_0_cp_enable(adev, false);
- adev->gfx.rlc.funcs->stop(adev);
+ /* Skip suspend with A+A reset */
+ if (adev->gmc.xgmi.connected_to_cpu && amdgpu_in_reset(adev)) {
+ dev_dbg(adev->dev, "Device in reset. Skipping RLC halt\n");
+ return 0;
+ }
+
+ adev->gfx.rlc.funcs->stop(adev);
return 0;
}
@@ -4101,7 +4174,7 @@ static uint64_t gfx_v9_0_kiq_read_clock(struct amdgpu_device *adev)
*
* also don't wait anymore for IRQ context
* */
- if (r < 1 && (amdgpu_in_reset(adev) || in_interrupt()))
+ if (r < 1 && (amdgpu_in_reset(adev)))
goto failed_kiq_read;
might_sleep();
@@ -4486,7 +4559,8 @@ static int gfx_v9_0_do_edc_gpr_workarounds(struct amdgpu_device *adev)
if (!ring->sched.ready)
return 0;
- if (adev->asic_type == CHIP_ARCTURUS) {
+ if (adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN) {
vgpr_init_shader_ptr = vgpr_init_compute_shader_arcturus;
vgpr_init_shader_size = sizeof(vgpr_init_compute_shader_arcturus);
vgpr_init_regs_ptr = vgpr_init_regs_arcturus;
@@ -4636,7 +4710,8 @@ static int gfx_v9_0_early_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- if (adev->asic_type == CHIP_ARCTURUS)
+ if (adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN)
adev->gfx.num_gfx_rings = 0;
else
adev->gfx.num_gfx_rings = GFX9_NUM_GFX_RINGS;
@@ -4662,7 +4737,8 @@ static int gfx_v9_0_ecc_late_init(void *handle)
* to GDS in suspend/resume sequence on several cards. So just
* limit this operation in cold boot sequence.
*/
- if (!adev->in_suspend) {
+ if ((!adev->in_suspend) &&
+ (adev->gds.gds_size)) {
r = gfx_v9_0_do_edc_gds_workarounds(adev);
if (r)
return r;
@@ -4673,13 +4749,16 @@ static int gfx_v9_0_ecc_late_init(void *handle)
if (r)
return r;
- if (adev->gfx.funcs &&
- adev->gfx.funcs->reset_ras_error_count)
- adev->gfx.funcs->reset_ras_error_count(adev);
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->ras_late_init) {
+ r = adev->gfx.ras_funcs->ras_late_init(adev);
+ if (r)
+ return r;
+ }
- r = amdgpu_gfx_ras_late_init(adev);
- if (r)
- return r;
+ if (adev->gfx.ras_funcs &&
+ adev->gfx.ras_funcs->enable_watchdog_timer)
+ adev->gfx.ras_funcs->enable_watchdog_timer(adev);
return 0;
}
@@ -4858,7 +4937,7 @@ static void gfx_v9_0_update_3d_clock_gating(struct amdgpu_device *adev,
{
uint32_t data, def;
- if (adev->asic_type == CHIP_ARCTURUS)
+ if (!adev->gfx.num_gfx_rings)
return;
amdgpu_gfx_rlc_enter_safe_mode(adev);
@@ -5105,6 +5184,7 @@ static int gfx_v9_0_set_clockgating_state(void *handle,
case CHIP_RAVEN:
case CHIP_ARCTURUS:
case CHIP_RENOIR:
+ case CHIP_ALDEBARAN:
gfx_v9_0_update_gfx_clock_gating(adev,
state == AMD_CG_STATE_GATE);
break;
@@ -6924,6 +7004,7 @@ static void gfx_v9_0_set_rlc_funcs(struct amdgpu_device *adev)
case CHIP_RAVEN:
case CHIP_ARCTURUS:
case CHIP_RENOIR:
+ case CHIP_ALDEBARAN:
adev->gfx.rlc.funcs = &gfx_v9_0_rlc_funcs;
break;
default:
@@ -6944,6 +7025,12 @@ static void gfx_v9_0_set_gds_init(struct amdgpu_device *adev)
case CHIP_ARCTURUS:
adev->gds.gds_size = 0x1000;
break;
+ case CHIP_ALDEBARAN:
+ /* aldebaran removed all the GDS internal memory,
+ * only support GWS opcode in kernel, like barrier
+ * semaphore.etc */
+ adev->gds.gds_size = 0;
+ break;
default:
adev->gds.gds_size = 0x10000;
break;
@@ -6966,6 +7053,10 @@ static void gfx_v9_0_set_gds_init(struct amdgpu_device *adev)
case CHIP_ARCTURUS:
adev->gds.gds_compute_max_wave_id = 0xfff;
break;
+ case CHIP_ALDEBARAN:
+ /* deprecated for Aldebaran, no usage at all */
+ adev->gds.gds_compute_max_wave_id = 0;
+ break;
default:
/* this really depends on the chip */
adev->gds.gds_compute_max_wave_id = 0x7ff;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c
index bc699d680ce8..b4789dfc2bb9 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c
@@ -863,8 +863,8 @@ static int gfx_v9_4_ras_error_count(struct amdgpu_device *adev,
return 0;
}
-int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status)
+static int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
{
struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
uint32_t sec_count = 0, ded_count = 0;
@@ -906,7 +906,7 @@ int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
return 0;
}
-void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev)
+static void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev)
{
int i, j, k;
@@ -971,7 +971,8 @@ void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev)
WREG32_SOC15(GC, 0, mmATC_L2_CACHE_4K_DSM_INDEX, 255);
}
-int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev, void *inject_if)
+static int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev,
+ void *inject_if)
{
struct ras_inject_if *info = (struct ras_inject_if *)inject_if;
int ret;
@@ -993,10 +994,10 @@ int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev, void *inject_if)
return ret;
}
-static const struct soc15_reg_entry gfx_v9_4_rdrsp_status_regs =
+static const struct soc15_reg_entry gfx_v9_4_ea_err_status_regs =
{ SOC15_REG_ENTRY(GC, 0, mmGCEA_ERR_STATUS), 0, 1, 32 };
-void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev)
+static void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev)
{
uint32_t i, j;
uint32_t reg_value;
@@ -1006,18 +1007,33 @@ void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev)
mutex_lock(&adev->grbm_idx_mutex);
- for (i = 0; i < gfx_v9_4_rdrsp_status_regs.se_num; i++) {
- for (j = 0; j < gfx_v9_4_rdrsp_status_regs.instance;
+ for (i = 0; i < gfx_v9_4_ea_err_status_regs.se_num; i++) {
+ for (j = 0; j < gfx_v9_4_ea_err_status_regs.instance;
j++) {
gfx_v9_4_select_se_sh(adev, i, 0, j);
reg_value = RREG32(SOC15_REG_ENTRY_OFFSET(
- gfx_v9_4_rdrsp_status_regs));
- if (reg_value)
+ gfx_v9_4_ea_err_status_regs));
+ if (REG_GET_FIELD(reg_value, GCEA_ERR_STATUS, SDP_RDRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, GCEA_ERR_STATUS, SDP_WRRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, GCEA_ERR_STATUS, SDP_RDRSP_DATAPARITY_ERROR)) {
+ /* SDP read/write error/parity error in FUE_IS_FATAL mode
+ * can cause system fatal error in arcturas. Harvest the error
+ * status before GPU reset */
dev_warn(adev->dev, "GCEA err detected at instance: %d, status: 0x%x!\n",
j, reg_value);
+ }
}
}
gfx_v9_4_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
mutex_unlock(&adev->grbm_idx_mutex);
}
+
+const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs = {
+ .ras_late_init = amdgpu_gfx_ras_late_init,
+ .ras_fini = amdgpu_gfx_ras_fini,
+ .ras_error_inject = &gfx_v9_4_ras_error_inject,
+ .query_ras_error_count = &gfx_v9_4_query_ras_error_count,
+ .reset_ras_error_count = &gfx_v9_4_reset_ras_error_count,
+ .query_ras_error_status = &gfx_v9_4_query_ras_error_status,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.h b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.h
index 875f18473a98..bdd16b568021 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.h
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.h
@@ -24,16 +24,6 @@
#ifndef __GFX_V9_4_H__
#define __GFX_V9_4_H__
-void gfx_v9_4_clear_ras_edc_counter(struct amdgpu_device *adev);
-
-int gfx_v9_4_query_ras_error_count(struct amdgpu_device *adev,
- void *ras_error_status);
-
-int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev,
- void *inject_if);
-
-void gfx_v9_4_reset_ras_error_count(struct amdgpu_device *adev);
-
-void gfx_v9_4_query_ras_error_status(struct amdgpu_device *adev);
+extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_ras_funcs;
#endif /* __GFX_V9_4_H__ */
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c
new file mode 100644
index 000000000000..a30c7c10cd9a
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.c
@@ -0,0 +1,1301 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "amdgpu.h"
+#include "soc15.h"
+
+#include "gc/gc_9_4_2_offset.h"
+#include "gc/gc_9_4_2_sh_mask.h"
+#include "gfx_v9_0.h"
+
+#include "gfx_v9_4_2.h"
+#include "amdgpu_ras.h"
+#include "amdgpu_gfx.h"
+
+enum gfx_v9_4_2_utc_type {
+ VML2_MEM,
+ VML2_WALKER_MEM,
+ UTCL2_MEM,
+ ATC_L2_CACHE_2M,
+ ATC_L2_CACHE_32K,
+ ATC_L2_CACHE_4K
+};
+
+struct gfx_v9_4_2_utc_block {
+ enum gfx_v9_4_2_utc_type type;
+ uint32_t num_banks;
+ uint32_t num_ways;
+ uint32_t num_mem_blocks;
+ struct soc15_reg idx_reg;
+ struct soc15_reg data_reg;
+ uint32_t sec_count_mask;
+ uint32_t sec_count_shift;
+ uint32_t ded_count_mask;
+ uint32_t ded_count_shift;
+ uint32_t clear;
+};
+
+static const struct soc15_reg_golden golden_settings_gc_9_4_2_alde_die_0[] = {
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_0, 0x3fffffff, 0x141dc920),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_1, 0x3fffffff, 0x3b458b93),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_2, 0x3fffffff, 0x1a4f5583),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_3, 0x3fffffff, 0x317717f6),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_4, 0x3fffffff, 0x107cc1e6),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_5, 0x3ff, 0x351),
+};
+
+static const struct soc15_reg_golden golden_settings_gc_9_4_2_alde_die_1[] = {
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_0, 0x3fffffff, 0x2591aa38),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_1, 0x3fffffff, 0xac9e88b),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_2, 0x3fffffff, 0x2bc3369b),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_3, 0x3fffffff, 0xfb74ee),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_4, 0x3fffffff, 0x21f0a2fe),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_CHAN_STEER_5, 0x3ff, 0x49),
+};
+
+static const struct soc15_reg_golden golden_settings_gc_9_4_2_alde[] = {
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regGB_ADDR_CONFIG, 0xffff77ff, 0x2a114042),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTA_CNTL_AUX, 0xfffffeef, 0x10b0000),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCP_UTCL1_CNTL1, 0xffffffff, 0x30800400),
+ SOC15_REG_GOLDEN_VALUE(GC, 0, regTCI_CNTL_3, 0xff, 0x20),
+};
+
+static void gfx_v9_4_2_query_sq_timeout_status(struct amdgpu_device *adev);
+static void gfx_v9_4_2_reset_sq_timeout_status(struct amdgpu_device *adev);
+
+void gfx_v9_4_2_init_golden_registers(struct amdgpu_device *adev,
+ uint32_t die_id)
+{
+ soc15_program_register_sequence(adev,
+ golden_settings_gc_9_4_2_alde,
+ ARRAY_SIZE(golden_settings_gc_9_4_2_alde));
+
+ /* apply golden settings per die */
+ switch (die_id) {
+ case 0:
+ soc15_program_register_sequence(adev,
+ golden_settings_gc_9_4_2_alde_die_0,
+ ARRAY_SIZE(golden_settings_gc_9_4_2_alde_die_0));
+ break;
+ case 1:
+ soc15_program_register_sequence(adev,
+ golden_settings_gc_9_4_2_alde_die_1,
+ ARRAY_SIZE(golden_settings_gc_9_4_2_alde_die_1));
+ break;
+ default:
+ dev_warn(adev->dev,
+ "invalid die id %d, ignore channel fabricid remap settings\n",
+ die_id);
+ break;
+ }
+
+ return;
+}
+
+void gfx_v9_4_2_debug_trap_config_init(struct amdgpu_device *adev,
+ uint32_t first_vmid,
+ uint32_t last_vmid)
+{
+ uint32_t data;
+ int i;
+
+ mutex_lock(&adev->srbm_mutex);
+
+ for (i = first_vmid; i < last_vmid; i++) {
+ data = 0;
+ soc15_grbm_select(adev, 0, 0, 0, i);
+ data = REG_SET_FIELD(data, SPI_GDBG_PER_VMID_CNTL, TRAP_EN, 1);
+ data = REG_SET_FIELD(data, SPI_GDBG_PER_VMID_CNTL, EXCP_EN, 0);
+ data = REG_SET_FIELD(data, SPI_GDBG_PER_VMID_CNTL, EXCP_REPLACE,
+ 0);
+ WREG32(SOC15_REG_OFFSET(GC, 0, regSPI_GDBG_PER_VMID_CNTL), data);
+ }
+
+ soc15_grbm_select(adev, 0, 0, 0, 0);
+ mutex_unlock(&adev->srbm_mutex);
+}
+
+void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev)
+{
+ u32 tmp;
+
+ gfx_v9_0_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+
+ tmp = 0;
+ tmp = REG_SET_FIELD(tmp, GC_THROTTLE_CTRL, PATTERN_MODE, 1);
+ WREG32_SOC15(GC, 0, regGC_THROTTLE_CTRL, tmp);
+
+ tmp = 0;
+ tmp = REG_SET_FIELD(tmp, GC_THROTTLE_CTRL1, PWRBRK_STALL_EN, 1);
+ WREG32_SOC15(GC, 0, regGC_THROTTLE_CTRL1, tmp);
+
+ WREG32_SOC15(GC, 0, regDIDT_IND_INDEX, ixDIDT_SQ_THROTTLE_CTRL);
+ tmp = 0;
+ tmp = REG_SET_FIELD(tmp, DIDT_SQ_THROTTLE_CTRL, PWRBRK_STALL_EN, 1);
+ WREG32_SOC15(GC, 0, regDIDT_IND_DATA, tmp);
+
+ WREG32_SOC15(GC, 0, regGC_CAC_IND_INDEX, ixPWRBRK_STALL_PATTERN_CTRL);
+ tmp = 0;
+ tmp = REG_SET_FIELD(tmp, PWRBRK_STALL_PATTERN_CTRL, PWRBRK_END_STEP, 0x12);
+ WREG32_SOC15(GC, 0, regGC_CAC_IND_DATA, tmp);
+}
+
+static const struct soc15_reg_entry gfx_v9_4_2_edc_counter_regs[] = {
+ /* CPF */
+ { SOC15_REG_ENTRY(GC, 0, regCPF_EDC_ROQ_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regCPF_EDC_TAG_CNT), 0, 1, 1 },
+ /* CPC */
+ { SOC15_REG_ENTRY(GC, 0, regCPC_EDC_SCRATCH_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regCPC_EDC_UCODE_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regDC_EDC_STATE_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regDC_EDC_CSINVOC_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regDC_EDC_RESTORE_CNT), 0, 1, 1 },
+ /* GDS */
+ { SOC15_REG_ENTRY(GC, 0, regGDS_EDC_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regGDS_EDC_GRBM_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PHY_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PIPE_CNT), 0, 1, 1 },
+ /* RLC */
+ { SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT), 0, 1, 1 },
+ { SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2), 0, 1, 1 },
+ /* SPI */
+ { SOC15_REG_ENTRY(GC, 0, regSPI_EDC_CNT), 0, 8, 1 },
+ /* SQC */
+ { SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT), 0, 8, 7 },
+ { SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT2), 0, 8, 7 },
+ { SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT3), 0, 8, 7 },
+ { SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3), 0, 8, 7 },
+ /* SQ */
+ { SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT), 0, 8, 14 },
+ /* TCP */
+ { SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW), 0, 8, 14 },
+ /* TCI */
+ { SOC15_REG_ENTRY(GC, 0, regTCI_EDC_CNT), 0, 1, 69 },
+ /* TCC */
+ { SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT), 0, 1, 16 },
+ { SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2), 0, 1, 16 },
+ /* TCA */
+ { SOC15_REG_ENTRY(GC, 0, regTCA_EDC_CNT), 0, 1, 2 },
+ /* TCX */
+ { SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT), 0, 1, 2 },
+ { SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT2), 0, 1, 2 },
+ /* TD */
+ { SOC15_REG_ENTRY(GC, 0, regTD_EDC_CNT), 0, 8, 14 },
+ /* TA */
+ { SOC15_REG_ENTRY(GC, 0, regTA_EDC_CNT), 0, 8, 14 },
+ /* GCEA */
+ { SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT), 0, 1, 16 },
+ { SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2), 0, 1, 16 },
+ { SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3), 0, 1, 16 },
+};
+
+static void gfx_v9_4_2_select_se_sh(struct amdgpu_device *adev, u32 se_num,
+ u32 sh_num, u32 instance)
+{
+ u32 data;
+
+ if (instance == 0xffffffff)
+ data = REG_SET_FIELD(0, GRBM_GFX_INDEX,
+ INSTANCE_BROADCAST_WRITES, 1);
+ else
+ data = REG_SET_FIELD(0, GRBM_GFX_INDEX, INSTANCE_INDEX,
+ instance);
+
+ if (se_num == 0xffffffff)
+ data = REG_SET_FIELD(data, GRBM_GFX_INDEX, SE_BROADCAST_WRITES,
+ 1);
+ else
+ data = REG_SET_FIELD(data, GRBM_GFX_INDEX, SE_INDEX, se_num);
+
+ if (sh_num == 0xffffffff)
+ data = REG_SET_FIELD(data, GRBM_GFX_INDEX, SH_BROADCAST_WRITES,
+ 1);
+ else
+ data = REG_SET_FIELD(data, GRBM_GFX_INDEX, SH_INDEX, sh_num);
+
+ WREG32_SOC15_RLC_SHADOW_EX(reg, GC, 0, regGRBM_GFX_INDEX, data);
+}
+
+static const struct soc15_ras_field_entry gfx_v9_4_2_ras_fields[] = {
+ /* CPF */
+ { "CPF_ROQ_ME2", SOC15_REG_ENTRY(GC, 0, regCPF_EDC_ROQ_CNT),
+ SOC15_REG_FIELD(CPF_EDC_ROQ_CNT, SEC_COUNT_ME2),
+ SOC15_REG_FIELD(CPF_EDC_ROQ_CNT, DED_COUNT_ME2) },
+ { "CPF_ROQ_ME1", SOC15_REG_ENTRY(GC, 0, regCPF_EDC_ROQ_CNT),
+ SOC15_REG_FIELD(CPF_EDC_ROQ_CNT, SEC_COUNT_ME1),
+ SOC15_REG_FIELD(CPF_EDC_ROQ_CNT, DED_COUNT_ME1) },
+ { "CPF_TCIU_TAG", SOC15_REG_ENTRY(GC, 0, regCPF_EDC_TAG_CNT),
+ SOC15_REG_FIELD(CPF_EDC_TAG_CNT, SEC_COUNT),
+ SOC15_REG_FIELD(CPF_EDC_TAG_CNT, DED_COUNT) },
+
+ /* CPC */
+ { "CPC_SCRATCH", SOC15_REG_ENTRY(GC, 0, regCPC_EDC_SCRATCH_CNT),
+ SOC15_REG_FIELD(CPC_EDC_SCRATCH_CNT, SEC_COUNT),
+ SOC15_REG_FIELD(CPC_EDC_SCRATCH_CNT, DED_COUNT) },
+ { "CPC_UCODE", SOC15_REG_ENTRY(GC, 0, regCPC_EDC_UCODE_CNT),
+ SOC15_REG_FIELD(CPC_EDC_UCODE_CNT, SEC_COUNT),
+ SOC15_REG_FIELD(CPC_EDC_UCODE_CNT, DED_COUNT) },
+ { "CPC_DC_STATE_RAM_ME1", SOC15_REG_ENTRY(GC, 0, regDC_EDC_STATE_CNT),
+ SOC15_REG_FIELD(DC_EDC_STATE_CNT, SEC_COUNT_ME1),
+ SOC15_REG_FIELD(DC_EDC_STATE_CNT, DED_COUNT_ME1) },
+ { "CPC_DC_CSINVOC_RAM_ME1",
+ SOC15_REG_ENTRY(GC, 0, regDC_EDC_CSINVOC_CNT),
+ SOC15_REG_FIELD(DC_EDC_CSINVOC_CNT, SEC_COUNT_ME1),
+ SOC15_REG_FIELD(DC_EDC_CSINVOC_CNT, DED_COUNT_ME1) },
+ { "CPC_DC_RESTORE_RAM_ME1",
+ SOC15_REG_ENTRY(GC, 0, regDC_EDC_RESTORE_CNT),
+ SOC15_REG_FIELD(DC_EDC_RESTORE_CNT, SEC_COUNT_ME1),
+ SOC15_REG_FIELD(DC_EDC_RESTORE_CNT, DED_COUNT_ME1) },
+ { "CPC_DC_CSINVOC_RAM1_ME1",
+ SOC15_REG_ENTRY(GC, 0, regDC_EDC_CSINVOC_CNT),
+ SOC15_REG_FIELD(DC_EDC_CSINVOC_CNT, SEC_COUNT1_ME1),
+ SOC15_REG_FIELD(DC_EDC_CSINVOC_CNT, DED_COUNT1_ME1) },
+ { "CPC_DC_RESTORE_RAM1_ME1",
+ SOC15_REG_ENTRY(GC, 0, regDC_EDC_RESTORE_CNT),
+ SOC15_REG_FIELD(DC_EDC_RESTORE_CNT, SEC_COUNT1_ME1),
+ SOC15_REG_FIELD(DC_EDC_RESTORE_CNT, DED_COUNT1_ME1) },
+
+ /* GDS */
+ { "GDS_GRBM", SOC15_REG_ENTRY(GC, 0, regGDS_EDC_GRBM_CNT),
+ SOC15_REG_FIELD(GDS_EDC_GRBM_CNT, SEC),
+ SOC15_REG_FIELD(GDS_EDC_GRBM_CNT, DED) },
+ { "GDS_MEM", SOC15_REG_ENTRY(GC, 0, regGDS_EDC_CNT),
+ SOC15_REG_FIELD(GDS_EDC_CNT, GDS_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_CNT, GDS_MEM_DED) },
+ { "GDS_PHY_CMD_RAM_MEM", SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PHY_CNT),
+ SOC15_REG_FIELD(GDS_EDC_OA_PHY_CNT, PHY_CMD_RAM_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_OA_PHY_CNT, PHY_CMD_RAM_MEM_DED) },
+ { "GDS_PHY_DATA_RAM_MEM", SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PHY_CNT),
+ SOC15_REG_FIELD(GDS_EDC_OA_PHY_CNT, PHY_DATA_RAM_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_OA_PHY_CNT, PHY_DATA_RAM_MEM_DED) },
+ { "GDS_ME0_CS_PIPE_MEM", SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PHY_CNT),
+ SOC15_REG_FIELD(GDS_EDC_OA_PHY_CNT, ME0_CS_PIPE_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_OA_PHY_CNT, ME0_CS_PIPE_MEM_DED) },
+ { "GDS_ME1_PIPE0_PIPE_MEM",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PIPE_CNT),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE0_PIPE_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE0_PIPE_MEM_DED) },
+ { "GDS_ME1_PIPE1_PIPE_MEM",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PIPE_CNT),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE1_PIPE_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE1_PIPE_MEM_DED) },
+ { "GDS_ME1_PIPE2_PIPE_MEM",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PIPE_CNT),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE2_PIPE_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE2_PIPE_MEM_DED) },
+ { "GDS_ME1_PIPE3_PIPE_MEM",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_PIPE_CNT),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE3_PIPE_MEM_SEC),
+ SOC15_REG_FIELD(GDS_EDC_OA_PIPE_CNT, ME1_PIPE3_PIPE_MEM_DED) },
+ { "GDS_ME0_GFXHP3D_PIX_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME0_GFXHP3D_PIX_DED) },
+ { "GDS_ME0_GFXHP3D_VTX_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME0_GFXHP3D_VTX_DED) },
+ { "GDS_ME0_CS_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME0_CS_DED) },
+ { "GDS_ME0_GFXHP3D_GS_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME0_GFXHP3D_GS_DED) },
+ { "GDS_ME1_PIPE0_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME1_PIPE0_DED) },
+ { "GDS_ME1_PIPE1_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME1_PIPE1_DED) },
+ { "GDS_ME1_PIPE2_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME1_PIPE2_DED) },
+ { "GDS_ME1_PIPE3_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME1_PIPE3_DED) },
+ { "GDS_ME2_PIPE0_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME2_PIPE0_DED) },
+ { "GDS_ME2_PIPE1_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME2_PIPE1_DED) },
+ { "GDS_ME2_PIPE2_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME2_PIPE2_DED) },
+ { "GDS_ME2_PIPE3_DED",
+ SOC15_REG_ENTRY(GC, 0, regGDS_EDC_OA_DED), 0, 0,
+ SOC15_REG_FIELD(GDS_EDC_OA_DED, ME2_PIPE3_DED) },
+
+ /* RLC */
+ { "RLCG_INSTR_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCG_INSTR_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCG_INSTR_RAM_DED_COUNT) },
+ { "RLCG_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCG_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCG_SCRATCH_RAM_DED_COUNT) },
+ { "RLCV_INSTR_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCV_INSTR_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCV_INSTR_RAM_DED_COUNT) },
+ { "RLCV_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCV_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLCV_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_TCTAG_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_TCTAG_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_TCTAG_RAM_DED_COUNT) },
+ { "RLC_SPM_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_SPM_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_SPM_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SRM_DATA_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_SRM_DATA_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_SRM_DATA_RAM_DED_COUNT) },
+ { "RLC_SRM_ADDR_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_SRM_ADDR_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT, RLC_SRM_ADDR_RAM_DED_COUNT) },
+ { "RLC_SPM_SE0_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE0_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE0_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SPM_SE1_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE1_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE1_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SPM_SE2_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE2_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE2_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SPM_SE3_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE3_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE3_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SPM_SE4_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE4_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE4_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SPM_SE5_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE5_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE5_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SPM_SE6_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE6_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE6_SCRATCH_RAM_DED_COUNT) },
+ { "RLC_SPM_SE7_SCRATCH_RAM", SOC15_REG_ENTRY(GC, 0, regRLC_EDC_CNT2),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE7_SCRATCH_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(RLC_EDC_CNT2, RLC_SPM_SE7_SCRATCH_RAM_DED_COUNT) },
+
+ /* SPI */
+ { "SPI_SR_MEM", SOC15_REG_ENTRY(GC, 0, regSPI_EDC_CNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_SR_MEM_SEC_COUNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_SR_MEM_DED_COUNT) },
+ { "SPI_GDS_EXPREQ", SOC15_REG_ENTRY(GC, 0, regSPI_EDC_CNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_GDS_EXPREQ_SEC_COUNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_GDS_EXPREQ_DED_COUNT) },
+ { "SPI_WB_GRANT_30", SOC15_REG_ENTRY(GC, 0, regSPI_EDC_CNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_WB_GRANT_30_SEC_COUNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_WB_GRANT_30_DED_COUNT) },
+ { "SPI_LIFE_CNT", SOC15_REG_ENTRY(GC, 0, regSPI_EDC_CNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_LIFE_CNT_SEC_COUNT),
+ SOC15_REG_FIELD(SPI_EDC_CNT, SPI_LIFE_CNT_DED_COUNT) },
+
+ /* SQC - regSQC_EDC_CNT */
+ { "SQC_DATA_CU0_WRITE_DATA_BUF", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU0_WRITE_DATA_BUF_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU0_WRITE_DATA_BUF_DED_COUNT) },
+ { "SQC_DATA_CU0_UTCL1_LFIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU0_UTCL1_LFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU0_UTCL1_LFIFO_DED_COUNT) },
+ { "SQC_DATA_CU1_WRITE_DATA_BUF", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU1_WRITE_DATA_BUF_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU1_WRITE_DATA_BUF_DED_COUNT) },
+ { "SQC_DATA_CU1_UTCL1_LFIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU1_UTCL1_LFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU1_UTCL1_LFIFO_DED_COUNT) },
+ { "SQC_DATA_CU2_WRITE_DATA_BUF", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU2_WRITE_DATA_BUF_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU2_WRITE_DATA_BUF_DED_COUNT) },
+ { "SQC_DATA_CU2_UTCL1_LFIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU2_UTCL1_LFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU2_UTCL1_LFIFO_DED_COUNT) },
+ { "SQC_DATA_CU3_WRITE_DATA_BUF", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU3_WRITE_DATA_BUF_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU3_WRITE_DATA_BUF_DED_COUNT) },
+ { "SQC_DATA_CU3_UTCL1_LFIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU3_UTCL1_LFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT, DATA_CU3_UTCL1_LFIFO_DED_COUNT) },
+
+ /* SQC - regSQC_EDC_CNT2 */
+ { "SQC_INST_BANKA_TAG_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT2),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, INST_BANKA_TAG_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, INST_BANKA_TAG_RAM_DED_COUNT) },
+ { "SQC_INST_BANKA_BANK_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT2),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, INST_BANKA_BANK_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, INST_BANKA_BANK_RAM_DED_COUNT) },
+ { "SQC_DATA_BANKA_TAG_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT2),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, DATA_BANKA_TAG_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, DATA_BANKA_TAG_RAM_DED_COUNT) },
+ { "SQC_DATA_BANKA_BANK_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT2),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, DATA_BANKA_BANK_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, DATA_BANKA_BANK_RAM_DED_COUNT) },
+ { "SQC_INST_UTCL1_LFIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT2),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, INST_UTCL1_LFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, INST_UTCL1_LFIFO_DED_COUNT) },
+ { "SQC_DATA_BANKA_DIRTY_BIT_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT2),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, DATA_BANKA_DIRTY_BIT_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT2, DATA_BANKA_DIRTY_BIT_RAM_DED_COUNT) },
+
+ /* SQC - regSQC_EDC_CNT3 */
+ { "SQC_INST_BANKB_TAG_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, INST_BANKB_TAG_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, INST_BANKB_TAG_RAM_DED_COUNT) },
+ { "SQC_INST_BANKB_BANK_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, INST_BANKB_BANK_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, INST_BANKB_BANK_RAM_DED_COUNT) },
+ { "SQC_DATA_BANKB_TAG_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, DATA_BANKB_TAG_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, DATA_BANKB_TAG_RAM_DED_COUNT) },
+ { "SQC_DATA_BANKB_BANK_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, DATA_BANKB_BANK_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, DATA_BANKB_BANK_RAM_DED_COUNT) },
+ { "SQC_DATA_BANKB_DIRTY_BIT_RAM", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, DATA_BANKB_DIRTY_BIT_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_CNT3, DATA_BANKB_DIRTY_BIT_RAM_DED_COUNT) },
+
+ /* SQC - regSQC_EDC_PARITY_CNT3 */
+ { "SQC_INST_BANKA_UTCL1_MISS_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKA_UTCL1_MISS_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKA_UTCL1_MISS_FIFO_DED_COUNT) },
+ { "SQC_INST_BANKA_MISS_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKA_MISS_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKA_MISS_FIFO_DED_COUNT) },
+ { "SQC_DATA_BANKA_HIT_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKA_HIT_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKA_HIT_FIFO_DED_COUNT) },
+ { "SQC_DATA_BANKA_MISS_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKA_MISS_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKA_MISS_FIFO_DED_COUNT) },
+ { "SQC_INST_BANKB_UTCL1_MISS_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKB_UTCL1_MISS_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKB_UTCL1_MISS_FIFO_DED_COUNT) },
+ { "SQC_INST_BANKB_MISS_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKB_MISS_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, INST_BANKB_MISS_FIFO_DED_COUNT) },
+ { "SQC_DATA_BANKB_HIT_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKB_HIT_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKB_HIT_FIFO_DED_COUNT) },
+ { "SQC_DATA_BANKB_MISS_FIFO", SOC15_REG_ENTRY(GC, 0, regSQC_EDC_PARITY_CNT3),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKB_MISS_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(SQC_EDC_PARITY_CNT3, DATA_BANKB_MISS_FIFO_DED_COUNT) },
+
+ /* SQ */
+ { "SQ_LDS_D", SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, LDS_D_SEC_COUNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, LDS_D_DED_COUNT) },
+ { "SQ_LDS_I", SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, LDS_I_SEC_COUNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, LDS_I_DED_COUNT) },
+ { "SQ_SGPR", SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, SGPR_SEC_COUNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, SGPR_DED_COUNT) },
+ { "SQ_VGPR0", SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR0_SEC_COUNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR0_DED_COUNT) },
+ { "SQ_VGPR1", SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR1_SEC_COUNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR1_DED_COUNT) },
+ { "SQ_VGPR2", SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR2_SEC_COUNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR2_DED_COUNT) },
+ { "SQ_VGPR3", SOC15_REG_ENTRY(GC, 0, regSQ_EDC_CNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR3_SEC_COUNT),
+ SOC15_REG_FIELD(SQ_EDC_CNT, VGPR3_DED_COUNT) },
+
+ /* TCP */
+ { "TCP_CACHE_RAM", SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, CACHE_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, CACHE_RAM_DED_COUNT) },
+ { "TCP_LFIFO_RAM", SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, LFIFO_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, LFIFO_RAM_DED_COUNT) },
+ { "TCP_CMD_FIFO", SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, CMD_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, CMD_FIFO_DED_COUNT) },
+ { "TCP_VM_FIFO", SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, VM_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, VM_FIFO_DED_COUNT) },
+ { "TCP_DB_RAM", SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, DB_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, DB_RAM_DED_COUNT) },
+ { "TCP_UTCL1_LFIFO0", SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, UTCL1_LFIFO0_SEC_COUNT),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, UTCL1_LFIFO0_DED_COUNT) },
+ { "TCP_UTCL1_LFIFO1", SOC15_REG_ENTRY(GC, 0, regTCP_EDC_CNT_NEW),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, UTCL1_LFIFO1_SEC_COUNT),
+ SOC15_REG_FIELD(TCP_EDC_CNT_NEW, UTCL1_LFIFO1_DED_COUNT) },
+
+ /* TCI */
+ { "TCI_WRITE_RAM", SOC15_REG_ENTRY(GC, 0, regTCI_EDC_CNT),
+ SOC15_REG_FIELD(TCI_EDC_CNT, WRITE_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(TCI_EDC_CNT, WRITE_RAM_DED_COUNT) },
+
+ /* TCC */
+ { "TCC_CACHE_DATA", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, CACHE_DATA_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, CACHE_DATA_DED_COUNT) },
+ { "TCC_CACHE_DIRTY", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, CACHE_DIRTY_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, CACHE_DIRTY_DED_COUNT) },
+ { "TCC_HIGH_RATE_TAG", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, HIGH_RATE_TAG_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, HIGH_RATE_TAG_DED_COUNT) },
+ { "TCC_LOW_RATE_TAG", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, LOW_RATE_TAG_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, LOW_RATE_TAG_DED_COUNT) },
+ { "TCC_SRC_FIFO", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, SRC_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, SRC_FIFO_DED_COUNT) },
+ { "TCC_LATENCY_FIFO", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, LATENCY_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, LATENCY_FIFO_DED_COUNT) },
+ { "TCC_LATENCY_FIFO_NEXT_RAM", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, LATENCY_FIFO_NEXT_RAM_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT, LATENCY_FIFO_NEXT_RAM_DED_COUNT) },
+ { "TCC_CACHE_TAG_PROBE_FIFO", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, CACHE_TAG_PROBE_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, CACHE_TAG_PROBE_FIFO_DED_COUNT) },
+ { "TCC_UC_ATOMIC_FIFO", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, UC_ATOMIC_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, UC_ATOMIC_FIFO_DED_COUNT) },
+ { "TCC_WRITE_CACHE_READ", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, WRITE_CACHE_READ_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, WRITE_CACHE_READ_DED_COUNT) },
+ { "TCC_RETURN_CONTROL", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, RETURN_CONTROL_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, RETURN_CONTROL_DED_COUNT) },
+ { "TCC_IN_USE_TRANSFER", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, IN_USE_TRANSFER_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, IN_USE_TRANSFER_DED_COUNT) },
+ { "TCC_IN_USE_DEC", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, IN_USE_DEC_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, IN_USE_DEC_DED_COUNT) },
+ { "TCC_WRITE_RETURN", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, WRITE_RETURN_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, WRITE_RETURN_DED_COUNT) },
+ { "TCC_RETURN_DATA", SOC15_REG_ENTRY(GC, 0, regTCC_EDC_CNT2),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, RETURN_DATA_SEC_COUNT),
+ SOC15_REG_FIELD(TCC_EDC_CNT2, RETURN_DATA_DED_COUNT) },
+
+ /* TCA */
+ { "TCA_HOLE_FIFO", SOC15_REG_ENTRY(GC, 0, regTCA_EDC_CNT),
+ SOC15_REG_FIELD(TCA_EDC_CNT, HOLE_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCA_EDC_CNT, HOLE_FIFO_DED_COUNT) },
+ { "TCA_REQ_FIFO", SOC15_REG_ENTRY(GC, 0, regTCA_EDC_CNT),
+ SOC15_REG_FIELD(TCA_EDC_CNT, REQ_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TCA_EDC_CNT, REQ_FIFO_DED_COUNT) },
+
+ /* TCX */
+ { "TCX_GROUP0", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP0_SEC_COUNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP0_DED_COUNT) },
+ { "TCX_GROUP1", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP1_SEC_COUNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP1_DED_COUNT) },
+ { "TCX_GROUP2", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP2_SEC_COUNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP2_DED_COUNT) },
+ { "TCX_GROUP3", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP3_SEC_COUNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP3_DED_COUNT) },
+ { "TCX_GROUP4", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP4_SEC_COUNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP4_DED_COUNT) },
+ { "TCX_GROUP5", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP5_SED_COUNT), 0, 0 },
+ { "TCX_GROUP6", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP6_SED_COUNT), 0, 0 },
+ { "TCX_GROUP7", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP7_SED_COUNT), 0, 0 },
+ { "TCX_GROUP8", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP8_SED_COUNT), 0, 0 },
+ { "TCX_GROUP9", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP9_SED_COUNT), 0, 0 },
+ { "TCX_GROUP10", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT),
+ SOC15_REG_FIELD(TCX_EDC_CNT, GROUP10_SED_COUNT), 0, 0 },
+ { "TCX_GROUP11", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT2),
+ SOC15_REG_FIELD(TCX_EDC_CNT2, GROUP11_SED_COUNT), 0, 0 },
+ { "TCX_GROUP12", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT2),
+ SOC15_REG_FIELD(TCX_EDC_CNT2, GROUP12_SED_COUNT), 0, 0 },
+ { "TCX_GROUP13", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT2),
+ SOC15_REG_FIELD(TCX_EDC_CNT2, GROUP13_SED_COUNT), 0, 0 },
+ { "TCX_GROUP14", SOC15_REG_ENTRY(GC, 0, regTCX_EDC_CNT2),
+ SOC15_REG_FIELD(TCX_EDC_CNT2, GROUP14_SED_COUNT), 0, 0 },
+
+ /* TD */
+ { "TD_SS_FIFO_LO", SOC15_REG_ENTRY(GC, 0, regTD_EDC_CNT),
+ SOC15_REG_FIELD(TD_EDC_CNT, SS_FIFO_LO_SEC_COUNT),
+ SOC15_REG_FIELD(TD_EDC_CNT, SS_FIFO_LO_DED_COUNT) },
+ { "TD_SS_FIFO_HI", SOC15_REG_ENTRY(GC, 0, regTD_EDC_CNT),
+ SOC15_REG_FIELD(TD_EDC_CNT, SS_FIFO_HI_SEC_COUNT),
+ SOC15_REG_FIELD(TD_EDC_CNT, SS_FIFO_HI_DED_COUNT) },
+ { "TD_CS_FIFO", SOC15_REG_ENTRY(GC, 0, regTD_EDC_CNT),
+ SOC15_REG_FIELD(TD_EDC_CNT, CS_FIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TD_EDC_CNT, CS_FIFO_DED_COUNT) },
+
+ /* TA */
+ { "TA_FS_DFIFO", SOC15_REG_ENTRY(GC, 0, regTA_EDC_CNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_DFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_DFIFO_DED_COUNT) },
+ { "TA_FS_AFIFO_LO", SOC15_REG_ENTRY(GC, 0, regTA_EDC_CNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_AFIFO_LO_SEC_COUNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_AFIFO_LO_DED_COUNT) },
+ { "TA_FL_LFIFO", SOC15_REG_ENTRY(GC, 0, regTA_EDC_CNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FL_LFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FL_LFIFO_DED_COUNT) },
+ { "TA_FX_LFIFO", SOC15_REG_ENTRY(GC, 0, regTA_EDC_CNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FX_LFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FX_LFIFO_DED_COUNT) },
+ { "TA_FS_CFIFO", SOC15_REG_ENTRY(GC, 0, regTA_EDC_CNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_CFIFO_SEC_COUNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_CFIFO_DED_COUNT) },
+ { "TA_FS_AFIFO_HI", SOC15_REG_ENTRY(GC, 0, regTA_EDC_CNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_AFIFO_HI_SEC_COUNT),
+ SOC15_REG_FIELD(TA_EDC_CNT, TA_FS_AFIFO_HI_DED_COUNT) },
+
+ /* EA - regGCEA_EDC_CNT */
+ { "EA_DRAMRD_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMRD_CMDMEM_DED_COUNT) },
+ { "EA_DRAMWR_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMWR_CMDMEM_DED_COUNT) },
+ { "EA_DRAMWR_DATAMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMWR_DATAMEM_DED_COUNT) },
+ { "EA_RRET_TAGMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, RRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, RRET_TAGMEM_DED_COUNT) },
+ { "EA_WRET_TAGMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, WRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, WRET_TAGMEM_DED_COUNT) },
+ { "EA_IOWR_DATAMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, IOWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, IOWR_DATAMEM_DED_COUNT) },
+ { "EA_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMRD_PAGEMEM_SED_COUNT), 0, 0 },
+ { "EA_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, DRAMWR_PAGEMEM_SED_COUNT), 0, 0 },
+ { "EA_IORD_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, IORD_CMDMEM_SED_COUNT), 0, 0 },
+ { "EA_IOWR_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT, IOWR_CMDMEM_SED_COUNT), 0, 0 },
+
+ /* EA - regGCEA_EDC_CNT2 */
+ { "EA_GMIRD_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIRD_CMDMEM_DED_COUNT) },
+ { "EA_GMIWR_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIWR_CMDMEM_DED_COUNT) },
+ { "EA_GMIWR_DATAMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIWR_DATAMEM_DED_COUNT) },
+ { "EA_GMIRD_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIRD_PAGEMEM_SED_COUNT), 0, 0 },
+ { "EA_GMIWR_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, GMIWR_PAGEMEM_SED_COUNT), 0, 0 },
+ { "EA_MAM_D0MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D0MEM_SED_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D0MEM_DED_COUNT) },
+ { "EA_MAM_D1MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D1MEM_SED_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D1MEM_DED_COUNT) },
+ { "EA_MAM_D2MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D2MEM_SED_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D2MEM_DED_COUNT) },
+ { "EA_MAM_D3MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT2),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D3MEM_SED_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT2, MAM_D3MEM_DED_COUNT) },
+
+ /* EA - regGCEA_EDC_CNT3 */
+ { "EA_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3), 0, 0,
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, DRAMRD_PAGEMEM_DED_COUNT) },
+ { "EA_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3), 0, 0,
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, DRAMWR_PAGEMEM_DED_COUNT) },
+ { "EA_IORD_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3), 0, 0,
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, IORD_CMDMEM_DED_COUNT) },
+ { "EA_IOWR_CMDMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3), 0, 0,
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, IOWR_CMDMEM_DED_COUNT) },
+ { "EA_GMIRD_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3), 0, 0,
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, GMIRD_PAGEMEM_DED_COUNT) },
+ { "EA_GMIWR_PAGEMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3), 0, 0,
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, GMIWR_PAGEMEM_DED_COUNT) },
+ { "EA_MAM_A0MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A0MEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A0MEM_DED_COUNT) },
+ { "EA_MAM_A1MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A1MEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A1MEM_DED_COUNT) },
+ { "EA_MAM_A2MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A2MEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A2MEM_DED_COUNT) },
+ { "EA_MAM_A3MEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A3MEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_A3MEM_DED_COUNT) },
+ { "EA_MAM_AFMEM", SOC15_REG_ENTRY(GC, 0, regGCEA_EDC_CNT3),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_AFMEM_SEC_COUNT),
+ SOC15_REG_FIELD(GCEA_EDC_CNT3, MAM_AFMEM_DED_COUNT) },
+};
+
+static const char * const vml2_walker_mems[] = {
+ "UTC_VML2_CACHE_PDE0_MEM0",
+ "UTC_VML2_CACHE_PDE0_MEM1",
+ "UTC_VML2_CACHE_PDE1_MEM0",
+ "UTC_VML2_CACHE_PDE1_MEM1",
+ "UTC_VML2_CACHE_PDE2_MEM0",
+ "UTC_VML2_CACHE_PDE2_MEM1",
+ "UTC_VML2_RDIF_ARADDRS",
+ "UTC_VML2_RDIF_LOG_FIFO",
+ "UTC_VML2_QUEUE_REQ",
+ "UTC_VML2_QUEUE_RET",
+};
+
+static struct gfx_v9_4_2_utc_block gfx_v9_4_2_utc_blocks[] = {
+ { VML2_MEM, 8, 2, 2,
+ { SOC15_REG_ENTRY(GC, 0, regVML2_MEM_ECC_INDEX) },
+ { SOC15_REG_ENTRY(GC, 0, regVML2_MEM_ECC_CNTL) },
+ SOC15_REG_FIELD(VML2_MEM_ECC_CNTL, SEC_COUNT),
+ SOC15_REG_FIELD(VML2_MEM_ECC_CNTL, DED_COUNT),
+ REG_SET_FIELD(0, VML2_MEM_ECC_CNTL, WRITE_COUNTERS, 1) },
+ { VML2_WALKER_MEM, ARRAY_SIZE(vml2_walker_mems), 1, 1,
+ { SOC15_REG_ENTRY(GC, 0, regVML2_WALKER_MEM_ECC_INDEX) },
+ { SOC15_REG_ENTRY(GC, 0, regVML2_WALKER_MEM_ECC_CNTL) },
+ SOC15_REG_FIELD(VML2_WALKER_MEM_ECC_CNTL, SEC_COUNT),
+ SOC15_REG_FIELD(VML2_WALKER_MEM_ECC_CNTL, DED_COUNT),
+ REG_SET_FIELD(0, VML2_WALKER_MEM_ECC_CNTL, WRITE_COUNTERS, 1) },
+ { UTCL2_MEM, 18, 1, 2,
+ { SOC15_REG_ENTRY(GC, 0, regUTCL2_MEM_ECC_INDEX) },
+ { SOC15_REG_ENTRY(GC, 0, regUTCL2_MEM_ECC_CNTL) },
+ SOC15_REG_FIELD(UTCL2_MEM_ECC_CNTL, SEC_COUNT),
+ SOC15_REG_FIELD(UTCL2_MEM_ECC_CNTL, DED_COUNT),
+ REG_SET_FIELD(0, UTCL2_MEM_ECC_CNTL, WRITE_COUNTERS, 1) },
+ { ATC_L2_CACHE_2M, 8, 2, 1,
+ { SOC15_REG_ENTRY(GC, 0, regATC_L2_CACHE_2M_DSM_INDEX) },
+ { SOC15_REG_ENTRY(GC, 0, regATC_L2_CACHE_2M_DSM_CNTL) },
+ SOC15_REG_FIELD(ATC_L2_CACHE_2M_DSM_CNTL, SEC_COUNT),
+ SOC15_REG_FIELD(ATC_L2_CACHE_2M_DSM_CNTL, DED_COUNT),
+ REG_SET_FIELD(0, ATC_L2_CACHE_2M_DSM_CNTL, WRITE_COUNTERS, 1) },
+ { ATC_L2_CACHE_32K, 8, 2, 2,
+ { SOC15_REG_ENTRY(GC, 0, regATC_L2_CACHE_32K_DSM_INDEX) },
+ { SOC15_REG_ENTRY(GC, 0, regATC_L2_CACHE_32K_DSM_CNTL) },
+ SOC15_REG_FIELD(ATC_L2_CACHE_32K_DSM_CNTL, SEC_COUNT),
+ SOC15_REG_FIELD(ATC_L2_CACHE_32K_DSM_CNTL, DED_COUNT),
+ REG_SET_FIELD(0, ATC_L2_CACHE_32K_DSM_CNTL, WRITE_COUNTERS, 1) },
+ { ATC_L2_CACHE_4K, 8, 2, 8,
+ { SOC15_REG_ENTRY(GC, 0, regATC_L2_CACHE_4K_DSM_INDEX) },
+ { SOC15_REG_ENTRY(GC, 0, regATC_L2_CACHE_4K_DSM_CNTL) },
+ SOC15_REG_FIELD(ATC_L2_CACHE_4K_DSM_CNTL, SEC_COUNT),
+ SOC15_REG_FIELD(ATC_L2_CACHE_4K_DSM_CNTL, DED_COUNT),
+ REG_SET_FIELD(0, ATC_L2_CACHE_4K_DSM_CNTL, WRITE_COUNTERS, 1) },
+};
+
+static const struct soc15_reg_entry gfx_v9_4_2_ea_err_status_regs =
+ { SOC15_REG_ENTRY(GC, 0, regGCEA_ERR_STATUS), 0, 1, 16 };
+
+static int gfx_v9_4_2_get_reg_error_count(struct amdgpu_device *adev,
+ const struct soc15_reg_entry *reg,
+ uint32_t se_id, uint32_t inst_id,
+ uint32_t value, uint32_t *sec_count,
+ uint32_t *ded_count)
+{
+ uint32_t i;
+ uint32_t sec_cnt, ded_cnt;
+
+ for (i = 0; i < ARRAY_SIZE(gfx_v9_4_2_ras_fields); i++) {
+ if (gfx_v9_4_2_ras_fields[i].reg_offset != reg->reg_offset ||
+ gfx_v9_4_2_ras_fields[i].seg != reg->seg ||
+ gfx_v9_4_2_ras_fields[i].inst != reg->inst)
+ continue;
+
+ sec_cnt = SOC15_RAS_REG_FIELD_VAL(
+ value, gfx_v9_4_2_ras_fields[i], sec);
+ if (sec_cnt) {
+ dev_info(adev->dev,
+ "GFX SubBlock %s, Instance[%d][%d], SEC %d\n",
+ gfx_v9_4_2_ras_fields[i].name, se_id, inst_id,
+ sec_cnt);
+ *sec_count += sec_cnt;
+ }
+
+ ded_cnt = SOC15_RAS_REG_FIELD_VAL(
+ value, gfx_v9_4_2_ras_fields[i], ded);
+ if (ded_cnt) {
+ dev_info(adev->dev,
+ "GFX SubBlock %s, Instance[%d][%d], DED %d\n",
+ gfx_v9_4_2_ras_fields[i].name, se_id, inst_id,
+ ded_cnt);
+ *ded_count += ded_cnt;
+ }
+ }
+
+ return 0;
+}
+
+static int gfx_v9_4_2_query_sram_edc_count(struct amdgpu_device *adev,
+ uint32_t *sec_count, uint32_t *ded_count)
+{
+ uint32_t i, j, k, data;
+ uint32_t sec_cnt = 0, ded_cnt = 0;
+
+ if (sec_count && ded_count) {
+ *sec_count = 0;
+ *ded_count = 0;
+ }
+
+ mutex_lock(&adev->grbm_idx_mutex);
+
+ for (i = 0; i < ARRAY_SIZE(gfx_v9_4_2_edc_counter_regs); i++) {
+ for (j = 0; j < gfx_v9_4_2_edc_counter_regs[i].se_num; j++) {
+ for (k = 0; k < gfx_v9_4_2_edc_counter_regs[i].instance;
+ k++) {
+ gfx_v9_4_2_select_se_sh(adev, j, 0, k);
+
+ /* if sec/ded_count is null, just clear counter */
+ if (!sec_count || !ded_count) {
+ WREG32(SOC15_REG_ENTRY_OFFSET(
+ gfx_v9_4_2_edc_counter_regs[i]), 0);
+ continue;
+ }
+
+ data = RREG32(SOC15_REG_ENTRY_OFFSET(
+ gfx_v9_4_2_edc_counter_regs[i]));
+
+ if (!data)
+ continue;
+
+ gfx_v9_4_2_get_reg_error_count(adev,
+ &gfx_v9_4_2_edc_counter_regs[i],
+ j, k, data, &sec_cnt, &ded_cnt);
+
+ /* clear counter after read */
+ WREG32(SOC15_REG_ENTRY_OFFSET(
+ gfx_v9_4_2_edc_counter_regs[i]), 0);
+ }
+ }
+ }
+
+ if (sec_count && ded_count) {
+ *sec_count += sec_cnt;
+ *ded_count += ded_cnt;
+ }
+
+ gfx_v9_4_2_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ mutex_unlock(&adev->grbm_idx_mutex);
+
+ return 0;
+}
+
+static void gfx_v9_4_2_log_utc_edc_count(struct amdgpu_device *adev,
+ struct gfx_v9_4_2_utc_block *blk,
+ uint32_t instance, uint32_t sec_cnt,
+ uint32_t ded_cnt)
+{
+ uint32_t bank, way, mem;
+ static const char *vml2_way_str[] = { "BIGK", "4K" };
+ static const char *utcl2_rounter_str[] = { "VMC", "APT" };
+
+ mem = instance % blk->num_mem_blocks;
+ way = (instance / blk->num_mem_blocks) % blk->num_ways;
+ bank = instance / (blk->num_mem_blocks * blk->num_ways);
+
+ switch (blk->type) {
+ case VML2_MEM:
+ dev_info(
+ adev->dev,
+ "GFX SubBlock UTC_VML2_BANK_CACHE_%d_%s_MEM%d, SED %d, DED %d\n",
+ bank, vml2_way_str[way], mem, sec_cnt, ded_cnt);
+ break;
+ case VML2_WALKER_MEM:
+ dev_info(adev->dev, "GFX SubBlock %s, SED %d, DED %d\n",
+ vml2_walker_mems[bank], sec_cnt, ded_cnt);
+ break;
+ case UTCL2_MEM:
+ dev_info(
+ adev->dev,
+ "GFX SubBlock UTCL2_ROUTER_IFIF%d_GROUP0_%s, SED %d, DED %d\n",
+ bank, utcl2_rounter_str[mem], sec_cnt, ded_cnt);
+ break;
+ case ATC_L2_CACHE_2M:
+ dev_info(
+ adev->dev,
+ "GFX SubBlock UTC_ATCL2_CACHE_2M_BANK%d_WAY%d_MEM, SED %d, DED %d\n",
+ bank, way, sec_cnt, ded_cnt);
+ break;
+ case ATC_L2_CACHE_32K:
+ dev_info(
+ adev->dev,
+ "GFX SubBlock UTC_ATCL2_CACHE_32K_BANK%d_WAY%d_MEM%d, SED %d, DED %d\n",
+ bank, way, mem, sec_cnt, ded_cnt);
+ break;
+ case ATC_L2_CACHE_4K:
+ dev_info(
+ adev->dev,
+ "GFX SubBlock UTC_ATCL2_CACHE_4K_BANK%d_WAY%d_MEM%d, SED %d, DED %d\n",
+ bank, way, mem, sec_cnt, ded_cnt);
+ break;
+ }
+}
+
+static int gfx_v9_4_2_query_utc_edc_count(struct amdgpu_device *adev,
+ uint32_t *sec_count,
+ uint32_t *ded_count)
+{
+ uint32_t i, j, data;
+ uint32_t sec_cnt, ded_cnt;
+ uint32_t num_instances;
+ struct gfx_v9_4_2_utc_block *blk;
+
+ if (sec_count && ded_count) {
+ *sec_count = 0;
+ *ded_count = 0;
+ }
+
+ for (i = 0; i < ARRAY_SIZE(gfx_v9_4_2_utc_blocks); i++) {
+ blk = &gfx_v9_4_2_utc_blocks[i];
+ num_instances =
+ blk->num_banks * blk->num_ways * blk->num_mem_blocks;
+ for (j = 0; j < num_instances; j++) {
+ WREG32(SOC15_REG_ENTRY_OFFSET(blk->idx_reg), j);
+
+ /* if sec/ded_count is NULL, just clear counter */
+ if (!sec_count || !ded_count) {
+ WREG32(SOC15_REG_ENTRY_OFFSET(blk->data_reg),
+ blk->clear);
+ continue;
+ }
+
+ data = RREG32(SOC15_REG_ENTRY_OFFSET(blk->data_reg));
+ if (!data)
+ continue;
+
+ sec_cnt = SOC15_RAS_REG_FIELD_VAL(data, *blk, sec);
+ *sec_count += sec_cnt;
+ ded_cnt = SOC15_RAS_REG_FIELD_VAL(data, *blk, ded);
+ *ded_count += ded_cnt;
+
+ /* clear counter after read */
+ WREG32(SOC15_REG_ENTRY_OFFSET(blk->data_reg),
+ blk->clear);
+
+ /* print the edc count */
+ if (sec_cnt || ded_cnt)
+ gfx_v9_4_2_log_utc_edc_count(adev, blk, j, sec_cnt,
+ ded_cnt);
+ }
+ }
+
+ return 0;
+}
+
+int gfx_v9_4_2_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+ uint32_t sec_count = 0, ded_count = 0;
+
+ if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
+ return -EINVAL;
+
+ err_data->ue_count = 0;
+ err_data->ce_count = 0;
+
+ gfx_v9_4_2_query_sram_edc_count(adev, &sec_count, &ded_count);
+ err_data->ce_count += sec_count;
+ err_data->ue_count += ded_count;
+
+ gfx_v9_4_2_query_utc_edc_count(adev, &sec_count, &ded_count);
+ err_data->ce_count += sec_count;
+ err_data->ue_count += ded_count;
+
+ return 0;
+}
+
+static void gfx_v9_4_2_reset_utc_err_status(struct amdgpu_device *adev)
+{
+ WREG32_SOC15(GC, 0, regUTCL2_MEM_ECC_STATUS, 0x3);
+ WREG32_SOC15(GC, 0, regVML2_MEM_ECC_STATUS, 0x3);
+ WREG32_SOC15(GC, 0, regVML2_WALKER_MEM_ECC_STATUS, 0x3);
+}
+
+static void gfx_v9_4_2_reset_ea_err_status(struct amdgpu_device *adev)
+{
+ uint32_t i, j;
+
+ mutex_lock(&adev->grbm_idx_mutex);
+ for (i = 0; i < gfx_v9_4_2_ea_err_status_regs.se_num; i++) {
+ for (j = 0; j < gfx_v9_4_2_ea_err_status_regs.instance;
+ j++) {
+ gfx_v9_4_2_select_se_sh(adev, i, 0, j);
+ WREG32(SOC15_REG_ENTRY_OFFSET(gfx_v9_4_2_ea_err_status_regs), 0x10);
+ }
+ }
+ gfx_v9_4_2_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ mutex_unlock(&adev->grbm_idx_mutex);
+}
+
+void gfx_v9_4_2_reset_ras_error_count(struct amdgpu_device *adev)
+{
+ if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
+ return;
+
+ gfx_v9_4_2_query_sram_edc_count(adev, NULL, NULL);
+ gfx_v9_4_2_query_utc_edc_count(adev, NULL, NULL);
+}
+
+int gfx_v9_4_2_ras_error_inject(struct amdgpu_device *adev, void *inject_if)
+{
+ struct ras_inject_if *info = (struct ras_inject_if *)inject_if;
+ int ret;
+ struct ta_ras_trigger_error_input block_info = { 0 };
+
+ if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
+ return -EINVAL;
+
+ block_info.block_id = amdgpu_ras_block_to_ta(info->head.block);
+ block_info.sub_block_index = info->head.sub_block_index;
+ block_info.inject_error_type = amdgpu_ras_error_to_ta(info->head.type);
+ block_info.address = info->address;
+ block_info.value = info->value;
+
+ mutex_lock(&adev->grbm_idx_mutex);
+ ret = psp_ras_trigger_error(&adev->psp, &block_info);
+ mutex_unlock(&adev->grbm_idx_mutex);
+
+ return ret;
+}
+
+static void gfx_v9_4_2_query_ea_err_status(struct amdgpu_device *adev)
+{
+ uint32_t i, j;
+ uint32_t reg_value;
+
+ mutex_lock(&adev->grbm_idx_mutex);
+
+ for (i = 0; i < gfx_v9_4_2_ea_err_status_regs.se_num; i++) {
+ for (j = 0; j < gfx_v9_4_2_ea_err_status_regs.instance;
+ j++) {
+ gfx_v9_4_2_select_se_sh(adev, i, 0, j);
+ reg_value = RREG32(SOC15_REG_ENTRY_OFFSET(
+ gfx_v9_4_2_ea_err_status_regs));
+ if (REG_GET_FIELD(reg_value, GCEA_ERR_STATUS, SDP_RDRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, GCEA_ERR_STATUS, SDP_WRRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, GCEA_ERR_STATUS, SDP_RDRSP_DATAPARITY_ERROR)) {
+ dev_warn(adev->dev, "GCEA err detected at instance: %d, status: 0x%x!\n",
+ j, reg_value);
+ }
+ /* clear after read */
+ WREG32(SOC15_REG_ENTRY_OFFSET(gfx_v9_4_2_ea_err_status_regs), 0x10);
+ }
+ }
+
+ gfx_v9_4_2_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ mutex_unlock(&adev->grbm_idx_mutex);
+}
+
+static void gfx_v9_4_2_query_utc_err_status(struct amdgpu_device *adev)
+{
+ uint32_t data;
+
+ data = RREG32_SOC15(GC, 0, regUTCL2_MEM_ECC_STATUS);
+ if (data) {
+ dev_warn(adev->dev, "GFX UTCL2 Mem Ecc Status: 0x%x!\n", data);
+ WREG32_SOC15(GC, 0, regUTCL2_MEM_ECC_STATUS, 0x3);
+ }
+
+ data = RREG32_SOC15(GC, 0, regVML2_MEM_ECC_STATUS);
+ if (data) {
+ dev_warn(adev->dev, "GFX VML2 Mem Ecc Status: 0x%x!\n", data);
+ WREG32_SOC15(GC, 0, regVML2_MEM_ECC_STATUS, 0x3);
+ }
+
+ data = RREG32_SOC15(GC, 0, regVML2_WALKER_MEM_ECC_STATUS);
+ if (data) {
+ dev_warn(adev->dev, "GFX VML2 Walker Mem Ecc Status: 0x%x!\n", data);
+ WREG32_SOC15(GC, 0, regVML2_WALKER_MEM_ECC_STATUS, 0x3);
+ }
+}
+
+void gfx_v9_4_2_query_ras_error_status(struct amdgpu_device *adev)
+{
+ if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
+ return;
+
+ gfx_v9_4_2_query_ea_err_status(adev);
+ gfx_v9_4_2_query_utc_err_status(adev);
+ gfx_v9_4_2_query_sq_timeout_status(adev);
+}
+
+void gfx_v9_4_2_reset_ras_error_status(struct amdgpu_device *adev)
+{
+ if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
+ return;
+
+ gfx_v9_4_2_reset_utc_err_status(adev);
+ gfx_v9_4_2_reset_ea_err_status(adev);
+ gfx_v9_4_2_reset_sq_timeout_status(adev);
+}
+
+void gfx_v9_4_2_enable_watchdog_timer(struct amdgpu_device *adev)
+{
+ uint32_t i;
+ uint32_t data;
+
+ data = REG_SET_FIELD(0, SQ_TIMEOUT_CONFIG, TIMEOUT_FATAL_DISABLE,
+ amdgpu_watchdog_timer.timeout_fatal_disable ? 1 :
+ 0);
+
+ if (amdgpu_watchdog_timer.timeout_fatal_disable &&
+ (amdgpu_watchdog_timer.period < 1 ||
+ amdgpu_watchdog_timer.period > 0x23)) {
+ dev_warn(adev->dev, "Watchdog period range is 1 to 0x23\n");
+ amdgpu_watchdog_timer.period = 0x23;
+ }
+ data = REG_SET_FIELD(data, SQ_TIMEOUT_CONFIG, PERIOD_SEL,
+ amdgpu_watchdog_timer.period);
+
+ mutex_lock(&adev->grbm_idx_mutex);
+ for (i = 0; i < adev->gfx.config.max_shader_engines; i++) {
+ gfx_v9_4_2_select_se_sh(adev, i, 0xffffffff, 0xffffffff);
+ WREG32_SOC15(GC, 0, regSQ_TIMEOUT_CONFIG, data);
+ }
+ gfx_v9_4_2_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ mutex_unlock(&adev->grbm_idx_mutex);
+}
+
+static uint32_t wave_read_ind(struct amdgpu_device *adev, uint32_t simd, uint32_t wave, uint32_t address)
+{
+ WREG32_SOC15_RLC_EX(reg, GC, 0, regSQ_IND_INDEX,
+ (wave << SQ_IND_INDEX__WAVE_ID__SHIFT) |
+ (simd << SQ_IND_INDEX__SIMD_ID__SHIFT) |
+ (address << SQ_IND_INDEX__INDEX__SHIFT) |
+ (SQ_IND_INDEX__FORCE_READ_MASK));
+ return RREG32_SOC15(GC, 0, regSQ_IND_DATA);
+}
+
+static void gfx_v9_4_2_log_cu_timeout_status(struct amdgpu_device *adev,
+ uint32_t status)
+{
+ struct amdgpu_cu_info *cu_info = &adev->gfx.cu_info;
+ uint32_t i, simd, wave;
+ uint32_t wave_status;
+ uint32_t wave_pc_lo, wave_pc_hi;
+ uint32_t wave_exec_lo, wave_exec_hi;
+ uint32_t wave_inst_dw0, wave_inst_dw1;
+ uint32_t wave_ib_sts;
+
+ for (i = 0; i < 32; i++) {
+ if (!((i << 1) & status))
+ continue;
+
+ simd = i / cu_info->max_waves_per_simd;
+ wave = i % cu_info->max_waves_per_simd;
+
+ wave_status = wave_read_ind(adev, simd, wave, ixSQ_WAVE_STATUS);
+ wave_pc_lo = wave_read_ind(adev, simd, wave, ixSQ_WAVE_PC_LO);
+ wave_pc_hi = wave_read_ind(adev, simd, wave, ixSQ_WAVE_PC_HI);
+ wave_exec_lo =
+ wave_read_ind(adev, simd, wave, ixSQ_WAVE_EXEC_LO);
+ wave_exec_hi =
+ wave_read_ind(adev, simd, wave, ixSQ_WAVE_EXEC_HI);
+ wave_inst_dw0 =
+ wave_read_ind(adev, simd, wave, ixSQ_WAVE_INST_DW0);
+ wave_inst_dw1 =
+ wave_read_ind(adev, simd, wave, ixSQ_WAVE_INST_DW1);
+ wave_ib_sts = wave_read_ind(adev, simd, wave, ixSQ_WAVE_IB_STS);
+
+ dev_info(
+ adev->dev,
+ "\t SIMD %d, Wave %d: status 0x%x, pc 0x%llx, exec 0x%llx, inst 0x%llx, ib_sts 0x%x\n",
+ simd, wave, wave_status,
+ ((uint64_t)wave_pc_hi << 32 | wave_pc_lo),
+ ((uint64_t)wave_exec_hi << 32 | wave_exec_lo),
+ ((uint64_t)wave_inst_dw1 << 32 | wave_inst_dw0),
+ wave_ib_sts);
+ }
+}
+
+static void gfx_v9_4_2_query_sq_timeout_status(struct amdgpu_device *adev)
+{
+ uint32_t se_idx, sh_idx, cu_idx;
+ uint32_t status;
+
+ mutex_lock(&adev->grbm_idx_mutex);
+ for (se_idx = 0; se_idx < adev->gfx.config.max_shader_engines;
+ se_idx++) {
+ for (sh_idx = 0; sh_idx < adev->gfx.config.max_sh_per_se;
+ sh_idx++) {
+ for (cu_idx = 0;
+ cu_idx < adev->gfx.config.max_cu_per_sh;
+ cu_idx++) {
+ gfx_v9_4_2_select_se_sh(adev, se_idx, sh_idx,
+ cu_idx);
+ status = RREG32_SOC15(GC, 0,
+ regSQ_TIMEOUT_STATUS);
+ if (status != 0) {
+ dev_info(
+ adev->dev,
+ "GFX Watchdog Timeout: SE %d, SH %d, CU %d\n",
+ se_idx, sh_idx, cu_idx);
+ gfx_v9_4_2_log_cu_timeout_status(
+ adev, status);
+ }
+ /* clear old status */
+ WREG32_SOC15(GC, 0, regSQ_TIMEOUT_STATUS, 0);
+ }
+ }
+ }
+ gfx_v9_4_2_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ mutex_unlock(&adev->grbm_idx_mutex);
+}
+
+static void gfx_v9_4_2_reset_sq_timeout_status(struct amdgpu_device *adev)
+{
+ uint32_t se_idx, sh_idx, cu_idx;
+
+ mutex_lock(&adev->grbm_idx_mutex);
+ for (se_idx = 0; se_idx < adev->gfx.config.max_shader_engines;
+ se_idx++) {
+ for (sh_idx = 0; sh_idx < adev->gfx.config.max_sh_per_se;
+ sh_idx++) {
+ for (cu_idx = 0;
+ cu_idx < adev->gfx.config.max_cu_per_sh;
+ cu_idx++) {
+ gfx_v9_4_2_select_se_sh(adev, se_idx, sh_idx,
+ cu_idx);
+ WREG32_SOC15(GC, 0, regSQ_TIMEOUT_STATUS, 0);
+ }
+ }
+ }
+ gfx_v9_4_2_select_se_sh(adev, 0xffffffff, 0xffffffff, 0xffffffff);
+ mutex_unlock(&adev->grbm_idx_mutex);
+}
+
+const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs = {
+ .ras_late_init = amdgpu_gfx_ras_late_init,
+ .ras_fini = amdgpu_gfx_ras_fini,
+ .ras_error_inject = &gfx_v9_4_2_ras_error_inject,
+ .query_ras_error_count = &gfx_v9_4_2_query_ras_error_count,
+ .reset_ras_error_count = &gfx_v9_4_2_reset_ras_error_count,
+ .query_ras_error_status = &gfx_v9_4_2_query_ras_error_status,
+ .reset_ras_error_status = &gfx_v9_4_2_reset_ras_error_status,
+ .enable_watchdog_timer = &gfx_v9_4_2_enable_watchdog_timer,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.h b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.h
new file mode 100644
index 000000000000..81c5833b6b9f
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4_2.h
@@ -0,0 +1,35 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+
+#ifndef __GFX_V9_4_2_H__
+#define __GFX_V9_4_2_H__
+
+void gfx_v9_4_2_debug_trap_config_init(struct amdgpu_device *adev,
+ uint32_t first_vmid, uint32_t last_vmid);
+void gfx_v9_4_2_init_golden_registers(struct amdgpu_device *adev,
+ uint32_t die_id);
+void gfx_v9_4_2_set_power_brake_sequence(struct amdgpu_device *adev);
+
+extern const struct amdgpu_gfx_ras_funcs gfx_v9_4_2_ras_funcs;
+
+#endif /* __GFX_V9_4_2_H__ */
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
index 6ddd53ba8b77..1e4678cb98f0 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_0.c
@@ -53,19 +53,39 @@ static void gfxhub_v1_0_setup_vm_pt_regs(struct amdgpu_device *adev,
static void gfxhub_v1_0_init_gart_aperture_regs(struct amdgpu_device *adev)
{
- uint64_t pt_base = amdgpu_gmc_pd_addr(adev->gart.bo);
+ uint64_t pt_base;
- gfxhub_v1_0_setup_vm_pt_regs(adev, 0, pt_base);
+ if (adev->gmc.pdb0_bo)
+ pt_base = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo);
+ else
+ pt_base = amdgpu_gmc_pd_addr(adev->gart.bo);
- WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
- (u32)(adev->gmc.gart_start >> 12));
- WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
- (u32)(adev->gmc.gart_start >> 44));
+ gfxhub_v1_0_setup_vm_pt_regs(adev, 0, pt_base);
- WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
- (u32)(adev->gmc.gart_end >> 12));
- WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
- (u32)(adev->gmc.gart_end >> 44));
+ /* If use GART for FB translation, vmid0 page table covers both
+ * vram and system memory (gart)
+ */
+ if (adev->gmc.pdb0_bo) {
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
+ (u32)(adev->gmc.fb_start >> 12));
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
+ (u32)(adev->gmc.fb_start >> 44));
+
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
+ (u32)(adev->gmc.gart_end >> 12));
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
+ (u32)(adev->gmc.gart_end >> 44));
+ } else {
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
+ (u32)(adev->gmc.gart_start >> 12));
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
+ (u32)(adev->gmc.gart_start >> 44));
+
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
+ (u32)(adev->gmc.gart_end >> 12));
+ WREG32_SOC15(GC, 0, mmVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
+ (u32)(adev->gmc.gart_end >> 44));
+ }
}
static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
@@ -100,8 +120,7 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
/* Set default page address. */
- value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
- adev->vm_manager.vram_base_offset;
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
(u32)(value >> 12));
WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_MSB,
@@ -116,6 +135,18 @@ static void gfxhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
WREG32_FIELD15(GC, 0, VM_L2_PROTECTION_FAULT_CNTL2,
ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
}
+
+ /* In the case squeezing vram into GART aperture, we don't use
+ * FB aperture and AGP aperture. Disable them.
+ */
+ if (adev->gmc.pdb0_bo) {
+ WREG32_SOC15(GC, 0, mmMC_VM_FB_LOCATION_TOP, 0);
+ WREG32_SOC15(GC, 0, mmMC_VM_FB_LOCATION_BASE, 0x00FFFFFF);
+ WREG32_SOC15(GC, 0, mmMC_VM_AGP_TOP, 0);
+ WREG32_SOC15(GC, 0, mmMC_VM_AGP_BOT, 0xFFFFFF);
+ WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_LOW_ADDR, 0x3FFFFFFF);
+ WREG32_SOC15(GC, 0, mmMC_VM_SYSTEM_APERTURE_HIGH_ADDR, 0);
+ }
}
static void gfxhub_v1_0_init_tlb_regs(struct amdgpu_device *adev)
@@ -173,8 +204,13 @@ static void gfxhub_v1_0_init_cache_regs(struct amdgpu_device *adev)
WREG32_SOC15_RLC(GC, 0, mmVM_L2_CNTL3, tmp);
tmp = mmVM_L2_CNTL4_DEFAULT;
- tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4, VMC_TAP_PDE_REQUEST_PHYSICAL, 0);
- tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4, VMC_TAP_PTE_REQUEST_PHYSICAL, 0);
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4, VMC_TAP_PDE_REQUEST_PHYSICAL, 1);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4, VMC_TAP_PTE_REQUEST_PHYSICAL, 1);
+ } else {
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4, VMC_TAP_PDE_REQUEST_PHYSICAL, 0);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4, VMC_TAP_PTE_REQUEST_PHYSICAL, 0);
+ }
WREG32_SOC15_RLC(GC, 0, mmVM_L2_CNTL4, tmp);
}
@@ -184,7 +220,10 @@ static void gfxhub_v1_0_enable_system_domain(struct amdgpu_device *adev)
tmp = RREG32_SOC15(GC, 0, mmVM_CONTEXT0_CNTL);
tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
- tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH, 0);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH,
+ adev->gmc.vmid0_page_table_depth);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, PAGE_TABLE_BLOCK_SIZE,
+ adev->gmc.vmid0_page_table_block_size);
tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL,
RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
WREG32_SOC15(GC, 0, mmVM_CONTEXT0_CNTL, tmp);
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_1.c
index c0ab71df0d90..8fca72ebd11c 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v1_1.c
@@ -28,13 +28,42 @@
#include "soc15_common.h"
+#define mmMC_VM_XGMI_LFB_CNTL_ALDE 0x0978
+#define mmMC_VM_XGMI_LFB_CNTL_ALDE_BASE_IDX 0
+#define mmMC_VM_XGMI_LFB_SIZE_ALDE 0x0979
+#define mmMC_VM_XGMI_LFB_SIZE_ALDE_BASE_IDX 0
+//MC_VM_XGMI_LFB_CNTL
+#define MC_VM_XGMI_LFB_CNTL_ALDE__PF_LFB_REGION__SHIFT 0x0
+#define MC_VM_XGMI_LFB_CNTL_ALDE__PF_MAX_REGION__SHIFT 0x4
+#define MC_VM_XGMI_LFB_CNTL_ALDE__PF_LFB_REGION_MASK 0x0000000FL
+#define MC_VM_XGMI_LFB_CNTL_ALDE__PF_MAX_REGION_MASK 0x000000F0L
+//MC_VM_XGMI_LFB_SIZE
+#define MC_VM_XGMI_LFB_SIZE_ALDE__PF_LFB_SIZE__SHIFT 0x0
+#define MC_VM_XGMI_LFB_SIZE_ALDE__PF_LFB_SIZE_MASK 0x0001FFFFL
+
int gfxhub_v1_1_get_xgmi_info(struct amdgpu_device *adev)
{
- u32 xgmi_lfb_cntl = RREG32_SOC15(GC, 0, mmMC_VM_XGMI_LFB_CNTL);
- u32 max_region =
+ u32 max_num_physical_nodes;
+ u32 max_physical_node_id;
+ u32 xgmi_lfb_cntl;
+ u32 max_region;
+ u64 seg_size;
+
+ if (adev->asic_type == CHIP_ALDEBARAN) {
+ xgmi_lfb_cntl = RREG32_SOC15(GC, 0, mmMC_VM_XGMI_LFB_CNTL_ALDE);
+ seg_size = REG_GET_FIELD(
+ RREG32_SOC15(GC, 0, mmMC_VM_XGMI_LFB_SIZE_ALDE),
+ MC_VM_XGMI_LFB_SIZE, PF_LFB_SIZE) << 24;
+ } else {
+ xgmi_lfb_cntl = RREG32_SOC15(GC, 0, mmMC_VM_XGMI_LFB_CNTL);
+ seg_size = REG_GET_FIELD(
+ RREG32_SOC15(GC, 0, mmMC_VM_XGMI_LFB_SIZE),
+ MC_VM_XGMI_LFB_SIZE, PF_LFB_SIZE) << 24;
+ }
+
+ max_region =
REG_GET_FIELD(xgmi_lfb_cntl, MC_VM_XGMI_LFB_CNTL, PF_MAX_REGION);
- u32 max_num_physical_nodes = 0;
- u32 max_physical_node_id = 0;
+
switch (adev->asic_type) {
case CHIP_VEGA20:
@@ -45,23 +74,30 @@ int gfxhub_v1_1_get_xgmi_info(struct amdgpu_device *adev)
max_num_physical_nodes = 8;
max_physical_node_id = 7;
break;
+ case CHIP_ALDEBARAN:
+ /* just using duplicates for Aldebaran support, revisit later */
+ max_num_physical_nodes = 8;
+ max_physical_node_id = 7;
+ break;
default:
return -EINVAL;
}
/* PF_MAX_REGION=0 means xgmi is disabled */
- if (max_region) {
+ if (max_region || adev->gmc.xgmi.connected_to_cpu) {
adev->gmc.xgmi.num_physical_nodes = max_region + 1;
+
if (adev->gmc.xgmi.num_physical_nodes > max_num_physical_nodes)
return -EINVAL;
adev->gmc.xgmi.physical_node_id =
- REG_GET_FIELD(xgmi_lfb_cntl, MC_VM_XGMI_LFB_CNTL, PF_LFB_REGION);
+ REG_GET_FIELD(xgmi_lfb_cntl, MC_VM_XGMI_LFB_CNTL,
+ PF_LFB_REGION);
+
if (adev->gmc.xgmi.physical_node_id > max_physical_node_id)
return -EINVAL;
- adev->gmc.xgmi.node_segment_size = REG_GET_FIELD(
- RREG32_SOC15(GC, 0, mmMC_VM_XGMI_LFB_SIZE),
- MC_VM_XGMI_LFB_SIZE, PF_LFB_SIZE) << 24;
+
+ adev->gmc.xgmi.node_segment_size = seg_size;
}
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
index 2aecc6a243e8..14c1c1a297dd 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_0.c
@@ -165,8 +165,7 @@ static void gfxhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
/* Set default page address. */
- value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start
- + adev->vm_manager.vram_base_offset;
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
WREG32_SOC15(GC, 0, mmGCMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
(u32)(value >> 12));
WREG32_SOC15(GC, 0, mmGCMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_MSB,
diff --git a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
index 410fd3a1a388..41807817de7d 100644
--- a/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/gfxhub_v2_1.c
@@ -164,8 +164,7 @@ static void gfxhub_v2_1_init_system_aperture_regs(struct amdgpu_device *adev)
max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
/* Set default page address. */
- value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start
- + adev->vm_manager.vram_base_offset;
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
WREG32_SOC15(GC, 0, mmGCMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
(u32)(value >> 12));
WREG32_SOC15(GC, 0, mmGCMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_MSB,
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
index 3b7c6c31fce1..498b28a35f5b 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v10_0.c
@@ -113,7 +113,7 @@ static int gmc_v10_0_process_interrupt(struct amdgpu_device *adev,
/* Delegate it to a different ring if the hardware hasn't
* already done it.
*/
- if (in_interrupt()) {
+ if (entry->ih == &adev->irq.ih) {
amdgpu_irq_delegate(adev, entry, 8);
return 1;
}
@@ -152,8 +152,9 @@ static int gmc_v10_0_process_interrupt(struct amdgpu_device *adev,
entry->src_id, entry->ring_id, entry->vmid,
entry->pasid, task_info.process_name, task_info.tgid,
task_info.task_name, task_info.pid);
- dev_err(adev->dev, " in page starting at address 0x%012llx from client %d\n",
- addr, entry->client_id);
+ dev_err(adev->dev, " in page starting at address 0x%016llx from client 0x%x (%s)\n",
+ addr, entry->client_id,
+ soc15_ih_clientid_name[entry->client_id]);
if (!amdgpu_sriov_vf(adev))
hub->vmhub_funcs->print_l2_protection_fault_status(adev,
@@ -567,8 +568,7 @@ static void gmc_v10_0_get_vm_pde(struct amdgpu_device *adev, int level,
uint64_t *addr, uint64_t *flags)
{
if (!(*flags & AMDGPU_PDE_PTE) && !(*flags & AMDGPU_PTE_SYSTEM))
- *addr = adev->vm_manager.vram_base_offset + *addr -
- adev->gmc.vram_start;
+ *addr = amdgpu_gmc_vram_mc2pa(adev, *addr);
BUG_ON(*addr & 0xFFFF00000000003FULL);
if (!adev->gmc.translate_further)
@@ -654,7 +654,7 @@ static void gmc_v10_0_set_umc_funcs(struct amdgpu_device *adev)
adev->umc.umc_inst_num = UMC_V8_7_UMC_INSTANCE_NUM;
adev->umc.channel_offs = UMC_V8_7_PER_CHANNEL_OFFSET_SIENNA;
adev->umc.channel_idx_tbl = &umc_v8_7_channel_idx_tbl[0][0];
- adev->umc.funcs = &umc_v8_7_funcs;
+ adev->umc.ras_funcs = &umc_v8_7_ras_funcs;
break;
default:
break;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
index f5b69484c45a..405d6ad09022 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
@@ -346,6 +346,7 @@ static int gmc_v6_0_mc_init(struct amdgpu_device *adev)
adev->gmc.gart_size = (u64)amdgpu_gart_size << 20;
}
+ adev->gmc.gart_size += adev->pm.smu_prv_buffer_size;
gmc_v6_0_vram_gtt_location(adev, &adev->gmc);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
index dee2b34effb6..210ada2289ec 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
@@ -414,6 +414,7 @@ static int gmc_v7_0_mc_init(struct amdgpu_device *adev)
adev->gmc.gart_size = (u64)amdgpu_gart_size << 20;
}
+ adev->gmc.gart_size += adev->pm.smu_prv_buffer_size;
gmc_v7_0_vram_gtt_location(adev, &adev->gmc);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
index 2d832fc23119..e4f27b3f28fb 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v8_0.c
@@ -59,6 +59,7 @@ MODULE_FIRMWARE("amdgpu/tonga_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris11_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris10_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris12_mc.bin");
+MODULE_FIRMWARE("amdgpu/polaris12_32_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris11_k_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris10_k_mc.bin");
MODULE_FIRMWARE("amdgpu/polaris12_k_mc.bin");
@@ -243,10 +244,16 @@ static int gmc_v8_0_init_microcode(struct amdgpu_device *adev)
chip_name = "polaris10";
break;
case CHIP_POLARIS12:
- if (ASICID_IS_P23(adev->pdev->device, adev->pdev->revision))
+ if (ASICID_IS_P23(adev->pdev->device, adev->pdev->revision)) {
chip_name = "polaris12_k";
- else
- chip_name = "polaris12";
+ } else {
+ WREG32(mmMC_SEQ_IO_DEBUG_INDEX, ixMC_IO_DEBUG_UP_159);
+ /* Polaris12 32bit ASIC needs a special MC firmware */
+ if (RREG32(mmMC_SEQ_IO_DEBUG_DATA) == 0x05b4dc40)
+ chip_name = "polaris12_32";
+ else
+ chip_name = "polaris12";
+ }
break;
case CHIP_FIJI:
case CHIP_CARRIZO:
@@ -599,6 +606,7 @@ static int gmc_v8_0_mc_init(struct amdgpu_device *adev)
adev->gmc.gart_size = (u64)amdgpu_gart_size << 20;
}
+ adev->gmc.gart_size += adev->pm.smu_prv_buffer_size;
gmc_v8_0_vram_gtt_location(adev, &adev->gmc);
return 0;
diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
index 3686e777c76c..455bb91060d0 100644
--- a/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/gmc_v9_0.c
@@ -50,6 +50,7 @@
#include "athub_v1_0.h"
#include "gfxhub_v1_1.h"
#include "mmhub_v9_4.h"
+#include "mmhub_v1_7.h"
#include "umc_v6_1.h"
#include "umc_v6_0.h"
@@ -279,6 +280,47 @@ static const char *mmhub_client_ids_arcturus[][2] = {
[384][1] = "OSS",
};
+static const char *mmhub_client_ids_aldebaran[][2] = {
+ [2][0] = "MP1",
+ [3][0] = "MP0",
+ [32+1][0] = "DBGU_IO0",
+ [32+2][0] = "DBGU_IO2",
+ [32+4][0] = "MPIO",
+ [96+11][0] = "JPEG0",
+ [96+12][0] = "VCN0",
+ [96+13][0] = "VCNU0",
+ [128+11][0] = "JPEG1",
+ [128+12][0] = "VCN1",
+ [128+13][0] = "VCNU1",
+ [160+1][0] = "XDP",
+ [160+14][0] = "HDP",
+ [256+0][0] = "SDMA0",
+ [256+1][0] = "SDMA1",
+ [256+2][0] = "SDMA2",
+ [256+3][0] = "SDMA3",
+ [256+4][0] = "SDMA4",
+ [384+0][0] = "OSS",
+ [2][1] = "MP1",
+ [3][1] = "MP0",
+ [32+1][1] = "DBGU_IO0",
+ [32+2][1] = "DBGU_IO2",
+ [32+4][1] = "MPIO",
+ [96+11][1] = "JPEG0",
+ [96+12][1] = "VCN0",
+ [96+13][1] = "VCNU0",
+ [128+11][1] = "JPEG1",
+ [128+12][1] = "VCN1",
+ [128+13][1] = "VCNU1",
+ [160+1][1] = "XDP",
+ [160+14][1] = "HDP",
+ [256+0][1] = "SDMA0",
+ [256+1][1] = "SDMA1",
+ [256+2][1] = "SDMA2",
+ [256+3][1] = "SDMA3",
+ [256+4][1] = "SDMA4",
+ [384+0][1] = "OSS",
+};
+
static const struct soc15_reg_golden golden_settings_mmhub_1_0_0[] =
{
SOC15_REG_GOLDEN_VALUE(MMHUB, 0, mmDAGB1_WRCLI2, 0x00000007, 0xfe5fe0fa),
@@ -484,7 +526,7 @@ static int gmc_v9_0_process_interrupt(struct amdgpu_device *adev,
/* Delegate it to a different ring if the hardware hasn't
* already done it.
*/
- if (in_interrupt()) {
+ if (entry->ih == &adev->irq.ih) {
amdgpu_irq_delegate(adev, entry, 8);
return 1;
}
@@ -520,8 +562,9 @@ static int gmc_v9_0_process_interrupt(struct amdgpu_device *adev,
entry->src_id, entry->ring_id, entry->vmid,
entry->pasid, task_info.process_name, task_info.tgid,
task_info.task_name, task_info.pid);
- dev_err(adev->dev, " in page starting at address 0x%012llx from client %d\n",
- addr, entry->client_id);
+ dev_err(adev->dev, " in page starting at address 0x%016llx from IH client 0x%x (%s)\n",
+ addr, entry->client_id,
+ soc15_ih_clientid_name[entry->client_id]);
if (amdgpu_sriov_vf(adev))
return 0;
@@ -531,7 +574,8 @@ static int gmc_v9_0_process_interrupt(struct amdgpu_device *adev,
* be updated to avoid reading an incorrect value due to
* the new fast GRBM interface.
*/
- if (entry->vmid_src == AMDGPU_GFXHUB_0)
+ if ((entry->vmid_src == AMDGPU_GFXHUB_0) &&
+ (adev->asic_type < CHIP_ALDEBARAN))
RREG32(hub->vm_l2_pro_fault_status);
status = RREG32(hub->vm_l2_pro_fault_status);
@@ -568,6 +612,9 @@ static int gmc_v9_0_process_interrupt(struct amdgpu_device *adev,
case CHIP_RENOIR:
mmhub_cid = mmhub_client_ids_renoir[cid][rw];
break;
+ case CHIP_ALDEBARAN:
+ mmhub_cid = mmhub_client_ids_aldebaran[cid][rw];
+ break;
default:
mmhub_cid = NULL;
break;
@@ -607,7 +654,8 @@ static void gmc_v9_0_set_irq_funcs(struct amdgpu_device *adev)
adev->gmc.vm_fault.num_types = 1;
adev->gmc.vm_fault.funcs = &gmc_v9_0_irq_funcs;
- if (!amdgpu_sriov_vf(adev)) {
+ if (!amdgpu_sriov_vf(adev) &&
+ !adev->gmc.xgmi.connected_to_cpu) {
adev->gmc.ecc_irq.num_types = 1;
adev->gmc.ecc_irq.funcs = &gmc_v9_0_ecc_funcs;
}
@@ -642,6 +690,9 @@ static uint32_t gmc_v9_0_get_invalidate_req(unsigned int vmid,
static bool gmc_v9_0_use_invalidate_semaphore(struct amdgpu_device *adev,
uint32_t vmhub)
{
+ if (adev->asic_type == CHIP_ALDEBARAN)
+ return false;
+
return ((vmhub == AMDGPU_MMHUB_0 ||
vmhub == AMDGPU_MMHUB_1) &&
(!amdgpu_sriov_vf(adev)) &&
@@ -752,7 +803,8 @@ static void gmc_v9_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
* be cleared to avoid a false ACK due to the new fast
* GRBM interface.
*/
- if (vmhub == AMDGPU_GFXHUB_0)
+ if ((vmhub == AMDGPU_GFXHUB_0) &&
+ (adev->asic_type < CHIP_ALDEBARAN))
RREG32_NO_KIQ(hub->vm_inv_eng0_req +
hub->eng_distance * eng);
@@ -998,8 +1050,7 @@ static void gmc_v9_0_get_vm_pde(struct amdgpu_device *adev, int level,
uint64_t *addr, uint64_t *flags)
{
if (!(*flags & AMDGPU_PDE_PTE) && !(*flags & AMDGPU_PTE_SYSTEM))
- *addr = adev->vm_manager.vram_base_offset + *addr -
- adev->gmc.vram_start;
+ *addr = amdgpu_gmc_vram_mc2pa(adev, *addr);
BUG_ON(*addr & 0xFFFF00000000003FULL);
if (!adev->gmc.translate_further)
@@ -1033,10 +1084,14 @@ static void gmc_v9_0_get_vm_pte(struct amdgpu_device *adev,
*flags &= ~AMDGPU_PTE_VALID;
}
- if (adev->asic_type == CHIP_ARCTURUS &&
+ if ((adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN) &&
!(*flags & AMDGPU_PTE_SYSTEM) &&
mapping->bo_va->is_xgmi)
*flags |= AMDGPU_PTE_SNOOPED;
+
+ if (adev->asic_type == CHIP_ALDEBARAN)
+ *flags |= mapping->flags & AMDGPU_PTE_SNOOPED;
}
static unsigned gmc_v9_0_get_vbios_fb_size(struct amdgpu_device *adev)
@@ -1102,7 +1157,7 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev)
adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM;
adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_VG20;
adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0];
- adev->umc.funcs = &umc_v6_1_funcs;
+ adev->umc.ras_funcs = &umc_v6_1_ras_funcs;
break;
case CHIP_ARCTURUS:
adev->umc.max_ras_err_cnt_per_query = UMC_V6_1_TOTAL_CHANNEL_NUM;
@@ -1110,7 +1165,7 @@ static void gmc_v9_0_set_umc_funcs(struct amdgpu_device *adev)
adev->umc.umc_inst_num = UMC_V6_1_UMC_INSTANCE_NUM;
adev->umc.channel_offs = UMC_V6_1_PER_CHANNEL_OFFSET_ARCT;
adev->umc.channel_idx_tbl = &umc_v6_1_channel_idx_tbl[0][0];
- adev->umc.funcs = &umc_v6_1_funcs;
+ adev->umc.ras_funcs = &umc_v6_1_ras_funcs;
break;
default:
break;
@@ -1123,12 +1178,33 @@ static void gmc_v9_0_set_mmhub_funcs(struct amdgpu_device *adev)
case CHIP_ARCTURUS:
adev->mmhub.funcs = &mmhub_v9_4_funcs;
break;
+ case CHIP_ALDEBARAN:
+ adev->mmhub.funcs = &mmhub_v1_7_funcs;
+ break;
default:
adev->mmhub.funcs = &mmhub_v1_0_funcs;
break;
}
}
+static void gmc_v9_0_set_mmhub_ras_funcs(struct amdgpu_device *adev)
+{
+ switch (adev->asic_type) {
+ case CHIP_VEGA20:
+ adev->mmhub.ras_funcs = &mmhub_v1_0_ras_funcs;
+ break;
+ case CHIP_ARCTURUS:
+ adev->mmhub.ras_funcs = &mmhub_v9_4_ras_funcs;
+ break;
+ case CHIP_ALDEBARAN:
+ adev->mmhub.ras_funcs = &mmhub_v1_7_ras_funcs;
+ break;
+ default:
+ /* mmhub ras is not available */
+ break;
+ }
+}
+
static void gmc_v9_0_set_gfxhub_funcs(struct amdgpu_device *adev)
{
adev->gfxhub.funcs = &gfxhub_v1_0_funcs;
@@ -1138,10 +1214,21 @@ static int gmc_v9_0_early_init(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
+ if (adev->asic_type == CHIP_VEGA20 ||
+ adev->asic_type == CHIP_ARCTURUS)
+ adev->gmc.xgmi.supported = true;
+
+ if (adev->asic_type == CHIP_ALDEBARAN) {
+ adev->gmc.xgmi.supported = true;
+ adev->gmc.xgmi.connected_to_cpu =
+ adev->smuio.funcs->is_host_gpu_xgmi_supported(adev);
+ }
+
gmc_v9_0_set_gmc_funcs(adev);
gmc_v9_0_set_irq_funcs(adev);
gmc_v9_0_set_umc_funcs(adev);
gmc_v9_0_set_mmhub_funcs(adev);
+ gmc_v9_0_set_mmhub_ras_funcs(adev);
gmc_v9_0_set_gfxhub_funcs(adev);
adev->gmc.shared_aperture_start = 0x2000000000000000ULL;
@@ -1174,8 +1261,9 @@ static int gmc_v9_0_late_init(void *handle)
}
}
- if (adev->mmhub.funcs && adev->mmhub.funcs->reset_ras_error_count)
- adev->mmhub.funcs->reset_ras_error_count(adev);
+ if (adev->mmhub.ras_funcs &&
+ adev->mmhub.ras_funcs->reset_ras_error_count)
+ adev->mmhub.ras_funcs->reset_ras_error_count(adev);
r = amdgpu_gmc_ras_late_init(adev);
if (r)
@@ -1194,9 +1282,13 @@ static void gmc_v9_0_vram_gtt_location(struct amdgpu_device *adev,
/* add the xgmi offset of the physical node */
base += adev->gmc.xgmi.physical_node_id * adev->gmc.xgmi.node_segment_size;
- amdgpu_gmc_vram_location(adev, mc, base);
- amdgpu_gmc_gart_location(adev, mc);
- amdgpu_gmc_agp_location(adev, mc);
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ amdgpu_gmc_sysvm_location(adev, mc);
+ } else {
+ amdgpu_gmc_vram_location(adev, mc, base);
+ amdgpu_gmc_gart_location(adev, mc);
+ amdgpu_gmc_agp_location(adev, mc);
+ }
/* base offset of vram pages */
adev->vm_manager.vram_base_offset = adev->gfxhub.funcs->get_mc_fb_offset(adev);
@@ -1223,7 +1315,8 @@ static int gmc_v9_0_mc_init(struct amdgpu_device *adev)
adev->nbio.funcs->get_memsize(adev) * 1024ULL * 1024ULL;
adev->gmc.real_vram_size = adev->gmc.mc_vram_size;
- if (!(adev->flags & AMD_IS_APU)) {
+ if (!(adev->flags & AMD_IS_APU) &&
+ !adev->gmc.xgmi.connected_to_cpu) {
r = amdgpu_device_resize_fb_bar(adev);
if (r)
return r;
@@ -1232,10 +1325,28 @@ static int gmc_v9_0_mc_init(struct amdgpu_device *adev)
adev->gmc.aper_size = pci_resource_len(adev->pdev, 0);
#ifdef CONFIG_X86_64
- if (adev->flags & AMD_IS_APU) {
- adev->gmc.aper_base = adev->gfxhub.funcs->get_mc_fb_offset(adev);
+ /*
+ * AMD Accelerated Processing Platform (APP) supporting GPU-HOST xgmi
+ * interface can use VRAM through here as it appears system reserved
+ * memory in host address space.
+ *
+ * For APUs, VRAM is just the stolen system memory and can be accessed
+ * directly.
+ *
+ * Otherwise, use the legacy Host Data Path (HDP) through PCIe BAR.
+ */
+
+ /* check whether both host-gpu and gpu-gpu xgmi links exist */
+ if ((adev->flags & AMD_IS_APU) ||
+ (adev->gmc.xgmi.supported &&
+ adev->gmc.xgmi.connected_to_cpu)) {
+ adev->gmc.aper_base =
+ adev->gfxhub.funcs->get_mc_fb_offset(adev) +
+ adev->gmc.xgmi.physical_node_id *
+ adev->gmc.xgmi.node_segment_size;
adev->gmc.aper_size = adev->gmc.real_vram_size;
}
+
#endif
/* In case the PCI BAR is larger than the actual amount of vram */
adev->gmc.visible_vram_size = adev->gmc.aper_size;
@@ -1249,6 +1360,7 @@ static int gmc_v9_0_mc_init(struct amdgpu_device *adev)
case CHIP_VEGA12: /* all engines support GPUVM */
case CHIP_VEGA20:
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
default:
adev->gmc.gart_size = 512ULL << 20;
break;
@@ -1261,6 +1373,8 @@ static int gmc_v9_0_mc_init(struct amdgpu_device *adev)
adev->gmc.gart_size = (u64)amdgpu_gart_size << 20;
}
+ adev->gmc.gart_size += adev->pm.smu_prv_buffer_size;
+
gmc_v9_0_vram_gtt_location(adev, &adev->gmc);
return 0;
@@ -1274,6 +1388,15 @@ static int gmc_v9_0_gart_init(struct amdgpu_device *adev)
WARN(1, "VEGA10 PCIE GART already initialized\n");
return 0;
}
+
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ adev->gmc.vmid0_page_table_depth = 1;
+ adev->gmc.vmid0_page_table_block_size = 12;
+ } else {
+ adev->gmc.vmid0_page_table_depth = 0;
+ adev->gmc.vmid0_page_table_block_size = 0;
+ }
+
/* Initialize common gart structure */
r = amdgpu_gart_init(adev);
if (r)
@@ -1281,7 +1404,16 @@ static int gmc_v9_0_gart_init(struct amdgpu_device *adev)
adev->gart.table_size = adev->gart.num_gpu_pages * 8;
adev->gart.gart_pte_flags = AMDGPU_PTE_MTYPE_VG10(MTYPE_UC) |
AMDGPU_PTE_EXECUTABLE;
- return amdgpu_gart_table_vram_alloc(adev);
+
+ r = amdgpu_gart_table_vram_alloc(adev);
+ if (r)
+ return r;
+
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ r = amdgpu_gmc_pdb0_alloc(adev);
+ }
+
+ return r;
}
/**
@@ -1352,6 +1484,7 @@ static int gmc_v9_0_sw_init(void *handle)
case CHIP_VEGA12:
case CHIP_VEGA20:
case CHIP_RENOIR:
+ case CHIP_ALDEBARAN:
adev->num_vmhubs = 2;
@@ -1395,7 +1528,8 @@ static int gmc_v9_0_sw_init(void *handle)
if (r)
return r;
- if (!amdgpu_sriov_vf(adev)) {
+ if (!amdgpu_sriov_vf(adev) &&
+ !adev->gmc.xgmi.connected_to_cpu) {
/* interrupt sent to DF. */
r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DF, 0,
&adev->gmc.ecc_irq);
@@ -1448,7 +1582,8 @@ static int gmc_v9_0_sw_init(void *handle)
* for video processing.
*/
adev->vm_manager.first_kfd_vmid =
- adev->asic_type == CHIP_ARCTURUS ? 3 : 8;
+ (adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN) ? 3 : 8;
amdgpu_vm_manager_init(adev);
@@ -1465,6 +1600,7 @@ static int gmc_v9_0_sw_fini(void *handle)
amdgpu_gem_force_release(adev);
amdgpu_vm_manager_fini(adev);
amdgpu_gart_table_vram_free(adev);
+ amdgpu_bo_unref(&adev->gmc.pdb0_bo);
amdgpu_bo_fini(adev);
amdgpu_gart_fini(adev);
@@ -1525,10 +1661,14 @@ static int gmc_v9_0_gart_enable(struct amdgpu_device *adev)
{
int r;
+ if (adev->gmc.xgmi.connected_to_cpu)
+ amdgpu_gmc_init_pdb0(adev);
+
if (adev->gart.bo == NULL) {
dev_err(adev->dev, "No VRAM object for PCIE GART.\n");
return -EINVAL;
}
+
r = amdgpu_gart_table_vram_pin(adev);
if (r)
return r;
@@ -1541,9 +1681,14 @@ static int gmc_v9_0_gart_enable(struct amdgpu_device *adev)
if (r)
return r;
- DRM_INFO("PCIE GART of %uM enabled (table at 0x%016llX).\n",
- (unsigned)(adev->gmc.gart_size >> 20),
- (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo));
+ DRM_INFO("PCIE GART of %uM enabled.\n",
+ (unsigned)(adev->gmc.gart_size >> 20));
+ if (adev->gmc.pdb0_bo)
+ DRM_INFO("PDB0 located at 0x%016llX\n",
+ (unsigned long long)amdgpu_bo_gpu_offset(adev->gmc.pdb0_bo));
+ DRM_INFO("PTB located at 0x%016llX\n",
+ (unsigned long long)amdgpu_bo_gpu_offset(adev->gart.bo));
+
adev->gart.ready = true;
return 0;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/hdp_v4_0.c b/drivers/gpu/drm/amd/amdgpu/hdp_v4_0.c
index e46621fed5b9..edbd35d293eb 100644
--- a/drivers/gpu/drm/amd/amdgpu/hdp_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/hdp_v4_0.c
@@ -49,6 +49,9 @@ static void hdp_v4_0_flush_hdp(struct amdgpu_device *adev,
static void hdp_v4_0_invalidate_hdp(struct amdgpu_device *adev,
struct amdgpu_ring *ring)
{
+ if (adev->asic_type == CHIP_ALDEBARAN)
+ return;
+
if (!ring || !ring->funcs->emit_wreg)
WREG32_SOC15_NO_KIQ(HDP, 0, mmHDP_READ_CACHE_INVALIDATE, 1);
else
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c
index 7332a320ede8..9360204da7fb 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v1_0.c
@@ -487,7 +487,7 @@ int jpeg_v1_0_sw_init(void *handle)
ring = &adev->jpeg.inst->ring_dec;
sprintf(ring->name, "jpeg_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst->irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
index 3b22953aa62e..de5abceced0d 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_0.c
@@ -108,7 +108,7 @@ static int jpeg_v2_0_sw_init(void *handle)
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1;
sprintf(ring->name, "jpeg_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst->irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
index c6724a0e0c43..83531997aeba 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.c
@@ -115,7 +115,7 @@ static int jpeg_v2_5_sw_init(void *handle)
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1 + 8 * i;
sprintf(ring->name, "jpeg_dec_%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst[i].irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
@@ -565,6 +565,26 @@ static const struct amd_ip_funcs jpeg_v2_5_ip_funcs = {
.set_powergating_state = jpeg_v2_5_set_powergating_state,
};
+static const struct amd_ip_funcs jpeg_v2_6_ip_funcs = {
+ .name = "jpeg_v2_6",
+ .early_init = jpeg_v2_5_early_init,
+ .late_init = NULL,
+ .sw_init = jpeg_v2_5_sw_init,
+ .sw_fini = jpeg_v2_5_sw_fini,
+ .hw_init = jpeg_v2_5_hw_init,
+ .hw_fini = jpeg_v2_5_hw_fini,
+ .suspend = jpeg_v2_5_suspend,
+ .resume = jpeg_v2_5_resume,
+ .is_idle = jpeg_v2_5_is_idle,
+ .wait_for_idle = jpeg_v2_5_wait_for_idle,
+ .check_soft_reset = NULL,
+ .pre_soft_reset = NULL,
+ .soft_reset = NULL,
+ .post_soft_reset = NULL,
+ .set_clockgating_state = jpeg_v2_5_set_clockgating_state,
+ .set_powergating_state = jpeg_v2_5_set_powergating_state,
+};
+
static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_JPEG,
.align_mask = 0xf,
@@ -595,6 +615,36 @@ static const struct amdgpu_ring_funcs jpeg_v2_5_dec_ring_vm_funcs = {
.emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
};
+static const struct amdgpu_ring_funcs jpeg_v2_6_dec_ring_vm_funcs = {
+ .type = AMDGPU_RING_TYPE_VCN_JPEG,
+ .align_mask = 0xf,
+ .vmhub = AMDGPU_MMHUB_0,
+ .get_rptr = jpeg_v2_5_dec_ring_get_rptr,
+ .get_wptr = jpeg_v2_5_dec_ring_get_wptr,
+ .set_wptr = jpeg_v2_5_dec_ring_set_wptr,
+ .emit_frame_size =
+ SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 +
+ SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 +
+ 8 + /* jpeg_v2_5_dec_ring_emit_vm_flush */
+ 18 + 18 + /* jpeg_v2_5_dec_ring_emit_fence x2 vm fence */
+ 8 + 16,
+ .emit_ib_size = 22, /* jpeg_v2_5_dec_ring_emit_ib */
+ .emit_ib = jpeg_v2_0_dec_ring_emit_ib,
+ .emit_fence = jpeg_v2_0_dec_ring_emit_fence,
+ .emit_vm_flush = jpeg_v2_0_dec_ring_emit_vm_flush,
+ .test_ring = amdgpu_jpeg_dec_ring_test_ring,
+ .test_ib = amdgpu_jpeg_dec_ring_test_ib,
+ .insert_nop = jpeg_v2_0_dec_ring_nop,
+ .insert_start = jpeg_v2_0_dec_ring_insert_start,
+ .insert_end = jpeg_v2_0_dec_ring_insert_end,
+ .pad_ib = amdgpu_ring_generic_pad_ib,
+ .begin_use = amdgpu_jpeg_ring_begin_use,
+ .end_use = amdgpu_jpeg_ring_end_use,
+ .emit_wreg = jpeg_v2_0_dec_ring_emit_wreg,
+ .emit_reg_wait = jpeg_v2_0_dec_ring_emit_reg_wait,
+ .emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
+};
+
static void jpeg_v2_5_set_dec_ring_funcs(struct amdgpu_device *adev)
{
int i;
@@ -602,8 +652,10 @@ static void jpeg_v2_5_set_dec_ring_funcs(struct amdgpu_device *adev)
for (i = 0; i < adev->jpeg.num_jpeg_inst; ++i) {
if (adev->jpeg.harvest_config & (1 << i))
continue;
-
- adev->jpeg.inst[i].ring_dec.funcs = &jpeg_v2_5_dec_ring_vm_funcs;
+ if (adev->asic_type == CHIP_ARCTURUS)
+ adev->jpeg.inst[i].ring_dec.funcs = &jpeg_v2_5_dec_ring_vm_funcs;
+ else /* CHIP_ALDEBARAN */
+ adev->jpeg.inst[i].ring_dec.funcs = &jpeg_v2_6_dec_ring_vm_funcs;
adev->jpeg.inst[i].ring_dec.me = i;
DRM_INFO("JPEG(%d) JPEG decode is enabled in VM mode\n", i);
}
@@ -635,3 +687,12 @@ const struct amdgpu_ip_block_version jpeg_v2_5_ip_block =
.rev = 0,
.funcs = &jpeg_v2_5_ip_funcs,
};
+
+const struct amdgpu_ip_block_version jpeg_v2_6_ip_block =
+{
+ .type = AMD_IP_BLOCK_TYPE_JPEG,
+ .major = 2,
+ .minor = 6,
+ .rev = 0,
+ .funcs = &jpeg_v2_6_ip_funcs,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.h b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.h
index 2b4087c02620..3b0aa29b9879 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.h
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v2_5.h
@@ -25,5 +25,6 @@
#define __JPEG_V2_5_H__
extern const struct amdgpu_ip_block_version jpeg_v2_5_ip_block;
+extern const struct amdgpu_ip_block_version jpeg_v2_6_ip_block;
#endif /* __JPEG_V2_5_H__ */
diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c
index e8fbb2a0de34..de5dfcfb3859 100644
--- a/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v3_0.c
@@ -94,7 +94,7 @@ static int jpeg_v3_0_sw_init(void *handle)
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1;
sprintf(ring->name, "jpeg_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->jpeg.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
index 7f30629f21a2..a7ec4ac89da5 100644
--- a/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/mes_v10_1.c
@@ -848,7 +848,8 @@ static int mes_v10_1_ring_init(struct amdgpu_device *adev)
ring->no_scheduler = true;
sprintf(ring->name, "mes_%d.%d.%d", ring->me, ring->pipe, ring->queue);
- return amdgpu_ring_init(adev, ring, 1024, NULL, 0, AMDGPU_RING_PRIO_DEFAULT);
+ return amdgpu_ring_init(adev, ring, 1024, NULL, 0,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
}
static int mes_v10_1_mqd_sw_init(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
index d7b39c07de20..a99953833820 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.c
@@ -114,8 +114,7 @@ static void mmhub_v1_0_init_system_aperture_regs(struct amdgpu_device *adev)
return;
/* Set default page address. */
- value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
- adev->vm_manager.vram_base_offset;
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
(u32)(value >> 12));
WREG32_SOC15(MMHUB, 0, mmMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_MSB,
@@ -776,10 +775,14 @@ static void mmhub_v1_0_reset_ras_error_count(struct amdgpu_device *adev)
}
}
-const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = {
+const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs = {
.ras_late_init = amdgpu_mmhub_ras_late_init,
+ .ras_fini = amdgpu_mmhub_ras_fini,
.query_ras_error_count = mmhub_v1_0_query_ras_error_count,
.reset_ras_error_count = mmhub_v1_0_reset_ras_error_count,
+};
+
+const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs = {
.get_fb_location = mmhub_v1_0_get_fb_location,
.init = mmhub_v1_0_init,
.gart_enable = mmhub_v1_0_gart_enable,
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.h b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.h
index d77f5b65a618..4661b094e007 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.h
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_0.h
@@ -24,5 +24,6 @@
#define __MMHUB_V1_0_H__
extern const struct amdgpu_mmhub_funcs mmhub_v1_0_funcs;
+extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_0_ras_funcs;
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.c
new file mode 100644
index 000000000000..0103a5ab28e6
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.c
@@ -0,0 +1,1333 @@
+/*
+ * Copyright 2019 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "amdgpu.h"
+#include "amdgpu_ras.h"
+#include "mmhub_v1_7.h"
+
+#include "mmhub/mmhub_1_7_offset.h"
+#include "mmhub/mmhub_1_7_sh_mask.h"
+#include "vega10_enum.h"
+
+#include "soc15_common.h"
+#include "soc15.h"
+
+#define regVM_L2_CNTL3_DEFAULT 0x80100007
+#define regVM_L2_CNTL4_DEFAULT 0x000000c1
+
+static u64 mmhub_v1_7_get_fb_location(struct amdgpu_device *adev)
+{
+ u64 base = RREG32_SOC15(MMHUB, 0, regMC_VM_FB_LOCATION_BASE);
+ u64 top = RREG32_SOC15(MMHUB, 0, regMC_VM_FB_LOCATION_TOP);
+
+ base &= MC_VM_FB_LOCATION_BASE__FB_BASE_MASK;
+ base <<= 24;
+
+ top &= MC_VM_FB_LOCATION_TOP__FB_TOP_MASK;
+ top <<= 24;
+
+ adev->gmc.fb_start = base;
+ adev->gmc.fb_end = top;
+
+ return base;
+}
+
+static void mmhub_v1_7_setup_vm_pt_regs(struct amdgpu_device *adev, uint32_t vmid,
+ uint64_t page_table_base)
+{
+ struct amdgpu_vmhub *hub = &adev->vmhub[AMDGPU_MMHUB_0];
+
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32,
+ hub->ctx_addr_distance * vmid, lower_32_bits(page_table_base));
+
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32,
+ hub->ctx_addr_distance * vmid, upper_32_bits(page_table_base));
+}
+
+static void mmhub_v1_7_init_gart_aperture_regs(struct amdgpu_device *adev)
+{
+ uint64_t pt_base;
+
+ if (adev->gmc.pdb0_bo)
+ pt_base = amdgpu_gmc_pd_addr(adev->gmc.pdb0_bo);
+ else
+ pt_base = amdgpu_gmc_pd_addr(adev->gart.bo);
+
+ mmhub_v1_7_setup_vm_pt_regs(adev, 0, pt_base);
+
+ /* If use GART for FB translation, vmid0 page table covers both
+ * vram and system memory (gart)
+ */
+ if (adev->gmc.pdb0_bo) {
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
+ (u32)(adev->gmc.fb_start >> 12));
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
+ (u32)(adev->gmc.fb_start >> 44));
+
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
+ (u32)(adev->gmc.gart_end >> 12));
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
+ (u32)(adev->gmc.gart_end >> 44));
+
+ } else {
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_START_ADDR_LO32,
+ (u32)(adev->gmc.gart_start >> 12));
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_START_ADDR_HI32,
+ (u32)(adev->gmc.gart_start >> 44));
+
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_END_ADDR_LO32,
+ (u32)(adev->gmc.gart_end >> 12));
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_PAGE_TABLE_END_ADDR_HI32,
+ (u32)(adev->gmc.gart_end >> 44));
+ }
+}
+
+static void mmhub_v1_7_init_system_aperture_regs(struct amdgpu_device *adev)
+{
+ uint64_t value;
+ uint32_t tmp;
+
+ /* Program the AGP BAR */
+ WREG32_SOC15(MMHUB, 0, regMC_VM_AGP_BASE, 0);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_AGP_BOT, adev->gmc.agp_start >> 24);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_AGP_TOP, adev->gmc.agp_end >> 24);
+
+ /* Program the system aperture low logical page number. */
+ WREG32_SOC15(MMHUB, 0, regMC_VM_SYSTEM_APERTURE_LOW_ADDR,
+ min(adev->gmc.fb_start, adev->gmc.agp_start) >> 18);
+
+ WREG32_SOC15(MMHUB, 0, regMC_VM_SYSTEM_APERTURE_HIGH_ADDR,
+ max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
+
+ /* In the case squeezing vram into GART aperture, we don't use
+ * FB aperture and AGP aperture. Disable them.
+ */
+ if (adev->gmc.pdb0_bo) {
+ WREG32_SOC15(MMHUB, 0, regMC_VM_AGP_BOT, 0xFFFFFF);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_AGP_TOP, 0);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_FB_LOCATION_TOP, 0);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_FB_LOCATION_BASE, 0x00FFFFFF);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_SYSTEM_APERTURE_LOW_ADDR, 0x3FFFFFFF);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_SYSTEM_APERTURE_HIGH_ADDR, 0);
+ }
+ if (amdgpu_sriov_vf(adev))
+ return;
+
+ /* Set default page address. */
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
+ (u32)(value >> 12));
+ WREG32_SOC15(MMHUB, 0, regMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_MSB,
+ (u32)(value >> 44));
+
+ /* Program "protection fault". */
+ WREG32_SOC15(MMHUB, 0, regVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_LO32,
+ (u32)(adev->dummy_page_addr >> 12));
+ WREG32_SOC15(MMHUB, 0, regVM_L2_PROTECTION_FAULT_DEFAULT_ADDR_HI32,
+ (u32)((u64)adev->dummy_page_addr >> 44));
+
+ tmp = RREG32_SOC15(MMHUB, 0, regVM_L2_PROTECTION_FAULT_CNTL2);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL2,
+ ACTIVE_PAGE_MIGRATION_PTE_READ_RETRY, 1);
+ WREG32_SOC15(MMHUB, 0, regVM_L2_PROTECTION_FAULT_CNTL2, tmp);
+}
+
+static void mmhub_v1_7_init_tlb_regs(struct amdgpu_device *adev)
+{
+ uint32_t tmp;
+
+ /* Setup TLB control */
+ tmp = RREG32_SOC15(MMHUB, 0, regMC_VM_MX_L1_TLB_CNTL);
+
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL, ENABLE_L1_TLB, 1);
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL, SYSTEM_ACCESS_MODE, 3);
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL,
+ ENABLE_ADVANCED_DRIVER_MODEL, 1);
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL,
+ SYSTEM_APERTURE_UNMAPPED_ACCESS, 0);
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL, ECO_BITS, 0);
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL,
+ MTYPE, MTYPE_UC);/* XXX for emulation. */
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL, ATC_EN, 1);
+
+ WREG32_SOC15(MMHUB, 0, regMC_VM_MX_L1_TLB_CNTL, tmp);
+}
+
+static void mmhub_v1_7_init_cache_regs(struct amdgpu_device *adev)
+{
+ uint32_t tmp;
+
+ if (amdgpu_sriov_vf(adev))
+ return;
+
+ /* Setup L2 cache */
+ tmp = RREG32_SOC15(MMHUB, 0, regVM_L2_CNTL);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 1);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_FRAGMENT_PROCESSING, 1);
+ /* XXX for emulation, Refer to closed source code.*/
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, L2_PDE0_CACHE_TAG_GENERATION_MODE,
+ 0);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, PDE_FAULT_CLASSIFICATION, 0);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, CONTEXT1_IDENTITY_ACCESS_MODE, 1);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, IDENTITY_MODE_FRAGMENT_SIZE, 0);
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CNTL, tmp);
+
+ tmp = RREG32_SOC15(MMHUB, 0, regVM_L2_CNTL2);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_ALL_L1_TLBS, 1);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL2, INVALIDATE_L2_CACHE, 1);
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CNTL2, tmp);
+
+ tmp = regVM_L2_CNTL3_DEFAULT;
+ if (adev->gmc.translate_further) {
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL3, BANK_SELECT, 12);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL3,
+ L2_CACHE_BIGK_FRAGMENT_SIZE, 9);
+ } else {
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL3, BANK_SELECT, 9);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL3,
+ L2_CACHE_BIGK_FRAGMENT_SIZE, 6);
+ }
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CNTL3, tmp);
+
+ tmp = regVM_L2_CNTL4_DEFAULT;
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4,
+ VMC_TAP_PDE_REQUEST_PHYSICAL, 1);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4,
+ VMC_TAP_PTE_REQUEST_PHYSICAL, 1);
+ } else {
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4,
+ VMC_TAP_PDE_REQUEST_PHYSICAL, 0);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL4,
+ VMC_TAP_PTE_REQUEST_PHYSICAL, 0);
+ }
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CNTL4, tmp);
+}
+
+static void mmhub_v1_7_enable_system_domain(struct amdgpu_device *adev)
+{
+ uint32_t tmp;
+
+ tmp = RREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_CNTL);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, ENABLE_CONTEXT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, PAGE_TABLE_DEPTH,
+ adev->gmc.vmid0_page_table_depth);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL, PAGE_TABLE_BLOCK_SIZE,
+ adev->gmc.vmid0_page_table_block_size);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT0_CNTL,
+ RETRY_PERMISSION_OR_INVALID_PAGE_FAULT, 0);
+ WREG32_SOC15(MMHUB, 0, regVM_CONTEXT0_CNTL, tmp);
+}
+
+static void mmhub_v1_7_disable_identity_aperture(struct amdgpu_device *adev)
+{
+ if (amdgpu_sriov_vf(adev))
+ return;
+
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR_LO32,
+ 0XFFFFFFFF);
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CONTEXT1_IDENTITY_APERTURE_LOW_ADDR_HI32,
+ 0x0000000F);
+
+ WREG32_SOC15(MMHUB, 0,
+ regVM_L2_CONTEXT1_IDENTITY_APERTURE_HIGH_ADDR_LO32, 0);
+ WREG32_SOC15(MMHUB, 0,
+ regVM_L2_CONTEXT1_IDENTITY_APERTURE_HIGH_ADDR_HI32, 0);
+
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CONTEXT_IDENTITY_PHYSICAL_OFFSET_LO32,
+ 0);
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CONTEXT_IDENTITY_PHYSICAL_OFFSET_HI32,
+ 0);
+}
+
+static void mmhub_v1_7_setup_vmid_config(struct amdgpu_device *adev)
+{
+ struct amdgpu_vmhub *hub = &adev->vmhub[AMDGPU_MMHUB_0];
+ unsigned num_level, block_size;
+ uint32_t tmp;
+ int i;
+
+ num_level = adev->vm_manager.num_level;
+ block_size = adev->vm_manager.block_size;
+ if (adev->gmc.translate_further)
+ num_level -= 1;
+ else
+ block_size -= 9;
+
+ for (i = 0; i <= 14; i++) {
+ tmp = RREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT1_CNTL, i);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL, ENABLE_CONTEXT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL, PAGE_TABLE_DEPTH,
+ num_level);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ RANGE_PROTECTION_FAULT_ENABLE_DEFAULT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ DUMMY_PAGE_PROTECTION_FAULT_ENABLE_DEFAULT,
+ 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ PDE0_PROTECTION_FAULT_ENABLE_DEFAULT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ VALID_PROTECTION_FAULT_ENABLE_DEFAULT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ READ_PROTECTION_FAULT_ENABLE_DEFAULT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ WRITE_PROTECTION_FAULT_ENABLE_DEFAULT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ EXECUTE_PROTECTION_FAULT_ENABLE_DEFAULT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ PAGE_TABLE_BLOCK_SIZE,
+ block_size);
+ /* Send no-retry XNACK on fault to suppress VM fault storm. */
+ tmp = REG_SET_FIELD(tmp, VM_CONTEXT1_CNTL,
+ RETRY_PERMISSION_OR_INVALID_PAGE_FAULT,
+ !adev->gmc.noretry);
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT1_CNTL,
+ i * hub->ctx_distance, tmp);
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT1_PAGE_TABLE_START_ADDR_LO32,
+ i * hub->ctx_addr_distance, 0);
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT1_PAGE_TABLE_START_ADDR_HI32,
+ i * hub->ctx_addr_distance, 0);
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT1_PAGE_TABLE_END_ADDR_LO32,
+ i * hub->ctx_addr_distance,
+ lower_32_bits(adev->vm_manager.max_pfn - 1));
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT1_PAGE_TABLE_END_ADDR_HI32,
+ i * hub->ctx_addr_distance,
+ upper_32_bits(adev->vm_manager.max_pfn - 1));
+ }
+}
+
+static void mmhub_v1_7_program_invalidation(struct amdgpu_device *adev)
+{
+ struct amdgpu_vmhub *hub = &adev->vmhub[AMDGPU_MMHUB_0];
+ unsigned i;
+
+ for (i = 0; i < 18; ++i) {
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_INVALIDATE_ENG0_ADDR_RANGE_LO32,
+ i * hub->eng_addr_distance, 0xffffffff);
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_INVALIDATE_ENG0_ADDR_RANGE_HI32,
+ i * hub->eng_addr_distance, 0x1f);
+ }
+}
+
+static int mmhub_v1_7_gart_enable(struct amdgpu_device *adev)
+{
+ if (amdgpu_sriov_vf(adev)) {
+ /*
+ * MC_VM_FB_LOCATION_BASE/TOP is NULL for VF, becuase they are
+ * VF copy registers so vbios post doesn't program them, for
+ * SRIOV driver need to program them
+ */
+ WREG32_SOC15(MMHUB, 0, regMC_VM_FB_LOCATION_BASE,
+ adev->gmc.vram_start >> 24);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_FB_LOCATION_TOP,
+ adev->gmc.vram_end >> 24);
+ }
+
+ /* GART Enable. */
+ mmhub_v1_7_init_gart_aperture_regs(adev);
+ mmhub_v1_7_init_system_aperture_regs(adev);
+ mmhub_v1_7_init_tlb_regs(adev);
+ mmhub_v1_7_init_cache_regs(adev);
+
+ mmhub_v1_7_enable_system_domain(adev);
+ mmhub_v1_7_disable_identity_aperture(adev);
+ mmhub_v1_7_setup_vmid_config(adev);
+ mmhub_v1_7_program_invalidation(adev);
+
+ return 0;
+}
+
+static void mmhub_v1_7_gart_disable(struct amdgpu_device *adev)
+{
+ struct amdgpu_vmhub *hub = &adev->vmhub[AMDGPU_MMHUB_0];
+ u32 tmp;
+ u32 i;
+
+ /* Disable all tables */
+ for (i = 0; i < 16; i++)
+ WREG32_SOC15_OFFSET(MMHUB, 0, regVM_CONTEXT0_CNTL,
+ i * hub->ctx_distance, 0);
+
+ /* Setup TLB control */
+ tmp = RREG32_SOC15(MMHUB, 0, regMC_VM_MX_L1_TLB_CNTL);
+ tmp = REG_SET_FIELD(tmp, MC_VM_MX_L1_TLB_CNTL, ENABLE_L1_TLB, 0);
+ tmp = REG_SET_FIELD(tmp,
+ MC_VM_MX_L1_TLB_CNTL,
+ ENABLE_ADVANCED_DRIVER_MODEL,
+ 0);
+ WREG32_SOC15(MMHUB, 0, regMC_VM_MX_L1_TLB_CNTL, tmp);
+
+ if (!amdgpu_sriov_vf(adev)) {
+ /* Setup L2 cache */
+ tmp = RREG32_SOC15(MMHUB, 0, regVM_L2_CNTL);
+ tmp = REG_SET_FIELD(tmp, VM_L2_CNTL, ENABLE_L2_CACHE, 0);
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CNTL, tmp);
+ WREG32_SOC15(MMHUB, 0, regVM_L2_CNTL3, 0);
+ }
+}
+
+/**
+ * mmhub_v1_7_set_fault_enable_default - update GART/VM fault handling
+ *
+ * @adev: amdgpu_device pointer
+ * @value: true redirects VM faults to the default page
+ */
+static void mmhub_v1_7_set_fault_enable_default(struct amdgpu_device *adev, bool value)
+{
+ u32 tmp;
+
+ if (amdgpu_sriov_vf(adev))
+ return;
+
+ tmp = RREG32_SOC15(MMHUB, 0, regVM_L2_PROTECTION_FAULT_CNTL);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ RANGE_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ PDE0_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ PDE1_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ PDE2_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp,
+ VM_L2_PROTECTION_FAULT_CNTL,
+ TRANSLATE_FURTHER_PROTECTION_FAULT_ENABLE_DEFAULT,
+ value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ NACK_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ DUMMY_PAGE_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ VALID_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ READ_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ WRITE_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ EXECUTE_PROTECTION_FAULT_ENABLE_DEFAULT, value);
+ if (!value) {
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ CRASH_ON_NO_RETRY_FAULT, 1);
+ tmp = REG_SET_FIELD(tmp, VM_L2_PROTECTION_FAULT_CNTL,
+ CRASH_ON_RETRY_FAULT, 1);
+ }
+
+ WREG32_SOC15(MMHUB, 0, regVM_L2_PROTECTION_FAULT_CNTL, tmp);
+}
+
+static void mmhub_v1_7_init(struct amdgpu_device *adev)
+{
+ struct amdgpu_vmhub *hub = &adev->vmhub[AMDGPU_MMHUB_0];
+
+ hub->ctx0_ptb_addr_lo32 =
+ SOC15_REG_OFFSET(MMHUB, 0,
+ regVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32);
+ hub->ctx0_ptb_addr_hi32 =
+ SOC15_REG_OFFSET(MMHUB, 0,
+ regVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_HI32);
+ hub->vm_inv_eng0_req =
+ SOC15_REG_OFFSET(MMHUB, 0, regVM_INVALIDATE_ENG0_REQ);
+ hub->vm_inv_eng0_ack =
+ SOC15_REG_OFFSET(MMHUB, 0, regVM_INVALIDATE_ENG0_ACK);
+ hub->vm_context0_cntl =
+ SOC15_REG_OFFSET(MMHUB, 0, regVM_CONTEXT0_CNTL);
+ hub->vm_l2_pro_fault_status =
+ SOC15_REG_OFFSET(MMHUB, 0, regVM_L2_PROTECTION_FAULT_STATUS);
+ hub->vm_l2_pro_fault_cntl =
+ SOC15_REG_OFFSET(MMHUB, 0, regVM_L2_PROTECTION_FAULT_CNTL);
+
+ hub->ctx_distance = regVM_CONTEXT1_CNTL - regVM_CONTEXT0_CNTL;
+ hub->ctx_addr_distance = regVM_CONTEXT1_PAGE_TABLE_BASE_ADDR_LO32 -
+ regVM_CONTEXT0_PAGE_TABLE_BASE_ADDR_LO32;
+ hub->eng_distance = regVM_INVALIDATE_ENG1_REQ - regVM_INVALIDATE_ENG0_REQ;
+ hub->eng_addr_distance = regVM_INVALIDATE_ENG1_ADDR_RANGE_LO32 -
+ regVM_INVALIDATE_ENG0_ADDR_RANGE_LO32;
+
+}
+
+static void mmhub_v1_7_update_medium_grain_clock_gating(struct amdgpu_device *adev,
+ bool enable)
+{
+ uint32_t def, data, def1, data1, def2 = 0, data2 = 0;
+
+ def = data = RREG32_SOC15(MMHUB, 0, regATC_L2_MISC_CG);
+
+ def1 = data1 = RREG32_SOC15(MMHUB, 0, regDAGB0_CNTL_MISC2);
+ def2 = data2 = RREG32_SOC15(MMHUB, 0, regDAGB1_CNTL_MISC2);
+
+ if (enable) {
+ data |= ATC_L2_MISC_CG__ENABLE_MASK;
+
+ data1 &= ~(DAGB0_CNTL_MISC2__DISABLE_WRREQ_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_WRRET_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_RDREQ_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_RDRET_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_TLBWR_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_TLBRD_CG_MASK);
+
+ data2 &= ~(DAGB1_CNTL_MISC2__DISABLE_WRREQ_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_WRRET_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_RDREQ_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_RDRET_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_TLBWR_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_TLBRD_CG_MASK);
+ } else {
+ data &= ~ATC_L2_MISC_CG__ENABLE_MASK;
+
+ data1 |= (DAGB0_CNTL_MISC2__DISABLE_WRREQ_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_WRRET_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_RDREQ_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_RDRET_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_TLBWR_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_TLBRD_CG_MASK);
+
+ data2 |= (DAGB1_CNTL_MISC2__DISABLE_WRREQ_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_WRRET_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_RDREQ_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_RDRET_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_TLBWR_CG_MASK |
+ DAGB1_CNTL_MISC2__DISABLE_TLBRD_CG_MASK);
+ }
+
+ if (def != data)
+ WREG32_SOC15(MMHUB, 0, regATC_L2_MISC_CG, data);
+
+ if (def1 != data1)
+ WREG32_SOC15(MMHUB, 0, regDAGB0_CNTL_MISC2, data1);
+
+ if (def2 != data2)
+ WREG32_SOC15(MMHUB, 0, regDAGB1_CNTL_MISC2, data2);
+}
+
+static void mmhub_v1_7_update_medium_grain_light_sleep(struct amdgpu_device *adev,
+ bool enable)
+{
+ uint32_t def, data;
+
+ def = data = RREG32_SOC15(MMHUB, 0, regATC_L2_MISC_CG);
+
+ if (enable)
+ data |= ATC_L2_MISC_CG__MEM_LS_ENABLE_MASK;
+ else
+ data &= ~ATC_L2_MISC_CG__MEM_LS_ENABLE_MASK;
+
+ if (def != data)
+ WREG32_SOC15(MMHUB, 0, regATC_L2_MISC_CG, data);
+}
+
+static int mmhub_v1_7_set_clockgating(struct amdgpu_device *adev,
+ enum amd_clockgating_state state)
+{
+ if (amdgpu_sriov_vf(adev))
+ return 0;
+
+ /* Change state only if MCCG support is enabled through driver */
+ if (adev->cg_flags & AMD_CG_SUPPORT_MC_MGCG)
+ mmhub_v1_7_update_medium_grain_clock_gating(adev,
+ state == AMD_CG_STATE_GATE);
+
+ /* Change state only if LS support is enabled through driver */
+ if (adev->cg_flags & AMD_CG_SUPPORT_MC_LS)
+ mmhub_v1_7_update_medium_grain_light_sleep(adev,
+ state == AMD_CG_STATE_GATE);
+
+ return 0;
+}
+
+static void mmhub_v1_7_get_clockgating(struct amdgpu_device *adev, u32 *flags)
+{
+ int data, data1;
+
+ if (amdgpu_sriov_vf(adev))
+ *flags = 0;
+
+ data = RREG32_SOC15(MMHUB, 0, regATC_L2_MISC_CG);
+
+ data1 = RREG32_SOC15(MMHUB, 0, regDAGB0_CNTL_MISC2);
+
+ /* AMD_CG_SUPPORT_MC_MGCG */
+ if ((data & ATC_L2_MISC_CG__ENABLE_MASK) &&
+ !(data1 & (DAGB0_CNTL_MISC2__DISABLE_WRREQ_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_WRRET_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_RDREQ_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_RDRET_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_TLBWR_CG_MASK |
+ DAGB0_CNTL_MISC2__DISABLE_TLBRD_CG_MASK)))
+ *flags |= AMD_CG_SUPPORT_MC_MGCG;
+
+ /* AMD_CG_SUPPORT_MC_LS */
+ if (data & ATC_L2_MISC_CG__MEM_LS_ENABLE_MASK)
+ *flags |= AMD_CG_SUPPORT_MC_LS;
+}
+
+static const struct soc15_ras_field_entry mmhub_v1_7_ras_fields[] = {
+ /* MMHUB Range 0 */
+ { "MMEA0_DRAMRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA0_DRAMWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA0_DRAMWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA0_RRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, RRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, RRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA0_WRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, WRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, WRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA0_IOWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, IOWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, IOWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA0_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA0_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, DRAMWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA0_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, IORD_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA0_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT, IOWR_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA0_GMIRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA0_GMIWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA0_GMIWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA0_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA0_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, GMIWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA0_MAM_D0MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D0MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D0MEM_DED_COUNT),
+ },
+ { "MMEA0_MAM_D1MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D1MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D1MEM_DED_COUNT),
+ },
+ { "MMEA0_MAM_D2MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D2MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D2MEM_DED_COUNT),
+ },
+ { "MMEA0_MAM_D3MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D3MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D3MEM_DED_COUNT),
+ },
+ { "MMEA0_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA0_EDC_CNT3, DRAMRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA0_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA0_EDC_CNT3, DRAMWR_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA0_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA0_EDC_CNT3, IORD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA0_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA0_EDC_CNT3, IOWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA0_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA0_EDC_CNT3, GMIRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA0_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA0_EDC_CNT3, GMIWR_PAGEMEM_DED_COUNT),
+ },
+
+ /* MMHUB Range 1 */
+ { "MMEA1_DRAMRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA1_DRAMWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA1_DRAMWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA1_RRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, RRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, RRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA1_WRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, WRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, WRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA1_IOWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, IOWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, IOWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA1_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA1_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, DRAMWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA1_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, IORD_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA1_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT, IOWR_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA1_GMIRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA1_GMIWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA1_GMIWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA1_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA1_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, GMIWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA1_MAM_D0MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D0MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D0MEM_DED_COUNT),
+ },
+ { "MMEA1_MAM_D1MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D1MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D1MEM_DED_COUNT),
+ },
+ { "MMEA1_MAM_D2MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D2MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D2MEM_DED_COUNT),
+ },
+ { "MMEA1_MAM_D3MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D3MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA1_EDC_CNT2, MAM_D3MEM_DED_COUNT),
+ },
+ { "MMEA1_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA1_EDC_CNT3, DRAMRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA1_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA1_EDC_CNT3, DRAMWR_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA1_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA1_EDC_CNT3, IORD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA1_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA1_EDC_CNT3, IOWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA1_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA1_EDC_CNT3, GMIRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA1_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA1_EDC_CNT3, GMIWR_PAGEMEM_DED_COUNT),
+ },
+
+ /* MMHAB Range 2*/
+ { "MMEA2_DRAMRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA2_DRAMWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA2_DRAMWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA2_RRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, RRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, RRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA2_WRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, WRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, WRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA2_IOWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, IOWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, IOWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA2_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA2_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, DRAMWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA2_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, IORD_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA2_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT, IOWR_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA2_GMIRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA2_GMIWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA2_GMIWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA2_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA2_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, GMIWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA2_MAM_D0MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D0MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA0_EDC_CNT2, MAM_D0MEM_DED_COUNT),
+ },
+ { "MMEA2_MAM_D1MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, MAM_D1MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, MAM_D1MEM_DED_COUNT),
+ },
+ { "MMEA2_MAM_D2MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, MAM_D2MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, MAM_D2MEM_DED_COUNT),
+ },
+ { "MMEA2_MAM_D3MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, MAM_D3MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA2_EDC_CNT2, MAM_D3MEM_DED_COUNT),
+ },
+ { "MMEA2_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA2_EDC_CNT3, DRAMRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA2_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA2_EDC_CNT3, DRAMWR_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA2_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA2_EDC_CNT3, IORD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA2_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA2_EDC_CNT3, IOWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA2_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA2_EDC_CNT3, GMIRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA2_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA2_EDC_CNT3, GMIWR_PAGEMEM_DED_COUNT),
+ },
+
+ /* MMHUB Rang 3 */
+ { "MMEA3_DRAMRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA3_DRAMWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA3_DRAMWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA3_RRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, RRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, RRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA3_WRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, WRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, WRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA3_IOWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, IOWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, IOWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA3_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA3_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, DRAMWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA3_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, IORD_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA3_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT, IOWR_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA3_GMIRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA3_GMIWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA3_GMIWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA3_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA3_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, GMIWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA3_MAM_D0MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D0MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D0MEM_DED_COUNT),
+ },
+ { "MMEA3_MAM_D1MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D1MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D1MEM_DED_COUNT),
+ },
+ { "MMEA3_MAM_D2MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D2MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D2MEM_DED_COUNT),
+ },
+ { "MMEA3_MAM_D3MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D3MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA3_EDC_CNT2, MAM_D3MEM_DED_COUNT),
+ },
+ { "MMEA3_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA3_EDC_CNT3, DRAMRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA3_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA3_EDC_CNT3, DRAMWR_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA3_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA3_EDC_CNT3, IORD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA3_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA3_EDC_CNT3, IOWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA3_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA3_EDC_CNT3, GMIRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA3_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA3_EDC_CNT3, GMIWR_PAGEMEM_DED_COUNT),
+ },
+
+ /* MMHUB Range 4 */
+ { "MMEA4_DRAMRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA4_DRAMWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA4_DRAMWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA4_RRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, RRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, RRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA4_WRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, WRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, WRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA4_IOWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, IOWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, IOWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA4_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA4_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, DRAMWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA4_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, IORD_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA4_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT, IOWR_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA4_GMIRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA4_GMIWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA4_GMIWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA4_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA4_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, GMIWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA4_MAM_D0MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D0MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D0MEM_DED_COUNT),
+ },
+ { "MMEA4_MAM_D1MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D1MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D1MEM_DED_COUNT),
+ },
+ { "MMEA4_MAM_D2MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D2MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D2MEM_DED_COUNT),
+ },
+ { "MMEA4_MAM_D3MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D3MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA4_EDC_CNT2, MAM_D3MEM_DED_COUNT),
+ },
+ { "MMEA4_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA4_EDC_CNT3, DRAMRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA4_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA4_EDC_CNT3, DRAMWR_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA4_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA4_EDC_CNT3, IORD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA4_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA4_EDC_CNT3, IOWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA4_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA4_EDC_CNT3, GMIRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA4_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA4_EDC_CNT3, GMIWR_PAGEMEM_DED_COUNT),
+ },
+
+ /* MMHUAB Range 5 */
+ { "MMEA5_DRAMRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA5_DRAMWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA5_DRAMWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA5_RRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, RRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, RRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA5_WRET_TAGMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, WRET_TAGMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, WRET_TAGMEM_DED_COUNT),
+ },
+ { "MMEA5_IOWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, IOWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, IOWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA5_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA5_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, DRAMWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA5_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, IORD_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA5_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT, IOWR_CMDMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA5_GMIRD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIRD_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIRD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA5_GMIWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIWR_CMDMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA5_GMIWR_DATAMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIWR_DATAMEM_SEC_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIWR_DATAMEM_DED_COUNT),
+ },
+ { "MMEA5_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIRD_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA5_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, GMIWR_PAGEMEM_SED_COUNT),
+ 0, 0,
+ },
+ { "MMEA5_MAM_D0MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D0MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D0MEM_DED_COUNT),
+ },
+ { "MMEA5_MAM_D1MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D1MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D1MEM_DED_COUNT),
+ },
+ { "MMEA5_MAM_D2MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D2MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D2MEM_DED_COUNT),
+ },
+ { "MMEA5_MAM_D3MEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D3MEM_SED_COUNT),
+ SOC15_REG_FIELD(MMEA5_EDC_CNT2, MAM_D3MEM_DED_COUNT),
+ },
+ { "MMEA5_DRAMRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA5_EDC_CNT3, DRAMRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA5_DRAMWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA5_EDC_CNT3, DRAMWR_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA5_IORD_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA5_EDC_CNT3, IORD_CMDMEM_DED_COUNT),
+ },
+ { "MMEA5_IOWR_CMDMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA5_EDC_CNT3, IOWR_CMDMEM_DED_COUNT),
+ },
+ { "MMEA5_GMIRD_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA5_EDC_CNT3, GMIRD_PAGEMEM_DED_COUNT),
+ },
+ { "MMEA5_GMIWR_PAGEMEM", SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT3),
+ 0, 0,
+ SOC15_REG_FIELD(MMEA5_EDC_CNT3, GMIWR_PAGEMEM_DED_COUNT),
+ },
+};
+
+static const struct soc15_reg_entry mmhub_v1_7_edc_cnt_regs[] = {
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT2), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_EDC_CNT3), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT2), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_EDC_CNT3), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT2), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_EDC_CNT3), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT2), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_EDC_CNT3), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT2), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_EDC_CNT3), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT2), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_EDC_CNT3), 0, 0, 0 },
+};
+
+static int mmhub_v1_7_get_ras_error_count(struct amdgpu_device *adev,
+ const struct soc15_reg_entry *reg,
+ uint32_t value,
+ uint32_t *sec_count,
+ uint32_t *ded_count)
+{
+ uint32_t i;
+ uint32_t sec_cnt, ded_cnt;
+
+ for (i = 0; i < ARRAY_SIZE(mmhub_v1_7_ras_fields); i++) {
+ if(mmhub_v1_7_ras_fields[i].reg_offset != reg->reg_offset)
+ continue;
+
+ sec_cnt = (value &
+ mmhub_v1_7_ras_fields[i].sec_count_mask) >>
+ mmhub_v1_7_ras_fields[i].sec_count_shift;
+ if (sec_cnt) {
+ dev_info(adev->dev, "MMHUB SubBlock %s, SEC %d\n",
+ mmhub_v1_7_ras_fields[i].name,
+ sec_cnt);
+ *sec_count += sec_cnt;
+ }
+
+ ded_cnt = (value &
+ mmhub_v1_7_ras_fields[i].ded_count_mask) >>
+ mmhub_v1_7_ras_fields[i].ded_count_shift;
+ if (ded_cnt) {
+ dev_info(adev->dev, "MMHUB SubBlock %s, DED %d\n",
+ mmhub_v1_7_ras_fields[i].name,
+ ded_cnt);
+ *ded_count += ded_cnt;
+ }
+ }
+
+ return 0;
+}
+
+static void mmhub_v1_7_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+ uint32_t sec_count = 0, ded_count = 0;
+ uint32_t i;
+ uint32_t reg_value;
+
+ err_data->ue_count = 0;
+ err_data->ce_count = 0;
+
+ for (i = 0; i < ARRAY_SIZE(mmhub_v1_7_edc_cnt_regs); i++) {
+ reg_value =
+ RREG32(SOC15_REG_ENTRY_OFFSET(mmhub_v1_7_edc_cnt_regs[i]));
+ if (reg_value)
+ mmhub_v1_7_get_ras_error_count(adev, &mmhub_v1_7_edc_cnt_regs[i],
+ reg_value, &sec_count, &ded_count);
+ }
+
+ err_data->ce_count += sec_count;
+ err_data->ue_count += ded_count;
+}
+
+static void mmhub_v1_7_reset_ras_error_count(struct amdgpu_device *adev)
+{
+ uint32_t i;
+
+ /* write 0 to reset the edc counters */
+ if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB)) {
+ for (i = 0; i < ARRAY_SIZE(mmhub_v1_7_edc_cnt_regs); i++)
+ WREG32(SOC15_REG_ENTRY_OFFSET(mmhub_v1_7_edc_cnt_regs[i]), 0);
+ }
+}
+
+static const struct soc15_reg_entry mmhub_v1_7_ea_err_status_regs[] = {
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA0_ERR_STATUS), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA1_ERR_STATUS), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA2_ERR_STATUS), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA3_ERR_STATUS), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA4_ERR_STATUS), 0, 0, 0 },
+ { SOC15_REG_ENTRY(MMHUB, 0, regMMEA5_ERR_STATUS), 0, 0, 0 },
+};
+
+static void mmhub_v1_7_query_ras_error_status(struct amdgpu_device *adev)
+{
+ int i;
+ uint32_t reg_value;
+
+ if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__MMHUB))
+ return;
+
+ for (i = 0; i < ARRAY_SIZE(mmhub_v1_7_ea_err_status_regs); i++) {
+ reg_value =
+ RREG32(SOC15_REG_ENTRY_OFFSET(mmhub_v1_7_ea_err_status_regs[i]));
+ if (REG_GET_FIELD(reg_value, MMEA0_ERR_STATUS, SDP_RDRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, MMEA0_ERR_STATUS, SDP_WRRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, MMEA0_ERR_STATUS, SDP_RDRSP_DATAPARITY_ERROR)) {
+ dev_warn(adev->dev, "MMHUB EA err detected at instance: %d, status: 0x%x!\n",
+ i, reg_value);
+ }
+ }
+}
+
+const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs = {
+ .ras_late_init = amdgpu_mmhub_ras_late_init,
+ .ras_fini = amdgpu_mmhub_ras_fini,
+ .query_ras_error_count = mmhub_v1_7_query_ras_error_count,
+ .reset_ras_error_count = mmhub_v1_7_reset_ras_error_count,
+ .query_ras_error_status = mmhub_v1_7_query_ras_error_status,
+};
+
+const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs = {
+ .get_fb_location = mmhub_v1_7_get_fb_location,
+ .init = mmhub_v1_7_init,
+ .gart_enable = mmhub_v1_7_gart_enable,
+ .set_fault_enable_default = mmhub_v1_7_set_fault_enable_default,
+ .gart_disable = mmhub_v1_7_gart_disable,
+ .set_clockgating = mmhub_v1_7_set_clockgating,
+ .get_clockgating = mmhub_v1_7_get_clockgating,
+ .setup_vm_pt_regs = mmhub_v1_7_setup_vm_pt_regs,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.h b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.h
new file mode 100644
index 000000000000..a7f9dfc24697
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v1_7.h
@@ -0,0 +1,29 @@
+/*
+ * Copyright 2016 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef __MMHUB_V1_7_H__
+#define __MMHUB_V1_7_H__
+
+extern const struct amdgpu_mmhub_funcs mmhub_v1_7_funcs;
+extern const struct amdgpu_mmhub_ras_funcs mmhub_v1_7_ras_funcs;
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
index f107385faba2..ac76081b91d5 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_0.c
@@ -210,8 +210,7 @@ static void mmhub_v2_0_init_system_aperture_regs(struct amdgpu_device *adev)
}
/* Set default page address. */
- value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
- adev->vm_manager.vram_base_offset;
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
WREG32_SOC15(MMHUB, 0, mmMMMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
(u32)(value >> 12));
WREG32_SOC15(MMHUB, 0, mmMMMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_MSB,
@@ -689,7 +688,6 @@ static void mmhub_v2_0_get_clockgating(struct amdgpu_device *adev, u32 *flags)
}
const struct amdgpu_mmhub_funcs mmhub_v2_0_funcs = {
- .ras_late_init = amdgpu_mmhub_ras_late_init,
.init = mmhub_v2_0_init,
.gart_enable = mmhub_v2_0_gart_enable,
.set_fault_enable_default = mmhub_v2_0_set_fault_enable_default,
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c
index ab9be5ad5a5f..a9899335d0b1 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v2_3.c
@@ -162,8 +162,7 @@ static void mmhub_v2_3_init_system_aperture_regs(struct amdgpu_device *adev)
max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
/* Set default page address. */
- value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
- adev->vm_manager.vram_base_offset;
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
WREG32_SOC15(MMHUB, 0, mmMMMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
(u32)(value >> 12));
WREG32_SOC15(MMHUB, 0, mmMMMC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_MSB,
@@ -616,7 +615,6 @@ static void mmhub_v2_3_get_clockgating(struct amdgpu_device *adev, u32 *flags)
}
const struct amdgpu_mmhub_funcs mmhub_v2_3_funcs = {
- .ras_late_init = amdgpu_mmhub_ras_late_init,
.init = mmhub_v2_3_init,
.gart_enable = mmhub_v2_3_gart_enable,
.set_fault_enable_default = mmhub_v2_3_set_fault_enable_default,
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c b/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c
index 4a31737b6bb0..47c8dd9d1c78 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.c
@@ -136,8 +136,7 @@ static void mmhub_v9_4_init_system_aperture_regs(struct amdgpu_device *adev,
max(adev->gmc.fb_end, adev->gmc.agp_end) >> 18);
/* Set default page address. */
- value = adev->vram_scratch.gpu_addr - adev->gmc.vram_start +
- adev->vm_manager.vram_base_offset;
+ value = amdgpu_gmc_vram_mc2pa(adev, adev->vram_scratch.gpu_addr);
WREG32_SOC15_OFFSET(
MMHUB, 0,
mmVMSHAREDPF0_MC_VM_SYSTEM_APERTURE_DEFAULT_ADDR_LSB,
@@ -1646,16 +1645,27 @@ static void mmhub_v9_4_query_ras_error_status(struct amdgpu_device *adev)
for (i = 0; i < ARRAY_SIZE(mmhub_v9_4_err_status_regs); i++) {
reg_value =
RREG32(SOC15_REG_ENTRY_OFFSET(mmhub_v9_4_err_status_regs[i]));
- if (reg_value)
+ if (REG_GET_FIELD(reg_value, MMEA0_ERR_STATUS, SDP_RDRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, MMEA0_ERR_STATUS, SDP_WRRSP_STATUS) ||
+ REG_GET_FIELD(reg_value, MMEA0_ERR_STATUS, SDP_RDRSP_DATAPARITY_ERROR)) {
+ /* SDP read/write error/parity error in FUE_IS_FATAL mode
+ * can cause system fatal error in arcturas. Harvest the error
+ * status before GPU reset */
dev_warn(adev->dev, "MMHUB EA err detected at instance: %d, status: 0x%x!\n",
i, reg_value);
+ }
}
}
-const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = {
+const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs = {
.ras_late_init = amdgpu_mmhub_ras_late_init,
+ .ras_fini = amdgpu_mmhub_ras_fini,
.query_ras_error_count = mmhub_v9_4_query_ras_error_count,
.reset_ras_error_count = mmhub_v9_4_reset_ras_error_count,
+ .query_ras_error_status = mmhub_v9_4_query_ras_error_status,
+};
+
+const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = {
.get_fb_location = mmhub_v9_4_get_fb_location,
.init = mmhub_v9_4_init,
.gart_enable = mmhub_v9_4_gart_enable,
@@ -1664,5 +1674,4 @@ const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs = {
.set_clockgating = mmhub_v9_4_set_clockgating,
.get_clockgating = mmhub_v9_4_get_clockgating,
.setup_vm_pt_regs = mmhub_v9_4_setup_vm_pt_regs,
- .query_ras_error_status = mmhub_v9_4_query_ras_error_status,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.h b/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.h
index 92404a8f66f3..90436efa92ef 100644
--- a/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.h
+++ b/drivers/gpu/drm/amd/amdgpu/mmhub_v9_4.h
@@ -24,5 +24,6 @@
#define __MMHUB_V9_4_H__
extern const struct amdgpu_mmhub_funcs mmhub_v9_4_funcs;
+extern const struct amdgpu_mmhub_ras_funcs mmhub_v9_4_ras_funcs;
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
index 83ea063388fd..0d2d629e2d6a 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v6_1.c
@@ -31,6 +31,28 @@
#include "vega10_enum.h"
#include <uapi/linux/kfd_ioctl.h>
+#define smnPCIE_LC_CNTL 0x11140280
+#define smnPCIE_LC_CNTL3 0x111402d4
+#define smnPCIE_LC_CNTL6 0x111402ec
+#define smnPCIE_LC_CNTL7 0x111402f0
+#define smnNBIF_MGCG_CTRL_LCLK 0x1013a05c
+#define NBIF_MGCG_CTRL_LCLK__NBIF_MGCG_REG_DIS_LCLK_MASK 0x00001000L
+#define RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER_MASK 0x0000FFFFL
+#define RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER_MASK 0xFFFF0000L
+#define smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL 0x10123530
+#define smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2 0x1014008c
+#define smnBIF_CFG_DEV0_EPF0_PCIE_LTR_CAP 0x10140324
+#define smnPSWUSP0_PCIE_LC_CNTL2 0x111402c4
+#define smnRCC_BIF_STRAP2 0x10123488
+#define smnRCC_BIF_STRAP3 0x1012348c
+#define smnRCC_BIF_STRAP5 0x10123494
+#define BIF_CFG_DEV0_EPF0_DEVICE_CNTL2__LTR_EN_MASK 0x0400L
+#define RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER_MASK 0x0000FFFFL
+#define RCC_BIF_STRAP2__STRAP_LTR_IN_ASPML1_DIS_MASK 0x00004000L
+#define RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER__SHIFT 0x0
+#define RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER__SHIFT 0x10
+#define RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER__SHIFT 0x0
+
static void nbio_v6_1_remap_hdp_registers(struct amdgpu_device *adev)
{
WREG32_SOC15(NBIO, 0, mmREMAP_HDP_MEM_FLUSH_CNTL,
@@ -256,6 +278,111 @@ static void nbio_v6_1_init_registers(struct amdgpu_device *adev)
WREG32_PCIE(smnPCIE_CI_CNTL, data);
}
+static void nbio_v6_1_program_ltr(struct amdgpu_device *adev)
+{
+ uint32_t def, data;
+
+ WREG32_PCIE(smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL, 0x75EB);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP2);
+ data &= ~RCC_BIF_STRAP2__STRAP_LTR_IN_ASPML1_DIS_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP2, data);
+
+ def = data = RREG32_PCIE(smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL);
+ data &= ~EP_PCIE_TX_LTR_CNTL__LTR_PRIV_MSG_DIS_IN_PM_NON_D0_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL, data);
+
+ def = data = RREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2);
+ data |= BIF_CFG_DEV0_EPF0_DEVICE_CNTL2__LTR_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2, data);
+}
+
+static void nbio_v6_1_program_aspm(struct amdgpu_device *adev)
+{
+ uint32_t def, data;
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL);
+ data &= ~PCIE_LC_CNTL__LC_L1_INACTIVITY_MASK;
+ data &= ~PCIE_LC_CNTL__LC_L0S_INACTIVITY_MASK;
+ data |= PCIE_LC_CNTL__LC_PMI_TO_L1_DIS_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL7);
+ data |= PCIE_LC_CNTL7__LC_NBIF_ASPM_INPUT_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL7, data);
+
+ def = data = RREG32_PCIE(smnNBIF_MGCG_CTRL_LCLK);
+ data |= NBIF_MGCG_CTRL_LCLK__NBIF_MGCG_REG_DIS_LCLK_MASK;
+ if (def != data)
+ WREG32_PCIE(smnNBIF_MGCG_CTRL_LCLK, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL3);
+ data |= PCIE_LC_CNTL3__LC_DSC_DONT_ENTER_L23_AFTER_PME_ACK_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL3, data);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP3);
+ data &= ~RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER_MASK;
+ data &= ~RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP3, data);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP5);
+ data &= ~RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP5, data);
+
+ def = data = RREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2);
+ data &= ~BIF_CFG_DEV0_EPF0_DEVICE_CNTL2__LTR_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2, data);
+
+ WREG32_PCIE(smnBIF_CFG_DEV0_EPF0_PCIE_LTR_CAP, 0x10011001);
+
+ def = data = RREG32_PCIE(smnPSWUSP0_PCIE_LC_CNTL2);
+ data |= PSWUSP0_PCIE_LC_CNTL2__LC_ALLOW_PDWN_IN_L1_MASK |
+ PSWUSP0_PCIE_LC_CNTL2__LC_ALLOW_PDWN_IN_L23_MASK;
+ data &= ~PSWUSP0_PCIE_LC_CNTL2__LC_RCV_L0_TO_RCV_L0S_DIS_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPSWUSP0_PCIE_LC_CNTL2, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL6);
+ data |= PCIE_LC_CNTL6__LC_L1_POWERDOWN_MASK |
+ PCIE_LC_CNTL6__LC_RX_L0S_STANDBY_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL6, data);
+
+ nbio_v6_1_program_ltr(adev);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP3);
+ data |= 0x5DE0 << RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER__SHIFT;
+ data |= 0x0010 << RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER__SHIFT;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP3, data);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP5);
+ data |= 0x0010 << RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER__SHIFT;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP5, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL);
+ data &= ~PCIE_LC_CNTL__LC_L0S_INACTIVITY_MASK;
+ data |= 0x9 << PCIE_LC_CNTL__LC_L1_INACTIVITY__SHIFT;
+ data |= 0x1 << PCIE_LC_CNTL__LC_PMI_TO_L1_DIS__SHIFT;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL3);
+ data &= ~PCIE_LC_CNTL3__LC_DSC_DONT_ENTER_L23_AFTER_PME_ACK_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL3, data);
+}
+
const struct amdgpu_nbio_funcs nbio_v6_1_funcs = {
.get_hdp_flush_req_offset = nbio_v6_1_get_hdp_flush_req_offset,
.get_hdp_flush_done_offset = nbio_v6_1_get_hdp_flush_done_offset,
@@ -274,4 +401,5 @@ const struct amdgpu_nbio_funcs nbio_v6_1_funcs = {
.ih_control = nbio_v6_1_ih_control,
.init_registers = nbio_v6_1_init_registers,
.remap_hdp_registers = nbio_v6_1_remap_hdp_registers,
+ .program_aspm = nbio_v6_1_program_aspm,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
index 4bc1d1434065..cef929746739 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.c
@@ -31,7 +31,26 @@
#include "ivsrcid/nbio/irqsrcs_nbif_7_4.h"
#include <uapi/linux/kfd_ioctl.h>
+#define smnPCIE_LC_CNTL 0x11140280
+#define smnPCIE_LC_CNTL3 0x111402d4
+#define smnPCIE_LC_CNTL6 0x111402ec
+#define smnPCIE_LC_CNTL7 0x111402f0
#define smnNBIF_MGCG_CTRL_LCLK 0x1013a21c
+#define smnRCC_BIF_STRAP3 0x1012348c
+#define RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER_MASK 0x0000FFFFL
+#define RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER_MASK 0xFFFF0000L
+#define smnRCC_BIF_STRAP5 0x10123494
+#define RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER_MASK 0x0000FFFFL
+#define smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2 0x1014008c
+#define BIF_CFG_DEV0_EPF0_DEVICE_CNTL2__LTR_EN_MASK 0x0400L
+#define smnBIF_CFG_DEV0_EPF0_PCIE_LTR_CAP 0x10140324
+#define smnPSWUSP0_PCIE_LC_CNTL2 0x111402c4
+#define smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL 0x10123538
+#define smnRCC_BIF_STRAP2 0x10123488
+#define RCC_BIF_STRAP2__STRAP_LTR_IN_ASPML1_DIS_MASK 0x00004000L
+#define RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER__SHIFT 0x0
+#define RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER__SHIFT 0x10
+#define RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER__SHIFT 0x0
/*
* These are nbio v7_4_1 registers mask. Temporarily define these here since
@@ -52,6 +71,20 @@
#define BIF_MMSCH1_DOORBELL_RANGE__OFFSET_MASK 0x00000FFCL
#define BIF_MMSCH1_DOORBELL_RANGE__SIZE_MASK 0x001F0000L
+#define BIF_MMSCH1_DOORBELL_RANGE__OFFSET_MASK 0x00000FFCL
+#define BIF_MMSCH1_DOORBELL_RANGE__SIZE_MASK 0x001F0000L
+
+#define mmBIF_MMSCH1_DOORBELL_RANGE_ALDE 0x01d8
+#define mmBIF_MMSCH1_DOORBELL_RANGE_ALDE_BASE_IDX 2
+//BIF_MMSCH1_DOORBELL_ALDE_RANGE
+#define BIF_MMSCH1_DOORBELL_RANGE_ALDE__OFFSET__SHIFT 0x2
+#define BIF_MMSCH1_DOORBELL_RANGE_ALDE__SIZE__SHIFT 0x10
+#define BIF_MMSCH1_DOORBELL_RANGE_ALDE__OFFSET_MASK 0x00000FFCL
+#define BIF_MMSCH1_DOORBELL_RANGE_ALDE__SIZE_MASK 0x001F0000L
+
+#define mmRCC_DEV0_EPF0_STRAP0_ALDE 0x0015
+#define mmRCC_DEV0_EPF0_STRAP0_ALDE_BASE_IDX 2
+
static void nbio_v7_4_query_ras_error_count(struct amdgpu_device *adev,
void *ras_error_status);
@@ -65,7 +98,12 @@ static void nbio_v7_4_remap_hdp_registers(struct amdgpu_device *adev)
static u32 nbio_v7_4_get_rev_id(struct amdgpu_device *adev)
{
- u32 tmp = RREG32_SOC15(NBIO, 0, mmRCC_DEV0_EPF0_STRAP0);
+ u32 tmp;
+
+ if (adev->asic_type == CHIP_ALDEBARAN)
+ tmp = RREG32_SOC15(NBIO, 0, mmRCC_DEV0_EPF0_STRAP0_ALDE);
+ else
+ tmp = RREG32_SOC15(NBIO, 0, mmRCC_DEV0_EPF0_STRAP0);
tmp &= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0_MASK;
tmp >>= RCC_DEV0_EPF0_STRAP0__STRAP_ATI_REV_ID_DEV0_F0__SHIFT;
@@ -92,10 +130,10 @@ static void nbio_v7_4_sdma_doorbell_range(struct amdgpu_device *adev, int instan
{
u32 reg, doorbell_range;
- if (instance < 2)
+ if (instance < 2) {
reg = instance +
SOC15_REG_OFFSET(NBIO, 0, mmBIF_SDMA0_DOORBELL_RANGE);
- else
+ } else {
/*
* These registers address of SDMA2~7 is not consecutive
* from SDMA0~1. Need plus 4 dwords offset.
@@ -103,9 +141,19 @@ static void nbio_v7_4_sdma_doorbell_range(struct amdgpu_device *adev, int instan
* BIF_SDMA0_DOORBELL_RANGE: 0x3bc0
* BIF_SDMA1_DOORBELL_RANGE: 0x3bc4
* BIF_SDMA2_DOORBELL_RANGE: 0x3bd8
++ * BIF_SDMA4_DOORBELL_RANGE:
++ * ARCTURUS: 0x3be0
++ * ALDEBARAN: 0x3be4
*/
- reg = instance + 0x4 +
- SOC15_REG_OFFSET(NBIO, 0, mmBIF_SDMA0_DOORBELL_RANGE);
+ if (adev->asic_type == CHIP_ALDEBARAN && instance == 4)
+ reg = instance + 0x4 + 0x1 +
+ SOC15_REG_OFFSET(NBIO, 0,
+ mmBIF_SDMA0_DOORBELL_RANGE);
+ else
+ reg = instance + 0x4 +
+ SOC15_REG_OFFSET(NBIO, 0,
+ mmBIF_SDMA0_DOORBELL_RANGE);
+ }
doorbell_range = RREG32(reg);
@@ -124,9 +172,12 @@ static void nbio_v7_4_vcn_doorbell_range(struct amdgpu_device *adev, bool use_do
u32 reg;
u32 doorbell_range;
- if (instance)
- reg = SOC15_REG_OFFSET(NBIO, 0, mmBIF_MMSCH1_DOORBELL_RANGE);
- else
+ if (instance) {
+ if (adev->asic_type == CHIP_ALDEBARAN)
+ reg = SOC15_REG_OFFSET(NBIO, 0, mmBIF_MMSCH1_DOORBELL_RANGE_ALDE);
+ else
+ reg = SOC15_REG_OFFSET(NBIO, 0, mmBIF_MMSCH1_DOORBELL_RANGE);
+ } else
reg = SOC15_REG_OFFSET(NBIO, 0, mmBIF_MMSCH0_DOORBELL_RANGE);
doorbell_range = RREG32(reg);
@@ -525,6 +576,121 @@ static void nbio_v7_4_enable_doorbell_interrupt(struct amdgpu_device *adev,
DOORBELL_INTERRUPT_DISABLE, enable ? 0 : 1);
}
+const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs = {
+ .handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring,
+ .handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring,
+ .init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt,
+ .init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt,
+ .query_ras_error_count = nbio_v7_4_query_ras_error_count,
+ .ras_late_init = amdgpu_nbio_ras_late_init,
+ .ras_fini = amdgpu_nbio_ras_fini,
+};
+
+static void nbio_v7_4_program_ltr(struct amdgpu_device *adev)
+{
+ uint32_t def, data;
+
+ WREG32_PCIE(smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL, 0x75EB);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP2);
+ data &= ~RCC_BIF_STRAP2__STRAP_LTR_IN_ASPML1_DIS_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP2, data);
+
+ def = data = RREG32_PCIE(smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL);
+ data &= ~EP_PCIE_TX_LTR_CNTL__LTR_PRIV_MSG_DIS_IN_PM_NON_D0_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_EP_DEV0_0_EP_PCIE_TX_LTR_CNTL, data);
+
+ def = data = RREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2);
+ data |= BIF_CFG_DEV0_EPF0_DEVICE_CNTL2__LTR_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2, data);
+}
+
+static void nbio_v7_4_program_aspm(struct amdgpu_device *adev)
+{
+ uint32_t def, data;
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL);
+ data &= ~PCIE_LC_CNTL__LC_L1_INACTIVITY_MASK;
+ data &= ~PCIE_LC_CNTL__LC_L0S_INACTIVITY_MASK;
+ data |= PCIE_LC_CNTL__LC_PMI_TO_L1_DIS_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL7);
+ data |= PCIE_LC_CNTL7__LC_NBIF_ASPM_INPUT_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL7, data);
+
+ def = data = RREG32_PCIE(smnNBIF_MGCG_CTRL_LCLK);
+ data |= NBIF_MGCG_CTRL_LCLK__NBIF_MGCG_REG_DIS_LCLK_MASK;
+ if (def != data)
+ WREG32_PCIE(smnNBIF_MGCG_CTRL_LCLK, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL3);
+ data |= PCIE_LC_CNTL3__LC_DSC_DONT_ENTER_L23_AFTER_PME_ACK_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL3, data);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP3);
+ data &= ~RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER_MASK;
+ data &= ~RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP3, data);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP5);
+ data &= ~RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER_MASK;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP5, data);
+
+ def = data = RREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2);
+ data &= ~BIF_CFG_DEV0_EPF0_DEVICE_CNTL2__LTR_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnBIF_CFG_DEV0_EPF0_DEVICE_CNTL2, data);
+
+ WREG32_PCIE(smnBIF_CFG_DEV0_EPF0_PCIE_LTR_CAP, 0x10011001);
+
+ def = data = RREG32_PCIE(smnPSWUSP0_PCIE_LC_CNTL2);
+ data |= PSWUSP0_PCIE_LC_CNTL2__LC_ALLOW_PDWN_IN_L1_MASK |
+ PSWUSP0_PCIE_LC_CNTL2__LC_ALLOW_PDWN_IN_L23_MASK;
+ data &= ~PSWUSP0_PCIE_LC_CNTL2__LC_RCV_L0_TO_RCV_L0S_DIS_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPSWUSP0_PCIE_LC_CNTL2, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL6);
+ data |= PCIE_LC_CNTL6__LC_L1_POWERDOWN_MASK |
+ PCIE_LC_CNTL6__LC_RX_L0S_STANDBY_EN_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL6, data);
+
+ nbio_v7_4_program_ltr(adev);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP3);
+ data |= 0x5DE0 << RCC_BIF_STRAP3__STRAP_VLINK_ASPM_IDLE_TIMER__SHIFT;
+ data |= 0x0010 << RCC_BIF_STRAP3__STRAP_VLINK_PM_L1_ENTRY_TIMER__SHIFT;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP3, data);
+
+ def = data = RREG32_PCIE(smnRCC_BIF_STRAP5);
+ data |= 0x0010 << RCC_BIF_STRAP5__STRAP_VLINK_LDN_ENTRY_TIMER__SHIFT;
+ if (def != data)
+ WREG32_PCIE(smnRCC_BIF_STRAP5, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL);
+ data &= ~PCIE_LC_CNTL__LC_L0S_INACTIVITY_MASK;
+ data |= 0x9 << PCIE_LC_CNTL__LC_L1_INACTIVITY__SHIFT;
+ data |= 0x1 << PCIE_LC_CNTL__LC_PMI_TO_L1_DIS__SHIFT;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL, data);
+
+ def = data = RREG32_PCIE(smnPCIE_LC_CNTL3);
+ data &= ~PCIE_LC_CNTL3__LC_DSC_DONT_ENTER_L23_AFTER_PME_ACK_MASK;
+ if (def != data)
+ WREG32_PCIE(smnPCIE_LC_CNTL3, data);
+}
+
const struct amdgpu_nbio_funcs nbio_v7_4_funcs = {
.get_hdp_flush_req_offset = nbio_v7_4_get_hdp_flush_req_offset,
.get_hdp_flush_done_offset = nbio_v7_4_get_hdp_flush_done_offset,
@@ -545,10 +711,5 @@ const struct amdgpu_nbio_funcs nbio_v7_4_funcs = {
.ih_control = nbio_v7_4_ih_control,
.init_registers = nbio_v7_4_init_registers,
.remap_hdp_registers = nbio_v7_4_remap_hdp_registers,
- .handle_ras_controller_intr_no_bifring = nbio_v7_4_handle_ras_controller_intr_no_bifring,
- .handle_ras_err_event_athub_intr_no_bifring = nbio_v7_4_handle_ras_err_event_athub_intr_no_bifring,
- .init_ras_controller_interrupt = nbio_v7_4_init_ras_controller_interrupt,
- .init_ras_err_event_athub_interrupt = nbio_v7_4_init_ras_err_event_athub_interrupt,
- .query_ras_error_count = nbio_v7_4_query_ras_error_count,
- .ras_late_init = amdgpu_nbio_ras_late_init,
+ .program_aspm = nbio_v7_4_program_aspm,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.h b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.h
index b1ac82872752..b8216581ec8d 100644
--- a/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.h
+++ b/drivers/gpu/drm/amd/amdgpu/nbio_v7_4.h
@@ -28,5 +28,6 @@
extern const struct nbio_hdp_flush_reg nbio_v7_4_hdp_flush_reg;
extern const struct amdgpu_nbio_funcs nbio_v7_4_funcs;
+extern const struct amdgpu_nbio_ras_funcs nbio_v7_4_ras_funcs;
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/nv.c b/drivers/gpu/drm/amd/amdgpu/nv.c
index c625c5d8ed89..d290ca0b06da 100644
--- a/drivers/gpu/drm/amd/amdgpu/nv.c
+++ b/drivers/gpu/drm/amd/amdgpu/nv.c
@@ -25,6 +25,8 @@
#include <linux/module.h>
#include <linux/pci.h>
+#include <drm/amdgpu_drm.h>
+
#include "amdgpu.h"
#include "amdgpu_atombios.h"
#include "amdgpu_ih.h"
@@ -32,7 +34,6 @@
#include "amdgpu_vce.h"
#include "amdgpu_ucode.h"
#include "amdgpu_psp.h"
-#include "amdgpu_smu.h"
#include "atom.h"
#include "amd_pcie.h"
@@ -65,6 +66,184 @@
static const struct amd_ip_funcs nv_common_ip_funcs;
+/* Navi */
+static const struct amdgpu_video_codec_info nv_video_codecs_encode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 2304,
+ .max_pixels_per_frame = 4096 * 2304,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 4096,
+ .max_height = 2304,
+ .max_pixels_per_frame = 4096 * 2304,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs nv_video_codecs_encode =
+{
+ .codec_count = ARRAY_SIZE(nv_video_codecs_encode_array),
+ .codec_array = nv_video_codecs_encode_array,
+};
+
+/* Navi1x */
+static const struct amdgpu_video_codec_info nv_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 52,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 4,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 8192,
+ .max_height = 4352,
+ .max_pixels_per_frame = 8192 * 4352,
+ .max_level = 186,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9,
+ .max_width = 8192,
+ .max_height = 4352,
+ .max_pixels_per_frame = 8192 * 4352,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs nv_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(nv_video_codecs_decode_array),
+ .codec_array = nv_video_codecs_decode_array,
+};
+
+/* Sienna Cichlid */
+static const struct amdgpu_video_codec_info sc_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 52,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 4,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 8192,
+ .max_height = 4352,
+ .max_pixels_per_frame = 8192 * 4352,
+ .max_level = 186,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9,
+ .max_width = 8192,
+ .max_height = 4352,
+ .max_pixels_per_frame = 8192 * 4352,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_AV1,
+ .max_width = 8192,
+ .max_height = 4352,
+ .max_pixels_per_frame = 8192 * 4352,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs sc_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(sc_video_codecs_decode_array),
+ .codec_array = sc_video_codecs_decode_array,
+};
+
+static int nv_query_video_codecs(struct amdgpu_device *adev, bool encode,
+ const struct amdgpu_video_codecs **codecs)
+{
+ switch (adev->asic_type) {
+ case CHIP_SIENNA_CICHLID:
+ case CHIP_NAVY_FLOUNDER:
+ case CHIP_DIMGREY_CAVEFISH:
+ case CHIP_VANGOGH:
+ if (encode)
+ *codecs = &nv_video_codecs_encode;
+ else
+ *codecs = &sc_video_codecs_decode;
+ return 0;
+ case CHIP_NAVI10:
+ case CHIP_NAVI14:
+ case CHIP_NAVI12:
+ if (encode)
+ *codecs = &nv_video_codecs_encode;
+ else
+ *codecs = &nv_video_codecs_decode;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
/*
* Indirect registers accessor
*/
@@ -304,44 +483,6 @@ static int nv_read_register(struct amdgpu_device *adev, u32 se_num,
return -EINVAL;
}
-static int nv_asic_mode1_reset(struct amdgpu_device *adev)
-{
- u32 i;
- int ret = 0;
-
- amdgpu_atombios_scratch_regs_engine_hung(adev, true);
-
- /* disable BM */
- pci_clear_master(adev->pdev);
-
- amdgpu_device_cache_pci_state(adev->pdev);
-
- if (amdgpu_dpm_is_mode1_reset_supported(adev)) {
- dev_info(adev->dev, "GPU smu mode1 reset\n");
- ret = amdgpu_dpm_mode1_reset(adev);
- } else {
- dev_info(adev->dev, "GPU psp mode1 reset\n");
- ret = psp_gpu_reset(adev);
- }
-
- if (ret)
- dev_err(adev->dev, "GPU mode1 reset failed\n");
- amdgpu_device_load_pci_state(adev->pdev);
-
- /* wait for asic to come out of reset */
- for (i = 0; i < adev->usec_timeout; i++) {
- u32 memsize = adev->nbio.funcs->get_memsize(adev);
-
- if (memsize != 0xffffffff)
- break;
- udelay(1);
- }
-
- amdgpu_atombios_scratch_regs_engine_hung(adev, false);
-
- return ret;
-}
-
static int nv_asic_mode2_reset(struct amdgpu_device *adev)
{
u32 i;
@@ -374,21 +515,9 @@ static int nv_asic_mode2_reset(struct amdgpu_device *adev)
return ret;
}
-static bool nv_asic_supports_baco(struct amdgpu_device *adev)
-{
- struct smu_context *smu = &adev->smu;
-
- if (smu_baco_is_support(smu))
- return true;
- else
- return false;
-}
-
static enum amd_reset_method
nv_asic_reset_method(struct amdgpu_device *adev)
{
- struct smu_context *smu = &adev->smu;
-
if (amdgpu_reset_method == AMD_RESET_METHOD_MODE1 ||
amdgpu_reset_method == AMD_RESET_METHOD_MODE2 ||
amdgpu_reset_method == AMD_RESET_METHOD_BACO ||
@@ -407,7 +536,7 @@ nv_asic_reset_method(struct amdgpu_device *adev)
case CHIP_DIMGREY_CAVEFISH:
return AMD_RESET_METHOD_MODE1;
default:
- if (smu_baco_is_support(smu))
+ if (amdgpu_dpm_is_baco_supported(adev))
return AMD_RESET_METHOD_BACO;
else
return AMD_RESET_METHOD_MODE1;
@@ -417,11 +546,6 @@ nv_asic_reset_method(struct amdgpu_device *adev)
static int nv_asic_reset(struct amdgpu_device *adev)
{
int ret = 0;
- struct smu_context *smu = &adev->smu;
-
- /* skip reset on vangogh for now */
- if (adev->asic_type == CHIP_VANGOGH)
- return 0;
switch (nv_asic_reset_method(adev)) {
case AMD_RESET_METHOD_PCI:
@@ -430,13 +554,7 @@ static int nv_asic_reset(struct amdgpu_device *adev)
break;
case AMD_RESET_METHOD_BACO:
dev_info(adev->dev, "BACO reset\n");
-
- ret = smu_baco_enter(smu);
- if (ret)
- return ret;
- ret = smu_baco_exit(smu);
- if (ret)
- return ret;
+ ret = amdgpu_dpm_baco_reset(adev);
break;
case AMD_RESET_METHOD_MODE2:
dev_info(adev->dev, "MODE2 reset\n");
@@ -444,7 +562,7 @@ static int nv_asic_reset(struct amdgpu_device *adev)
break;
default:
dev_info(adev->dev, "MODE1 reset\n");
- ret = nv_asic_mode1_reset(adev);
+ ret = amdgpu_device_mode1_reset(adev);
break;
}
@@ -483,8 +601,7 @@ static void nv_program_aspm(struct amdgpu_device *adev)
if (amdgpu_aspm != 1)
return;
- if ((adev->asic_type >= CHIP_SIENNA_CICHLID) &&
- !(adev->flags & AMD_IS_APU) &&
+ if (!(adev->flags & AMD_IS_APU) &&
(adev->nbio.funcs->program_aspm))
adev->nbio.funcs->program_aspm(adev);
@@ -506,6 +623,16 @@ static const struct amdgpu_ip_block_version nv_common_ip_block =
.funcs = &nv_common_ip_funcs,
};
+static bool nv_is_headless_sku(struct pci_dev *pdev)
+{
+ if ((pdev->device == 0x731E &&
+ (pdev->revision == 0xC6 || pdev->revision == 0xC7)) ||
+ (pdev->device == 0x7340 && pdev->revision == 0xC9) ||
+ (pdev->device == 0x7360 && pdev->revision == 0xC7))
+ return true;
+ return false;
+}
+
static int nv_reg_base_init(struct amdgpu_device *adev)
{
int r;
@@ -518,6 +645,12 @@ static int nv_reg_base_init(struct amdgpu_device *adev)
goto legacy_init;
}
+ amdgpu_discovery_harvest_ip(adev);
+ if (nv_is_headless_sku(adev->pdev)) {
+ adev->harvest_ip_mask |= AMD_HARVEST_IP_VCN_MASK;
+ adev->harvest_ip_mask |= AMD_HARVEST_IP_JPEG_MASK;
+ }
+
return 0;
}
@@ -554,16 +687,6 @@ void nv_set_virt_ops(struct amdgpu_device *adev)
adev->virt.ops = &xgpu_nv_virt_ops;
}
-static bool nv_is_headless_sku(struct pci_dev *pdev)
-{
- if ((pdev->device == 0x731E &&
- (pdev->revision == 0xC6 || pdev->revision == 0xC7)) ||
- (pdev->device == 0x7340 && pdev->revision == 0xC9) ||
- (pdev->device == 0x7360 && pdev->revision == 0xC7))
- return true;
- return false;
-}
-
int nv_set_ip_blocks(struct amdgpu_device *adev)
{
int r;
@@ -611,8 +734,7 @@ int nv_set_ip_blocks(struct amdgpu_device *adev)
if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT &&
!amdgpu_sriov_vf(adev))
amdgpu_device_ip_block_add(adev, &smu_v11_0_ip_block);
- if (!nv_is_headless_sku(adev->pdev))
- amdgpu_device_ip_block_add(adev, &vcn_v2_0_ip_block);
+ amdgpu_device_ip_block_add(adev, &vcn_v2_0_ip_block);
amdgpu_device_ip_block_add(adev, &jpeg_v2_0_ip_block);
if (adev->enable_mes)
amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block);
@@ -635,8 +757,7 @@ int nv_set_ip_blocks(struct amdgpu_device *adev)
if (adev->firmware.load_type == AMDGPU_FW_LOAD_DIRECT &&
!amdgpu_sriov_vf(adev))
amdgpu_device_ip_block_add(adev, &smu_v11_0_ip_block);
- if (!nv_is_headless_sku(adev->pdev))
- amdgpu_device_ip_block_add(adev, &vcn_v2_0_ip_block);
+ amdgpu_device_ip_block_add(adev, &vcn_v2_0_ip_block);
if (!amdgpu_sriov_vf(adev))
amdgpu_device_ip_block_add(adev, &jpeg_v2_0_ip_block);
break;
@@ -660,7 +781,6 @@ int nv_set_ip_blocks(struct amdgpu_device *adev)
amdgpu_device_ip_block_add(adev, &vcn_v3_0_ip_block);
if (!amdgpu_sriov_vf(adev))
amdgpu_device_ip_block_add(adev, &jpeg_v3_0_ip_block);
-
if (adev->enable_mes)
amdgpu_device_ip_block_add(adev, &mes_v10_1_ip_block);
break;
@@ -816,12 +936,7 @@ static int nv_update_umd_stable_pstate(struct amdgpu_device *adev,
if (adev->gfx.funcs->update_perfmon_mgcg)
adev->gfx.funcs->update_perfmon_mgcg(adev, !enter);
- /*
- * The ASPM function is not fully enabled and verified on
- * Navi yet. Temporarily skip this until ASPM enabled.
- */
- if ((adev->asic_type >= CHIP_SIENNA_CICHLID) &&
- !(adev->flags & AMD_IS_APU) &&
+ if (!(adev->flags & AMD_IS_APU) &&
(adev->nbio.funcs->enable_aspm))
adev->nbio.funcs->enable_aspm(adev, !enter);
@@ -844,9 +959,10 @@ static const struct amdgpu_asic_funcs nv_asic_funcs =
.need_full_reset = &nv_need_full_reset,
.need_reset_on_init = &nv_need_reset_on_init,
.get_pcie_replay_count = &nv_get_pcie_replay_count,
- .supports_baco = &nv_asic_supports_baco,
+ .supports_baco = &amdgpu_dpm_is_baco_supported,
.pre_asic_init = &nv_pre_asic_init,
.update_umd_stable_pstate = &nv_update_umd_stable_pstate,
+ .query_video_codecs = &nv_query_video_codecs,
};
static int nv_common_early_init(void *handle)
@@ -1036,6 +1152,11 @@ static int nv_common_early_init(void *handle)
return -EINVAL;
}
+ if (adev->harvest_ip_mask & AMD_HARVEST_IP_VCN_MASK)
+ adev->pg_flags &= ~(AMD_PG_SUPPORT_VCN |
+ AMD_PG_SUPPORT_VCN_DPG |
+ AMD_PG_SUPPORT_JPEG);
+
if (amdgpu_sriov_vf(adev)) {
amdgpu_virt_init_setting(adev);
xgpu_nv_mailbox_set_irq_funcs(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h b/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h
index 3ba7bdfde65d..96064c343163 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h
+++ b/drivers/gpu/drm/amd/amdgpu/psp_gfx_if.h
@@ -102,6 +102,21 @@ enum psp_gfx_cmd_id
/* IDs upto 0x1F are reserved for older programs (Raven, Vega 10/12/20) */
GFX_CMD_ID_LOAD_TOC = 0x00000020, /* Load TOC and obtain TMR size */
GFX_CMD_ID_AUTOLOAD_RLC = 0x00000021, /* Indicates all graphics fw loaded, start RLC autoload */
+ GFX_CMD_ID_BOOT_CFG = 0x00000022, /* Boot Config */
+};
+
+/* PSP boot config sub-commands */
+enum psp_gfx_boot_config_cmd
+{
+ BOOTCFG_CMD_SET = 1, /* Set boot configuration settings */
+ BOOTCFG_CMD_GET = 2, /* Get boot configuration settings */
+ BOOTCFG_CMD_INVALIDATE = 3 /* Reset current boot configuration settings to VBIOS defaults */
+};
+
+/* PSP boot config bitmask values */
+enum psp_gfx_boot_config
+{
+ BOOT_CONFIG_GECC = 0x1,
};
/* Command to load Trusted Application binary into PSP OS. */
@@ -170,10 +185,19 @@ struct psp_gfx_cmd_setup_tmr
uint32_t buf_phy_addr_lo; /* bits [31:0] of GPU Virtual address of TMR buffer (must be 4 KB aligned) */
uint32_t buf_phy_addr_hi; /* bits [63:32] of GPU Virtual address of TMR buffer */
uint32_t buf_size; /* buffer size in bytes (must be multiple of 4 KB) */
+ union {
+ struct {
+ uint32_t sriov_enabled:1; /* whether the device runs under SR-IOV*/
+ uint32_t virt_phy_addr:1; /* driver passes both virtual and physical address to PSP*/
+ uint32_t reserved:30;
+ } bitfield;
+ uint32_t tmr_flags;
+ };
+ uint32_t system_phy_addr_lo; /* bits [31:0] of system physical address of TMR buffer (must be 4 KB aligned) */
+ uint32_t system_phy_addr_hi; /* bits [63:32] of system physical address of TMR buffer */
};
-
/* FW types for GFX_CMD_ID_LOAD_IP_FW command. Limit 31. */
enum psp_gfx_fw_type {
GFX_FW_TYPE_NONE = 0, /* */
@@ -235,6 +259,7 @@ enum psp_gfx_fw_type {
GFX_FW_TYPE_SDMA6 = 56, /* SDMA6 MI */
GFX_FW_TYPE_SDMA7 = 57, /* SDMA7 MI */
GFX_FW_TYPE_VCN1 = 58, /* VCN1 MI */
+ GFX_FW_TYPE_REG_LIST = 67, /* REG_LIST MI */
GFX_FW_TYPE_MAX
};
@@ -272,6 +297,15 @@ struct psp_gfx_cmd_load_toc
uint32_t toc_size; /* FW buffer size in bytes */
};
+/* Dynamic boot configuration */
+struct psp_gfx_cmd_boot_cfg
+{
+ uint32_t timestamp; /* calendar time as number of seconds */
+ enum psp_gfx_boot_config_cmd sub_cmd; /* sub-command indicating how to process command data */
+ uint32_t boot_config; /* dynamic boot configuration bitmask */
+ uint32_t boot_config_valid; /* dynamic boot configuration valid bits bitmask */
+};
+
/* All GFX ring buffer commands. */
union psp_gfx_commands
{
@@ -284,6 +318,7 @@ union psp_gfx_commands
struct psp_gfx_cmd_reg_prog cmd_setup_reg_prog;
struct psp_gfx_cmd_setup_tmr cmd_setup_vmr;
struct psp_gfx_cmd_load_toc cmd_load_toc;
+ struct psp_gfx_cmd_boot_cfg boot_cfg;
};
struct psp_gfx_uresp_reserved
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
index c325d6f53a71..589410c32d09 100644
--- a/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v11_0.c
@@ -598,7 +598,7 @@ static int psp_v11_0_memory_training_send_msg(struct psp_context *psp, int msg)
}
/*
- * save and restore proces
+ * save and restore process
*/
static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops)
{
@@ -661,9 +661,9 @@ static int psp_v11_0_memory_training(struct psp_context *psp, uint32_t ops)
if (ops & PSP_MEM_TRAIN_SEND_LONG_MSG) {
/*
- * Long traing will encroach certain mount of bottom VRAM,
- * saving the content of this bottom VRAM to system memory
- * before training, and restoring it after training to avoid
+ * Long training will encroach a certain amount on the bottom of VRAM;
+ * save the content from the bottom of VRAM to system memory
+ * before training, and restore it after training to avoid
* VRAM corruption.
*/
sz = GDDR6_MEM_TRAINING_ENCROACHED_SIZE;
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v13_0.c b/drivers/gpu/drm/amd/amdgpu/psp_v13_0.c
new file mode 100644
index 000000000000..fcdce46445d6
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v13_0.c
@@ -0,0 +1,378 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "amdgpu.h"
+#include "amdgpu_psp.h"
+#include "amdgpu_ucode.h"
+#include "soc15_common.h"
+#include "psp_v13_0.h"
+
+#include "mp/mp_13_0_2_offset.h"
+#include "mp/mp_13_0_2_sh_mask.h"
+
+MODULE_FIRMWARE("amdgpu/aldebaran_sos.bin");
+MODULE_FIRMWARE("amdgpu/aldebaran_ta.bin");
+
+static int psp_v13_0_init_microcode(struct psp_context *psp)
+{
+ struct amdgpu_device *adev = psp->adev;
+ const char *chip_name;
+ int err = 0;
+
+ switch (adev->asic_type) {
+ case CHIP_ALDEBARAN:
+ chip_name = "aldebaran";
+ break;
+ default:
+ BUG();
+ }
+
+ err = psp_init_sos_microcode(psp, chip_name);
+ if (err)
+ return err;
+
+ err = psp_init_ta_microcode(&adev->psp, chip_name);
+
+ return err;
+}
+
+static bool psp_v13_0_is_sos_alive(struct psp_context *psp)
+{
+ struct amdgpu_device *adev = psp->adev;
+ uint32_t sol_reg;
+
+ sol_reg = RREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_81);
+
+ return sol_reg != 0x0;
+}
+
+static int psp_v13_0_wait_for_bootloader(struct psp_context *psp)
+{
+ struct amdgpu_device *adev = psp->adev;
+
+ int ret;
+ int retry_loop;
+
+ for (retry_loop = 0; retry_loop < 10; retry_loop++) {
+ /* Wait for bootloader to signify that is
+ ready having bit 31 of C2PMSG_35 set to 1 */
+ ret = psp_wait_for(psp,
+ SOC15_REG_OFFSET(MP0, 0, regMP0_SMN_C2PMSG_35),
+ 0x80000000,
+ 0x80000000,
+ false);
+
+ if (ret == 0)
+ return 0;
+ }
+
+ return ret;
+}
+
+static int psp_v13_0_bootloader_load_kdb(struct psp_context *psp)
+{
+ int ret;
+ uint32_t psp_gfxdrv_command_reg = 0;
+ struct amdgpu_device *adev = psp->adev;
+
+ /* Check tOS sign of life register to confirm sys driver and sOS
+ * are already been loaded.
+ */
+ if (psp_v13_0_is_sos_alive(psp))
+ return 0;
+
+ ret = psp_v13_0_wait_for_bootloader(psp);
+ if (ret)
+ return ret;
+
+ memset(psp->fw_pri_buf, 0, PSP_1_MEG);
+
+ /* Copy PSP KDB binary to memory */
+ memcpy(psp->fw_pri_buf, psp->kdb_start_addr, psp->kdb_bin_size);
+
+ /* Provide the PSP KDB to bootloader */
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_36,
+ (uint32_t)(psp->fw_pri_mc_addr >> 20));
+ psp_gfxdrv_command_reg = PSP_BL__LOAD_KEY_DATABASE;
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_35,
+ psp_gfxdrv_command_reg);
+
+ ret = psp_v13_0_wait_for_bootloader(psp);
+
+ return ret;
+}
+
+static int psp_v13_0_bootloader_load_sysdrv(struct psp_context *psp)
+{
+ int ret;
+ uint32_t psp_gfxdrv_command_reg = 0;
+ struct amdgpu_device *adev = psp->adev;
+
+ /* Check sOS sign of life register to confirm sys driver and sOS
+ * are already been loaded.
+ */
+ if (psp_v13_0_is_sos_alive(psp))
+ return 0;
+
+ ret = psp_v13_0_wait_for_bootloader(psp);
+ if (ret)
+ return ret;
+
+ memset(psp->fw_pri_buf, 0, PSP_1_MEG);
+
+ /* Copy PSP System Driver binary to memory */
+ memcpy(psp->fw_pri_buf, psp->sys_start_addr, psp->sys_bin_size);
+
+ /* Provide the sys driver to bootloader */
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_36,
+ (uint32_t)(psp->fw_pri_mc_addr >> 20));
+ psp_gfxdrv_command_reg = PSP_BL__LOAD_SYSDRV;
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_35,
+ psp_gfxdrv_command_reg);
+
+ /* there might be handshake issue with hardware which needs delay */
+ mdelay(20);
+
+ ret = psp_v13_0_wait_for_bootloader(psp);
+
+ return ret;
+}
+
+static int psp_v13_0_bootloader_load_sos(struct psp_context *psp)
+{
+ int ret;
+ unsigned int psp_gfxdrv_command_reg = 0;
+ struct amdgpu_device *adev = psp->adev;
+
+ /* Check sOS sign of life register to confirm sys driver and sOS
+ * are already been loaded.
+ */
+ if (psp_v13_0_is_sos_alive(psp))
+ return 0;
+
+ ret = psp_v13_0_wait_for_bootloader(psp);
+ if (ret)
+ return ret;
+
+ memset(psp->fw_pri_buf, 0, PSP_1_MEG);
+
+ /* Copy Secure OS binary to PSP memory */
+ memcpy(psp->fw_pri_buf, psp->sos_start_addr, psp->sos_bin_size);
+
+ /* Provide the PSP secure OS to bootloader */
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_36,
+ (uint32_t)(psp->fw_pri_mc_addr >> 20));
+ psp_gfxdrv_command_reg = PSP_BL__LOAD_SOSDRV;
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_35,
+ psp_gfxdrv_command_reg);
+
+ /* there might be handshake issue with hardware which needs delay */
+ mdelay(20);
+ ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMP0_SMN_C2PMSG_81),
+ RREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_81),
+ 0, true);
+
+ return ret;
+}
+
+static int psp_v13_0_ring_init(struct psp_context *psp,
+ enum psp_ring_type ring_type)
+{
+ int ret = 0;
+ struct psp_ring *ring;
+ struct amdgpu_device *adev = psp->adev;
+
+ ring = &psp->km_ring;
+
+ ring->ring_type = ring_type;
+
+ /* allocate 4k Page of Local Frame Buffer memory for ring */
+ ring->ring_size = 0x1000;
+ ret = amdgpu_bo_create_kernel(adev, ring->ring_size, PAGE_SIZE,
+ AMDGPU_GEM_DOMAIN_VRAM,
+ &adev->firmware.rbuf,
+ &ring->ring_mem_mc_addr,
+ (void **)&ring->ring_mem);
+ if (ret) {
+ ring->ring_size = 0;
+ return ret;
+ }
+
+ return 0;
+}
+
+static int psp_v13_0_ring_stop(struct psp_context *psp,
+ enum psp_ring_type ring_type)
+{
+ int ret = 0;
+ struct amdgpu_device *adev = psp->adev;
+
+ if (amdgpu_sriov_vf(adev)) {
+ /* Write the ring destroy command*/
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_101,
+ GFX_CTRL_CMD_ID_DESTROY_GPCOM_RING);
+ /* there might be handshake issue with hardware which needs delay */
+ mdelay(20);
+ /* Wait for response flag (bit 31) */
+ ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMP0_SMN_C2PMSG_101),
+ 0x80000000, 0x80000000, false);
+ } else {
+ /* Write the ring destroy command*/
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_64,
+ GFX_CTRL_CMD_ID_DESTROY_RINGS);
+ /* there might be handshake issue with hardware which needs delay */
+ mdelay(20);
+ /* Wait for response flag (bit 31) */
+ ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMP0_SMN_C2PMSG_64),
+ 0x80000000, 0x80000000, false);
+ }
+
+ return ret;
+}
+
+static int psp_v13_0_ring_create(struct psp_context *psp,
+ enum psp_ring_type ring_type)
+{
+ int ret = 0;
+ unsigned int psp_ring_reg = 0;
+ struct psp_ring *ring = &psp->km_ring;
+ struct amdgpu_device *adev = psp->adev;
+
+ if (amdgpu_sriov_vf(adev)) {
+ ret = psp_v13_0_ring_stop(psp, ring_type);
+ if (ret) {
+ DRM_ERROR("psp_v13_0_ring_stop_sriov failed!\n");
+ return ret;
+ }
+
+ /* Write low address of the ring to C2PMSG_102 */
+ psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr);
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_102, psp_ring_reg);
+ /* Write high address of the ring to C2PMSG_103 */
+ psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr);
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_103, psp_ring_reg);
+
+ /* Write the ring initialization command to C2PMSG_101 */
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_101,
+ GFX_CTRL_CMD_ID_INIT_GPCOM_RING);
+
+ /* there might be handshake issue with hardware which needs delay */
+ mdelay(20);
+
+ /* Wait for response flag (bit 31) in C2PMSG_101 */
+ ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMP0_SMN_C2PMSG_101),
+ 0x80000000, 0x8000FFFF, false);
+
+ } else {
+ /* Wait for sOS ready for ring creation */
+ ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMP0_SMN_C2PMSG_64),
+ 0x80000000, 0x80000000, false);
+ if (ret) {
+ DRM_ERROR("Failed to wait for trust OS ready for ring creation\n");
+ return ret;
+ }
+
+ /* Write low address of the ring to C2PMSG_69 */
+ psp_ring_reg = lower_32_bits(ring->ring_mem_mc_addr);
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_69, psp_ring_reg);
+ /* Write high address of the ring to C2PMSG_70 */
+ psp_ring_reg = upper_32_bits(ring->ring_mem_mc_addr);
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_70, psp_ring_reg);
+ /* Write size of ring to C2PMSG_71 */
+ psp_ring_reg = ring->ring_size;
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_71, psp_ring_reg);
+ /* Write the ring initialization command to C2PMSG_64 */
+ psp_ring_reg = ring_type;
+ psp_ring_reg = psp_ring_reg << 16;
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_64, psp_ring_reg);
+
+ /* there might be handshake issue with hardware which needs delay */
+ mdelay(20);
+
+ /* Wait for response flag (bit 31) in C2PMSG_64 */
+ ret = psp_wait_for(psp, SOC15_REG_OFFSET(MP0, 0, regMP0_SMN_C2PMSG_64),
+ 0x80000000, 0x8000FFFF, false);
+ }
+
+ return ret;
+}
+
+static int psp_v13_0_ring_destroy(struct psp_context *psp,
+ enum psp_ring_type ring_type)
+{
+ int ret = 0;
+ struct psp_ring *ring = &psp->km_ring;
+ struct amdgpu_device *adev = psp->adev;
+
+ ret = psp_v13_0_ring_stop(psp, ring_type);
+ if (ret)
+ DRM_ERROR("Fail to stop psp ring\n");
+
+ amdgpu_bo_free_kernel(&adev->firmware.rbuf,
+ &ring->ring_mem_mc_addr,
+ (void **)&ring->ring_mem);
+
+ return ret;
+}
+
+static uint32_t psp_v13_0_ring_get_wptr(struct psp_context *psp)
+{
+ uint32_t data;
+ struct amdgpu_device *adev = psp->adev;
+
+ if (amdgpu_sriov_vf(adev))
+ data = RREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_102);
+ else
+ data = RREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_67);
+
+ return data;
+}
+
+static void psp_v13_0_ring_set_wptr(struct psp_context *psp, uint32_t value)
+{
+ struct amdgpu_device *adev = psp->adev;
+
+ if (amdgpu_sriov_vf(adev)) {
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_102, value);
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_101,
+ GFX_CTRL_CMD_ID_CONSUME_CMD);
+ } else
+ WREG32_SOC15(MP0, 0, regMP0_SMN_C2PMSG_67, value);
+}
+
+static const struct psp_funcs psp_v13_0_funcs = {
+ .init_microcode = psp_v13_0_init_microcode,
+ .bootloader_load_kdb = psp_v13_0_bootloader_load_kdb,
+ .bootloader_load_sysdrv = psp_v13_0_bootloader_load_sysdrv,
+ .bootloader_load_sos = psp_v13_0_bootloader_load_sos,
+ .ring_init = psp_v13_0_ring_init,
+ .ring_create = psp_v13_0_ring_create,
+ .ring_stop = psp_v13_0_ring_stop,
+ .ring_destroy = psp_v13_0_ring_destroy,
+ .ring_get_wptr = psp_v13_0_ring_get_wptr,
+ .ring_set_wptr = psp_v13_0_ring_set_wptr,
+};
+
+void psp_v13_0_set_psp_funcs(struct psp_context *psp)
+{
+ psp->funcs = &psp_v13_0_funcs;
+}
diff --git a/drivers/gpu/drm/amd/amdgpu/psp_v13_0.h b/drivers/gpu/drm/amd/amdgpu/psp_v13_0.h
new file mode 100644
index 000000000000..b2414a729ca1
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/psp_v13_0.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef __PSP_V13_0_H__
+#define __PSP_V13_0_H__
+
+#include "amdgpu_psp.h"
+
+void psp_v13_0_set_psp_funcs(struct psp_context *psp);
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
index eb5dc6c5b46e..9f0dda040ec8 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v2_4.c
@@ -876,12 +876,10 @@ static int sdma_v2_4_sw_init(void *handle)
ring->ring_obj = NULL;
ring->use_doorbell = false;
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
index ad308d8c6d30..135727b59c41 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v3_0.c
@@ -1160,12 +1160,10 @@ static int sdma_v3_0_sw_init(void *handle)
}
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
index c8c22c1d1e65..5715be6770ec 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c
@@ -56,6 +56,7 @@
#include "ivsrcid/sdma1/irqsrcs_sdma1_4_0.h"
#include "amdgpu_ras.h"
+#include "sdma_v4_4.h"
MODULE_FIRMWARE("amdgpu/vega10_sdma.bin");
MODULE_FIRMWARE("amdgpu/vega10_sdma1.bin");
@@ -69,6 +70,7 @@ MODULE_FIRMWARE("amdgpu/raven2_sdma.bin");
MODULE_FIRMWARE("amdgpu/arcturus_sdma.bin");
MODULE_FIRMWARE("amdgpu/renoir_sdma.bin");
MODULE_FIRMWARE("amdgpu/green_sardine_sdma.bin");
+MODULE_FIRMWARE("amdgpu/aldebaran_sdma.bin");
#define SDMA0_POWER_CNTL__ON_OFF_CONDITION_HOLD_TIME_MASK 0x000000F8L
#define SDMA0_POWER_CNTL__ON_OFF_STATUS_DURATION_TIME_MASK 0xFC000000L
@@ -259,6 +261,24 @@ static const struct soc15_reg_golden golden_settings_sdma_arct[] =
SOC15_REG_GOLDEN_VALUE(SDMA7, 0, mmSDMA7_UTCL1_TIMEOUT, 0xffffffff, 0x00010001)
};
+static const struct soc15_reg_golden golden_settings_sdma_aldebaran[] = {
+ SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_GB_ADDR_CONFIG_READ, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_UTCL1_TIMEOUT, 0xffffffff, 0x00010001),
+ SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_GB_ADDR_CONFIG_READ, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA1, 0, mmSDMA1_UTCL1_TIMEOUT, 0xffffffff, 0x00010001),
+ SOC15_REG_GOLDEN_VALUE(SDMA2, 0, mmSDMA2_GB_ADDR_CONFIG, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA2, 0, mmSDMA2_GB_ADDR_CONFIG_READ, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA3, 0, mmSDMA2_UTCL1_TIMEOUT, 0xffffffff, 0x00010001),
+ SOC15_REG_GOLDEN_VALUE(SDMA3, 0, mmSDMA3_GB_ADDR_CONFIG, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA3, 0, mmSDMA3_GB_ADDR_CONFIG_READ, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA3, 0, mmSDMA3_UTCL1_TIMEOUT, 0xffffffff, 0x00010001),
+ SOC15_REG_GOLDEN_VALUE(SDMA4, 0, mmSDMA4_GB_ADDR_CONFIG, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA4, 0, mmSDMA4_GB_ADDR_CONFIG_READ, 0x0018773f, 0x00104002),
+ SOC15_REG_GOLDEN_VALUE(SDMA4, 0, mmSDMA4_UTCL1_TIMEOUT, 0xffffffff, 0x00010001),
+};
+
static const struct soc15_reg_golden golden_settings_sdma_4_3[] = {
SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CHICKEN_BITS, 0xfe931f07, 0x02831f07),
SOC15_REG_GOLDEN_VALUE(SDMA0, 0, mmSDMA0_CLK_CTRL, 0xffffffff, 0x3f000100),
@@ -482,6 +502,11 @@ static void sdma_v4_0_init_golden_registers(struct amdgpu_device *adev)
golden_settings_sdma_arct,
ARRAY_SIZE(golden_settings_sdma_arct));
break;
+ case CHIP_ALDEBARAN:
+ soc15_program_register_sequence(adev,
+ golden_settings_sdma_aldebaran,
+ ARRAY_SIZE(golden_settings_sdma_aldebaran));
+ break;
case CHIP_RAVEN:
soc15_program_register_sequence(adev,
golden_settings_sdma_4_1,
@@ -564,7 +589,8 @@ static void sdma_v4_0_destroy_inst_ctx(struct amdgpu_device *adev)
/* arcturus shares the same FW memory across
all SDMA isntances */
- if (adev->asic_type == CHIP_ARCTURUS)
+ if (adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN)
break;
}
@@ -621,6 +647,9 @@ static int sdma_v4_0_init_microcode(struct amdgpu_device *adev)
else
chip_name = "green_sardine";
break;
+ case CHIP_ALDEBARAN:
+ chip_name = "aldebaran";
+ break;
default:
BUG();
}
@@ -636,8 +665,9 @@ static int sdma_v4_0_init_microcode(struct amdgpu_device *adev)
goto out;
for (i = 1; i < adev->sdma.num_instances; i++) {
- if (adev->asic_type == CHIP_ARCTURUS) {
- /* Acturus will leverage the same FW memory
+ if (adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN) {
+ /* Acturus & Aldebaran will leverage the same FW memory
for every SDMA instance */
memcpy((void *)&adev->sdma.instance[i],
(void *)&adev->sdma.instance[0],
@@ -1825,6 +1855,8 @@ static int sdma_v4_0_early_init(void *handle)
adev->sdma.num_instances = 1;
else if (adev->asic_type == CHIP_ARCTURUS)
adev->sdma.num_instances = 8;
+ else if (adev->asic_type == CHIP_ALDEBARAN)
+ adev->sdma.num_instances = 5;
else
adev->sdma.num_instances = 2;
@@ -1895,6 +1927,33 @@ static int sdma_v4_0_sw_init(void *handle)
return r;
}
+ /* SDMA VM_HOLE/DOORBELL_INV/POLL_TIMEOUT/SRBM_WRITE_PROTECTION event*/
+ for (i = 0; i < adev->sdma.num_instances; i++) {
+ r = amdgpu_irq_add_id(adev, sdma_v4_0_seq_to_irq_id(i),
+ SDMA0_4_0__SRCID__SDMA_VM_HOLE,
+ &adev->sdma.vm_hole_irq);
+ if (r)
+ return r;
+
+ r = amdgpu_irq_add_id(adev, sdma_v4_0_seq_to_irq_id(i),
+ SDMA0_4_0__SRCID__SDMA_DOORBELL_INVALID,
+ &adev->sdma.doorbell_invalid_irq);
+ if (r)
+ return r;
+
+ r = amdgpu_irq_add_id(adev, sdma_v4_0_seq_to_irq_id(i),
+ SDMA0_4_0__SRCID__SDMA_POLL_TIMEOUT,
+ &adev->sdma.pool_timeout_irq);
+ if (r)
+ return r;
+
+ r = amdgpu_irq_add_id(adev, sdma_v4_0_seq_to_irq_id(i),
+ SDMA0_4_0__SRCID__SDMA_SRBMWRITE,
+ &adev->sdma.srbm_write_irq);
+ if (r)
+ return r;
+ }
+
for (i = 0; i < adev->sdma.num_instances; i++) {
ring = &adev->sdma.instance[i].ring;
ring->ring_obj = NULL;
@@ -1909,7 +1968,7 @@ static int sdma_v4_0_sw_init(void *handle)
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
AMDGPU_SDMA_IRQ_INSTANCE0 + i,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
@@ -1928,7 +1987,7 @@ static int sdma_v4_0_sw_init(void *handle)
r = amdgpu_ring_init(adev, ring, 1024,
&adev->sdma.trap_irq,
AMDGPU_SDMA_IRQ_INSTANCE0 + i,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -2149,6 +2208,72 @@ static int sdma_v4_0_set_ecc_irq_state(struct amdgpu_device *adev,
return 0;
}
+static int sdma_v4_0_print_iv_entry(struct amdgpu_device *adev,
+ struct amdgpu_iv_entry *entry)
+{
+ int instance;
+ struct amdgpu_task_info task_info;
+ u64 addr;
+
+ instance = sdma_v4_0_irq_id_to_seq(entry->client_id);
+ if (instance < 0 || instance >= adev->sdma.num_instances) {
+ dev_err(adev->dev, "sdma instance invalid %d\n", instance);
+ return -EINVAL;
+ }
+
+ addr = (u64)entry->src_data[0] << 12;
+ addr |= ((u64)entry->src_data[1] & 0xf) << 44;
+
+ memset(&task_info, 0, sizeof(struct amdgpu_task_info));
+ amdgpu_vm_get_task_info(adev, entry->pasid, &task_info);
+
+ dev_info(adev->dev,
+ "[sdma%d] address:0x%016llx src_id:%u ring:%u vmid:%u "
+ "pasid:%u, for process %s pid %d thread %s pid %d\n",
+ instance, addr, entry->src_id, entry->ring_id, entry->vmid,
+ entry->pasid, task_info.process_name, task_info.tgid,
+ task_info.task_name, task_info.pid);
+ return 0;
+}
+
+static int sdma_v4_0_process_vm_hole_irq(struct amdgpu_device *adev,
+ struct amdgpu_irq_src *source,
+ struct amdgpu_iv_entry *entry)
+{
+ dev_err(adev->dev, "MC or SEM address in VM hole\n");
+ sdma_v4_0_print_iv_entry(adev, entry);
+ return 0;
+}
+
+static int sdma_v4_0_process_doorbell_invalid_irq(struct amdgpu_device *adev,
+ struct amdgpu_irq_src *source,
+ struct amdgpu_iv_entry *entry)
+{
+ dev_err(adev->dev, "SDMA received a doorbell from BIF with byte_enable !=0xff\n");
+ sdma_v4_0_print_iv_entry(adev, entry);
+ return 0;
+}
+
+static int sdma_v4_0_process_pool_timeout_irq(struct amdgpu_device *adev,
+ struct amdgpu_irq_src *source,
+ struct amdgpu_iv_entry *entry)
+{
+ dev_err(adev->dev,
+ "Polling register/memory timeout executing POLL_REG/MEM with finite timer\n");
+ sdma_v4_0_print_iv_entry(adev, entry);
+ return 0;
+}
+
+static int sdma_v4_0_process_srbm_write_irq(struct amdgpu_device *adev,
+ struct amdgpu_irq_src *source,
+ struct amdgpu_iv_entry *entry)
+{
+ dev_err(adev->dev,
+ "SDMA gets an Register Write SRBM_WRITE command in non-privilege command buffer\n");
+ sdma_v4_0_print_iv_entry(adev, entry);
+ return 0;
+}
+
static void sdma_v4_0_update_medium_grain_clock_gating(
struct amdgpu_device *adev,
bool enable)
@@ -2222,21 +2347,10 @@ static int sdma_v4_0_set_clockgating_state(void *handle,
if (amdgpu_sriov_vf(adev))
return 0;
- switch (adev->asic_type) {
- case CHIP_VEGA10:
- case CHIP_VEGA12:
- case CHIP_VEGA20:
- case CHIP_RAVEN:
- case CHIP_ARCTURUS:
- case CHIP_RENOIR:
- sdma_v4_0_update_medium_grain_clock_gating(adev,
- state == AMD_CG_STATE_GATE);
- sdma_v4_0_update_medium_grain_light_sleep(adev,
- state == AMD_CG_STATE_GATE);
- break;
- default:
- break;
- }
+ sdma_v4_0_update_medium_grain_clock_gating(adev,
+ state == AMD_CG_STATE_GATE);
+ sdma_v4_0_update_medium_grain_light_sleep(adev,
+ state == AMD_CG_STATE_GATE);
return 0;
}
@@ -2249,7 +2363,7 @@ static int sdma_v4_0_set_powergating_state(void *handle,
case CHIP_RAVEN:
case CHIP_RENOIR:
sdma_v4_1_update_power_gating(adev,
- state == AMD_PG_STATE_GATE ? true : false);
+ state == AMD_PG_STATE_GATE);
break;
default:
break;
@@ -2465,7 +2579,21 @@ static const struct amdgpu_irq_src_funcs sdma_v4_0_ecc_irq_funcs = {
.process = amdgpu_sdma_process_ecc_irq,
};
+static const struct amdgpu_irq_src_funcs sdma_v4_0_vm_hole_irq_funcs = {
+ .process = sdma_v4_0_process_vm_hole_irq,
+};
+
+static const struct amdgpu_irq_src_funcs sdma_v4_0_doorbell_invalid_irq_funcs = {
+ .process = sdma_v4_0_process_doorbell_invalid_irq,
+};
+
+static const struct amdgpu_irq_src_funcs sdma_v4_0_pool_timeout_irq_funcs = {
+ .process = sdma_v4_0_process_pool_timeout_irq,
+};
+static const struct amdgpu_irq_src_funcs sdma_v4_0_srbm_write_irq_funcs = {
+ .process = sdma_v4_0_process_srbm_write_irq,
+};
static void sdma_v4_0_set_irq_funcs(struct amdgpu_device *adev)
{
@@ -2474,9 +2602,17 @@ static void sdma_v4_0_set_irq_funcs(struct amdgpu_device *adev)
adev->sdma.trap_irq.num_types = AMDGPU_SDMA_IRQ_INSTANCE1;
adev->sdma.ecc_irq.num_types = AMDGPU_SDMA_IRQ_INSTANCE1;
break;
+ case 5:
+ adev->sdma.trap_irq.num_types = AMDGPU_SDMA_IRQ_INSTANCE5;
+ adev->sdma.ecc_irq.num_types = AMDGPU_SDMA_IRQ_INSTANCE5;
+ break;
case 8:
adev->sdma.trap_irq.num_types = AMDGPU_SDMA_IRQ_LAST;
adev->sdma.ecc_irq.num_types = AMDGPU_SDMA_IRQ_LAST;
+ adev->sdma.vm_hole_irq.num_types = AMDGPU_SDMA_IRQ_INSTANCE5;
+ adev->sdma.doorbell_invalid_irq.num_types = AMDGPU_SDMA_IRQ_LAST;
+ adev->sdma.pool_timeout_irq.num_types = AMDGPU_SDMA_IRQ_LAST;
+ adev->sdma.srbm_write_irq.num_types = AMDGPU_SDMA_IRQ_LAST;
break;
case 2:
default:
@@ -2487,6 +2623,10 @@ static void sdma_v4_0_set_irq_funcs(struct amdgpu_device *adev)
adev->sdma.trap_irq.funcs = &sdma_v4_0_trap_irq_funcs;
adev->sdma.illegal_inst_irq.funcs = &sdma_v4_0_illegal_inst_irq_funcs;
adev->sdma.ecc_irq.funcs = &sdma_v4_0_ecc_irq_funcs;
+ adev->sdma.vm_hole_irq.funcs = &sdma_v4_0_vm_hole_irq_funcs;
+ adev->sdma.doorbell_invalid_irq.funcs = &sdma_v4_0_doorbell_invalid_irq_funcs;
+ adev->sdma.pool_timeout_irq.funcs = &sdma_v4_0_pool_timeout_irq_funcs;
+ adev->sdma.srbm_write_irq.funcs = &sdma_v4_0_srbm_write_irq_funcs;
}
/**
@@ -2655,6 +2795,9 @@ static void sdma_v4_0_set_ras_funcs(struct amdgpu_device *adev)
case CHIP_ARCTURUS:
adev->sdma.funcs = &sdma_v4_0_ras_funcs;
break;
+ case CHIP_ALDEBARAN:
+ adev->sdma.funcs = &sdma_v4_4_ras_funcs;
+ break;
default:
break;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4.c b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4.c
new file mode 100644
index 000000000000..bf95007f0843
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4.c
@@ -0,0 +1,253 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "amdgpu.h"
+#include "sdma/sdma_4_4_0_offset.h"
+#include "sdma/sdma_4_4_0_sh_mask.h"
+#include "soc15.h"
+#include "amdgpu_ras.h"
+
+#define SDMA1_REG_OFFSET 0x600
+#define SDMA2_REG_OFFSET 0x1cda0
+#define SDMA3_REG_OFFSET 0x1d1a0
+#define SDMA4_REG_OFFSET 0x1d5a0
+
+/* helper function that allow only use sdma0 register offset
+ * to calculate register offset for all the sdma instances */
+static uint32_t sdma_v4_4_get_reg_offset(struct amdgpu_device *adev,
+ uint32_t instance,
+ uint32_t offset)
+{
+ uint32_t sdma_base = adev->reg_offset[SDMA0_HWIP][0][0];
+
+ switch (instance) {
+ case 0:
+ return (sdma_base + offset);
+ case 1:
+ return (sdma_base + SDMA1_REG_OFFSET + offset);
+ case 2:
+ return (sdma_base + SDMA2_REG_OFFSET + offset);
+ case 3:
+ return (sdma_base + SDMA3_REG_OFFSET + offset);
+ case 4:
+ return (sdma_base + SDMA4_REG_OFFSET + offset);
+ default:
+ break;
+ }
+ return 0;
+}
+
+static const struct soc15_ras_field_entry sdma_v4_4_ras_fields[] = {
+ { "SDMA_MBANK_DATA_BUF0_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF0_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF1_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF1_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF2_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF2_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF3_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF3_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF4_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF4_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF5_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF5_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF6_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF6_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF7_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF7_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF8_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF8_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF9_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF9_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF10_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF10_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF11_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF11_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF12_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF12_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF13_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF13_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF14_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF14_SED),
+ 0, 0,
+ },
+ { "SDMA_MBANK_DATA_BUF15_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER, SDMA_MBANK_DATA_BUF15_SED),
+ 0, 0,
+ },
+ { "SDMA_UCODE_BUF_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_UCODE_BUF_SED),
+ 0, 0,
+ },
+ { "SDMA_RB_CMD_BUF_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_RB_CMD_BUF_SED),
+ 0, 0,
+ },
+ { "SDMA_IB_CMD_BUF_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_IB_CMD_BUF_SED),
+ 0, 0,
+ },
+ { "SDMA_UTCL1_RD_FIFO_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_UTCL1_RD_FIFO_SED),
+ 0, 0,
+ },
+ { "SDMA_UTCL1_RDBST_FIFO_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_UTCL1_RDBST_FIFO_SED),
+ 0, 0,
+ },
+ { "SDMA_DATA_LUT_FIFO_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_DATA_LUT_FIFO_SED),
+ 0, 0,
+ },
+ { "SDMA_SPLIT_DATA_BUF_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_SPLIT_DATA_BUF_SED),
+ 0, 0,
+ },
+ { "SDMA_MC_WR_ADDR_FIFO_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_MC_WR_ADDR_FIFO_SED),
+ 0, 0,
+ },
+ { "SDMA_MC_RDRET_BUF_SED", SOC15_REG_ENTRY(SDMA0, 0, regSDMA0_EDC_COUNTER2),
+ SOC15_REG_FIELD(SDMA0_EDC_COUNTER2, SDMA_MC_WR_ADDR_FIFO_SED),
+ 0, 0,
+ },
+};
+
+static void sdma_v4_4_get_ras_error_count(struct amdgpu_device *adev,
+ uint32_t reg_offset,
+ uint32_t value,
+ uint32_t instance,
+ uint32_t *sec_count)
+{
+ uint32_t i;
+ uint32_t sec_cnt;
+
+ /* double bits error (multiple bits) error detection is not supported */
+ for (i = 0; i < ARRAY_SIZE(sdma_v4_4_ras_fields); i++) {
+ if (sdma_v4_4_ras_fields[i].reg_offset != reg_offset)
+ continue;
+
+ /* the SDMA_EDC_COUNTER register in each sdma instance
+ * shares the same sed shift_mask
+ * */
+ sec_cnt = (value &
+ sdma_v4_4_ras_fields[i].sec_count_mask) >>
+ sdma_v4_4_ras_fields[i].sec_count_shift;
+ if (sec_cnt) {
+ dev_info(adev->dev, "Detected %s in SDMA%d, SED %d\n",
+ sdma_v4_4_ras_fields[i].name,
+ instance, sec_cnt);
+ *sec_count += sec_cnt;
+ }
+ }
+}
+
+static int sdma_v4_4_query_ras_error_count(struct amdgpu_device *adev,
+ uint32_t instance,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+ uint32_t sec_count = 0;
+ uint32_t reg_value = 0;
+ uint32_t reg_offset = 0;
+
+ reg_offset = sdma_v4_4_get_reg_offset(adev, instance, regSDMA0_EDC_COUNTER);
+ reg_value = RREG32(reg_offset);
+ /* double bit error is not supported */
+ if (reg_value)
+ sdma_v4_4_get_ras_error_count(adev, regSDMA0_EDC_COUNTER, reg_value,
+ instance, &sec_count);
+
+ reg_offset = sdma_v4_4_get_reg_offset(adev, instance, regSDMA0_EDC_COUNTER2);
+ reg_value = RREG32(reg_offset);
+ /* double bit error is not supported */
+ if (reg_value)
+ sdma_v4_4_get_ras_error_count(adev, regSDMA0_EDC_COUNTER2, reg_value,
+ instance, &sec_count);
+
+ /*
+ * err_data->ue_count should be initialized to 0
+ * before calling into this function
+ *
+ * SDMA RAS supports single bit uncorrectable error detection.
+ * So, increment uncorrectable error count.
+ */
+ err_data->ue_count += sec_count;
+
+ /*
+ * SDMA RAS does not support correctable errors.
+ * Set ce count to 0.
+ */
+ err_data->ce_count = 0;
+
+ return 0;
+};
+
+static void sdma_v4_4_reset_ras_error_count(struct amdgpu_device *adev)
+{
+ int i;
+ uint32_t reg_offset;
+
+ /* write 0 to EDC_COUNTER reg to clear sdma edc counters */
+ if (amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__SDMA)) {
+ for (i = 0; i < adev->sdma.num_instances; i++) {
+ reg_offset = sdma_v4_4_get_reg_offset(adev, i, regSDMA0_EDC_COUNTER);
+ WREG32(reg_offset, 0);
+ reg_offset = sdma_v4_4_get_reg_offset(adev, i, regSDMA0_EDC_COUNTER2);
+ WREG32(reg_offset, 0);
+ }
+ }
+}
+
+const struct amdgpu_sdma_ras_funcs sdma_v4_4_ras_funcs = {
+ .ras_late_init = amdgpu_sdma_ras_late_init,
+ .ras_fini = amdgpu_sdma_ras_fini,
+ .query_ras_error_count = sdma_v4_4_query_ras_error_count,
+ .reset_ras_error_count = sdma_v4_4_reset_ras_error_count,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v4_4.h b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4.h
new file mode 100644
index 000000000000..74a6e5b5e949
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v4_4.h
@@ -0,0 +1,28 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef __SDMA_V4_4_H__
+#define __SDMA_V4_4_H__
+
+extern const struct amdgpu_sdma_ras_funcs sdma_v4_4_ras_funcs;
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
index d345e324837d..920fc6d4a127 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c
@@ -1273,12 +1273,10 @@ static int sdma_v5_0_sw_init(void *handle)
: (adev->doorbell_index.sdma_engine[1] << 1); // get DWORD offset
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
index 690a5090475a..b1ad9e52b234 100644
--- a/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c
@@ -370,6 +370,33 @@ static void sdma_v5_2_ring_emit_ib(struct amdgpu_ring *ring,
}
/**
+ * sdma_v5_2_ring_emit_mem_sync - flush the IB by graphics cache rinse
+ *
+ * @ring: amdgpu ring pointer
+ * @job: job to retrieve vmid from
+ * @ib: IB object to schedule
+ *
+ * flush the IB by graphics cache rinse.
+ */
+static void sdma_v5_2_ring_emit_mem_sync(struct amdgpu_ring *ring)
+{
+ uint32_t gcr_cntl =
+ SDMA_GCR_GL2_INV | SDMA_GCR_GL2_WB | SDMA_GCR_GLM_INV |
+ SDMA_GCR_GL1_INV | SDMA_GCR_GLV_INV | SDMA_GCR_GLK_INV |
+ SDMA_GCR_GLI_INV(1);
+
+ /* flush entire cache L0/L1/L2, this can be optimized by performance requirement */
+ amdgpu_ring_write(ring, SDMA_PKT_HEADER_OP(SDMA_OP_GCR_REQ));
+ amdgpu_ring_write(ring, SDMA_PKT_GCR_REQ_PAYLOAD1_BASE_VA_31_7(0));
+ amdgpu_ring_write(ring, SDMA_PKT_GCR_REQ_PAYLOAD2_GCR_CONTROL_15_0(gcr_cntl) |
+ SDMA_PKT_GCR_REQ_PAYLOAD2_BASE_VA_47_32(0));
+ amdgpu_ring_write(ring, SDMA_PKT_GCR_REQ_PAYLOAD3_LIMIT_VA_31_7(0) |
+ SDMA_PKT_GCR_REQ_PAYLOAD3_GCR_CONTROL_18_16(gcr_cntl >> 16));
+ amdgpu_ring_write(ring, SDMA_PKT_GCR_REQ_PAYLOAD4_LIMIT_VA_47_32(0) |
+ SDMA_PKT_GCR_REQ_PAYLOAD4_VMID(0));
+}
+
+/**
* sdma_v5_2_ring_emit_hdp_flush - emit an hdp flush on the DMA ring
*
* @ring: amdgpu ring pointer
@@ -1283,10 +1310,9 @@ static int sdma_v5_2_sw_init(void *handle)
(adev->doorbell_index.sdma_engine[i] << 1); //get DWORD offset
sprintf(ring->name, "sdma%d", i);
- r = amdgpu_ring_init(adev, ring, 1024,
- &adev->sdma.trap_irq,
+ r = amdgpu_ring_init(adev, ring, 1024, &adev->sdma.trap_irq,
AMDGPU_SDMA_IRQ_INSTANCE0 + i,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -1595,9 +1621,9 @@ static int sdma_v5_2_set_clockgating_state(void *handle,
case CHIP_VANGOGH:
case CHIP_DIMGREY_CAVEFISH:
sdma_v5_2_update_medium_grain_clock_gating(adev,
- state == AMD_CG_STATE_GATE ? true : false);
+ state == AMD_CG_STATE_GATE);
sdma_v5_2_update_medium_grain_light_sleep(adev,
- state == AMD_CG_STATE_GATE ? true : false);
+ state == AMD_CG_STATE_GATE);
break;
default:
break;
@@ -1664,6 +1690,7 @@ static const struct amdgpu_ring_funcs sdma_v5_2_ring_funcs = {
10 + 10 + 10, /* sdma_v5_2_ring_emit_fence x3 for user fence, vm fence */
.emit_ib_size = 7 + 6, /* sdma_v5_2_ring_emit_ib */
.emit_ib = sdma_v5_2_ring_emit_ib,
+ .emit_mem_sync = sdma_v5_2_ring_emit_mem_sync,
.emit_fence = sdma_v5_2_ring_emit_fence,
.emit_pipeline_sync = sdma_v5_2_ring_emit_pipeline_sync,
.emit_vm_flush = sdma_v5_2_ring_emit_vm_flush,
diff --git a/drivers/gpu/drm/amd/amdgpu/si.c b/drivers/gpu/drm/amd/amdgpu/si.c
index 6b5cf7882a12..7cbc2bb03bc6 100644
--- a/drivers/gpu/drm/amd/amdgpu/si.c
+++ b/drivers/gpu/drm/amd/amdgpu/si.c
@@ -26,6 +26,8 @@
#include <linux/module.h>
#include <linux/pci.h>
+#include <drm/amdgpu_drm.h>
+
#include "amdgpu.h"
#include "amdgpu_atombios.h"
#include "amdgpu_ih.h"
@@ -905,6 +907,114 @@ static const u32 hainan_mgcg_cgcg_init[] =
0x3630, 0xfffffff0, 0x00000100,
};
+/* XXX: update when we support VCE */
+#if 0
+/* tahiti, pitcarin, verde */
+static const struct amdgpu_video_codec_info tahiti_video_codecs_encode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs tahiti_video_codecs_encode =
+{
+ .codec_count = ARRAY_SIZE(tahiti_video_codecs_encode_array),
+ .codec_array = tahiti_video_codecs_encode_array,
+};
+#else
+static const struct amdgpu_video_codecs tahiti_video_codecs_encode =
+{
+ .codec_count = 0,
+ .codec_array = NULL,
+};
+#endif
+/* oland and hainan don't support encode */
+static const struct amdgpu_video_codecs hainan_video_codecs_encode =
+{
+ .codec_count = 0,
+ .codec_array = NULL,
+};
+
+/* tahiti, pitcarin, verde, oland */
+static const struct amdgpu_video_codec_info tahiti_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 41,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 2048,
+ .max_height = 1152,
+ .max_pixels_per_frame = 2048 * 1152,
+ .max_level = 4,
+ },
+};
+
+static const struct amdgpu_video_codecs tahiti_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(tahiti_video_codecs_decode_array),
+ .codec_array = tahiti_video_codecs_decode_array,
+};
+
+/* hainan doesn't support decode */
+static const struct amdgpu_video_codecs hainan_video_codecs_decode =
+{
+ .codec_count = 0,
+ .codec_array = NULL,
+};
+
+static int si_query_video_codecs(struct amdgpu_device *adev, bool encode,
+ const struct amdgpu_video_codecs **codecs)
+{
+ switch (adev->asic_type) {
+ case CHIP_VERDE:
+ case CHIP_TAHITI:
+ case CHIP_PITCAIRN:
+ if (encode)
+ *codecs = &tahiti_video_codecs_encode;
+ else
+ *codecs = &tahiti_video_codecs_decode;
+ return 0;
+ case CHIP_OLAND:
+ if (encode)
+ *codecs = &hainan_video_codecs_encode;
+ else
+ *codecs = &tahiti_video_codecs_decode;
+ return 0;
+ case CHIP_HAINAN:
+ if (encode)
+ *codecs = &hainan_video_codecs_encode;
+ else
+ *codecs = &hainan_video_codecs_decode;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
static u32 si_pcie_rreg(struct amdgpu_device *adev, u32 reg)
{
unsigned long flags;
@@ -1903,6 +2013,7 @@ static const struct amdgpu_asic_funcs si_asic_funcs =
.get_pcie_replay_count = &si_get_pcie_replay_count,
.supports_baco = &si_asic_supports_baco,
.pre_asic_init = &si_pre_asic_init,
+ .query_video_codecs = &si_query_video_codecs,
};
static uint32_t si_get_rev_id(struct amdgpu_device *adev)
diff --git a/drivers/gpu/drm/amd/amdgpu/si_dma.c b/drivers/gpu/drm/amd/amdgpu/si_dma.c
index 488497ad5e0c..cb703e307238 100644
--- a/drivers/gpu/drm/amd/amdgpu/si_dma.c
+++ b/drivers/gpu/drm/amd/amdgpu/si_dma.c
@@ -507,10 +507,9 @@ static int si_dma_sw_init(void *handle)
sprintf(ring->name, "sdma%d", i);
r = amdgpu_ring_init(adev, ring, 1024,
&adev->sdma.trap_irq,
- (i == 0) ?
- AMDGPU_SDMA_IRQ_INSTANCE0 :
+ (i == 0) ? AMDGPU_SDMA_IRQ_INSTANCE0 :
AMDGPU_SDMA_IRQ_INSTANCE1,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/smuio_v13_0.c b/drivers/gpu/drm/amd/amdgpu/smuio_v13_0.c
new file mode 100644
index 000000000000..079b094c48ad
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/smuio_v13_0.c
@@ -0,0 +1,121 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "amdgpu.h"
+#include "smuio_v13_0.h"
+#include "smuio/smuio_13_0_2_offset.h"
+#include "smuio/smuio_13_0_2_sh_mask.h"
+
+#define SMUIO_MCM_CONFIG__HOST_GPU_XGMI_MASK 0x00000001L
+
+static u32 smuio_v13_0_get_rom_index_offset(struct amdgpu_device *adev)
+{
+ return SOC15_REG_OFFSET(SMUIO, 0, regROM_INDEX);
+}
+
+static u32 smuio_v13_0_get_rom_data_offset(struct amdgpu_device *adev)
+{
+ return SOC15_REG_OFFSET(SMUIO, 0, regROM_DATA);
+}
+
+static void smuio_v13_0_update_rom_clock_gating(struct amdgpu_device *adev, bool enable)
+{
+ u32 def, data;
+
+ /* enable/disable ROM CG is not supported on APU */
+ if (adev->flags & AMD_IS_APU)
+ return;
+
+ def = data = RREG32_SOC15(SMUIO, 0, regCGTT_ROM_CLK_CTRL0);
+
+ if (enable && (adev->cg_flags & AMD_CG_SUPPORT_ROM_MGCG))
+ data &= ~(CGTT_ROM_CLK_CTRL0__SOFT_OVERRIDE0_MASK |
+ CGTT_ROM_CLK_CTRL0__SOFT_OVERRIDE1_MASK);
+ else
+ data |= CGTT_ROM_CLK_CTRL0__SOFT_OVERRIDE0_MASK |
+ CGTT_ROM_CLK_CTRL0__SOFT_OVERRIDE1_MASK;
+
+ if (def != data)
+ WREG32_SOC15(SMUIO, 0, regCGTT_ROM_CLK_CTRL0, data);
+}
+
+static void smuio_v13_0_get_clock_gating_state(struct amdgpu_device *adev, u32 *flags)
+{
+ u32 data;
+
+ /* CGTT_ROM_CLK_CTRL0 is not available for APU */
+ if (adev->flags & AMD_IS_APU)
+ return;
+
+ data = RREG32_SOC15(SMUIO, 0, regCGTT_ROM_CLK_CTRL0);
+ if (!(data & CGTT_ROM_CLK_CTRL0__SOFT_OVERRIDE0_MASK))
+ *flags |= AMD_CG_SUPPORT_ROM_MGCG;
+}
+
+/**
+ * smuio_v13_0_get_die_id - query die id from FCH.
+ *
+ * @adev: amdgpu device pointer
+ *
+ * Returns die id
+ */
+static u32 smuio_v13_0_get_die_id(struct amdgpu_device *adev)
+{
+ u32 data, die_id;
+
+ data = RREG32_SOC15(SMUIO, 0, regSMUIO_MCM_CONFIG);
+ die_id = REG_GET_FIELD(data, SMUIO_MCM_CONFIG, DIE_ID);
+
+ return die_id;
+}
+
+/**
+ * smuio_v13_0_supports_host_gpu_xgmi - detect xgmi interface between cpu and gpu/s.
+ *
+ * @adev: amdgpu device pointer
+ *
+ * Returns true on success or false otherwise.
+ */
+static bool smuio_v13_0_is_host_gpu_xgmi_supported(struct amdgpu_device *adev)
+{
+ u32 data;
+
+ data = RREG32_SOC15(SMUIO, 0, regSMUIO_MCM_CONFIG);
+ data = REG_GET_FIELD(data, SMUIO_MCM_CONFIG, TOPOLOGY_ID);
+ /* data[4:0]
+ * bit 0 == 0 host-gpu interface is PCIE
+ * bit 0 == 1 host-gpu interface is Alternate Protocal
+ * for AMD, this is XGMI
+ */
+ data &= SMUIO_MCM_CONFIG__HOST_GPU_XGMI_MASK;
+
+ return data ? true : false;
+}
+
+const struct amdgpu_smuio_funcs smuio_v13_0_funcs = {
+ .get_rom_index_offset = smuio_v13_0_get_rom_index_offset,
+ .get_rom_data_offset = smuio_v13_0_get_rom_data_offset,
+ .get_die_id = smuio_v13_0_get_die_id,
+ .is_host_gpu_xgmi_supported = smuio_v13_0_is_host_gpu_xgmi_supported,
+ .update_rom_clock_gating = smuio_v13_0_update_rom_clock_gating,
+ .get_clock_gating_state = smuio_v13_0_get_clock_gating_state,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/smuio_v13_0.h b/drivers/gpu/drm/amd/amdgpu/smuio_v13_0.h
new file mode 100644
index 000000000000..a3bfe3e4fb46
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/smuio_v13_0.h
@@ -0,0 +1,30 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef __SMUIO_V13_0_H__
+#define __SMUIO_V13_0_H__
+
+#include "soc15_common.h"
+
+extern const struct amdgpu_smuio_funcs smuio_v13_0_funcs;
+
+#endif /* __SMUIO_V13_0_H__ */
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.c b/drivers/gpu/drm/amd/amdgpu/soc15.c
index 1221aa6b40a9..8e1b9a40839f 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.c
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.c
@@ -25,6 +25,8 @@
#include <linux/module.h>
#include <linux/pci.h>
+#include <drm/amdgpu_drm.h>
+
#include "amdgpu.h"
#include "amdgpu_atombios.h"
#include "amdgpu_ih.h"
@@ -71,9 +73,9 @@
#include "jpeg_v2_5.h"
#include "smuio_v9_0.h"
#include "smuio_v11_0.h"
+#include "smuio_v13_0.h"
#include "dce_virtual.h"
#include "mxgpu_ai.h"
-#include "amdgpu_smu.h"
#include "amdgpu_ras.h"
#include "amdgpu_xgmi.h"
#include <uapi/linux/kfd_ioctl.h>
@@ -83,6 +85,234 @@
#define mmMP0_MISC_LIGHT_SLEEP_CTRL 0x01ba
#define mmMP0_MISC_LIGHT_SLEEP_CTRL_BASE_IDX 0
+/* Vega, Raven, Arcturus */
+static const struct amdgpu_video_codec_info vega_video_codecs_encode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 2304,
+ .max_pixels_per_frame = 4096 * 2304,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 4096,
+ .max_height = 2304,
+ .max_pixels_per_frame = 4096 * 2304,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs vega_video_codecs_encode =
+{
+ .codec_count = ARRAY_SIZE(vega_video_codecs_encode_array),
+ .codec_array = vega_video_codecs_encode_array,
+};
+
+/* Vega */
+static const struct amdgpu_video_codec_info vega_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 52,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 4,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 186,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs vega_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(vega_video_codecs_decode_array),
+ .codec_array = vega_video_codecs_decode_array,
+};
+
+/* Raven */
+static const struct amdgpu_video_codec_info rv_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 52,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 4,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 186,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs rv_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(rv_video_codecs_decode_array),
+ .codec_array = rv_video_codecs_decode_array,
+};
+
+/* Renoir, Arcturus */
+static const struct amdgpu_video_codec_info rn_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 52,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 4,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 8192,
+ .max_height = 4352,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 186,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VP9,
+ .max_width = 8192,
+ .max_height = 4352,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs rn_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(rn_video_codecs_decode_array),
+ .codec_array = rn_video_codecs_decode_array,
+};
+
+static int soc15_query_video_codecs(struct amdgpu_device *adev, bool encode,
+ const struct amdgpu_video_codecs **codecs)
+{
+ switch (adev->asic_type) {
+ case CHIP_VEGA20:
+ case CHIP_VEGA10:
+ case CHIP_VEGA12:
+ if (encode)
+ *codecs = &vega_video_codecs_encode;
+ else
+ *codecs = &vega_video_codecs_decode;
+ return 0;
+ case CHIP_RAVEN:
+ if (encode)
+ *codecs = &vega_video_codecs_encode;
+ else
+ *codecs = &rv_video_codecs_decode;
+ return 0;
+ case CHIP_ARCTURUS:
+ case CHIP_RENOIR:
+ if (encode)
+ *codecs = &vega_video_codecs_encode;
+ else
+ *codecs = &rn_video_codecs_decode;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
/*
* Indirect registers accessor
*/
@@ -419,40 +649,6 @@ void soc15_program_register_sequence(struct amdgpu_device *adev,
}
-static int soc15_asic_mode1_reset(struct amdgpu_device *adev)
-{
- u32 i;
- int ret = 0;
-
- amdgpu_atombios_scratch_regs_engine_hung(adev, true);
-
- dev_info(adev->dev, "GPU mode1 reset\n");
-
- /* disable BM */
- pci_clear_master(adev->pdev);
-
- amdgpu_device_cache_pci_state(adev->pdev);
-
- ret = psp_gpu_reset(adev);
- if (ret)
- dev_err(adev->dev, "GPU mode1 reset failed\n");
-
- amdgpu_device_load_pci_state(adev->pdev);
-
- /* wait for asic to come out of reset */
- for (i = 0; i < adev->usec_timeout; i++) {
- u32 memsize = adev->nbio.funcs->get_memsize(adev);
-
- if (memsize != 0xffffffff)
- break;
- udelay(1);
- }
-
- amdgpu_atombios_scratch_regs_engine_hung(adev, false);
-
- return ret;
-}
-
static int soc15_asic_baco_reset(struct amdgpu_device *adev)
{
struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
@@ -477,13 +673,21 @@ static enum amd_reset_method
soc15_asic_reset_method(struct amdgpu_device *adev)
{
bool baco_reset = false;
+ bool connected_to_cpu = false;
struct amdgpu_ras *ras = amdgpu_ras_get_context(adev);
+ if (adev->gmc.xgmi.supported && adev->gmc.xgmi.connected_to_cpu)
+ connected_to_cpu = true;
+
if (amdgpu_reset_method == AMD_RESET_METHOD_MODE1 ||
amdgpu_reset_method == AMD_RESET_METHOD_MODE2 ||
amdgpu_reset_method == AMD_RESET_METHOD_BACO ||
- amdgpu_reset_method == AMD_RESET_METHOD_PCI)
- return amdgpu_reset_method;
+ amdgpu_reset_method == AMD_RESET_METHOD_PCI) {
+ /* If connected to cpu, driver only support mode2 */
+ if (connected_to_cpu)
+ return AMD_RESET_METHOD_MODE2;
+ return amdgpu_reset_method;
+ }
if (amdgpu_reset_method != -1)
dev_warn(adev->dev, "Specified reset method:%d isn't supported, using AUTO instead.\n",
@@ -509,6 +713,14 @@ soc15_asic_reset_method(struct amdgpu_device *adev)
if ((ras && ras->supported) && adev->pm.fw_version <= 0x283400)
baco_reset = false;
break;
+ case CHIP_ALDEBARAN:
+ /*
+ * 1.connected to cpu: driver issue mode2 reset
+ * 2.discret gpu: driver issue mode1 reset
+ */
+ if (connected_to_cpu)
+ return AMD_RESET_METHOD_MODE2;
+ break;
default:
break;
}
@@ -538,7 +750,7 @@ static int soc15_asic_reset(struct amdgpu_device *adev)
return amdgpu_dpm_mode2_reset(adev);
default:
dev_info(adev->dev, "MODE1 reset\n");
- return soc15_asic_mode1_reset(adev);
+ return amdgpu_device_mode1_reset(adev);
}
}
@@ -604,11 +816,12 @@ static void soc15_pcie_gen3_enable(struct amdgpu_device *adev)
static void soc15_program_aspm(struct amdgpu_device *adev)
{
-
- if (amdgpu_aspm == 0)
+ if (amdgpu_aspm != 1)
return;
- /* todo */
+ if (!(adev->flags & AMD_IS_APU) &&
+ (adev->nbio.funcs->program_aspm))
+ adev->nbio.funcs->program_aspm(adev);
}
static void soc15_enable_doorbell_aperture(struct amdgpu_device *adev,
@@ -661,6 +874,9 @@ static void soc15_reg_base_init(struct amdgpu_device *adev)
case CHIP_ARCTURUS:
arct_reg_base_init(adev);
break;
+ case CHIP_ALDEBARAN:
+ aldebaran_reg_base_init(adev);
+ break;
default:
DRM_ERROR("Unsupported asic type: %d!\n", adev->asic_type);
break;
@@ -683,14 +899,12 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev)
if (!amdgpu_sriov_vf(adev))
soc15_reg_base_init(adev);
- if (adev->asic_type == CHIP_VEGA20 || adev->asic_type == CHIP_ARCTURUS)
- adev->gmc.xgmi.supported = true;
-
if (adev->flags & AMD_IS_APU) {
adev->nbio.funcs = &nbio_v7_0_funcs;
adev->nbio.hdp_flush_reg = &nbio_v7_0_hdp_flush_reg;
} else if (adev->asic_type == CHIP_VEGA20 ||
- adev->asic_type == CHIP_ARCTURUS) {
+ adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN) {
adev->nbio.funcs = &nbio_v7_4_funcs;
adev->nbio.hdp_flush_reg = &nbio_v7_4_hdp_flush_reg;
} else {
@@ -699,7 +913,9 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev)
}
adev->hdp.funcs = &hdp_v4_0_funcs;
- if (adev->asic_type == CHIP_VEGA20 || adev->asic_type == CHIP_ARCTURUS)
+ if (adev->asic_type == CHIP_VEGA20 ||
+ adev->asic_type == CHIP_ARCTURUS ||
+ adev->asic_type == CHIP_ALDEBARAN)
adev->df.funcs = &df_v3_6_funcs;
else
adev->df.funcs = &df_v1_7_funcs;
@@ -707,6 +923,8 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev)
if (adev->asic_type == CHIP_VEGA20 ||
adev->asic_type == CHIP_ARCTURUS)
adev->smuio.funcs = &smuio_v11_0_funcs;
+ else if (adev->asic_type == CHIP_ALDEBARAN)
+ adev->smuio.funcs = &smuio_v13_0_funcs;
else
adev->smuio.funcs = &smuio_v9_0_funcs;
@@ -826,6 +1044,27 @@ int soc15_set_ip_blocks(struct amdgpu_device *adev)
amdgpu_device_ip_block_add(adev, &vcn_v2_0_ip_block);
amdgpu_device_ip_block_add(adev, &jpeg_v2_0_ip_block);
break;
+ case CHIP_ALDEBARAN:
+ amdgpu_device_ip_block_add(adev, &vega10_common_ip_block);
+ amdgpu_device_ip_block_add(adev, &gmc_v9_0_ip_block);
+
+ if (amdgpu_sriov_vf(adev)) {
+ if (likely(adev->firmware.load_type == AMDGPU_FW_LOAD_PSP))
+ amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block);
+ amdgpu_device_ip_block_add(adev, &vega20_ih_ip_block);
+ } else {
+ amdgpu_device_ip_block_add(adev, &vega20_ih_ip_block);
+ if (likely(adev->firmware.load_type == AMDGPU_FW_LOAD_PSP))
+ amdgpu_device_ip_block_add(adev, &psp_v13_0_ip_block);
+ }
+
+ amdgpu_device_ip_block_add(adev, &gfx_v9_0_ip_block);
+ amdgpu_device_ip_block_add(adev, &sdma_v4_0_ip_block);
+
+ amdgpu_device_ip_block_add(adev, &smu_v13_0_ip_block);
+ amdgpu_device_ip_block_add(adev, &vcn_v2_6_ip_block);
+ amdgpu_device_ip_block_add(adev, &jpeg_v2_6_ip_block);
+ break;
default:
return -EINVAL;
}
@@ -994,6 +1233,7 @@ static const struct amdgpu_asic_funcs soc15_asic_funcs =
.get_pcie_replay_count = &soc15_get_pcie_replay_count,
.supports_baco = &soc15_supports_baco,
.pre_asic_init = &soc15_pre_asic_init,
+ .query_video_codecs = &soc15_query_video_codecs,
};
static const struct amdgpu_asic_funcs vega20_asic_funcs =
@@ -1015,6 +1255,7 @@ static const struct amdgpu_asic_funcs vega20_asic_funcs =
.get_pcie_replay_count = &soc15_get_pcie_replay_count,
.supports_baco = &soc15_supports_baco,
.pre_asic_init = &soc15_pre_asic_init,
+ .query_video_codecs = &soc15_query_video_codecs,
};
static int soc15_common_early_init(void *handle)
@@ -1160,7 +1401,8 @@ static int soc15_common_early_init(void *handle)
AMD_CG_SUPPORT_MC_MGCG |
AMD_CG_SUPPORT_MC_LS |
AMD_CG_SUPPORT_SDMA_MGCG |
- AMD_CG_SUPPORT_SDMA_LS;
+ AMD_CG_SUPPORT_SDMA_LS |
+ AMD_CG_SUPPORT_VCN_MGCG;
adev->pg_flags = AMD_PG_SUPPORT_SDMA |
AMD_PG_SUPPORT_MMHUB |
@@ -1244,6 +1486,21 @@ static int soc15_common_early_init(void *handle)
AMD_PG_SUPPORT_JPEG |
AMD_PG_SUPPORT_VCN_DPG;
break;
+ case CHIP_ALDEBARAN:
+ adev->asic_funcs = &vega20_asic_funcs;
+ adev->cg_flags = AMD_CG_SUPPORT_GFX_MGCG |
+ AMD_CG_SUPPORT_GFX_MGLS |
+ AMD_CG_SUPPORT_GFX_CGCG |
+ AMD_CG_SUPPORT_GFX_CGLS |
+ AMD_CG_SUPPORT_GFX_CP_LS |
+ AMD_CG_SUPPORT_HDP_LS |
+ AMD_CG_SUPPORT_SDMA_MGCG |
+ AMD_CG_SUPPORT_SDMA_LS |
+ AMD_CG_SUPPORT_IH_CG |
+ AMD_CG_SUPPORT_VCN_MGCG | AMD_CG_SUPPORT_JPEG_MGCG;
+ adev->pg_flags = AMD_PG_SUPPORT_VCN_DPG;
+ adev->external_rev_id = adev->rev_id + 0x3c;
+ break;
default:
/* FIXME: not supported yet */
return -EINVAL;
@@ -1268,8 +1525,9 @@ static int soc15_common_late_init(void *handle)
if (adev->hdp.funcs->reset_ras_error_count)
adev->hdp.funcs->reset_ras_error_count(adev);
- if (adev->nbio.funcs->ras_late_init)
- r = adev->nbio.funcs->ras_late_init(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->ras_late_init)
+ r = adev->nbio.ras_funcs->ras_late_init(adev);
return r;
}
@@ -1290,7 +1548,9 @@ static int soc15_common_sw_fini(void *handle)
{
struct amdgpu_device *adev = (struct amdgpu_device *)handle;
- amdgpu_nbio_ras_fini(adev);
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->ras_fini)
+ adev->nbio.ras_funcs->ras_fini(adev);
adev->df.funcs->sw_fini(adev);
return 0;
}
@@ -1354,9 +1614,11 @@ static int soc15_common_hw_fini(void *handle)
if (adev->nbio.ras_if &&
amdgpu_ras_is_supported(adev, adev->nbio.ras_if->block)) {
- if (adev->nbio.funcs->init_ras_controller_interrupt)
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_controller_interrupt)
amdgpu_irq_put(adev, &adev->nbio.ras_controller_irq, 0);
- if (adev->nbio.funcs->init_ras_err_event_athub_interrupt)
+ if (adev->nbio.ras_funcs &&
+ adev->nbio.ras_funcs->init_ras_err_event_athub_interrupt)
amdgpu_irq_put(adev, &adev->nbio.ras_err_event_athub_irq, 0);
}
@@ -1477,6 +1739,7 @@ static int soc15_common_set_clockgating_state(void *handle,
state == AMD_CG_STATE_GATE);
break;
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
adev->hdp.funcs->update_clock_gating(adev,
state == AMD_CG_STATE_GATE);
break;
@@ -1498,15 +1761,18 @@ static void soc15_common_get_clockgating_state(void *handle, u32 *flags)
adev->hdp.funcs->get_clock_gating_state(adev, flags);
- /* AMD_CG_SUPPORT_DRM_MGCG */
- data = RREG32(SOC15_REG_OFFSET(MP0, 0, mmMP0_MISC_CGTT_CTRL0));
- if (!(data & 0x01000000))
- *flags |= AMD_CG_SUPPORT_DRM_MGCG;
+ if (adev->asic_type != CHIP_ALDEBARAN) {
+
+ /* AMD_CG_SUPPORT_DRM_MGCG */
+ data = RREG32(SOC15_REG_OFFSET(MP0, 0, mmMP0_MISC_CGTT_CTRL0));
+ if (!(data & 0x01000000))
+ *flags |= AMD_CG_SUPPORT_DRM_MGCG;
- /* AMD_CG_SUPPORT_DRM_LS */
- data = RREG32(SOC15_REG_OFFSET(MP0, 0, mmMP0_MISC_LIGHT_SLEEP_CTRL));
- if (data & 0x1)
- *flags |= AMD_CG_SUPPORT_DRM_LS;
+ /* AMD_CG_SUPPORT_DRM_LS */
+ data = RREG32(SOC15_REG_OFFSET(MP0, 0, mmMP0_MISC_LIGHT_SLEEP_CTRL));
+ if (data & 0x1)
+ *flags |= AMD_CG_SUPPORT_DRM_LS;
+ }
/* AMD_CG_SUPPORT_ROM_MGCG */
adev->smuio.funcs->get_clock_gating_state(adev, flags);
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15.h b/drivers/gpu/drm/amd/amdgpu/soc15.h
index 8f38f047265b..034cfdfc4dbe 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15.h
+++ b/drivers/gpu/drm/amd/amdgpu/soc15.h
@@ -49,6 +49,13 @@ struct soc15_reg_rlcg {
u32 reg;
};
+struct soc15_reg {
+ uint32_t hwip;
+ uint32_t inst;
+ uint32_t seg;
+ uint32_t reg_offset;
+};
+
struct soc15_reg_entry {
uint32_t hwip;
uint32_t inst;
@@ -88,6 +95,10 @@ struct soc15_ras_field_entry {
#define SOC15_REG_FIELD(reg, field) reg##__##field##_MASK, reg##__##field##__SHIFT
+#define SOC15_REG_FIELD_VAL(val, mask, shift) (((val) & mask) >> shift)
+
+#define SOC15_RAS_REG_FIELD_VAL(val, entry, field) SOC15_REG_FIELD_VAL((val), (entry).field##_count_mask, (entry).field##_count_shift)
+
void soc15_grbm_select(struct amdgpu_device *adev,
u32 me, u32 pipe, u32 queue, u32 vmid);
void soc15_set_virt_ops(struct amdgpu_device *adev);
@@ -100,6 +111,7 @@ void soc15_program_register_sequence(struct amdgpu_device *adev,
int vega10_reg_base_init(struct amdgpu_device *adev);
int vega20_reg_base_init(struct amdgpu_device *adev);
int arct_reg_base_init(struct amdgpu_device *adev);
+int aldebaran_reg_base_init(struct amdgpu_device *adev);
void vega10_doorbell_index_init(struct amdgpu_device *adev);
void vega20_doorbell_index_init(struct amdgpu_device *adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/soc15_common.h b/drivers/gpu/drm/amd/amdgpu/soc15_common.h
index a5c00ab8b021..14bd794bbea6 100644
--- a/drivers/gpu/drm/amd/amdgpu/soc15_common.h
+++ b/drivers/gpu/drm/amd/amdgpu/soc15_common.h
@@ -77,13 +77,21 @@
})
#define WREG32_RLC(reg, value) \
+ do { \
+ if (adev->gfx.rlc.funcs->rlcg_wreg) \
+ adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, value, 0); \
+ else \
+ WREG32(reg, value); \
+ } while (0)
+
+#define WREG32_RLC_EX(prefix, reg, value) \
do { \
if (amdgpu_sriov_fullaccess(adev)) { \
uint32_t i = 0; \
uint32_t retries = 50000; \
- uint32_t r0 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG0_BASE_IDX] + mmSCRATCH_REG0; \
- uint32_t r1 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG1; \
- uint32_t spare_int = adev->reg_offset[GC_HWIP][0][mmRLC_SPARE_INT_BASE_IDX] + mmRLC_SPARE_INT; \
+ uint32_t r0 = adev->reg_offset[GC_HWIP][0][prefix##SCRATCH_REG0_BASE_IDX] + prefix##SCRATCH_REG0; \
+ uint32_t r1 = adev->reg_offset[GC_HWIP][0][prefix##SCRATCH_REG1_BASE_IDX] + prefix##SCRATCH_REG1; \
+ uint32_t spare_int = adev->reg_offset[GC_HWIP][0][prefix##RLC_SPARE_INT_BASE_IDX] + prefix##RLC_SPARE_INT; \
WREG32(r0, value); \
WREG32(r1, (reg | 0x80000000)); \
WREG32(spare_int, 0x1); \
@@ -101,13 +109,32 @@
} while (0)
#define WREG32_SOC15_RLC_SHADOW(ip, inst, reg, value) \
+ WREG32_RLC((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg), value)
+
+#define RREG32_RLC(reg) \
+ (adev->gfx.rlc.funcs->rlcg_rreg ? \
+ adev->gfx.rlc.funcs->rlcg_rreg(adev, reg, 0) : RREG32(reg))
+
+#define WREG32_RLC_NO_KIQ(reg, value) \
+ do { \
+ if (adev->gfx.rlc.funcs->rlcg_wreg) \
+ adev->gfx.rlc.funcs->rlcg_wreg(adev, reg, value, AMDGPU_REGS_NO_KIQ); \
+ else \
+ WREG32_NO_KIQ(reg, value); \
+ } while (0)
+
+#define RREG32_RLC_NO_KIQ(reg) \
+ (adev->gfx.rlc.funcs->rlcg_rreg ? \
+ adev->gfx.rlc.funcs->rlcg_rreg(adev, reg, AMDGPU_REGS_NO_KIQ) : RREG32_NO_KIQ(reg))
+
+#define WREG32_SOC15_RLC_SHADOW_EX(prefix, ip, inst, reg, value) \
do { \
uint32_t target_reg = adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg;\
if (amdgpu_sriov_fullaccess(adev)) { \
- uint32_t r2 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG2; \
- uint32_t r3 = adev->reg_offset[GC_HWIP][0][mmSCRATCH_REG1_BASE_IDX] + mmSCRATCH_REG3; \
- uint32_t grbm_cntl = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_CNTL_BASE_IDX] + mmGRBM_GFX_CNTL; \
- uint32_t grbm_idx = adev->reg_offset[GC_HWIP][0][mmGRBM_GFX_INDEX_BASE_IDX] + mmGRBM_GFX_INDEX; \
+ uint32_t r2 = adev->reg_offset[GC_HWIP][0][prefix##SCRATCH_REG1_BASE_IDX] + prefix##SCRATCH_REG2; \
+ uint32_t r3 = adev->reg_offset[GC_HWIP][0][prefix##SCRATCH_REG1_BASE_IDX] + prefix##SCRATCH_REG3; \
+ uint32_t grbm_cntl = adev->reg_offset[GC_HWIP][0][prefix##GRBM_GFX_CNTL_BASE_IDX] + prefix##GRBM_GFX_CNTL; \
+ uint32_t grbm_idx = adev->reg_offset[GC_HWIP][0][prefix##GRBM_GFX_INDEX_BASE_IDX] + prefix##GRBM_GFX_INDEX; \
if (target_reg == grbm_cntl) \
WREG32(r2, value); \
else if (target_reg == grbm_idx) \
@@ -118,18 +145,30 @@
} \
} while (0)
+#define RREG32_SOC15_RLC(ip, inst, reg) \
+ RREG32_RLC(adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg)
+
#define WREG32_SOC15_RLC(ip, inst, reg, value) \
do { \
+ uint32_t target_reg = adev->reg_offset[ip##_HWIP][0][reg##_BASE_IDX] + reg;\
+ WREG32_RLC(target_reg, value); \
+ } while (0)
+
+#define WREG32_SOC15_RLC_EX(prefix, ip, inst, reg, value) \
+ do { \
uint32_t target_reg = adev->reg_offset[GC_HWIP][0][reg##_BASE_IDX] + reg;\
- WREG32_RLC(target_reg, value); \
+ WREG32_RLC_EX(prefix, target_reg, value); \
} while (0)
#define WREG32_FIELD15_RLC(ip, idx, reg, field, val) \
- WREG32_RLC((adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg), \
- (RREG32(adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg) \
- & ~REG_FIELD_MASK(reg, field)) | (val) << REG_FIELD_SHIFT(reg, field))
+ WREG32_RLC((adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg), \
+ (RREG32_RLC(adev->reg_offset[ip##_HWIP][idx][mm##reg##_BASE_IDX] + mm##reg) \
+ & ~REG_FIELD_MASK(reg, field)) | (val) << REG_FIELD_SHIFT(reg, field))
#define WREG32_SOC15_OFFSET_RLC(ip, inst, reg, offset, value) \
- WREG32_RLC(((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) + offset), value)
+ WREG32_RLC(((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) + offset), value)
+
+#define RREG32_SOC15_OFFSET_RLC(ip, inst, reg, offset) \
+ RREG32_RLC(((adev->reg_offset[ip##_HWIP][inst][reg##_BASE_IDX] + reg) + offset))
#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/ta_secureDisplay_if.h b/drivers/gpu/drm/amd/amdgpu/ta_secureDisplay_if.h
index 5039375bb1d4..cf8ff064dc72 100644
--- a/drivers/gpu/drm/amd/amdgpu/ta_secureDisplay_if.h
+++ b/drivers/gpu/drm/amd/amdgpu/ta_secureDisplay_if.h
@@ -50,6 +50,7 @@ enum ta_securedisplay_status {
TA_SECUREDISPLAY_STATUS__I2C_WRITE_ERROR = 0x04, /* Fail to Write to I2C */
TA_SECUREDISPLAY_STATUS__READ_DIO_SCRATCH_ERROR = 0x05, /*Fail Read DIO Scratch Register*/
TA_SECUREDISPLAY_STATUS__READ_CRC_ERROR = 0x06, /* Fail to Read CRC*/
+ TA_SECUREDISPLAY_STATUS__I2C_INIT_ERROR = 0x07, /* Failed to initialize I2C */
TA_SECUREDISPLAY_STATUS__MAX = 0x7FFFFFFF,/* Maximum Value for status*/
};
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v6_1.c b/drivers/gpu/drm/amd/amdgpu/umc_v6_1.c
index 96d7769609f4..20b44983ac94 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v6_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v6_1.c
@@ -22,6 +22,7 @@
*/
#include "umc_v6_1.h"
#include "amdgpu_ras.h"
+#include "amdgpu_umc.h"
#include "amdgpu.h"
#include "rsmu/rsmu_0_0_2_offset.h"
@@ -464,9 +465,10 @@ static void umc_v6_1_err_cnt_init(struct amdgpu_device *adev)
umc_v6_1_enable_umc_index_mode(adev);
}
-const struct amdgpu_umc_funcs umc_v6_1_funcs = {
+const struct amdgpu_umc_ras_funcs umc_v6_1_ras_funcs = {
.err_cnt_init = umc_v6_1_err_cnt_init,
.ras_late_init = amdgpu_umc_ras_late_init,
+ .ras_fini = amdgpu_umc_ras_fini,
.query_ras_error_count = umc_v6_1_query_ras_error_count,
.query_ras_error_address = umc_v6_1_query_ras_error_address,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v6_1.h b/drivers/gpu/drm/amd/amdgpu/umc_v6_1.h
index 0ce1d323cfdd..5dc36c730bb2 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v6_1.h
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v6_1.h
@@ -45,7 +45,7 @@
/* umc ce count initial value */
#define UMC_V6_1_CE_CNT_INIT (UMC_V6_1_CE_CNT_MAX - UMC_V6_1_CE_INT_THRESHOLD)
-extern const struct amdgpu_umc_funcs umc_v6_1_funcs;
+extern const struct amdgpu_umc_ras_funcs umc_v6_1_ras_funcs;
extern const uint32_t
umc_v6_1_channel_idx_tbl[UMC_V6_1_UMC_INSTANCE_NUM][UMC_V6_1_CHANNEL_INSTANCE_NUM];
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
new file mode 100644
index 000000000000..3a8f787374c0
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.c
@@ -0,0 +1,281 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#include "umc_v6_7.h"
+#include "amdgpu_ras.h"
+#include "amdgpu_umc.h"
+#include "amdgpu.h"
+
+#include "umc/umc_6_7_0_offset.h"
+#include "umc/umc_6_7_0_sh_mask.h"
+
+static inline uint32_t get_umc_v6_7_reg_offset(struct amdgpu_device *adev,
+ uint32_t umc_inst,
+ uint32_t ch_inst)
+{
+ return adev->umc.channel_offs * ch_inst + UMC_V6_7_INST_DIST * umc_inst;
+}
+
+static void umc_v6_7_query_correctable_error_count(struct amdgpu_device *adev,
+ uint32_t umc_reg_offset,
+ unsigned long *error_count)
+{
+ uint32_t ecc_err_cnt_sel, ecc_err_cnt_sel_addr;
+ uint32_t ecc_err_cnt, ecc_err_cnt_addr;
+ uint64_t mc_umc_status;
+ uint32_t mc_umc_status_addr;
+
+ /* UMC 6_1_1 registers */
+ ecc_err_cnt_sel_addr =
+ SOC15_REG_OFFSET(UMC, 0, regUMCCH0_0_EccErrCntSel);
+ ecc_err_cnt_addr =
+ SOC15_REG_OFFSET(UMC, 0, regUMCCH0_0_EccErrCnt);
+ mc_umc_status_addr =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
+
+ /* select the lower chip and check the error count */
+ ecc_err_cnt_sel = RREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4);
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel, UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 0);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4, ecc_err_cnt_sel);
+
+ ecc_err_cnt = RREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4);
+ *error_count +=
+ (REG_GET_FIELD(ecc_err_cnt, UMCCH0_0_EccErrCnt, EccErrCnt) -
+ UMC_V6_7_CE_CNT_INIT);
+
+ /* select the higher chip and check the err counter */
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel, UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 1);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4, ecc_err_cnt_sel);
+
+ ecc_err_cnt = RREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4);
+ *error_count +=
+ (REG_GET_FIELD(ecc_err_cnt, UMCCH0_0_EccErrCnt, EccErrCnt) -
+ UMC_V6_7_CE_CNT_INIT);
+
+ /* check for SRAM correctable error
+ MCUMC_STATUS is a 64 bit register */
+ mc_umc_status = RREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4);
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)
+ *error_count += 1;
+}
+
+static void umc_v6_7_querry_uncorrectable_error_count(struct amdgpu_device *adev,
+ uint32_t umc_reg_offset,
+ unsigned long *error_count)
+{
+ uint64_t mc_umc_status;
+ uint32_t mc_umc_status_addr;
+
+ mc_umc_status_addr =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
+
+ /* check the MCUMC_STATUS */
+ mc_umc_status = RREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4);
+ if ((REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1) &&
+ (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Deferred) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, PCC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, TCC) == 1))
+ *error_count += 1;
+}
+
+static void umc_v6_7_reset_error_count_per_channel(struct amdgpu_device *adev,
+ uint32_t umc_reg_offset)
+{
+ uint32_t ecc_err_cnt_addr;
+ uint32_t ecc_err_cnt_sel, ecc_err_cnt_sel_addr;
+
+ ecc_err_cnt_sel_addr =
+ SOC15_REG_OFFSET(UMC, 0,
+ regUMCCH0_0_EccErrCntSel);
+ ecc_err_cnt_addr =
+ SOC15_REG_OFFSET(UMC, 0,
+ regUMCCH0_0_EccErrCnt);
+
+ /* select the lower chip */
+ ecc_err_cnt_sel = RREG32_PCIE((ecc_err_cnt_sel_addr +
+ umc_reg_offset) * 4);
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel,
+ UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 0);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4,
+ ecc_err_cnt_sel);
+
+ /* clear lower chip error count */
+ WREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4,
+ UMC_V6_7_CE_CNT_INIT);
+
+ /* select the higher chip */
+ ecc_err_cnt_sel = RREG32_PCIE((ecc_err_cnt_sel_addr +
+ umc_reg_offset) * 4);
+ ecc_err_cnt_sel = REG_SET_FIELD(ecc_err_cnt_sel,
+ UMCCH0_0_EccErrCntSel,
+ EccErrCntCsSel, 1);
+ WREG32_PCIE((ecc_err_cnt_sel_addr + umc_reg_offset) * 4,
+ ecc_err_cnt_sel);
+
+ /* clear higher chip error count */
+ WREG32_PCIE((ecc_err_cnt_addr + umc_reg_offset) * 4,
+ UMC_V6_7_CE_CNT_INIT);
+}
+
+static void umc_v6_7_reset_error_count(struct amdgpu_device *adev)
+{
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+
+ umc_v6_7_reset_error_count_per_channel(adev,
+ umc_reg_offset);
+ }
+}
+
+static void umc_v6_7_query_ras_error_count(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+
+ /*TODO: driver needs to toggle DF Cstate to ensure
+ * safe access of UMC registers. Will add the protection */
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+ umc_v6_7_query_correctable_error_count(adev,
+ umc_reg_offset,
+ &(err_data->ce_count));
+ umc_v6_7_querry_uncorrectable_error_count(adev,
+ umc_reg_offset,
+ &(err_data->ue_count));
+ }
+
+ umc_v6_7_reset_error_count(adev);
+}
+
+static void umc_v6_7_query_error_address(struct amdgpu_device *adev,
+ struct ras_err_data *err_data,
+ uint32_t umc_reg_offset,
+ uint32_t ch_inst,
+ uint32_t umc_inst)
+{
+ uint32_t mc_umc_status_addr;
+ uint64_t mc_umc_status, err_addr, retired_page, mc_umc_addrt0;
+ struct eeprom_table_record *err_rec;
+ uint32_t channel_index;
+
+ mc_umc_status_addr =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_STATUST0);
+ mc_umc_addrt0 =
+ SOC15_REG_OFFSET(UMC, 0, regMCA_UMC_UMC0_MCUMC_ADDRT0);
+
+ mc_umc_status = RREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4);
+
+ if (mc_umc_status == 0)
+ return;
+
+ if (!err_data->err_addr) {
+ /* clear umc status */
+ WREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4, 0x0ULL);
+ return;
+ }
+
+ err_rec = &err_data->err_addr[err_data->err_addr_cnt];
+
+ channel_index =
+ adev->umc.channel_idx_tbl[umc_inst * adev->umc.channel_inst_num + ch_inst];
+
+ /* calculate error address if ue/ce error is detected */
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, Val) == 1 &&
+ (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC) == 1 ||
+ REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, CECC) == 1)) {
+
+ err_addr = RREG64_PCIE((mc_umc_addrt0 + umc_reg_offset) * 4);
+ err_addr = REG_GET_FIELD(err_addr, MCA_UMC_UMC0_MCUMC_ADDRT0, ErrorAddr);
+
+ /* translate umc channel address to soc pa, 3 parts are included */
+ retired_page = ADDR_OF_8KB_BLOCK(err_addr) |
+ ADDR_OF_256B_BLOCK(channel_index) |
+ OFFSET_IN_256B_BLOCK(err_addr);
+
+ /* we only save ue error information currently, ce is skipped */
+ if (REG_GET_FIELD(mc_umc_status, MCA_UMC_UMC0_MCUMC_STATUST0, UECC)
+ == 1) {
+ err_rec->address = err_addr;
+ /* page frame address is saved */
+ err_rec->retired_page = retired_page >> AMDGPU_GPU_PAGE_SHIFT;
+ err_rec->ts = (uint64_t)ktime_get_real_seconds();
+ err_rec->err_type = AMDGPU_RAS_EEPROM_ERR_NON_RECOVERABLE;
+ err_rec->cu = 0;
+ err_rec->mem_channel = channel_index;
+ err_rec->mcumc_id = umc_inst;
+
+ err_data->err_addr_cnt++;
+ }
+ }
+
+ /* clear umc status */
+ WREG64_PCIE((mc_umc_status_addr + umc_reg_offset) * 4, 0x0ULL);
+}
+
+static void umc_v6_7_query_ras_error_address(struct amdgpu_device *adev,
+ void *ras_error_status)
+{
+ struct ras_err_data *err_data = (struct ras_err_data *)ras_error_status;
+
+ uint32_t umc_inst = 0;
+ uint32_t ch_inst = 0;
+ uint32_t umc_reg_offset = 0;
+
+ /*TODO: driver needs to toggle DF Cstate to ensure
+ * safe access of UMC resgisters. Will add the protection
+ * when firmware interface is ready */
+ LOOP_UMC_INST_AND_CH(umc_inst, ch_inst) {
+ umc_reg_offset = get_umc_v6_7_reg_offset(adev,
+ umc_inst,
+ ch_inst);
+ umc_v6_7_query_error_address(adev,
+ err_data,
+ umc_reg_offset,
+ ch_inst,
+ umc_inst);
+ }
+}
+
+const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs = {
+ .ras_late_init = amdgpu_umc_ras_late_init,
+ .ras_fini = amdgpu_umc_ras_fini,
+ .query_ras_error_count = umc_v6_7_query_ras_error_count,
+ .query_ras_error_address = umc_v6_7_query_ras_error_address,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v6_7.h b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.h
new file mode 100644
index 000000000000..4eb85f247e96
--- /dev/null
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v6_7.h
@@ -0,0 +1,37 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef __UMC_V6_7_H__
+#define __UMC_V6_7_H__
+
+/* EccErrCnt max value */
+#define UMC_V6_7_CE_CNT_MAX 0xffff
+/* umc ce interrupt threshold */
+#define UMC_V6_7_CE_INT_THRESHOLD 0xffff
+/* umc ce count initial value */
+#define UMC_V6_7_CE_CNT_INIT (UMC_V6_7_CE_CNT_MAX - UMC_V6_7_CE_INT_THRESHOLD)
+
+#define UMC_V6_7_INST_DIST 0x40000
+
+extern const struct amdgpu_umc_ras_funcs umc_v6_7_ras_funcs;
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v8_7.c b/drivers/gpu/drm/amd/amdgpu/umc_v8_7.c
index a064c097690c..89d20adfa001 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v8_7.c
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v8_7.c
@@ -22,6 +22,7 @@
*/
#include "umc_v8_7.h"
#include "amdgpu_ras.h"
+#include "amdgpu_umc.h"
#include "amdgpu.h"
#include "rsmu/rsmu_0_0_2_offset.h"
@@ -323,9 +324,10 @@ static void umc_v8_7_err_cnt_init(struct amdgpu_device *adev)
}
}
-const struct amdgpu_umc_funcs umc_v8_7_funcs = {
+const struct amdgpu_umc_ras_funcs umc_v8_7_ras_funcs = {
.err_cnt_init = umc_v8_7_err_cnt_init,
.ras_late_init = amdgpu_umc_ras_late_init,
+ .ras_fini = amdgpu_umc_ras_fini,
.query_ras_error_count = umc_v8_7_query_ras_error_count,
.query_ras_error_address = umc_v8_7_query_ras_error_address,
};
diff --git a/drivers/gpu/drm/amd/amdgpu/umc_v8_7.h b/drivers/gpu/drm/amd/amdgpu/umc_v8_7.h
index d4d0468e3df5..37e6dc7c28e0 100644
--- a/drivers/gpu/drm/amd/amdgpu/umc_v8_7.h
+++ b/drivers/gpu/drm/amd/amdgpu/umc_v8_7.h
@@ -44,7 +44,7 @@
/* umc ce count initial value */
#define UMC_V8_7_CE_CNT_INIT (UMC_V8_7_CE_CNT_MAX - UMC_V8_7_CE_INT_THRESHOLD)
-extern const struct amdgpu_umc_funcs umc_v8_7_funcs;
+extern const struct amdgpu_umc_ras_funcs umc_v8_7_ras_funcs;
extern const uint32_t
umc_v8_7_channel_idx_tbl[UMC_V8_7_UMC_INSTANCE_NUM][UMC_V8_7_CHANNEL_INSTANCE_NUM];
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c b/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c
index 10ecae257b18..284447d7a579 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v3_1.c
@@ -562,7 +562,7 @@ static int uvd_v3_1_sw_init(void *handle)
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
index a70d2a0de316..a301518e4957 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v4_2.c
@@ -119,7 +119,7 @@ static int uvd_v4_2_sw_init(void *handle)
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
index f3b0a927101b..a4d5bd21c83c 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v5_0.c
@@ -117,7 +117,7 @@ static int uvd_v5_0_sw_init(void *handle)
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
index 760859880c1e..2bab9c77952f 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v6_0.c
@@ -420,7 +420,7 @@ static int uvd_v6_0_sw_init(void *handle)
ring = &adev->uvd.inst->ring;
sprintf(ring->name, "uvd");
r = amdgpu_ring_init(adev, ring, 512, &adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
@@ -434,7 +434,7 @@ static int uvd_v6_0_sw_init(void *handle)
sprintf(ring->name, "uvd_enc%d", i);
r = amdgpu_ring_init(adev, ring, 512,
&adev->uvd.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
index 7cd67cb2ac5f..0cd98fcb1f9f 100644
--- a/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/uvd_v7_0.c
@@ -454,7 +454,7 @@ static int uvd_v7_0_sw_init(void *handle)
sprintf(ring->name, "uvd_%d", ring->me);
r = amdgpu_ring_init(adev, ring, 512,
&adev->uvd.inst[j].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -475,7 +475,7 @@ static int uvd_v7_0_sw_init(void *handle)
}
r = amdgpu_ring_init(adev, ring, 512,
&adev->uvd.inst[j].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
index 0e2945baf0f1..c7d28c169be5 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v2_0.c
@@ -433,9 +433,8 @@ static int vce_v2_0_sw_init(void *handle)
for (i = 0; i < adev->vce.num_rings; i++) {
ring = &adev->vce.ring[i];
sprintf(ring->name, "vce%d", i);
- r = amdgpu_ring_init(adev, ring, 512,
- &adev->vce.irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ r = amdgpu_ring_init(adev, ring, 512, &adev->vce.irq, 0,
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
index 6d9108fa22e0..3b82fb289ef6 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v3_0.c
@@ -443,7 +443,7 @@ static int vce_v3_0_sw_init(void *handle)
ring = &adev->vce.ring[i];
sprintf(ring->name, "vce%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vce.irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
index 37fa163393fd..8e238dea7bef 100644
--- a/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vce_v4_0.c
@@ -477,7 +477,7 @@ static int vce_v4_0_sw_init(void *handle)
ring->doorbell_index = adev->doorbell_index.uvd_vce.vce_ring2_3 * 2 + 1;
}
r = amdgpu_ring_init(adev, ring, 512, &adev->vce.irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
index 6117931fa8d7..0c1beefa3e49 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v1_0.c
@@ -129,7 +129,7 @@ static int vcn_v1_0_sw_init(void *handle)
ring = &adev->vcn.inst->ring_dec;
sprintf(ring->name, "vcn_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
@@ -148,7 +148,7 @@ static int vcn_v1_0_sw_init(void *handle)
ring = &adev->vcn.inst->ring_enc[i];
sprintf(ring->name, "vcn_enc%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -1119,10 +1119,10 @@ static int vcn_v1_0_stop_spg_mode(struct amdgpu_device *adev)
UVD_LMI_STATUS__WRITE_CLEAN_RAW_MASK;
SOC15_WAIT_ON_RREG(UVD, 0, mmUVD_LMI_STATUS, tmp, tmp);
- /* put VCPU into reset */
- WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_SOFT_RESET),
- UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK,
- ~UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK);
+ /* stall UMC channel */
+ WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_LMI_CTRL2),
+ UVD_LMI_CTRL2__STALL_ARB_UMC_MASK,
+ ~UVD_LMI_CTRL2__STALL_ARB_UMC_MASK);
tmp = UVD_LMI_STATUS__UMC_READ_CLEAN_RAW_MASK |
UVD_LMI_STATUS__UMC_WRITE_CLEAN_RAW_MASK;
@@ -1141,6 +1141,11 @@ static int vcn_v1_0_stop_spg_mode(struct amdgpu_device *adev)
UVD_SOFT_RESET__LMI_SOFT_RESET_MASK,
~UVD_SOFT_RESET__LMI_SOFT_RESET_MASK);
+ /* put VCPU into reset */
+ WREG32_P(SOC15_REG_OFFSET(UVD, 0, mmUVD_SOFT_RESET),
+ UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK,
+ ~UVD_SOFT_RESET__VCPU_SOFT_RESET_MASK);
+
WREG32_SOC15(UVD, 0, mmUVD_STATUS, 0);
vcn_v1_0_enable_clock_gating(adev);
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c
index d63198c945bf..116b9643d5ba 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v2_0.c
@@ -136,7 +136,7 @@ static int vcn_v2_0_sw_init(void *handle)
sprintf(ring->name, "vcn_dec");
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
@@ -167,7 +167,7 @@ static int vcn_v2_0_sw_init(void *handle)
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 1 + i;
sprintf(ring->name, "vcn_enc%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst->irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
index b6e0f4ba6272..948813d7caa0 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.c
@@ -189,7 +189,7 @@ static int vcn_v2_5_sw_init(void *handle)
(amdgpu_sriov_vf(adev) ? 2*j : 8*j);
sprintf(ring->name, "vcn_dec_%d", j);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst[j].irq,
- 0, AMDGPU_RING_PRIO_DEFAULT);
+ 0, AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
@@ -203,7 +203,7 @@ static int vcn_v2_5_sw_init(void *handle)
sprintf(ring->name, "vcn_enc_%d.%d", j, i);
r = amdgpu_ring_init(adev, ring, 512,
&adev->vcn.inst[j].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT, NULL);
if (r)
return r;
}
@@ -1545,6 +1545,36 @@ static const struct amdgpu_ring_funcs vcn_v2_5_dec_ring_vm_funcs = {
.emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
};
+static const struct amdgpu_ring_funcs vcn_v2_6_dec_ring_vm_funcs = {
+ .type = AMDGPU_RING_TYPE_VCN_DEC,
+ .align_mask = 0xf,
+ .vmhub = AMDGPU_MMHUB_0,
+ .get_rptr = vcn_v2_5_dec_ring_get_rptr,
+ .get_wptr = vcn_v2_5_dec_ring_get_wptr,
+ .set_wptr = vcn_v2_5_dec_ring_set_wptr,
+ .emit_frame_size =
+ SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 +
+ SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 +
+ 8 + /* vcn_v2_0_dec_ring_emit_vm_flush */
+ 14 + 14 + /* vcn_v2_0_dec_ring_emit_fence x2 vm fence */
+ 6,
+ .emit_ib_size = 8, /* vcn_v2_0_dec_ring_emit_ib */
+ .emit_ib = vcn_v2_0_dec_ring_emit_ib,
+ .emit_fence = vcn_v2_0_dec_ring_emit_fence,
+ .emit_vm_flush = vcn_v2_0_dec_ring_emit_vm_flush,
+ .test_ring = vcn_v2_0_dec_ring_test_ring,
+ .test_ib = amdgpu_vcn_dec_ring_test_ib,
+ .insert_nop = vcn_v2_0_dec_ring_insert_nop,
+ .insert_start = vcn_v2_0_dec_ring_insert_start,
+ .insert_end = vcn_v2_0_dec_ring_insert_end,
+ .pad_ib = amdgpu_ring_generic_pad_ib,
+ .begin_use = amdgpu_vcn_ring_begin_use,
+ .end_use = amdgpu_vcn_ring_end_use,
+ .emit_wreg = vcn_v2_0_dec_ring_emit_wreg,
+ .emit_reg_wait = vcn_v2_0_dec_ring_emit_reg_wait,
+ .emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
+};
+
/**
* vcn_v2_5_enc_ring_get_rptr - get enc read pointer
*
@@ -1644,6 +1674,36 @@ static const struct amdgpu_ring_funcs vcn_v2_5_enc_ring_vm_funcs = {
.emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
};
+static const struct amdgpu_ring_funcs vcn_v2_6_enc_ring_vm_funcs = {
+ .type = AMDGPU_RING_TYPE_VCN_ENC,
+ .align_mask = 0x3f,
+ .nop = VCN_ENC_CMD_NO_OP,
+ .vmhub = AMDGPU_MMHUB_0,
+ .get_rptr = vcn_v2_5_enc_ring_get_rptr,
+ .get_wptr = vcn_v2_5_enc_ring_get_wptr,
+ .set_wptr = vcn_v2_5_enc_ring_set_wptr,
+ .emit_frame_size =
+ SOC15_FLUSH_GPU_TLB_NUM_WREG * 3 +
+ SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 4 +
+ 4 + /* vcn_v2_0_enc_ring_emit_vm_flush */
+ 5 + 5 + /* vcn_v2_0_enc_ring_emit_fence x2 vm fence */
+ 1, /* vcn_v2_0_enc_ring_insert_end */
+ .emit_ib_size = 5, /* vcn_v2_0_enc_ring_emit_ib */
+ .emit_ib = vcn_v2_0_enc_ring_emit_ib,
+ .emit_fence = vcn_v2_0_enc_ring_emit_fence,
+ .emit_vm_flush = vcn_v2_0_enc_ring_emit_vm_flush,
+ .test_ring = amdgpu_vcn_enc_ring_test_ring,
+ .test_ib = amdgpu_vcn_enc_ring_test_ib,
+ .insert_nop = amdgpu_ring_insert_nop,
+ .insert_end = vcn_v2_0_enc_ring_insert_end,
+ .pad_ib = amdgpu_ring_generic_pad_ib,
+ .begin_use = amdgpu_vcn_ring_begin_use,
+ .end_use = amdgpu_vcn_ring_end_use,
+ .emit_wreg = vcn_v2_0_enc_ring_emit_wreg,
+ .emit_reg_wait = vcn_v2_0_enc_ring_emit_reg_wait,
+ .emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
+};
+
static void vcn_v2_5_set_dec_ring_funcs(struct amdgpu_device *adev)
{
int i;
@@ -1651,7 +1711,10 @@ static void vcn_v2_5_set_dec_ring_funcs(struct amdgpu_device *adev)
for (i = 0; i < adev->vcn.num_vcn_inst; ++i) {
if (adev->vcn.harvest_config & (1 << i))
continue;
- adev->vcn.inst[i].ring_dec.funcs = &vcn_v2_5_dec_ring_vm_funcs;
+ if (adev->asic_type == CHIP_ARCTURUS)
+ adev->vcn.inst[i].ring_dec.funcs = &vcn_v2_5_dec_ring_vm_funcs;
+ else /* CHIP_ALDEBARAN */
+ adev->vcn.inst[i].ring_dec.funcs = &vcn_v2_6_dec_ring_vm_funcs;
adev->vcn.inst[i].ring_dec.me = i;
DRM_INFO("VCN(%d) decode is enabled in VM mode\n", i);
}
@@ -1665,7 +1728,10 @@ static void vcn_v2_5_set_enc_ring_funcs(struct amdgpu_device *adev)
if (adev->vcn.harvest_config & (1 << j))
continue;
for (i = 0; i < adev->vcn.num_enc_rings; ++i) {
- adev->vcn.inst[j].ring_enc[i].funcs = &vcn_v2_5_enc_ring_vm_funcs;
+ if (adev->asic_type == CHIP_ARCTURUS)
+ adev->vcn.inst[j].ring_enc[i].funcs = &vcn_v2_5_enc_ring_vm_funcs;
+ else /* CHIP_ALDEBARAN */
+ adev->vcn.inst[j].ring_enc[i].funcs = &vcn_v2_6_enc_ring_vm_funcs;
adev->vcn.inst[j].ring_enc[i].me = j;
}
DRM_INFO("VCN(%d) encode is enabled in VM mode\n", j);
@@ -1830,6 +1896,26 @@ static const struct amd_ip_funcs vcn_v2_5_ip_funcs = {
.set_powergating_state = vcn_v2_5_set_powergating_state,
};
+static const struct amd_ip_funcs vcn_v2_6_ip_funcs = {
+ .name = "vcn_v2_6",
+ .early_init = vcn_v2_5_early_init,
+ .late_init = NULL,
+ .sw_init = vcn_v2_5_sw_init,
+ .sw_fini = vcn_v2_5_sw_fini,
+ .hw_init = vcn_v2_5_hw_init,
+ .hw_fini = vcn_v2_5_hw_fini,
+ .suspend = vcn_v2_5_suspend,
+ .resume = vcn_v2_5_resume,
+ .is_idle = vcn_v2_5_is_idle,
+ .wait_for_idle = vcn_v2_5_wait_for_idle,
+ .check_soft_reset = NULL,
+ .pre_soft_reset = NULL,
+ .soft_reset = NULL,
+ .post_soft_reset = NULL,
+ .set_clockgating_state = vcn_v2_5_set_clockgating_state,
+ .set_powergating_state = vcn_v2_5_set_powergating_state,
+};
+
const struct amdgpu_ip_block_version vcn_v2_5_ip_block =
{
.type = AMD_IP_BLOCK_TYPE_VCN,
@@ -1838,3 +1924,12 @@ const struct amdgpu_ip_block_version vcn_v2_5_ip_block =
.rev = 0,
.funcs = &vcn_v2_5_ip_funcs,
};
+
+const struct amdgpu_ip_block_version vcn_v2_6_ip_block =
+{
+ .type = AMD_IP_BLOCK_TYPE_VCN,
+ .major = 2,
+ .minor = 6,
+ .rev = 0,
+ .funcs = &vcn_v2_6_ip_funcs,
+};
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.h b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.h
index 8d9c0800b8e0..e72f799ed0fd 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.h
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v2_5.h
@@ -25,5 +25,6 @@
#define __VCN_V2_5_H__
extern const struct amdgpu_ip_block_version vcn_v2_5_ip_block;
+extern const struct amdgpu_ip_block_version vcn_v2_6_ip_block;
#endif /* __VCN_V2_5_H__ */
diff --git a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
index def583916294..cf165ab5dd26 100644
--- a/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
+++ b/drivers/gpu/drm/amd/amdgpu/vcn_v3_0.c
@@ -50,6 +50,9 @@
#define VCN_INSTANCES_SIENNA_CICHLID 2
#define DEC_SW_RING_ENABLED FALSE
+#define RDECODE_MSG_CREATE 0x00000000
+#define RDECODE_MESSAGE_CREATE 0x00000001
+
static int amdgpu_ih_clientid_vcns[] = {
SOC15_IH_CLIENTID_VCN,
SOC15_IH_CLIENTID_VCN1
@@ -171,6 +174,7 @@ static int vcn_v3_0_sw_init(void *handle)
for (i = 0; i < adev->vcn.num_vcn_inst; i++) {
volatile struct amdgpu_fw_shared *fw_shared;
+
if (adev->vcn.harvest_config & (1 << i))
continue;
@@ -198,6 +202,8 @@ static int vcn_v3_0_sw_init(void *handle)
if (r)
return r;
+ atomic_set(&adev->vcn.inst[i].sched_score, 0);
+
ring = &adev->vcn.inst[i].ring_dec;
ring->use_doorbell = true;
if (amdgpu_sriov_vf(adev)) {
@@ -205,11 +211,10 @@ static int vcn_v3_0_sw_init(void *handle)
} else {
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 8 * i;
}
- if (adev->asic_type == CHIP_SIENNA_CICHLID && i != 0)
- ring->no_scheduler = true;
sprintf(ring->name, "vcn_dec_%d", i);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst[i].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT,
+ &adev->vcn.inst[i].sched_score);
if (r)
return r;
@@ -227,18 +232,18 @@ static int vcn_v3_0_sw_init(void *handle)
} else {
ring->doorbell_index = (adev->doorbell_index.vcn.vcn_ring0_1 << 1) + 2 + j + 8 * i;
}
- if (adev->asic_type == CHIP_SIENNA_CICHLID && i != 1)
- ring->no_scheduler = true;
sprintf(ring->name, "vcn_enc_%d.%d", i, j);
r = amdgpu_ring_init(adev, ring, 512, &adev->vcn.inst[i].irq, 0,
- AMDGPU_RING_PRIO_DEFAULT);
+ AMDGPU_RING_PRIO_DEFAULT,
+ &adev->vcn.inst[i].sched_score);
if (r)
return r;
}
fw_shared = adev->vcn.inst[i].fw_shared_cpu_addr;
fw_shared->present_flag_0 |= cpu_to_le32(AMDGPU_VCN_SW_RING_FLAG) |
- cpu_to_le32(AMDGPU_VCN_MULTI_QUEUE_FLAG);
+ cpu_to_le32(AMDGPU_VCN_MULTI_QUEUE_FLAG) |
+ cpu_to_le32(AMDGPU_VCN_FW_SHARED_FLAG_0_RB);
fw_shared->sw_ring.is_enabled = cpu_to_le32(DEC_SW_RING_ENABLED);
}
@@ -584,6 +589,10 @@ static void vcn_v3_0_mc_resume_dpg_mode(struct amdgpu_device *adev, int inst_idx
WREG32_SOC15_DPG_MODE(inst_idx, SOC15_DPG_MODE_OFFSET(
VCN, inst_idx, mmUVD_VCPU_NONCACHE_SIZE0),
AMDGPU_GPU_PAGE_ALIGN(sizeof(struct amdgpu_fw_shared)), 0, indirect);
+
+ /* VCN global tiling registers */
+ WREG32_SOC15_DPG_MODE(0, SOC15_DPG_MODE_OFFSET(
+ UVD, 0, mmUVD_GFX10_ADDR_CONFIG), adev->gfx.config.gb_addr_config, 0, indirect);
}
static void vcn_v3_0_disable_static_power_gating(struct amdgpu_device *adev, int inst)
@@ -1074,7 +1083,13 @@ static int vcn_v3_0_start_dpg_mode(struct amdgpu_device *adev, int inst_idx, boo
WREG32_SOC15(VCN, inst_idx, mmUVD_RBC_RB_WPTR,
lower_32_bits(ring->wptr));
+ /* Reset FW shared memory RBC WPTR/RPTR */
+ fw_shared->rb.rptr = 0;
+ fw_shared->rb.wptr = lower_32_bits(ring->wptr);
+
+ /*resetting done, fw can check RB ring */
fw_shared->multi_queue.decode_queue_mode &= cpu_to_le32(~FW_QUEUE_RING_RESET);
+
/* Unstall DPG */
WREG32_P(SOC15_REG_OFFSET(VCN, inst_idx, mmUVD_POWER_STATUS),
0, ~UVD_POWER_STATUS__STALL_DPG_POWER_UP_MASK);
@@ -1239,9 +1254,11 @@ static int vcn_v3_0_start(struct amdgpu_device *adev)
/* Initialize the ring buffer's read and write pointers */
WREG32_SOC15(VCN, i, mmUVD_RBC_RB_RPTR, 0);
+ WREG32_SOC15(VCN, i, mmUVD_SCRATCH2, 0);
ring->wptr = RREG32_SOC15(VCN, i, mmUVD_RBC_RB_RPTR);
WREG32_SOC15(VCN, i, mmUVD_RBC_RB_WPTR,
lower_32_bits(ring->wptr));
+ fw_shared->rb.wptr = lower_32_bits(ring->wptr);
fw_shared->multi_queue.decode_queue_mode &= cpu_to_le32(~FW_QUEUE_RING_RESET);
fw_shared->multi_queue.encode_generalpurpose_queue_mode |= cpu_to_le32(FW_QUEUE_RING_RESET);
@@ -1662,6 +1679,10 @@ static int vcn_v3_0_pause_dpg_mode(struct amdgpu_device *adev,
WREG32_SOC15(VCN, inst_idx, mmUVD_RB_WPTR2, lower_32_bits(ring->wptr));
fw_shared->multi_queue.encode_lowlatency_queue_mode &= cpu_to_le32(~FW_QUEUE_RING_RESET);
+ /* restore wptr/rptr with pointers saved in FW shared memory*/
+ WREG32_SOC15(VCN, inst_idx, mmUVD_RBC_RB_RPTR, fw_shared->rb.rptr);
+ WREG32_SOC15(VCN, inst_idx, mmUVD_RBC_RB_WPTR, fw_shared->rb.wptr);
+
/* Unstall DPG */
WREG32_P(SOC15_REG_OFFSET(VCN, inst_idx, mmUVD_POWER_STATUS),
0, ~UVD_POWER_STATUS__STALL_DPG_POWER_UP_MASK);
@@ -1721,6 +1742,15 @@ static uint64_t vcn_v3_0_dec_ring_get_wptr(struct amdgpu_ring *ring)
static void vcn_v3_0_dec_ring_set_wptr(struct amdgpu_ring *ring)
{
struct amdgpu_device *adev = ring->adev;
+ volatile struct amdgpu_fw_shared *fw_shared;
+
+ if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG) {
+ /*whenever update RBC_RB_WPTR, we save the wptr in shared rb.wptr and scratch2 */
+ fw_shared = adev->vcn.inst[ring->me].fw_shared_cpu_addr;
+ fw_shared->rb.wptr = lower_32_bits(ring->wptr);
+ WREG32_SOC15(VCN, ring->me, mmUVD_SCRATCH2,
+ lower_32_bits(ring->wptr));
+ }
if (ring->use_doorbell) {
adev->wb.wb[ring->wptr_offs] = lower_32_bits(ring->wptr);
@@ -1822,6 +1852,132 @@ static const struct amdgpu_ring_funcs vcn_v3_0_dec_sw_ring_vm_funcs = {
.emit_reg_write_reg_wait = amdgpu_ring_emit_reg_write_reg_wait_helper,
};
+static int vcn_v3_0_limit_sched(struct amdgpu_cs_parser *p)
+{
+ struct drm_gpu_scheduler **scheds;
+
+ /* The create msg must be in the first IB submitted */
+ if (atomic_read(&p->entity->fence_seq))
+ return -EINVAL;
+
+ scheds = p->adev->gpu_sched[AMDGPU_HW_IP_VCN_DEC]
+ [AMDGPU_RING_PRIO_DEFAULT].sched;
+ drm_sched_entity_modify_sched(p->entity, scheds, 1);
+ return 0;
+}
+
+static int vcn_v3_0_dec_msg(struct amdgpu_cs_parser *p, uint64_t addr)
+{
+ struct ttm_operation_ctx ctx = { false, false };
+ struct amdgpu_bo_va_mapping *map;
+ uint32_t *msg, num_buffers;
+ struct amdgpu_bo *bo;
+ uint64_t start, end;
+ unsigned int i;
+ void * ptr;
+ int r;
+
+ addr &= AMDGPU_GMC_HOLE_MASK;
+ r = amdgpu_cs_find_mapping(p, addr, &bo, &map);
+ if (r) {
+ DRM_ERROR("Can't find BO for addr 0x%08Lx\n", addr);
+ return r;
+ }
+
+ start = map->start * AMDGPU_GPU_PAGE_SIZE;
+ end = (map->last + 1) * AMDGPU_GPU_PAGE_SIZE;
+ if (addr & 0x7) {
+ DRM_ERROR("VCN messages must be 8 byte aligned!\n");
+ return -EINVAL;
+ }
+
+ bo->flags |= AMDGPU_GEM_CREATE_CPU_ACCESS_REQUIRED;
+ amdgpu_bo_placement_from_domain(bo, bo->allowed_domains);
+ r = ttm_bo_validate(&bo->tbo, &bo->placement, &ctx);
+ if (r) {
+ DRM_ERROR("Failed validating the VCN message BO (%d)!\n", r);
+ return r;
+ }
+
+ r = amdgpu_bo_kmap(bo, &ptr);
+ if (r) {
+ DRM_ERROR("Failed mapping the VCN message (%d)!\n", r);
+ return r;
+ }
+
+ msg = ptr + addr - start;
+
+ /* Check length */
+ if (msg[1] > end - addr) {
+ r = -EINVAL;
+ goto out;
+ }
+
+ if (msg[3] != RDECODE_MSG_CREATE)
+ goto out;
+
+ num_buffers = msg[2];
+ for (i = 0, msg = &msg[6]; i < num_buffers; ++i, msg += 4) {
+ uint32_t offset, size, *create;
+
+ if (msg[0] != RDECODE_MESSAGE_CREATE)
+ continue;
+
+ offset = msg[1];
+ size = msg[2];
+
+ if (offset + size > end) {
+ r = -EINVAL;
+ goto out;
+ }
+
+ create = ptr + addr + offset - start;
+
+ /* H246, HEVC and VP9 can run on any instance */
+ if (create[0] == 0x7 || create[0] == 0x10 || create[0] == 0x11)
+ continue;
+
+ r = vcn_v3_0_limit_sched(p);
+ if (r)
+ goto out;
+ }
+
+out:
+ amdgpu_bo_kunmap(bo);
+ return r;
+}
+
+static int vcn_v3_0_ring_patch_cs_in_place(struct amdgpu_cs_parser *p,
+ uint32_t ib_idx)
+{
+ struct amdgpu_ring *ring = to_amdgpu_ring(p->entity->rq->sched);
+ struct amdgpu_ib *ib = &p->job->ibs[ib_idx];
+ uint32_t msg_lo = 0, msg_hi = 0;
+ unsigned i;
+ int r;
+
+ /* The first instance can decode anything */
+ if (!ring->me)
+ return 0;
+
+ for (i = 0; i < ib->length_dw; i += 2) {
+ uint32_t reg = amdgpu_get_ib_value(p, ib_idx, i);
+ uint32_t val = amdgpu_get_ib_value(p, ib_idx, i + 1);
+
+ if (reg == PACKET0(p->adev->vcn.internal.data0, 0)) {
+ msg_lo = val;
+ } else if (reg == PACKET0(p->adev->vcn.internal.data1, 0)) {
+ msg_hi = val;
+ } else if (reg == PACKET0(p->adev->vcn.internal.cmd, 0) &&
+ val == 0) {
+ r = vcn_v3_0_dec_msg(p, ((u64)msg_hi) << 32 | msg_lo);
+ if (r)
+ return r;
+ }
+ }
+ return 0;
+}
+
static const struct amdgpu_ring_funcs vcn_v3_0_dec_ring_vm_funcs = {
.type = AMDGPU_RING_TYPE_VCN_DEC,
.align_mask = 0xf,
@@ -1829,6 +1985,7 @@ static const struct amdgpu_ring_funcs vcn_v3_0_dec_ring_vm_funcs = {
.get_rptr = vcn_v3_0_dec_ring_get_rptr,
.get_wptr = vcn_v3_0_dec_ring_get_wptr,
.set_wptr = vcn_v3_0_dec_ring_set_wptr,
+ .patch_cs_in_place = vcn_v3_0_ring_patch_cs_in_place,
.emit_frame_size =
SOC15_FLUSH_GPU_TLB_NUM_WREG * 6 +
SOC15_FLUSH_GPU_TLB_NUM_REG_WAIT * 8 +
diff --git a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
index 88626d83e07b..ca8efa5c6978 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega10_ih.c
@@ -220,10 +220,8 @@ static int vega10_ih_enable_ring(struct amdgpu_device *adev,
tmp = vega10_ih_rb_cntl(ih, tmp);
if (ih == &adev->irq.ih)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RPTR_REARM, !!adev->irq.msi_enabled);
- if (ih == &adev->irq.ih1) {
- tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_ENABLE, 0);
+ if (ih == &adev->irq.ih1)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_FULL_DRAIN_ENABLE, 1);
- }
if (amdgpu_sriov_vf(adev)) {
if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
dev_err(adev->dev, "PSP program IH_RB_CNTL failed!\n");
@@ -265,7 +263,6 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
u32 ih_chicken;
int ret;
int i;
- u32 tmp;
/* disable irqs */
ret = vega10_ih_toggle_interrupts(adev, false);
@@ -291,15 +288,6 @@ static int vega10_ih_irq_init(struct amdgpu_device *adev)
}
}
- tmp = RREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL);
- tmp = REG_SET_FIELD(tmp, IH_STORM_CLIENT_LIST_CNTL,
- CLIENT18_IS_STORM_CLIENT, 1);
- WREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL, tmp);
-
- tmp = RREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL);
- tmp = REG_SET_FIELD(tmp, IH_INT_FLOOD_CNTL, FLOOD_CNTL_ENABLE, 1);
- WREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL, tmp);
-
pci_set_master(adev->pdev);
/* enable interrupts */
@@ -345,11 +333,17 @@ static u32 vega10_ih_get_wptr(struct amdgpu_device *adev,
u32 wptr, tmp;
struct amdgpu_ih_regs *ih_regs;
- wptr = le32_to_cpu(*ih->wptr_cpu);
- ih_regs = &ih->ih_regs;
+ if (ih == &adev->irq.ih) {
+ /* Only ring0 supports writeback. On other rings fall back
+ * to register-based code with overflow checking below.
+ */
+ wptr = le32_to_cpu(*ih->wptr_cpu);
- if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
- goto out;
+ if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
+ goto out;
+ }
+
+ ih_regs = &ih->ih_regs;
/* Double check that the overflow wasn't already cleared. */
wptr = RREG32_NO_KIQ(ih_regs->ih_rb_wptr);
@@ -440,15 +434,11 @@ static int vega10_ih_self_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry)
{
- uint32_t wptr = cpu_to_le32(entry->src_data[0]);
-
switch (entry->ring_id) {
case 1:
- *adev->irq.ih1.wptr_cpu = wptr;
schedule_work(&adev->irq.ih1_work);
break;
case 2:
- *adev->irq.ih2.wptr_cpu = wptr;
schedule_work(&adev->irq.ih2_work);
break;
default: break;
diff --git a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
index 5a3c867d5881..8a122b413bf5 100644
--- a/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
+++ b/drivers/gpu/drm/amd/amdgpu/vega20_ih.c
@@ -35,6 +35,9 @@
#define MAX_REARM_RETRY 10
+#define mmIH_CHICKEN_ALDEBARAN 0x18d
+#define mmIH_CHICKEN_ALDEBARAN_BASE_IDX 0
+
static void vega20_ih_set_interrupt_funcs(struct amdgpu_device *adev);
/**
@@ -104,6 +107,8 @@ static int vega20_ih_toggle_ring_interrupts(struct amdgpu_device *adev,
tmp = RREG32(ih_regs->ih_rb_cntl);
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_ENABLE, (enable ? 1 : 0));
+ tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_GPU_TS_ENABLE, 1);
+
/* enable_intr field is only valid in ring0 */
if (ih == &adev->irq.ih)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, ENABLE_INTR, (enable ? 1 : 0));
@@ -220,10 +225,8 @@ static int vega20_ih_enable_ring(struct amdgpu_device *adev,
tmp = vega20_ih_rb_cntl(ih, tmp);
if (ih == &adev->irq.ih)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RPTR_REARM, !!adev->irq.msi_enabled);
- if (ih == &adev->irq.ih1) {
- tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, WPTR_OVERFLOW_ENABLE, 0);
+ if (ih == &adev->irq.ih1)
tmp = REG_SET_FIELD(tmp, IH_RB_CNTL, RB_FULL_DRAIN_ENABLE, 1);
- }
if (amdgpu_sriov_vf(adev)) {
if (psp_reg_program(&adev->psp, ih_regs->psp_reg_id, tmp)) {
dev_err(adev->dev, "PSP program IH_RB_CNTL failed!\n");
@@ -261,10 +264,10 @@ static void vega20_ih_reroute_ih(struct amdgpu_device *adev)
{
uint32_t tmp;
- /* vega20 ih reroute will go through psp
- * this function is only used for arcturus
+ /* vega20 ih reroute will go through psp this
+ * function is used for newer asics starting arcturus
*/
- if (adev->asic_type == CHIP_ARCTURUS) {
+ if (adev->asic_type >= CHIP_ARCTURUS) {
/* Reroute to IH ring 1 for VMC */
WREG32_SOC15(OSSSYS, 0, mmIH_CLIENT_CFG_INDEX, 0x12);
tmp = RREG32_SOC15(OSSSYS, 0, mmIH_CLIENT_CFG_DATA);
@@ -297,7 +300,6 @@ static int vega20_ih_irq_init(struct amdgpu_device *adev)
u32 ih_chicken;
int ret;
int i;
- u32 tmp;
/* disable irqs */
ret = vega20_ih_toggle_interrupts(adev, false);
@@ -316,6 +318,18 @@ static int vega20_ih_irq_init(struct amdgpu_device *adev)
WREG32_SOC15(OSSSYS, 0, mmIH_CHICKEN, ih_chicken);
}
+ /* psp firmware won't program IH_CHICKEN for aldebaran
+ * driver needs to program it properly according to
+ * MC_SPACE type in IH_RB_CNTL */
+ if (adev->asic_type == CHIP_ALDEBARAN) {
+ ih_chicken = RREG32_SOC15(OSSSYS, 0, mmIH_CHICKEN_ALDEBARAN);
+ if (adev->irq.ih.use_bus_addr) {
+ ih_chicken = REG_SET_FIELD(ih_chicken, IH_CHICKEN,
+ MC_SPACE_GPA_ENABLE, 1);
+ }
+ WREG32_SOC15(OSSSYS, 0, mmIH_CHICKEN_ALDEBARAN, ih_chicken);
+ }
+
for (i = 0; i < ARRAY_SIZE(ih); i++) {
if (ih[i]->ring_size) {
if (i == 1)
@@ -326,15 +340,6 @@ static int vega20_ih_irq_init(struct amdgpu_device *adev)
}
}
- tmp = RREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL);
- tmp = REG_SET_FIELD(tmp, IH_STORM_CLIENT_LIST_CNTL,
- CLIENT18_IS_STORM_CLIENT, 1);
- WREG32_SOC15(OSSSYS, 0, mmIH_STORM_CLIENT_LIST_CNTL, tmp);
-
- tmp = RREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL);
- tmp = REG_SET_FIELD(tmp, IH_INT_FLOOD_CNTL, FLOOD_CNTL_ENABLE, 1);
- WREG32_SOC15(OSSSYS, 0, mmIH_INT_FLOOD_CNTL, tmp);
-
pci_set_master(adev->pdev);
/* enable interrupts */
@@ -380,11 +385,17 @@ static u32 vega20_ih_get_wptr(struct amdgpu_device *adev,
u32 wptr, tmp;
struct amdgpu_ih_regs *ih_regs;
- wptr = le32_to_cpu(*ih->wptr_cpu);
- ih_regs = &ih->ih_regs;
+ if (ih == &adev->irq.ih) {
+ /* Only ring0 supports writeback. On other rings fall back
+ * to register-based code with overflow checking below.
+ */
+ wptr = le32_to_cpu(*ih->wptr_cpu);
- if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
- goto out;
+ if (!REG_GET_FIELD(wptr, IH_RB_WPTR, RB_OVERFLOW))
+ goto out;
+ }
+
+ ih_regs = &ih->ih_regs;
/* Double check that the overflow wasn't already cleared. */
wptr = RREG32_NO_KIQ(ih_regs->ih_rb_wptr);
@@ -476,15 +487,11 @@ static int vega20_ih_self_irq(struct amdgpu_device *adev,
struct amdgpu_irq_src *source,
struct amdgpu_iv_entry *entry)
{
- uint32_t wptr = cpu_to_le32(entry->src_data[0]);
-
switch (entry->ring_id) {
case 1:
- *adev->irq.ih1.wptr_cpu = wptr;
schedule_work(&adev->irq.ih1_work);
break;
case 2:
- *adev->irq.ih2.wptr_cpu = wptr;
schedule_work(&adev->irq.ih2_work);
break;
default: break;
diff --git a/drivers/gpu/drm/amd/amdgpu/vi.c b/drivers/gpu/drm/amd/amdgpu/vi.c
index eafb76aebd00..735ebbd1148f 100644
--- a/drivers/gpu/drm/amd/amdgpu/vi.c
+++ b/drivers/gpu/drm/amd/amdgpu/vi.c
@@ -24,6 +24,8 @@
#include <linux/pci.h>
#include <linux/slab.h>
+#include <drm/amdgpu_drm.h>
+
#include "amdgpu.h"
#include "amdgpu_atombios.h"
#include "amdgpu_ih.h"
@@ -79,6 +81,217 @@
#include "mxgpu_vi.h"
#include "amdgpu_dm.h"
+#define ixPCIE_LC_L1_PM_SUBSTATE 0x100100C6
+#define PCIE_LC_L1_PM_SUBSTATE__LC_L1_SUBSTATES_OVERRIDE_EN_MASK 0x00000001L
+#define PCIE_LC_L1_PM_SUBSTATE__LC_PCI_PM_L1_2_OVERRIDE_MASK 0x00000002L
+#define PCIE_LC_L1_PM_SUBSTATE__LC_PCI_PM_L1_1_OVERRIDE_MASK 0x00000004L
+#define PCIE_LC_L1_PM_SUBSTATE__LC_ASPM_L1_2_OVERRIDE_MASK 0x00000008L
+#define PCIE_LC_L1_PM_SUBSTATE__LC_ASPM_L1_1_OVERRIDE_MASK 0x00000010L
+#define ixPCIE_L1_PM_SUB_CNTL 0x378
+#define PCIE_L1_PM_SUB_CNTL__ASPM_L1_2_EN_MASK 0x00000004L
+#define PCIE_L1_PM_SUB_CNTL__ASPM_L1_1_EN_MASK 0x00000008L
+#define PCIE_L1_PM_SUB_CNTL__PCI_PM_L1_2_EN_MASK 0x00000001L
+#define PCIE_L1_PM_SUB_CNTL__PCI_PM_L1_1_EN_MASK 0x00000002L
+#define PCIE_LC_CNTL6__LC_L1_POWERDOWN_MASK 0x00200000L
+#define LINK_CAP 0x64
+#define PCIE_LINK_CAP__CLOCK_POWER_MANAGEMENT_MASK 0x00040000L
+#define ixCPM_CONTROL 0x1400118
+#define ixPCIE_LC_CNTL7 0x100100BC
+#define PCIE_LC_CNTL7__LC_L1_SIDEBAND_CLKREQ_PDWN_EN_MASK 0x00000400L
+#define PCIE_LC_CNTL__LC_L0S_INACTIVITY_DEFAULT 0x00000007
+#define PCIE_LC_CNTL__LC_L1_INACTIVITY_DEFAULT 0x00000009
+#define CPM_CONTROL__CLKREQb_UNGATE_TXCLK_ENABLE_MASK 0x01000000L
+#define PCIE_L1_PM_SUB_CNTL 0x378
+#define ASIC_IS_P22(asic_type, rid) ((asic_type >= CHIP_POLARIS10) && \
+ (asic_type <= CHIP_POLARIS12) && \
+ (rid >= 0x6E))
+/* Topaz */
+static const struct amdgpu_video_codecs topaz_video_codecs_encode =
+{
+ .codec_count = 0,
+ .codec_array = NULL,
+};
+
+/* Tonga, CZ, ST, Fiji */
+static const struct amdgpu_video_codec_info tonga_video_codecs_encode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 2304,
+ .max_pixels_per_frame = 4096 * 2304,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs tonga_video_codecs_encode =
+{
+ .codec_count = ARRAY_SIZE(tonga_video_codecs_encode_array),
+ .codec_array = tonga_video_codecs_encode_array,
+};
+
+/* Polaris */
+static const struct amdgpu_video_codec_info polaris_video_codecs_encode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 2304,
+ .max_pixels_per_frame = 4096 * 2304,
+ .max_level = 0,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 4096,
+ .max_height = 2304,
+ .max_pixels_per_frame = 4096 * 2304,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs polaris_video_codecs_encode =
+{
+ .codec_count = ARRAY_SIZE(polaris_video_codecs_encode_array),
+ .codec_array = polaris_video_codecs_encode_array,
+};
+
+/* Topaz */
+static const struct amdgpu_video_codecs topaz_video_codecs_decode =
+{
+ .codec_count = 0,
+ .codec_array = NULL,
+};
+
+/* Tonga */
+static const struct amdgpu_video_codec_info tonga_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 52,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 4,
+ },
+};
+
+static const struct amdgpu_video_codecs tonga_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(tonga_video_codecs_decode_array),
+ .codec_array = tonga_video_codecs_decode_array,
+};
+
+/* CZ, ST, Fiji, Polaris */
+static const struct amdgpu_video_codec_info cz_video_codecs_decode_array[] =
+{
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG2,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 3,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 5,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_MPEG4_AVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 52,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_VC1,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 4,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_HEVC,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 186,
+ },
+ {
+ .codec_type = AMDGPU_INFO_VIDEO_CAPS_CODEC_IDX_JPEG,
+ .max_width = 4096,
+ .max_height = 4096,
+ .max_pixels_per_frame = 4096 * 4096,
+ .max_level = 0,
+ },
+};
+
+static const struct amdgpu_video_codecs cz_video_codecs_decode =
+{
+ .codec_count = ARRAY_SIZE(cz_video_codecs_decode_array),
+ .codec_array = cz_video_codecs_decode_array,
+};
+
+static int vi_query_video_codecs(struct amdgpu_device *adev, bool encode,
+ const struct amdgpu_video_codecs **codecs)
+{
+ switch (adev->asic_type) {
+ case CHIP_TOPAZ:
+ if (encode)
+ *codecs = &topaz_video_codecs_encode;
+ else
+ *codecs = &topaz_video_codecs_decode;
+ return 0;
+ case CHIP_TONGA:
+ if (encode)
+ *codecs = &tonga_video_codecs_encode;
+ else
+ *codecs = &tonga_video_codecs_decode;
+ return 0;
+ case CHIP_POLARIS10:
+ case CHIP_POLARIS11:
+ case CHIP_POLARIS12:
+ case CHIP_VEGAM:
+ if (encode)
+ *codecs = &polaris_video_codecs_encode;
+ else
+ *codecs = &cz_video_codecs_decode;
+ return 0;
+ case CHIP_FIJI:
+ case CHIP_CARRIZO:
+ case CHIP_STONEY:
+ if (encode)
+ *codecs = &tonga_video_codecs_encode;
+ else
+ *codecs = &cz_video_codecs_decode;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+
/*
* Indirect registers accessor
*/
@@ -902,13 +1115,178 @@ static void vi_pcie_gen3_enable(struct amdgpu_device *adev)
/* todo */
}
+static void vi_enable_aspm(struct amdgpu_device *adev)
+{
+ u32 data, orig;
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_CNTL);
+ data |= PCIE_LC_CNTL__LC_L0S_INACTIVITY_DEFAULT <<
+ PCIE_LC_CNTL__LC_L0S_INACTIVITY__SHIFT;
+ data |= PCIE_LC_CNTL__LC_L1_INACTIVITY_DEFAULT <<
+ PCIE_LC_CNTL__LC_L1_INACTIVITY__SHIFT;
+ data &= ~PCIE_LC_CNTL__LC_PMI_TO_L1_DIS_MASK;
+ data |= PCIE_LC_CNTL__LC_DELAY_L1_EXIT_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_CNTL, data);
+}
+
static void vi_program_aspm(struct amdgpu_device *adev)
{
+ u32 data, data1, orig;
+ bool bL1SS = false;
+ bool bClkReqSupport = true;
- if (amdgpu_aspm == 0)
+ if (amdgpu_aspm != 1)
return;
- /* todo */
+ if (adev->flags & AMD_IS_APU ||
+ adev->asic_type < CHIP_POLARIS10)
+ return;
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_CNTL);
+ data &= ~PCIE_LC_CNTL__LC_L1_INACTIVITY_MASK;
+ data &= ~PCIE_LC_CNTL__LC_L0S_INACTIVITY_MASK;
+ data |= PCIE_LC_CNTL__LC_PMI_TO_L1_DIS_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_CNTL, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_N_FTS_CNTL);
+ data &= ~PCIE_LC_N_FTS_CNTL__LC_XMIT_N_FTS_MASK;
+ data |= 0x0024 << PCIE_LC_N_FTS_CNTL__LC_XMIT_N_FTS__SHIFT;
+ data |= PCIE_LC_N_FTS_CNTL__LC_XMIT_N_FTS_OVERRIDE_EN_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_N_FTS_CNTL, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_CNTL3);
+ data |= PCIE_LC_CNTL3__LC_GO_TO_RECOVERY_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_CNTL3, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_P_CNTL);
+ data |= PCIE_P_CNTL__P_IGNORE_EDB_ERR_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_P_CNTL, data);
+
+ data = RREG32_PCIE(ixPCIE_LC_L1_PM_SUBSTATE);
+ pci_read_config_dword(adev->pdev, PCIE_L1_PM_SUB_CNTL, &data1);
+ if (data & PCIE_LC_L1_PM_SUBSTATE__LC_L1_SUBSTATES_OVERRIDE_EN_MASK &&
+ (data & (PCIE_LC_L1_PM_SUBSTATE__LC_PCI_PM_L1_2_OVERRIDE_MASK |
+ PCIE_LC_L1_PM_SUBSTATE__LC_PCI_PM_L1_1_OVERRIDE_MASK |
+ PCIE_LC_L1_PM_SUBSTATE__LC_ASPM_L1_2_OVERRIDE_MASK |
+ PCIE_LC_L1_PM_SUBSTATE__LC_ASPM_L1_1_OVERRIDE_MASK))) {
+ bL1SS = true;
+ } else if (data1 & (PCIE_L1_PM_SUB_CNTL__ASPM_L1_2_EN_MASK |
+ PCIE_L1_PM_SUB_CNTL__ASPM_L1_1_EN_MASK |
+ PCIE_L1_PM_SUB_CNTL__PCI_PM_L1_2_EN_MASK |
+ PCIE_L1_PM_SUB_CNTL__PCI_PM_L1_1_EN_MASK)) {
+ bL1SS = true;
+ }
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_CNTL6);
+ data |= PCIE_LC_CNTL6__LC_L1_POWERDOWN_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_CNTL6, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_LINK_WIDTH_CNTL);
+ data |= PCIE_LC_LINK_WIDTH_CNTL__LC_DYN_LANES_PWR_STATE_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_LINK_WIDTH_CNTL, data);
+
+ pci_read_config_dword(adev->pdev, LINK_CAP, &data);
+ if (!(data & PCIE_LINK_CAP__CLOCK_POWER_MANAGEMENT_MASK))
+ bClkReqSupport = false;
+
+ if (bClkReqSupport) {
+ orig = data = RREG32_SMC(ixTHM_CLK_CNTL);
+ data &= ~(THM_CLK_CNTL__CMON_CLK_SEL_MASK | THM_CLK_CNTL__TMON_CLK_SEL_MASK);
+ data |= (1 << THM_CLK_CNTL__CMON_CLK_SEL__SHIFT) |
+ (1 << THM_CLK_CNTL__TMON_CLK_SEL__SHIFT);
+ if (orig != data)
+ WREG32_SMC(ixTHM_CLK_CNTL, data);
+
+ orig = data = RREG32_SMC(ixMISC_CLK_CTRL);
+ data &= ~(MISC_CLK_CTRL__DEEP_SLEEP_CLK_SEL_MASK |
+ MISC_CLK_CTRL__ZCLK_SEL_MASK | MISC_CLK_CTRL__DFT_SMS_PG_CLK_SEL_MASK);
+ data |= (1 << MISC_CLK_CTRL__DEEP_SLEEP_CLK_SEL__SHIFT) |
+ (1 << MISC_CLK_CTRL__ZCLK_SEL__SHIFT);
+ data |= (0x20 << MISC_CLK_CTRL__DFT_SMS_PG_CLK_SEL__SHIFT);
+ if (orig != data)
+ WREG32_SMC(ixMISC_CLK_CTRL, data);
+
+ orig = data = RREG32_SMC(ixCG_CLKPIN_CNTL);
+ data |= CG_CLKPIN_CNTL__XTALIN_DIVIDE_MASK;
+ if (orig != data)
+ WREG32_SMC(ixCG_CLKPIN_CNTL, data);
+
+ orig = data = RREG32_SMC(ixCG_CLKPIN_CNTL_2);
+ data |= CG_CLKPIN_CNTL_2__ENABLE_XCLK_MASK;
+ if (orig != data)
+ WREG32_SMC(ixCG_CLKPIN_CNTL, data);
+
+ orig = data = RREG32_SMC(ixMPLL_BYPASSCLK_SEL);
+ data &= ~MPLL_BYPASSCLK_SEL__MPLL_CLKOUT_SEL_MASK;
+ data |= (4 << MPLL_BYPASSCLK_SEL__MPLL_CLKOUT_SEL__SHIFT);
+ if (orig != data)
+ WREG32_SMC(ixMPLL_BYPASSCLK_SEL, data);
+
+ orig = data = RREG32_PCIE(ixCPM_CONTROL);
+ data |= (CPM_CONTROL__REFCLK_XSTCLK_ENABLE_MASK |
+ CPM_CONTROL__CLKREQb_UNGATE_TXCLK_ENABLE_MASK);
+ if (orig != data)
+ WREG32_PCIE(ixCPM_CONTROL, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_CONFIG_CNTL);
+ data &= ~PCIE_CONFIG_CNTL__DYN_CLK_LATENCY_MASK;
+ data |= (0xE << PCIE_CONFIG_CNTL__DYN_CLK_LATENCY__SHIFT);
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_CONFIG_CNTL, data);
+
+ orig = data = RREG32(mmBIF_CLK_CTRL);
+ data |= BIF_CLK_CTRL__BIF_XSTCLK_READY_MASK;
+ if (orig != data)
+ WREG32(mmBIF_CLK_CTRL, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_CNTL7);
+ data |= PCIE_LC_CNTL7__LC_L1_SIDEBAND_CLKREQ_PDWN_EN_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_CNTL7, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_HW_DEBUG);
+ data |= PCIE_HW_DEBUG__HW_01_DEBUG_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_HW_DEBUG, data);
+
+ orig = data = RREG32_PCIE(ixPCIE_LC_CNTL2);
+ data |= PCIE_LC_CNTL2__LC_ALLOW_PDWN_IN_L23_MASK;
+ data |= PCIE_LC_CNTL2__LC_ALLOW_PDWN_IN_L1_MASK;
+ if (bL1SS)
+ data &= ~PCIE_LC_CNTL2__LC_ALLOW_PDWN_IN_L1_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_CNTL2, data);
+
+ }
+
+ vi_enable_aspm(adev);
+
+ data = RREG32_PCIE(ixPCIE_LC_N_FTS_CNTL);
+ data1 = RREG32_PCIE(ixPCIE_LC_STATUS1);
+ if (((data & PCIE_LC_N_FTS_CNTL__LC_N_FTS_MASK) == PCIE_LC_N_FTS_CNTL__LC_N_FTS_MASK) &&
+ data1 & PCIE_LC_STATUS1__LC_REVERSE_XMIT_MASK &&
+ data1 & PCIE_LC_STATUS1__LC_REVERSE_RCVR_MASK) {
+ orig = data = RREG32_PCIE(ixPCIE_LC_CNTL);
+ data &= ~PCIE_LC_CNTL__LC_L0S_INACTIVITY_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_CNTL, data);
+ }
+
+ if ((adev->asic_type == CHIP_POLARIS12 &&
+ !(ASICID_IS_P23(adev->pdev->device, adev->pdev->revision))) ||
+ ASIC_IS_P22(adev->asic_type, adev->external_rev_id)) {
+ orig = data = RREG32_PCIE(ixPCIE_LC_TRAINING_CNTL);
+ data &= ~PCIE_LC_TRAINING_CNTL__LC_DISABLE_TRAINING_BIT_ARCH_MASK;
+ if (orig != data)
+ WREG32_PCIE(ixPCIE_LC_TRAINING_CNTL, data);
+ }
}
static void vi_enable_doorbell_aperture(struct amdgpu_device *adev,
@@ -1085,6 +1463,7 @@ static const struct amdgpu_asic_funcs vi_asic_funcs =
.get_pcie_replay_count = &vi_get_pcie_replay_count,
.supports_baco = &vi_asic_supports_baco,
.pre_asic_init = &vi_pre_asic_init,
+ .query_video_codecs = &vi_query_video_codecs,
};
#define CZ_REV_BRISTOL(rev) \
diff --git a/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c b/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c
index fe14e473f026..f6233019f042 100644
--- a/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c
+++ b/drivers/gpu/drm/amd/amdkfd/cik_event_interrupt.c
@@ -80,8 +80,9 @@ static bool cik_event_interrupt_isr(struct kfd_dev *dev,
ihre->source_id == CIK_INTSRC_SDMA_TRAP ||
ihre->source_id == CIK_INTSRC_SQ_INTERRUPT_MSG ||
ihre->source_id == CIK_INTSRC_CP_BAD_OPCODE ||
- ihre->source_id == CIK_INTSRC_GFX_PAGE_INV_FAULT ||
- ihre->source_id == CIK_INTSRC_GFX_MEM_PROT_FAULT;
+ ((ihre->source_id == CIK_INTSRC_GFX_PAGE_INV_FAULT ||
+ ihre->source_id == CIK_INTSRC_GFX_MEM_PROT_FAULT) &&
+ !amdgpu_no_queue_eviction_on_vm_fault);
}
static void cik_event_interrupt_wq(struct kfd_dev *dev,
diff --git a/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler.h b/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler.h
index affbca7c0050..475f89700c74 100644
--- a/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler.h
+++ b/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler.h
@@ -1575,6 +1575,498 @@ static const uint32_t cwsr_trap_arcturus_hex[] = {
0xbf810000, 0x00000000,
};
+static const uint32_t cwsr_trap_aldebaran_hex[] = {
+ 0xbf820001, 0xbf8202ce,
+ 0xb8f8f802, 0x89788678,
+ 0xb8eef801, 0x866eff6e,
+ 0x00000800, 0xbf840003,
+ 0x866eff78, 0x00002000,
+ 0xbf840016, 0xb8fbf803,
+ 0x866eff7b, 0x00000400,
+ 0xbf85003b, 0x866eff7b,
+ 0x00000800, 0xbf850003,
+ 0x866eff7b, 0x00000100,
+ 0xbf84000c, 0x866eff78,
+ 0x00002000, 0xbf840005,
+ 0xbf8e0010, 0xb8eef803,
+ 0x866eff6e, 0x00000400,
+ 0xbf84fffb, 0x8778ff78,
+ 0x00002000, 0x80ec886c,
+ 0x82ed806d, 0xb8eef807,
+ 0x866fff6e, 0x001f8000,
+ 0x8e6f8b6f, 0x8977ff77,
+ 0xfc000000, 0x87776f77,
+ 0x896eff6e, 0x001f8000,
+ 0xb96ef807, 0xb8faf812,
+ 0xb8fbf813, 0x8efa887a,
+ 0xc0071bbd, 0x00000000,
+ 0xbf8cc07f, 0xc0071ebd,
+ 0x00000008, 0xbf8cc07f,
+ 0x86ee6e6e, 0xbf840001,
+ 0xbe801d6e, 0xb8fbf803,
+ 0x867bff7b, 0x000001ff,
+ 0xbf850002, 0x806c846c,
+ 0x826d806d, 0x866dff6d,
+ 0x0000ffff, 0x8f6e8b77,
+ 0x866eff6e, 0x001f8000,
+ 0xb96ef807, 0x86fe7e7e,
+ 0x86ea6a6a, 0x8f6e8378,
+ 0xb96ee0c2, 0xbf800002,
+ 0xb9780002, 0xbe801f6c,
+ 0x866dff6d, 0x0000ffff,
+ 0xbefa0080, 0xb97a0283,
+ 0xb8fa2407, 0x8e7a9b7a,
+ 0x876d7a6d, 0xb8fa03c7,
+ 0x8e7a9a7a, 0x876d7a6d,
+ 0xb8faf807, 0x867aff7a,
+ 0x00007fff, 0xb97af807,
+ 0xbeee007e, 0xbeef007f,
+ 0xbefe0180, 0xbf900004,
+ 0x877a8478, 0xb97af802,
+ 0xbf8e0002, 0xbf88fffe,
+ 0xb8fa2985, 0x807a817a,
+ 0x8e7a8a7a, 0x8e7a817a,
+ 0xb8fb1605, 0x807b817b,
+ 0x8e7b867b, 0x807a7b7a,
+ 0x807a7e7a, 0x827b807f,
+ 0x867bff7b, 0x0000ffff,
+ 0xc04b1c3d, 0x00000050,
+ 0xbf8cc07f, 0xc04b1d3d,
+ 0x00000060, 0xbf8cc07f,
+ 0xc0431e7d, 0x00000074,
+ 0xbf8cc07f, 0xbef4007e,
+ 0x8675ff7f, 0x0000ffff,
+ 0x8775ff75, 0x00040000,
+ 0xbef60080, 0xbef700ff,
+ 0x00807fac, 0x867aff7f,
+ 0x08000000, 0x8f7a837a,
+ 0x87777a77, 0x867aff7f,
+ 0x70000000, 0x8f7a817a,
+ 0x87777a77, 0xbef1007c,
+ 0xbef00080, 0xb8f02985,
+ 0x80708170, 0x8e708a70,
+ 0x8e708170, 0xb8fa1605,
+ 0x807a817a, 0x8e7a867a,
+ 0x80707a70, 0xbef60084,
+ 0xbef600ff, 0x01000000,
+ 0xbefe007c, 0xbefc0070,
+ 0xc0611c7a, 0x0000007c,
+ 0xbf8cc07f, 0x80708470,
+ 0xbefc007e, 0xbefe007c,
+ 0xbefc0070, 0xc0611b3a,
+ 0x0000007c, 0xbf8cc07f,
+ 0x80708470, 0xbefc007e,
+ 0xbefe007c, 0xbefc0070,
+ 0xc0611b7a, 0x0000007c,
+ 0xbf8cc07f, 0x80708470,
+ 0xbefc007e, 0xbefe007c,
+ 0xbefc0070, 0xc0611bba,
+ 0x0000007c, 0xbf8cc07f,
+ 0x80708470, 0xbefc007e,
+ 0xbefe007c, 0xbefc0070,
+ 0xc0611bfa, 0x0000007c,
+ 0xbf8cc07f, 0x80708470,
+ 0xbefc007e, 0xbefe007c,
+ 0xbefc0070, 0xc0611e3a,
+ 0x0000007c, 0xbf8cc07f,
+ 0x80708470, 0xbefc007e,
+ 0xb8fbf803, 0xbefe007c,
+ 0xbefc0070, 0xc0611efa,
+ 0x0000007c, 0xbf8cc07f,
+ 0x80708470, 0xbefc007e,
+ 0xbefe007c, 0xbefc0070,
+ 0xc0611a3a, 0x0000007c,
+ 0xbf8cc07f, 0x80708470,
+ 0xbefc007e, 0xbefe007c,
+ 0xbefc0070, 0xc0611a7a,
+ 0x0000007c, 0xbf8cc07f,
+ 0x80708470, 0xbefc007e,
+ 0xb8f1f801, 0xbefe007c,
+ 0xbefc0070, 0xc0611c7a,
+ 0x0000007c, 0xbf8cc07f,
+ 0x80708470, 0xbefc007e,
+ 0x867aff7f, 0x04000000,
+ 0xbeef0080, 0x876f6f7a,
+ 0xb8f02985, 0x80708170,
+ 0x8e708a70, 0x8e708170,
+ 0xb8fb1605, 0x807b817b,
+ 0x8e7b847b, 0x8e76827b,
+ 0xbef600ff, 0x01000000,
+ 0xbef20174, 0x80747074,
+ 0x82758075, 0xbefc0080,
+ 0xbf800000, 0xbe802b00,
+ 0xbe822b02, 0xbe842b04,
+ 0xbe862b06, 0xbe882b08,
+ 0xbe8a2b0a, 0xbe8c2b0c,
+ 0xbe8e2b0e, 0xc06b003a,
+ 0x00000000, 0xbf8cc07f,
+ 0xc06b013a, 0x00000010,
+ 0xbf8cc07f, 0xc06b023a,
+ 0x00000020, 0xbf8cc07f,
+ 0xc06b033a, 0x00000030,
+ 0xbf8cc07f, 0x8074c074,
+ 0x82758075, 0x807c907c,
+ 0xbf0a7b7c, 0xbf85ffe7,
+ 0xbef40172, 0xbef00080,
+ 0xbefe00c1, 0xbeff00c1,
+ 0xbee80080, 0xbee90080,
+ 0xbef600ff, 0x01000000,
+ 0x867aff78, 0x00400000,
+ 0xbf850003, 0xb8faf803,
+ 0x897a7aff, 0x10000000,
+ 0xbf85004d, 0xbe840080,
+ 0xd2890000, 0x00000900,
+ 0x80048104, 0xd2890001,
+ 0x00000900, 0x80048104,
+ 0xd2890002, 0x00000900,
+ 0x80048104, 0xd2890003,
+ 0x00000900, 0x80048104,
+ 0xc069003a, 0x00000070,
+ 0xbf8cc07f, 0x80709070,
+ 0xbf06c004, 0xbf84ffee,
+ 0xbe840080, 0xd2890000,
+ 0x00000901, 0x80048104,
+ 0xd2890001, 0x00000901,
+ 0x80048104, 0xd2890002,
+ 0x00000901, 0x80048104,
+ 0xd2890003, 0x00000901,
+ 0x80048104, 0xc069003a,
+ 0x00000070, 0xbf8cc07f,
+ 0x80709070, 0xbf06c004,
+ 0xbf84ffee, 0xbe840080,
+ 0xd2890000, 0x00000902,
+ 0x80048104, 0xd2890001,
+ 0x00000902, 0x80048104,
+ 0xd2890002, 0x00000902,
+ 0x80048104, 0xd2890003,
+ 0x00000902, 0x80048104,
+ 0xc069003a, 0x00000070,
+ 0xbf8cc07f, 0x80709070,
+ 0xbf06c004, 0xbf84ffee,
+ 0xbe840080, 0xd2890000,
+ 0x00000903, 0x80048104,
+ 0xd2890001, 0x00000903,
+ 0x80048104, 0xd2890002,
+ 0x00000903, 0x80048104,
+ 0xd2890003, 0x00000903,
+ 0x80048104, 0xc069003a,
+ 0x00000070, 0xbf8cc07f,
+ 0x80709070, 0xbf06c004,
+ 0xbf84ffee, 0xbf820008,
+ 0xe0724000, 0x701d0000,
+ 0xe0724100, 0x701d0100,
+ 0xe0724200, 0x701d0200,
+ 0xe0724300, 0x701d0300,
+ 0xbefe00c1, 0xbeff00c1,
+ 0xb8fb4306, 0x867bc17b,
+ 0xbf840064, 0xbf8a0000,
+ 0x867aff6f, 0x04000000,
+ 0xbf840060, 0x8e7b867b,
+ 0x8e7b827b, 0xbef6007b,
+ 0xb8f02985, 0x80708170,
+ 0x8e708a70, 0x8e708170,
+ 0xb8fa1605, 0x807a817a,
+ 0x8e7a867a, 0x80707a70,
+ 0x8070ff70, 0x00000080,
+ 0xbef600ff, 0x01000000,
+ 0xbefc0080, 0xd28c0002,
+ 0x000100c1, 0xd28d0003,
+ 0x000204c1, 0x867aff78,
+ 0x00400000, 0xbf850003,
+ 0xb8faf803, 0x897a7aff,
+ 0x10000000, 0xbf850030,
+ 0x24040682, 0xd86e4000,
+ 0x00000002, 0xbf8cc07f,
+ 0xbe840080, 0xd2890000,
+ 0x00000900, 0x80048104,
+ 0xd2890001, 0x00000900,
+ 0x80048104, 0xd2890002,
+ 0x00000900, 0x80048104,
+ 0xd2890003, 0x00000900,
+ 0x80048104, 0xc069003a,
+ 0x00000070, 0xbf8cc07f,
+ 0x80709070, 0xbf06c004,
+ 0xbf84ffee, 0xbe840080,
+ 0xd2890000, 0x00000901,
+ 0x80048104, 0xd2890001,
+ 0x00000901, 0x80048104,
+ 0xd2890002, 0x00000901,
+ 0x80048104, 0xd2890003,
+ 0x00000901, 0x80048104,
+ 0xc069003a, 0x00000070,
+ 0xbf8cc07f, 0x80709070,
+ 0xbf06c004, 0xbf84ffee,
+ 0x680404ff, 0x00000200,
+ 0xd0c9006a, 0x0000f702,
+ 0xbf87ffd2, 0xbf820015,
+ 0xd1060002, 0x00011103,
+ 0x7e0602ff, 0x00000200,
+ 0xbefc00ff, 0x00010000,
+ 0xbe800077, 0x8677ff77,
+ 0xff7fffff, 0x8777ff77,
+ 0x00058000, 0xd8ec0000,
+ 0x00000002, 0xbf8cc07f,
+ 0xe0765000, 0x701d0002,
+ 0x68040702, 0xd0c9006a,
+ 0x0000f702, 0xbf87fff7,
+ 0xbef70000, 0xbef000ff,
+ 0x00000400, 0xbefe00c1,
+ 0xbeff00c1, 0xb8fb2b05,
+ 0x807b817b, 0x8e7b827b,
+ 0xbef600ff, 0x01000000,
+ 0xbefc0084, 0xbf0a7b7c,
+ 0xbf84006d, 0xbf11017c,
+ 0x807bff7b, 0x00001000,
+ 0x867aff78, 0x00400000,
+ 0xbf850003, 0xb8faf803,
+ 0x897a7aff, 0x10000000,
+ 0xbf850051, 0xbe840080,
+ 0xd2890000, 0x00000900,
+ 0x80048104, 0xd2890001,
+ 0x00000900, 0x80048104,
+ 0xd2890002, 0x00000900,
+ 0x80048104, 0xd2890003,
+ 0x00000900, 0x80048104,
+ 0xc069003a, 0x00000070,
+ 0xbf8cc07f, 0x80709070,
+ 0xbf06c004, 0xbf84ffee,
+ 0xbe840080, 0xd2890000,
+ 0x00000901, 0x80048104,
+ 0xd2890001, 0x00000901,
+ 0x80048104, 0xd2890002,
+ 0x00000901, 0x80048104,
+ 0xd2890003, 0x00000901,
+ 0x80048104, 0xc069003a,
+ 0x00000070, 0xbf8cc07f,
+ 0x80709070, 0xbf06c004,
+ 0xbf84ffee, 0xbe840080,
+ 0xd2890000, 0x00000902,
+ 0x80048104, 0xd2890001,
+ 0x00000902, 0x80048104,
+ 0xd2890002, 0x00000902,
+ 0x80048104, 0xd2890003,
+ 0x00000902, 0x80048104,
+ 0xc069003a, 0x00000070,
+ 0xbf8cc07f, 0x80709070,
+ 0xbf06c004, 0xbf84ffee,
+ 0xbe840080, 0xd2890000,
+ 0x00000903, 0x80048104,
+ 0xd2890001, 0x00000903,
+ 0x80048104, 0xd2890002,
+ 0x00000903, 0x80048104,
+ 0xd2890003, 0x00000903,
+ 0x80048104, 0xc069003a,
+ 0x00000070, 0xbf8cc07f,
+ 0x80709070, 0xbf06c004,
+ 0xbf84ffee, 0x807c847c,
+ 0xbf0a7b7c, 0xbf85ffb1,
+ 0xbf9c0000, 0xbf820012,
+ 0x7e000300, 0x7e020301,
+ 0x7e040302, 0x7e060303,
+ 0xe0724000, 0x701d0000,
+ 0xe0724100, 0x701d0100,
+ 0xe0724200, 0x701d0200,
+ 0xe0724300, 0x701d0300,
+ 0x807c847c, 0x8070ff70,
+ 0x00000400, 0xbf0a7b7c,
+ 0xbf85ffef, 0xbf9c0000,
+ 0xb8fb2985, 0x807b817b,
+ 0x8e7b837b, 0xb8fa2b05,
+ 0x807a817a, 0x8e7a827a,
+ 0x80fb7a7b, 0x867b7b7b,
+ 0xbf84007a, 0x807bff7b,
+ 0x00001000, 0xbefc0080,
+ 0xbf11017c, 0x867aff78,
+ 0x00400000, 0xbf850003,
+ 0xb8faf803, 0x897a7aff,
+ 0x10000000, 0xbf850059,
+ 0xd3d84000, 0x18000100,
+ 0xd3d84001, 0x18000101,
+ 0xd3d84002, 0x18000102,
+ 0xd3d84003, 0x18000103,
+ 0xbe840080, 0xd2890000,
+ 0x00000900, 0x80048104,
+ 0xd2890001, 0x00000900,
+ 0x80048104, 0xd2890002,
+ 0x00000900, 0x80048104,
+ 0xd2890003, 0x00000900,
+ 0x80048104, 0xc069003a,
+ 0x00000070, 0xbf8cc07f,
+ 0x80709070, 0xbf06c004,
+ 0xbf84ffee, 0xbe840080,
+ 0xd2890000, 0x00000901,
+ 0x80048104, 0xd2890001,
+ 0x00000901, 0x80048104,
+ 0xd2890002, 0x00000901,
+ 0x80048104, 0xd2890003,
+ 0x00000901, 0x80048104,
+ 0xc069003a, 0x00000070,
+ 0xbf8cc07f, 0x80709070,
+ 0xbf06c004, 0xbf84ffee,
+ 0xbe840080, 0xd2890000,
+ 0x00000902, 0x80048104,
+ 0xd2890001, 0x00000902,
+ 0x80048104, 0xd2890002,
+ 0x00000902, 0x80048104,
+ 0xd2890003, 0x00000902,
+ 0x80048104, 0xc069003a,
+ 0x00000070, 0xbf8cc07f,
+ 0x80709070, 0xbf06c004,
+ 0xbf84ffee, 0xbe840080,
+ 0xd2890000, 0x00000903,
+ 0x80048104, 0xd2890001,
+ 0x00000903, 0x80048104,
+ 0xd2890002, 0x00000903,
+ 0x80048104, 0xd2890003,
+ 0x00000903, 0x80048104,
+ 0xc069003a, 0x00000070,
+ 0xbf8cc07f, 0x80709070,
+ 0xbf06c004, 0xbf84ffee,
+ 0x807c847c, 0xbf0a7b7c,
+ 0xbf85ffa9, 0xbf9c0000,
+ 0xbf820016, 0xd3d84000,
+ 0x18000100, 0xd3d84001,
+ 0x18000101, 0xd3d84002,
+ 0x18000102, 0xd3d84003,
+ 0x18000103, 0xe0724000,
+ 0x701d0000, 0xe0724100,
+ 0x701d0100, 0xe0724200,
+ 0x701d0200, 0xe0724300,
+ 0x701d0300, 0x807c847c,
+ 0x8070ff70, 0x00000400,
+ 0xbf0a7b7c, 0xbf85ffeb,
+ 0xbf9c0000, 0xbf820101,
+ 0xbef4007e, 0x8675ff7f,
+ 0x0000ffff, 0x8775ff75,
+ 0x00040000, 0xbef60080,
+ 0xbef700ff, 0x00807fac,
+ 0x866eff7f, 0x08000000,
+ 0x8f6e836e, 0x87776e77,
+ 0x866eff7f, 0x70000000,
+ 0x8f6e816e, 0x87776e77,
+ 0x866eff7f, 0x04000000,
+ 0xbf84001f, 0xbefe00c1,
+ 0xbeff00c1, 0xb8ef4306,
+ 0x866fc16f, 0xbf84001a,
+ 0x8e6f866f, 0x8e6f826f,
+ 0xbef6006f, 0xb8f82985,
+ 0x80788178, 0x8e788a78,
+ 0x8e788178, 0xb8ee1605,
+ 0x806e816e, 0x8e6e866e,
+ 0x80786e78, 0x8078ff78,
+ 0x00000080, 0xbef600ff,
+ 0x01000000, 0xbefc0080,
+ 0xe0510000, 0x781d0000,
+ 0xe0510100, 0x781d0000,
+ 0x807cff7c, 0x00000200,
+ 0x8078ff78, 0x00000200,
+ 0xbf0a6f7c, 0xbf85fff6,
+ 0xbefe00c1, 0xbeff00c1,
+ 0xbef600ff, 0x01000000,
+ 0xb8ef2b05, 0x806f816f,
+ 0x8e6f826f, 0x806fff6f,
+ 0x00008000, 0xbef80080,
+ 0xbeee0078, 0x8078ff78,
+ 0x00000400, 0xbefc0084,
+ 0xbf11087c, 0xe0524000,
+ 0x781d0000, 0xe0524100,
+ 0x781d0100, 0xe0524200,
+ 0x781d0200, 0xe0524300,
+ 0x781d0300, 0xbf8c0f70,
+ 0x7e000300, 0x7e020301,
+ 0x7e040302, 0x7e060303,
+ 0x807c847c, 0x8078ff78,
+ 0x00000400, 0xbf0a6f7c,
+ 0xbf85ffee, 0xb8ef2985,
+ 0x806f816f, 0x8e6f836f,
+ 0xb8f92b05, 0x80798179,
+ 0x8e798279, 0x80ef796f,
+ 0x866f6f6f, 0xbf84001a,
+ 0x806fff6f, 0x00008000,
+ 0xbefc0080, 0xbf11087c,
+ 0xe0524000, 0x781d0000,
+ 0xe0524100, 0x781d0100,
+ 0xe0524200, 0x781d0200,
+ 0xe0524300, 0x781d0300,
+ 0xbf8c0f70, 0xd3d94000,
+ 0x18000100, 0xd3d94001,
+ 0x18000101, 0xd3d94002,
+ 0x18000102, 0xd3d94003,
+ 0x18000103, 0x807c847c,
+ 0x8078ff78, 0x00000400,
+ 0xbf0a6f7c, 0xbf85ffea,
+ 0xbf9c0000, 0xe0524000,
+ 0x6e1d0000, 0xe0524100,
+ 0x6e1d0100, 0xe0524200,
+ 0x6e1d0200, 0xe0524300,
+ 0x6e1d0300, 0xbf8c0f70,
+ 0xb8f82985, 0x80788178,
+ 0x8e788a78, 0x8e788178,
+ 0xb8ee1605, 0x806e816e,
+ 0x8e6e866e, 0x80786e78,
+ 0x80f8c078, 0xb8ef1605,
+ 0x806f816f, 0x8e6f846f,
+ 0x8e76826f, 0xbef600ff,
+ 0x01000000, 0xbefc006f,
+ 0xc031003a, 0x00000078,
+ 0x80f8c078, 0xbf8cc07f,
+ 0x80fc907c, 0xbf800000,
+ 0xbe802d00, 0xbe822d02,
+ 0xbe842d04, 0xbe862d06,
+ 0xbe882d08, 0xbe8a2d0a,
+ 0xbe8c2d0c, 0xbe8e2d0e,
+ 0xbf06807c, 0xbf84fff0,
+ 0xb8f82985, 0x80788178,
+ 0x8e788a78, 0x8e788178,
+ 0xb8ee1605, 0x806e816e,
+ 0x8e6e866e, 0x80786e78,
+ 0xbef60084, 0xbef600ff,
+ 0x01000000, 0xc0211bfa,
+ 0x00000078, 0x80788478,
+ 0xc0211b3a, 0x00000078,
+ 0x80788478, 0xc0211b7a,
+ 0x00000078, 0x80788478,
+ 0xc0211c3a, 0x00000078,
+ 0x80788478, 0xc0211c7a,
+ 0x00000078, 0x80788478,
+ 0xc0211eba, 0x00000078,
+ 0x80788478, 0xc0211efa,
+ 0x00000078, 0x80788478,
+ 0xc0211a3a, 0x00000078,
+ 0x80788478, 0xc0211a7a,
+ 0x00000078, 0x80788478,
+ 0xc0211cfa, 0x00000078,
+ 0x80788478, 0xbf8cc07f,
+ 0xbefc006f, 0xbefe0070,
+ 0xbeff0071, 0x866f7bff,
+ 0x000003ff, 0xb96f4803,
+ 0x866f7bff, 0xfffff800,
+ 0x8f6f8b6f, 0xb96fa2c3,
+ 0xb973f801, 0xb8ee2985,
+ 0x806e816e, 0x8e6e8a6e,
+ 0x8e6e816e, 0xb8ef1605,
+ 0x806f816f, 0x8e6f866f,
+ 0x806e6f6e, 0x806e746e,
+ 0x826f8075, 0x866fff6f,
+ 0x0000ffff, 0xc00b1c37,
+ 0x00000050, 0xc00b1d37,
+ 0x00000060, 0xc0031e77,
+ 0x00000074, 0xbf8cc07f,
+ 0x866fff6d, 0xf8000000,
+ 0x8f6f9b6f, 0x8e6f906f,
+ 0xbeee0080, 0x876e6f6e,
+ 0x866fff6d, 0x04000000,
+ 0x8f6f9a6f, 0x8e6f8f6f,
+ 0x876e6f6e, 0x866fff7a,
+ 0x00800000, 0x8f6f976f,
+ 0xb96ef807, 0x866dff6d,
+ 0x0000ffff, 0x86fe7e7e,
+ 0x86ea6a6a, 0x8f6e837a,
+ 0xb96ee0c2, 0xbf800002,
+ 0xb97a0002, 0xbf8a0000,
+ 0x95806f6c, 0xbf810000,
+};
+
static const uint32_t cwsr_trap_gfx10_hex[] = {
0xbf820001, 0xbf8201cf,
0xb0804004, 0xb978f802,
diff --git a/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx8.asm b/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx8.asm
index b195b7cd8a17..ac8edef09ca5 100644
--- a/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx8.asm
+++ b/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx8.asm
@@ -563,6 +563,7 @@ L_RESTORE:
buffer_load_dword v1, v0, s_restore_buf_rsrc0, s_restore_mem_offset_save slc:1 glc:1 offset:256
buffer_load_dword v2, v0, s_restore_buf_rsrc0, s_restore_mem_offset_save slc:1 glc:1 offset:256*2
buffer_load_dword v3, v0, s_restore_buf_rsrc0, s_restore_mem_offset_save slc:1 glc:1 offset:256*3
+ s_waitcnt vmcnt(0)
/* restore SGPRs */
//////////////////////////////
diff --git a/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx9.asm b/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx9.asm
index 75f29d13c90f..eed78a04e7c7 100644
--- a/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx9.asm
+++ b/drivers/gpu/drm/amd/amdkfd/cwsr_trap_handler_gfx9.asm
@@ -21,9 +21,24 @@
*/
/* To compile this assembly code:
- * PROJECT=greenland ./sp3 cwsr_trap_handler_gfx9.asm -hex tmp.hex
+ *
+ * gfx9:
+ * cpp -DASIC_FAMILY=CHIP_VEGAM cwsr_trap_handler_gfx9.asm -P -o gfx9.sp3
+ * sp3 gfx9.sp3 -hex gfx9.hex
+ *
+ * arcturus:
+ * cpp -DASIC_FAMILY=CHIP_ARCTURUS cwsr_trap_handler_gfx9.asm -P -o arcturus.sp3
+ * sp3 arcturus.sp3 -hex arcturus.hex
+ *
+ * aldebaran:
+ * cpp -DASIC_FAMILY=CHIP_ALDEBARAN cwsr_trap_handler_gfx9.asm -P -o aldebaran.sp3
+ * sp3 aldebaran.sp3 -hex aldebaran.hex
*/
+#define CHIP_VEGAM 18
+#define CHIP_ARCTURUS 23
+#define CHIP_ALDEBARAN 25
+
var ACK_SQC_STORE = 1 //workaround for suspected SQC store bug causing incorrect stores under concurrency
var SAVE_AFTER_XNACK_ERROR = 1 //workaround for TCP store failure after XNACK error when ALLOW_REPLAY=0, for debugger
var SINGLE_STEP_MISSED_WORKAROUND = 1 //workaround for lost MODE.DEBUG_EN exception when SAVECTX raised
@@ -44,10 +59,17 @@ var SQ_WAVE_STATUS_ALLOW_REPLAY_MASK = 0x400000
var SQ_WAVE_LDS_ALLOC_LDS_SIZE_SHIFT = 12
var SQ_WAVE_LDS_ALLOC_LDS_SIZE_SIZE = 9
-var SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SHIFT = 8
var SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SIZE = 6
-var SQ_WAVE_GPR_ALLOC_SGPR_SIZE_SHIFT = 24
var SQ_WAVE_GPR_ALLOC_SGPR_SIZE_SIZE = 3 //FIXME sq.blk still has 4 bits at this time while SQ programming guide has 3 bits
+var SQ_WAVE_GPR_ALLOC_SGPR_SIZE_SHIFT = 24
+
+#if ASIC_FAMILY >= CHIP_ALDEBARAN
+var SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SHIFT = 6
+var SQ_WAVE_GPR_ALLOC_ACCV_OFFSET_SHIFT = 12
+var SQ_WAVE_GPR_ALLOC_ACCV_OFFSET_SIZE = 6
+#else
+var SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SHIFT = 8
+#endif
var SQ_WAVE_TRAPSTS_SAVECTX_MASK = 0x400
var SQ_WAVE_TRAPSTS_EXCE_MASK = 0x1FF // Exception mask
@@ -134,7 +156,7 @@ var s_restore_spi_init_lo = exec_lo
var s_restore_spi_init_hi = exec_hi
var s_restore_mem_offset = ttmp12
-var s_restore_accvgpr_offset = ttmp13
+var s_restore_tmp2 = ttmp13
var s_restore_alloc_size = ttmp3
var s_restore_tmp = ttmp2
var s_restore_mem_offset_save = s_restore_tmp //no conflict
@@ -466,12 +488,7 @@ if SAVE_AFTER_XNACK_ERROR
L_SAVE_FIRST_VGPRS_WITH_TCP:
end
- buffer_store_dword v0, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1
- buffer_store_dword v1, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256
- buffer_store_dword v2, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256*2
- buffer_store_dword v3, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256*3
-
-
+ write_4vgprs_to_mem(s_save_buf_rsrc0, s_save_mem_offset)
/* save LDS */
//////////////////////////////
@@ -565,11 +582,8 @@ L_SAVE_LDS_DONE:
s_mov_b32 exec_lo, 0xFFFFFFFF //need every thread from now on
s_mov_b32 exec_hi, 0xFFFFFFFF
- s_getreg_b32 s_save_alloc_size, hwreg(HW_REG_GPR_ALLOC,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SHIFT,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SIZE) //vpgr_size
- s_add_u32 s_save_alloc_size, s_save_alloc_size, 1
- s_lshl_b32 s_save_alloc_size, s_save_alloc_size, 2 //Number of VGPRs = (vgpr_size + 1) * 4 (non-zero value) //FIXME for GFX, zero is possible
- s_lshl_b32 s_save_buf_rsrc2, s_save_alloc_size, 8 //NUM_RECORDS in bytes (64 threads*4)
- s_mov_b32 s_save_buf_rsrc2, 0x1000000 //NUM_RECORDS in bytes
+ get_num_arch_vgprs(s_save_alloc_size)
+ s_mov_b32 s_save_buf_rsrc2, 0x1000000 //NUM_RECORDS in bytes
// VGPR store using dw burst
@@ -602,10 +616,7 @@ end
v_mov_b32 v2, v2 //v0 = v[0+m0]
v_mov_b32 v3, v3 //v0 = v[0+m0]
- buffer_store_dword v0, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1
- buffer_store_dword v1, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256
- buffer_store_dword v2, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256*2
- buffer_store_dword v3, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256*3
+ write_4vgprs_to_mem(s_save_buf_rsrc0, s_save_mem_offset)
s_add_u32 m0, m0, 4 //next vgpr index
s_add_u32 s_save_mem_offset, s_save_mem_offset, 256*4 //every buffer_store_dword does 256 bytes
@@ -615,8 +626,17 @@ end
L_SAVE_VGPR_END:
-if ASIC_TARGET_ARCTURUS
+#if ASIC_FAMILY >= CHIP_ARCTURUS
// Save ACC VGPRs
+
+#if ASIC_FAMILY >= CHIP_ALDEBARAN
+ // ACC VGPR count may differ from ARCH VGPR count.
+ get_num_acc_vgprs(s_save_alloc_size, s_save_tmp)
+ s_and_b32 s_save_alloc_size, s_save_alloc_size, s_save_alloc_size
+ s_cbranch_scc0 L_SAVE_ACCVGPR_END
+ s_add_u32 s_save_alloc_size, s_save_alloc_size, 0x1000 //add 0x1000 since we compare m0 against it later
+#endif
+
s_mov_b32 m0, 0x0 //VGPR initial index value =0
s_set_gpr_idx_on m0, 0x1 //M0[7:0] = M0[7:0] and M0[15:12] = 0x1
@@ -644,10 +664,7 @@ L_SAVE_ACCVGPR_LOOP:
v_accvgpr_read v[vgpr], acc[vgpr] // v[N] = acc[N+m0]
end
- buffer_store_dword v0, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1
- buffer_store_dword v1, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256
- buffer_store_dword v2, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256*2
- buffer_store_dword v3, v0, s_save_buf_rsrc0, s_save_mem_offset slc:1 glc:1 offset:256*3
+ write_4vgprs_to_mem(s_save_buf_rsrc0, s_save_mem_offset)
s_add_u32 m0, m0, 4
s_add_u32 s_save_mem_offset, s_save_mem_offset, 256*4
@@ -656,7 +673,7 @@ L_SAVE_ACCVGPR_LOOP:
s_set_gpr_idx_off
L_SAVE_ACCVGPR_END:
-end
+#endif
s_branch L_END_PGM
@@ -724,53 +741,23 @@ L_RESTORE:
/* restore VGPRs */
//////////////////////////////
L_RESTORE_VGPR:
- // VGPR SR memory offset : 0
- s_mov_b32 s_restore_mem_offset, 0x0
s_mov_b32 exec_lo, 0xFFFFFFFF //need every thread from now on //be consistent with SAVE although can be moved ahead
s_mov_b32 exec_hi, 0xFFFFFFFF
+ s_mov_b32 s_restore_buf_rsrc2, 0x1000000 //NUM_RECORDS in bytes
- s_getreg_b32 s_restore_alloc_size, hwreg(HW_REG_GPR_ALLOC,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SHIFT,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SIZE) //vpgr_size
- s_add_u32 s_restore_alloc_size, s_restore_alloc_size, 1
- s_lshl_b32 s_restore_alloc_size, s_restore_alloc_size, 2 //Number of VGPRs = (vgpr_size + 1) * 4 (non-zero value)
- s_lshl_b32 s_restore_buf_rsrc2, s_restore_alloc_size, 8 //NUM_RECORDS in bytes (64 threads*4)
-
-if ASIC_TARGET_ARCTURUS
- s_mov_b32 s_restore_accvgpr_offset, s_restore_buf_rsrc2 //ACC VGPRs at end of VGPRs
-end
-
- s_mov_b32 s_restore_buf_rsrc2, 0x1000000 //NUM_RECORDS in bytes
+ // Save ARCH VGPRs 4-N, then all ACC VGPRs, then ARCH VGPRs 0-3.
+ get_num_arch_vgprs(s_restore_alloc_size)
+ s_add_u32 s_restore_alloc_size, s_restore_alloc_size, 0x8000 //add 0x8000 since we compare m0 against it later
- // VGPR load using dw burst
+ // ARCH VGPRs at offset: 0
+ s_mov_b32 s_restore_mem_offset, 0x0
s_mov_b32 s_restore_mem_offset_save, s_restore_mem_offset // restore start with v1, v0 will be the last
s_add_u32 s_restore_mem_offset, s_restore_mem_offset, 256*4
-if ASIC_TARGET_ARCTURUS
- s_mov_b32 s_restore_accvgpr_offset_save, s_restore_accvgpr_offset
- s_add_u32 s_restore_accvgpr_offset, s_restore_accvgpr_offset, 256*4
-end
s_mov_b32 m0, 4 //VGPR initial index value = 1
- s_set_gpr_idx_on m0, 0x8 //M0[7:0] = M0[7:0] and M0[15:12] = 0x8
- s_add_u32 s_restore_alloc_size, s_restore_alloc_size, 0x8000 //add 0x8000 since we compare m0 against it later
+ s_set_gpr_idx_on m0, 0x8 //M0[7:0] = M0[7:0] and M0[15:12] = 0x8
L_RESTORE_VGPR_LOOP:
-
-if ASIC_TARGET_ARCTURUS
- buffer_load_dword v0, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset slc:1 glc:1
- buffer_load_dword v1, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset slc:1 glc:1 offset:256
- buffer_load_dword v2, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset slc:1 glc:1 offset:256*2
- buffer_load_dword v3, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset slc:1 glc:1 offset:256*3
- s_add_u32 s_restore_accvgpr_offset, s_restore_accvgpr_offset, 256*4
- s_waitcnt vmcnt(0)
-
- for var vgpr = 0; vgpr < 4; ++ vgpr
- v_accvgpr_write acc[vgpr], v[vgpr]
- end
-end
-
- buffer_load_dword v0, v0, s_restore_buf_rsrc0, s_restore_mem_offset slc:1 glc:1
- buffer_load_dword v1, v0, s_restore_buf_rsrc0, s_restore_mem_offset slc:1 glc:1 offset:256
- buffer_load_dword v2, v0, s_restore_buf_rsrc0, s_restore_mem_offset slc:1 glc:1 offset:256*2
- buffer_load_dword v3, v0, s_restore_buf_rsrc0, s_restore_mem_offset slc:1 glc:1 offset:256*3
- s_waitcnt vmcnt(0) //ensure data ready
+ read_4vgprs_from_mem(s_restore_buf_rsrc0, s_restore_mem_offset)
v_mov_b32 v0, v0 //v[0+m0] = v0
v_mov_b32 v1, v1
v_mov_b32 v2, v2
@@ -779,24 +766,38 @@ end
s_add_u32 s_restore_mem_offset, s_restore_mem_offset, 256*4 //every buffer_load_dword does 256 bytes
s_cmp_lt_u32 m0, s_restore_alloc_size //scc = (m0 < s_restore_alloc_size) ? 1 : 0
s_cbranch_scc1 L_RESTORE_VGPR_LOOP //VGPR restore (except v0) is complete?
- s_set_gpr_idx_off
- /* VGPR restore on v0 */
-if ASIC_TARGET_ARCTURUS
- buffer_load_dword v0, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset_save slc:1 glc:1
- buffer_load_dword v1, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset_save slc:1 glc:1 offset:256
- buffer_load_dword v2, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset_save slc:1 glc:1 offset:256*2
- buffer_load_dword v3, v0, s_restore_buf_rsrc0, s_restore_accvgpr_offset_save slc:1 glc:1 offset:256*3
- s_waitcnt vmcnt(0)
- for var vgpr = 0; vgpr < 4; ++ vgpr
- v_accvgpr_write acc[vgpr], v[vgpr]
- end
-end
+#if ASIC_FAMILY >= CHIP_ALDEBARAN
+ // ACC VGPR count may differ from ARCH VGPR count.
+ get_num_acc_vgprs(s_restore_alloc_size, s_restore_tmp2)
+ s_and_b32 s_restore_alloc_size, s_restore_alloc_size, s_restore_alloc_size
+ s_cbranch_scc0 L_RESTORE_ACCVGPR_END
+ s_add_u32 s_restore_alloc_size, s_restore_alloc_size, 0x8000 //add 0x8000 since we compare m0 against it later
+#endif
- buffer_load_dword v0, v0, s_restore_buf_rsrc0, s_restore_mem_offset_save slc:1 glc:1
- buffer_load_dword v1, v0, s_restore_buf_rsrc0, s_restore_mem_offset_save slc:1 glc:1 offset:256
- buffer_load_dword v2, v0, s_restore_buf_rsrc0, s_restore_mem_offset_save slc:1 glc:1 offset:256*2
- buffer_load_dword v3, v0, s_restore_buf_rsrc0, s_restore_mem_offset_save slc:1 glc:1 offset:256*3
+#if ASIC_FAMILY >= CHIP_ARCTURUS
+ // ACC VGPRs at offset: size(ARCH VGPRs)
+ s_mov_b32 m0, 0
+ s_set_gpr_idx_on m0, 0x8 //M0[7:0] = M0[7:0] and M0[15:12] = 0x8
+
+ L_RESTORE_ACCVGPR_LOOP:
+ read_4vgprs_from_mem(s_restore_buf_rsrc0, s_restore_mem_offset)
+
+ for var vgpr = 0; vgpr < 4; ++ vgpr
+ v_accvgpr_write acc[vgpr], v[vgpr]
+ end
+
+ s_add_u32 m0, m0, 4 //next vgpr index
+ s_add_u32 s_restore_mem_offset, s_restore_mem_offset, 256*4 //every buffer_load_dword does 256 bytes
+ s_cmp_lt_u32 m0, s_restore_alloc_size //scc = (m0 < s_restore_alloc_size) ? 1 : 0
+ s_cbranch_scc1 L_RESTORE_ACCVGPR_LOOP //VGPR restore (except v0) is complete?
+ L_RESTORE_ACCVGPR_END:
+#endif
+
+ s_set_gpr_idx_off
+
+ // Restore VGPRs 0-3 last, no longer needed.
+ read_4vgprs_from_mem(s_restore_buf_rsrc0, s_restore_mem_offset_save)
/* restore SGPRs */
//////////////////////////////
@@ -974,6 +975,21 @@ function check_if_tcp_store_ok
L_TCP_STORE_CHECK_DONE:
end
+function write_4vgprs_to_mem(s_rsrc, s_mem_offset)
+ buffer_store_dword v0, v0, s_rsrc, s_mem_offset slc:1 glc:1
+ buffer_store_dword v1, v0, s_rsrc, s_mem_offset slc:1 glc:1 offset:256
+ buffer_store_dword v2, v0, s_rsrc, s_mem_offset slc:1 glc:1 offset:256*2
+ buffer_store_dword v3, v0, s_rsrc, s_mem_offset slc:1 glc:1 offset:256*3
+end
+
+function read_4vgprs_from_mem(s_rsrc, s_mem_offset)
+ buffer_load_dword v0, v0, s_rsrc, s_mem_offset slc:1 glc:1
+ buffer_load_dword v1, v0, s_rsrc, s_mem_offset slc:1 glc:1 offset:256
+ buffer_load_dword v2, v0, s_rsrc, s_mem_offset slc:1 glc:1 offset:256*2
+ buffer_load_dword v3, v0, s_rsrc, s_mem_offset slc:1 glc:1 offset:256*3
+ s_waitcnt vmcnt(0)
+end
+
function write_vgpr_to_mem_with_sqc(v, s_rsrc, s_mem_offset)
s_mov_b32 s4, 0
@@ -1008,9 +1024,9 @@ function get_vgpr_size_bytes(s_vgpr_size_byte)
s_add_u32 s_vgpr_size_byte, s_vgpr_size_byte, 1
s_lshl_b32 s_vgpr_size_byte, s_vgpr_size_byte, (2+8) //Number of VGPRs = (vgpr_size + 1) * 4 * 64 * 4 (non-zero value) //FIXME for GFX, zero is possible
-if ASIC_TARGET_ARCTURUS
+#if ASIC_FAMILY >= CHIP_ARCTURUS
s_lshl_b32 s_vgpr_size_byte, s_vgpr_size_byte, 1 // Double size for ACC VGPRs
-end
+#endif
end
function get_sgpr_size_bytes(s_sgpr_size_byte)
@@ -1023,6 +1039,32 @@ function get_hwreg_size_bytes
return 128 //HWREG size 128 bytes
end
+function get_num_arch_vgprs(s_num_arch_vgprs)
+#if ASIC_FAMILY >= CHIP_ALDEBARAN
+ // VGPR count includes ACC VGPRs, use ACC VGPR offset for ARCH VGPR count.
+ s_getreg_b32 s_num_arch_vgprs, hwreg(HW_REG_GPR_ALLOC,SQ_WAVE_GPR_ALLOC_ACCV_OFFSET_SHIFT,SQ_WAVE_GPR_ALLOC_ACCV_OFFSET_SIZE)
+#else
+ s_getreg_b32 s_num_arch_vgprs, hwreg(HW_REG_GPR_ALLOC,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SHIFT,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SIZE)
+#endif
+
+ // Number of VGPRs = (vgpr_size + 1) * 4
+ s_add_u32 s_num_arch_vgprs, s_num_arch_vgprs, 1
+ s_lshl_b32 s_num_arch_vgprs, s_num_arch_vgprs, 2
+end
+
+#if ASIC_FAMILY >= CHIP_ALDEBARAN
+function get_num_acc_vgprs(s_num_acc_vgprs, s_tmp)
+ // VGPR count = (GPR_ALLOC.VGPR_SIZE + 1) * 8
+ s_getreg_b32 s_num_acc_vgprs, hwreg(HW_REG_GPR_ALLOC,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SHIFT,SQ_WAVE_GPR_ALLOC_VGPR_SIZE_SIZE)
+ s_add_u32 s_num_acc_vgprs, s_num_acc_vgprs, 1
+ s_lshl_b32 s_num_acc_vgprs, s_num_acc_vgprs, 3
+
+ // ACC VGPR count = VGPR count - ARCH VGPR count.
+ get_num_arch_vgprs(s_tmp)
+ s_sub_u32 s_num_acc_vgprs, s_num_acc_vgprs, s_tmp
+end
+#endif
+
function ack_sqc_store_workaround
if ACK_SQC_STORE
s_waitcnt lgkmcnt(0)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
index 8cc51cec988a..43de260b2230 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -556,11 +556,7 @@ static int kfd_ioctl_set_trap_handler(struct file *filep,
goto out;
}
- if (dev->dqm->ops.set_trap_handler(dev->dqm,
- &pdd->qpd,
- args->tba_addr,
- args->tma_addr))
- err = -EINVAL;
+ kfd_process_set_trap_handler(&pdd->qpd, args->tba_addr, args->tma_addr);
out:
mutex_unlock(&p->mutex);
@@ -874,52 +870,47 @@ static int kfd_ioctl_get_process_apertures(struct file *filp,
{
struct kfd_ioctl_get_process_apertures_args *args = data;
struct kfd_process_device_apertures *pAperture;
- struct kfd_process_device *pdd;
+ int i;
dev_dbg(kfd_device, "get apertures for PASID 0x%x", p->pasid);
args->num_of_nodes = 0;
mutex_lock(&p->mutex);
+ /* Run over all pdd of the process */
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
+ pAperture =
+ &args->process_apertures[args->num_of_nodes];
+ pAperture->gpu_id = pdd->dev->id;
+ pAperture->lds_base = pdd->lds_base;
+ pAperture->lds_limit = pdd->lds_limit;
+ pAperture->gpuvm_base = pdd->gpuvm_base;
+ pAperture->gpuvm_limit = pdd->gpuvm_limit;
+ pAperture->scratch_base = pdd->scratch_base;
+ pAperture->scratch_limit = pdd->scratch_limit;
- /*if the process-device list isn't empty*/
- if (kfd_has_process_device_data(p)) {
- /* Run over all pdd of the process */
- pdd = kfd_get_first_process_device_data(p);
- do {
- pAperture =
- &args->process_apertures[args->num_of_nodes];
- pAperture->gpu_id = pdd->dev->id;
- pAperture->lds_base = pdd->lds_base;
- pAperture->lds_limit = pdd->lds_limit;
- pAperture->gpuvm_base = pdd->gpuvm_base;
- pAperture->gpuvm_limit = pdd->gpuvm_limit;
- pAperture->scratch_base = pdd->scratch_base;
- pAperture->scratch_limit = pdd->scratch_limit;
-
- dev_dbg(kfd_device,
- "node id %u\n", args->num_of_nodes);
- dev_dbg(kfd_device,
- "gpu id %u\n", pdd->dev->id);
- dev_dbg(kfd_device,
- "lds_base %llX\n", pdd->lds_base);
- dev_dbg(kfd_device,
- "lds_limit %llX\n", pdd->lds_limit);
- dev_dbg(kfd_device,
- "gpuvm_base %llX\n", pdd->gpuvm_base);
- dev_dbg(kfd_device,
- "gpuvm_limit %llX\n", pdd->gpuvm_limit);
- dev_dbg(kfd_device,
- "scratch_base %llX\n", pdd->scratch_base);
- dev_dbg(kfd_device,
- "scratch_limit %llX\n", pdd->scratch_limit);
-
- args->num_of_nodes++;
-
- pdd = kfd_get_next_process_device_data(p, pdd);
- } while (pdd && (args->num_of_nodes < NUM_OF_SUPPORTED_GPUS));
- }
+ dev_dbg(kfd_device,
+ "node id %u\n", args->num_of_nodes);
+ dev_dbg(kfd_device,
+ "gpu id %u\n", pdd->dev->id);
+ dev_dbg(kfd_device,
+ "lds_base %llX\n", pdd->lds_base);
+ dev_dbg(kfd_device,
+ "lds_limit %llX\n", pdd->lds_limit);
+ dev_dbg(kfd_device,
+ "gpuvm_base %llX\n", pdd->gpuvm_base);
+ dev_dbg(kfd_device,
+ "gpuvm_limit %llX\n", pdd->gpuvm_limit);
+ dev_dbg(kfd_device,
+ "scratch_base %llX\n", pdd->scratch_base);
+ dev_dbg(kfd_device,
+ "scratch_limit %llX\n", pdd->scratch_limit);
+ if (++args->num_of_nodes >= NUM_OF_SUPPORTED_GPUS)
+ break;
+ }
mutex_unlock(&p->mutex);
return 0;
@@ -930,9 +921,8 @@ static int kfd_ioctl_get_process_apertures_new(struct file *filp,
{
struct kfd_ioctl_get_process_apertures_new_args *args = data;
struct kfd_process_device_apertures *pa;
- struct kfd_process_device *pdd;
- uint32_t nodes = 0;
int ret;
+ int i;
dev_dbg(kfd_device, "get apertures for PASID 0x%x", p->pasid);
@@ -941,17 +931,7 @@ static int kfd_ioctl_get_process_apertures_new(struct file *filp,
* sufficient memory
*/
mutex_lock(&p->mutex);
-
- if (!kfd_has_process_device_data(p))
- goto out_unlock;
-
- /* Run over all pdd of the process */
- pdd = kfd_get_first_process_device_data(p);
- do {
- args->num_of_nodes++;
- pdd = kfd_get_next_process_device_data(p, pdd);
- } while (pdd);
-
+ args->num_of_nodes = p->n_pdds;
goto out_unlock;
}
@@ -966,22 +946,23 @@ static int kfd_ioctl_get_process_apertures_new(struct file *filp,
mutex_lock(&p->mutex);
- if (!kfd_has_process_device_data(p)) {
+ if (!p->n_pdds) {
args->num_of_nodes = 0;
kfree(pa);
goto out_unlock;
}
/* Run over all pdd of the process */
- pdd = kfd_get_first_process_device_data(p);
- do {
- pa[nodes].gpu_id = pdd->dev->id;
- pa[nodes].lds_base = pdd->lds_base;
- pa[nodes].lds_limit = pdd->lds_limit;
- pa[nodes].gpuvm_base = pdd->gpuvm_base;
- pa[nodes].gpuvm_limit = pdd->gpuvm_limit;
- pa[nodes].scratch_base = pdd->scratch_base;
- pa[nodes].scratch_limit = pdd->scratch_limit;
+ for (i = 0; i < min(p->n_pdds, args->num_of_nodes); i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
+ pa[i].gpu_id = pdd->dev->id;
+ pa[i].lds_base = pdd->lds_base;
+ pa[i].lds_limit = pdd->lds_limit;
+ pa[i].gpuvm_base = pdd->gpuvm_base;
+ pa[i].gpuvm_limit = pdd->gpuvm_limit;
+ pa[i].scratch_base = pdd->scratch_base;
+ pa[i].scratch_limit = pdd->scratch_limit;
dev_dbg(kfd_device,
"gpu id %u\n", pdd->dev->id);
@@ -997,17 +978,14 @@ static int kfd_ioctl_get_process_apertures_new(struct file *filp,
"scratch_base %llX\n", pdd->scratch_base);
dev_dbg(kfd_device,
"scratch_limit %llX\n", pdd->scratch_limit);
- nodes++;
-
- pdd = kfd_get_next_process_device_data(p, pdd);
- } while (pdd && (nodes < args->num_of_nodes));
+ }
mutex_unlock(&p->mutex);
- args->num_of_nodes = nodes;
+ args->num_of_nodes = i;
ret = copy_to_user(
(void __user *)args->kfd_process_device_apertures_ptr,
pa,
- (nodes * sizeof(struct kfd_process_device_apertures)));
+ (i * sizeof(struct kfd_process_device_apertures)));
kfree(pa);
return ret ? -EFAULT : 0;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
index a5640a6138cf..c60e82697385 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_crat.c
@@ -26,6 +26,7 @@
#include "kfd_priv.h"
#include "kfd_topology.h"
#include "kfd_iommu.h"
+#include "amdgpu.h"
#include "amdgpu_amdkfd.h"
/* GPU Processor ID base for dGPUs for which VCRAT needs to be created.
@@ -665,6 +666,7 @@ static int kfd_fill_gpu_cache_info(struct kfd_dev *kdev,
case CHIP_VEGA12:
case CHIP_VEGA20:
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
pcache_info = vega10_cache_info;
num_of_cache_types = ARRAY_SIZE(vega10_cache_info);
break;
@@ -1112,6 +1114,8 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size,
struct crat_subtype_iolink *sub_type_hdr,
uint32_t proximity_domain)
{
+ struct amdgpu_device *adev = (struct amdgpu_device *)kdev->kgd;
+
*avail_size -= sizeof(struct crat_subtype_iolink);
if (*avail_size < 0)
return -ENOMEM;
@@ -1128,7 +1132,18 @@ static int kfd_fill_gpu_direct_io_link_to_cpu(int *avail_size,
/* Fill in IOLINK subtype.
* TODO: Fill-in other fields of iolink subtype
*/
- sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_PCIEXPRESS;
+ if (adev->gmc.xgmi.connected_to_cpu) {
+ /*
+ * with host gpu xgmi link, host can access gpu memory whether
+ * or not pcie bar type is large, so always create bidirectional
+ * io link.
+ */
+ sub_type_hdr->flags |= CRAT_IOLINK_FLAGS_BI_DIRECTIONAL;
+ sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_XGMI;
+ } else {
+ sub_type_hdr->io_interface_type = CRAT_IOLINK_TYPE_PCIEXPRESS;
+ }
+
sub_type_hdr->proximity_domain_from = proximity_domain;
#ifdef CONFIG_NUMA
if (kdev->pdev->dev.numa_node == NUMA_NO_NODE)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c b/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c
index b258a3dae767..159add0f5aaa 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_dbgdev.c
@@ -155,7 +155,7 @@ static int dbgdev_diq_submit_ib(struct kfd_dbgdev *dbgdev,
/* Wait till CP writes sync code: */
status = amdkfd_fence_wait_timeout(
- (unsigned int *) rm_state,
+ rm_state,
QUEUESTATE__ACTIVE, 1500);
kfd_gtt_sa_free(dbgdev->dev, mem_obj);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_debugfs.c b/drivers/gpu/drm/amd/amdkfd/kfd_debugfs.c
index 511712c2e382..673d5e34f213 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_debugfs.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_debugfs.c
@@ -33,6 +33,11 @@ static int kfd_debugfs_open(struct inode *inode, struct file *file)
return single_open(file, show, NULL);
}
+static int kfd_debugfs_hang_hws_read(struct seq_file *m, void *data)
+{
+ seq_printf(m, "echo gpu_id > hang_hws\n");
+ return 0;
+}
static ssize_t kfd_debugfs_hang_hws_write(struct file *file,
const char __user *user_buf, size_t size, loff_t *ppos)
@@ -94,7 +99,7 @@ void kfd_debugfs_init(void)
debugfs_create_file("rls", S_IFREG | 0444, debugfs_root,
kfd_debugfs_rls_by_device, &kfd_debugfs_fops);
debugfs_create_file("hang_hws", S_IFREG | 0200, debugfs_root,
- NULL, &kfd_debugfs_hang_hws_fops);
+ kfd_debugfs_hang_hws_read, &kfd_debugfs_hang_hws_fops);
}
void kfd_debugfs_fini(void)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device.c b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
index 72c893fff61a..357b9bf62a1c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device.c
@@ -46,6 +46,7 @@ extern const struct kfd2kgd_calls gfx_v7_kfd2kgd;
extern const struct kfd2kgd_calls gfx_v8_kfd2kgd;
extern const struct kfd2kgd_calls gfx_v9_kfd2kgd;
extern const struct kfd2kgd_calls arcturus_kfd2kgd;
+extern const struct kfd2kgd_calls aldebaran_kfd2kgd;
extern const struct kfd2kgd_calls gfx_v10_kfd2kgd;
extern const struct kfd2kgd_calls gfx_v10_3_kfd2kgd;
@@ -71,6 +72,7 @@ static const struct kfd2kgd_calls *kfd2kgd_funcs[] = {
[CHIP_VEGA20] = &gfx_v9_kfd2kgd,
[CHIP_RENOIR] = &gfx_v9_kfd2kgd,
[CHIP_ARCTURUS] = &arcturus_kfd2kgd,
+ [CHIP_ALDEBARAN] = &aldebaran_kfd2kgd,
[CHIP_NAVI10] = &gfx_v10_kfd2kgd,
[CHIP_NAVI12] = &gfx_v10_kfd2kgd,
[CHIP_NAVI14] = &gfx_v10_kfd2kgd,
@@ -392,6 +394,24 @@ static const struct kfd_device_info arcturus_device_info = {
.num_sdma_queues_per_engine = 8,
};
+static const struct kfd_device_info aldebaran_device_info = {
+ .asic_family = CHIP_ALDEBARAN,
+ .asic_name = "aldebaran",
+ .max_pasid_bits = 16,
+ .max_no_of_hqd = 24,
+ .doorbell_size = 8,
+ .ih_ring_entry_size = 8 * sizeof(uint32_t),
+ .event_interrupt_class = &event_interrupt_class_v9,
+ .num_of_watch_points = 4,
+ .mqd_size_aligned = MQD_SIZE_ALIGNED,
+ .supports_cwsr = true,
+ .needs_iommu_device = false,
+ .needs_pci_atomics = false,
+ .num_sdma_engines = 2,
+ .num_xgmi_sdma_engines = 3,
+ .num_sdma_queues_per_engine = 8,
+};
+
static const struct kfd_device_info renoir_device_info = {
.asic_family = CHIP_RENOIR,
.asic_name = "renoir",
@@ -556,6 +576,7 @@ static const struct kfd_device_info *kfd_supported_devices[][2] = {
[CHIP_VEGA20] = {&vega20_device_info, NULL},
[CHIP_RENOIR] = {&renoir_device_info, NULL},
[CHIP_ARCTURUS] = {&arcturus_device_info, &arcturus_device_info},
+ [CHIP_ALDEBARAN] = {&aldebaran_device_info, NULL},
[CHIP_NAVI10] = {&navi10_device_info, NULL},
[CHIP_NAVI12] = {&navi12_device_info, &navi12_device_info},
[CHIP_NAVI14] = {&navi14_device_info, NULL},
@@ -640,6 +661,10 @@ static void kfd_cwsr_init(struct kfd_dev *kfd)
BUILD_BUG_ON(sizeof(cwsr_trap_arcturus_hex) > PAGE_SIZE);
kfd->cwsr_isa = cwsr_trap_arcturus_hex;
kfd->cwsr_isa_size = sizeof(cwsr_trap_arcturus_hex);
+ } else if (kfd->device_info->asic_family == CHIP_ALDEBARAN) {
+ BUILD_BUG_ON(sizeof(cwsr_trap_aldebaran_hex) > PAGE_SIZE);
+ kfd->cwsr_isa = cwsr_trap_aldebaran_hex;
+ kfd->cwsr_isa_size = sizeof(cwsr_trap_aldebaran_hex);
} else if (kfd->device_info->asic_family < CHIP_NAVI10) {
BUILD_BUG_ON(sizeof(cwsr_trap_gfx9_hex) > PAGE_SIZE);
kfd->cwsr_isa = cwsr_trap_gfx9_hex;
@@ -1297,7 +1322,7 @@ void kfd_dec_compute_active(struct kfd_dev *kfd)
void kgd2kfd_smi_event_throttle(struct kfd_dev *kfd, uint32_t throttle_bitmask)
{
- if (kfd)
+ if (kfd && kfd->init_complete)
kfd_smi_event_update_thermal_throttling(kfd, throttle_bitmask);
}
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index e686ce2bf3b3..d3eaa1549bd7 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1128,6 +1128,9 @@ static int set_sched_resources(struct device_queue_manager *dqm)
static int initialize_cpsch(struct device_queue_manager *dqm)
{
+ uint64_t num_sdma_queues;
+ uint64_t num_xgmi_sdma_queues;
+
pr_debug("num of pipes: %d\n", get_pipes_per_mec(dqm));
mutex_init(&dqm->lock_hidden);
@@ -1136,8 +1139,18 @@ static int initialize_cpsch(struct device_queue_manager *dqm)
dqm->active_cp_queue_count = 0;
dqm->gws_queue_count = 0;
dqm->active_runlist = false;
- dqm->sdma_bitmap = ~0ULL >> (64 - get_num_sdma_queues(dqm));
- dqm->xgmi_sdma_bitmap = ~0ULL >> (64 - get_num_xgmi_sdma_queues(dqm));
+
+ num_sdma_queues = get_num_sdma_queues(dqm);
+ if (num_sdma_queues >= BITS_PER_TYPE(dqm->sdma_bitmap))
+ dqm->sdma_bitmap = ULLONG_MAX;
+ else
+ dqm->sdma_bitmap = (BIT_ULL(num_sdma_queues) - 1);
+
+ num_xgmi_sdma_queues = get_num_xgmi_sdma_queues(dqm);
+ if (num_xgmi_sdma_queues >= BITS_PER_TYPE(dqm->xgmi_sdma_bitmap))
+ dqm->xgmi_sdma_bitmap = ULLONG_MAX;
+ else
+ dqm->xgmi_sdma_bitmap = (BIT_ULL(num_xgmi_sdma_queues) - 1);
INIT_WORK(&dqm->hw_exception_work, kfd_process_hw_exception);
@@ -1167,7 +1180,7 @@ static int start_cpsch(struct device_queue_manager *dqm)
if (retval)
goto fail_allocate_vidmem;
- dqm->fence_addr = dqm->fence_mem->cpu_ptr;
+ dqm->fence_addr = (uint64_t *)dqm->fence_mem->cpu_ptr;
dqm->fence_gpu_addr = dqm->fence_mem->gpu_addr;
init_interrupts(dqm);
@@ -1340,8 +1353,8 @@ out:
return retval;
}
-int amdkfd_fence_wait_timeout(unsigned int *fence_addr,
- unsigned int fence_value,
+int amdkfd_fence_wait_timeout(uint64_t *fence_addr,
+ uint64_t fence_value,
unsigned int timeout_ms)
{
unsigned long end_jiffies = msecs_to_jiffies(timeout_ms) + jiffies;
@@ -1393,6 +1406,7 @@ static int unmap_queues_cpsch(struct device_queue_manager *dqm,
uint32_t filter_param)
{
int retval = 0;
+ struct mqd_manager *mqd_mgr;
if (!dqm->sched_running)
return 0;
@@ -1424,6 +1438,22 @@ static int unmap_queues_cpsch(struct device_queue_manager *dqm,
return retval;
}
+ /* In the current MEC firmware implementation, if compute queue
+ * doesn't response to the preemption request in time, HIQ will
+ * abandon the unmap request without returning any timeout error
+ * to driver. Instead, MEC firmware will log the doorbell of the
+ * unresponding compute queue to HIQ.MQD.queue_doorbell_id fields.
+ * To make sure the queue unmap was successful, driver need to
+ * check those fields
+ */
+ mqd_mgr = dqm->mqd_mgrs[KFD_MQD_TYPE_HIQ];
+ if (mqd_mgr->read_doorbell_id(dqm->packets.priv_queue->queue->mqd)) {
+ pr_err("HIQ MQD's queue_doorbell_id0 is not 0, Queue preemption time out\n");
+ while (halt_if_hws_hang)
+ schedule();
+ return -ETIME;
+ }
+
pm_release_ib(&dqm->packets);
dqm->active_runlist = false;
@@ -1596,26 +1626,6 @@ out:
return retval;
}
-static int set_trap_handler(struct device_queue_manager *dqm,
- struct qcm_process_device *qpd,
- uint64_t tba_addr,
- uint64_t tma_addr)
-{
- uint64_t *tma;
-
- if (dqm->dev->cwsr_enabled) {
- /* Jump from CWSR trap handler to user trap */
- tma = (uint64_t *)(qpd->cwsr_kaddr + KFD_CWSR_TMA_OFFSET);
- tma[0] = tba_addr;
- tma[1] = tma_addr;
- } else {
- qpd->tba_addr = tba_addr;
- qpd->tma_addr = tma_addr;
- }
-
- return 0;
-}
-
static int process_termination_nocpsch(struct device_queue_manager *dqm,
struct qcm_process_device *qpd)
{
@@ -1859,7 +1869,6 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev)
dqm->ops.create_kernel_queue = create_kernel_queue_cpsch;
dqm->ops.destroy_kernel_queue = destroy_kernel_queue_cpsch;
dqm->ops.set_cache_memory_policy = set_cache_memory_policy;
- dqm->ops.set_trap_handler = set_trap_handler;
dqm->ops.process_termination = process_termination_cpsch;
dqm->ops.evict_process_queues = evict_process_queues_cpsch;
dqm->ops.restore_process_queues = restore_process_queues_cpsch;
@@ -1878,7 +1887,6 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev)
dqm->ops.initialize = initialize_nocpsch;
dqm->ops.uninitialize = uninitialize;
dqm->ops.set_cache_memory_policy = set_cache_memory_policy;
- dqm->ops.set_trap_handler = set_trap_handler;
dqm->ops.process_termination = process_termination_nocpsch;
dqm->ops.evict_process_queues = evict_process_queues_nocpsch;
dqm->ops.restore_process_queues =
@@ -1918,6 +1926,7 @@ struct device_queue_manager *device_queue_manager_init(struct kfd_dev *dev)
case CHIP_RAVEN:
case CHIP_RENOIR:
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
device_queue_manager_init_v9(&dqm->asic_ops);
break;
case CHIP_NAVI10:
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
index 7351dd195274..71e2fde56b2b 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.h
@@ -121,11 +121,6 @@ struct device_queue_manager_ops {
void __user *alternate_aperture_base,
uint64_t alternate_aperture_size);
- int (*set_trap_handler)(struct device_queue_manager *dqm,
- struct qcm_process_device *qpd,
- uint64_t tba_addr,
- uint64_t tma_addr);
-
int (*process_termination)(struct device_queue_manager *dqm,
struct qcm_process_device *qpd);
@@ -192,7 +187,7 @@ struct device_queue_manager {
uint16_t vmid_pasid[VMID_NUM];
uint64_t pipelines_addr;
uint64_t fence_gpu_addr;
- unsigned int *fence_addr;
+ uint64_t *fence_addr;
struct kfd_mem_obj *fence_mem;
bool active_runlist;
int sched_policy;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
index 98a5e1d719c8..a2c9063076cc 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_flat_memory.c
@@ -412,6 +412,7 @@ int kfd_init_apertures(struct kfd_process *process)
case CHIP_RAVEN:
case CHIP_RENOIR:
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
case CHIP_NAVI10:
case CHIP_NAVI12:
case CHIP_NAVI14:
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c
index 74a460be077b..696944fa0177 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_int_process_v9.c
@@ -25,6 +25,70 @@
#include "soc15_int.h"
#include "kfd_device_queue_manager.h"
#include "kfd_smi_events.h"
+#include "amdgpu.h"
+
+enum SQ_INTERRUPT_WORD_ENCODING {
+ SQ_INTERRUPT_WORD_ENCODING_AUTO = 0x0,
+ SQ_INTERRUPT_WORD_ENCODING_INST,
+ SQ_INTERRUPT_WORD_ENCODING_ERROR,
+};
+
+enum SQ_INTERRUPT_ERROR_TYPE {
+ SQ_INTERRUPT_ERROR_TYPE_EDC_FUE = 0x0,
+ SQ_INTERRUPT_ERROR_TYPE_ILLEGAL_INST,
+ SQ_INTERRUPT_ERROR_TYPE_MEMVIOL,
+ SQ_INTERRUPT_ERROR_TYPE_EDC_FED,
+};
+
+/* SQ_INTERRUPT_WORD_AUTO_CTXID */
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__THREAD_TRACE__SHIFT 0
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__WLT__SHIFT 1
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__THREAD_TRACE_BUF_FULL__SHIFT 2
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__REG_TIMESTAMP__SHIFT 3
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__CMD_TIMESTAMP__SHIFT 4
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__HOST_CMD_OVERFLOW__SHIFT 5
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__HOST_REG_OVERFLOW__SHIFT 6
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__IMMED_OVERFLOW__SHIFT 7
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__THREAD_TRACE_UTC_ERROR__SHIFT 8
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__SE_ID__SHIFT 24
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__ENCODING__SHIFT 26
+
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__THREAD_TRACE_MASK 0x00000001
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__WLT_MASK 0x00000002
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__THREAD_TRACE_BUF_FULL_MASK 0x00000004
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__REG_TIMESTAMP_MASK 0x00000008
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__CMD_TIMESTAMP_MASK 0x00000010
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__HOST_CMD_OVERFLOW_MASK 0x00000020
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__HOST_REG_OVERFLOW_MASK 0x00000040
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__IMMED_OVERFLOW_MASK 0x00000080
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__THREAD_TRACE_UTC_ERROR_MASK 0x00000100
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__SE_ID_MASK 0x03000000
+#define SQ_INTERRUPT_WORD_AUTO_CTXID__ENCODING_MASK 0x0c000000
+
+/* SQ_INTERRUPT_WORD_WAVE_CTXID */
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__DATA__SHIFT 0
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__SH_ID__SHIFT 12
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__PRIV__SHIFT 13
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__WAVE_ID__SHIFT 14
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__SIMD_ID__SHIFT 18
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__CU_ID__SHIFT 20
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__SE_ID__SHIFT 24
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__ENCODING__SHIFT 26
+
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__DATA_MASK 0x00000fff
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__SH_ID_MASK 0x00001000
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__PRIV_MASK 0x00002000
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__WAVE_ID_MASK 0x0003c000
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__SIMD_ID_MASK 0x000c0000
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__CU_ID_MASK 0x00f00000
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__SE_ID_MASK 0x03000000
+#define SQ_INTERRUPT_WORD_WAVE_CTXID__ENCODING_MASK 0x0c000000
+
+#define KFD_CONTEXT_ID_GET_SQ_INT_DATA(ctx0, ctx1) \
+ ((ctx0 & 0xfff) | ((ctx0 >> 16) & 0xf000) | ((ctx1 << 16) & 0xff0000))
+
+#define KFD_SQ_INT_DATA__ERR_TYPE_MASK 0xF00000
+#define KFD_SQ_INT_DATA__ERR_TYPE__SHIFT 20
static bool event_interrupt_isr_v9(struct kfd_dev *dev,
const uint32_t *ih_ring_entry,
@@ -98,22 +162,25 @@ static bool event_interrupt_isr_v9(struct kfd_dev *dev,
source_id == SOC15_INTSRC_SDMA_TRAP ||
source_id == SOC15_INTSRC_SQ_INTERRUPT_MSG ||
source_id == SOC15_INTSRC_CP_BAD_OPCODE ||
- client_id == SOC15_IH_CLIENTID_VMC ||
+ ((client_id == SOC15_IH_CLIENTID_VMC ||
client_id == SOC15_IH_CLIENTID_VMC1 ||
- client_id == SOC15_IH_CLIENTID_UTCL2;
+ client_id == SOC15_IH_CLIENTID_UTCL2) &&
+ !amdgpu_no_queue_eviction_on_vm_fault);
}
static void event_interrupt_wq_v9(struct kfd_dev *dev,
const uint32_t *ih_ring_entry)
{
uint16_t source_id, client_id, pasid, vmid;
- uint32_t context_id;
+ uint32_t context_id0, context_id1;
+ uint32_t sq_intr_err, sq_int_data, encoding;
source_id = SOC15_SOURCE_ID_FROM_IH_ENTRY(ih_ring_entry);
client_id = SOC15_CLIENT_ID_FROM_IH_ENTRY(ih_ring_entry);
pasid = SOC15_PASID_FROM_IH_ENTRY(ih_ring_entry);
vmid = SOC15_VMID_FROM_IH_ENTRY(ih_ring_entry);
- context_id = SOC15_CONTEXT_ID0_FROM_IH_ENTRY(ih_ring_entry);
+ context_id0 = SOC15_CONTEXT_ID0_FROM_IH_ENTRY(ih_ring_entry);
+ context_id1 = SOC15_CONTEXT_ID1_FROM_IH_ENTRY(ih_ring_entry);
if (client_id == SOC15_IH_CLIENTID_GRBM_CP ||
client_id == SOC15_IH_CLIENTID_SE0SH ||
@@ -121,10 +188,59 @@ static void event_interrupt_wq_v9(struct kfd_dev *dev,
client_id == SOC15_IH_CLIENTID_SE2SH ||
client_id == SOC15_IH_CLIENTID_SE3SH) {
if (source_id == SOC15_INTSRC_CP_END_OF_PIPE)
- kfd_signal_event_interrupt(pasid, context_id, 32);
- else if (source_id == SOC15_INTSRC_SQ_INTERRUPT_MSG)
- kfd_signal_event_interrupt(pasid, context_id & 0xffffff, 24);
- else if (source_id == SOC15_INTSRC_CP_BAD_OPCODE)
+ kfd_signal_event_interrupt(pasid, context_id0, 32);
+ else if (source_id == SOC15_INTSRC_SQ_INTERRUPT_MSG) {
+ sq_int_data = KFD_CONTEXT_ID_GET_SQ_INT_DATA(context_id0, context_id1);
+ encoding = REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, ENCODING);
+ switch (encoding) {
+ case SQ_INTERRUPT_WORD_ENCODING_AUTO:
+ pr_debug(
+ "sq_intr: auto, se %d, ttrace %d, wlt %d, ttrac_buf_full %d, reg_tms %d, cmd_tms %d, host_cmd_ovf %d, host_reg_ovf %d, immed_ovf %d, ttrace_utc_err %d\n",
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, SE_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, THREAD_TRACE),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, WLT),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, THREAD_TRACE_BUF_FULL),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, REG_TIMESTAMP),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, CMD_TIMESTAMP),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, HOST_CMD_OVERFLOW),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, HOST_REG_OVERFLOW),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, IMMED_OVERFLOW),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_AUTO_CTXID, THREAD_TRACE_UTC_ERROR));
+ break;
+ case SQ_INTERRUPT_WORD_ENCODING_INST:
+ pr_debug("sq_intr: inst, se %d, data 0x%x, sh %d, priv %d, wave_id %d, simd_id %d, cu_id %d, intr_data 0x%x\n",
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, SE_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, DATA),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, SH_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, PRIV),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, WAVE_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, SIMD_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, CU_ID),
+ sq_int_data);
+ break;
+ case SQ_INTERRUPT_WORD_ENCODING_ERROR:
+ sq_intr_err = REG_GET_FIELD(sq_int_data, KFD_SQ_INT_DATA, ERR_TYPE);
+ pr_warn("sq_intr: error, se %d, data 0x%x, sh %d, priv %d, wave_id %d, simd_id %d, cu_id %d, err_type %d\n",
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, SE_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, DATA),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, SH_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, PRIV),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, WAVE_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, SIMD_ID),
+ REG_GET_FIELD(context_id0, SQ_INTERRUPT_WORD_WAVE_CTXID, CU_ID),
+ sq_intr_err);
+ if (sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_ILLEGAL_INST &&
+ sq_intr_err != SQ_INTERRUPT_ERROR_TYPE_MEMVIOL) {
+ kfd_signal_hw_exception_event(pasid);
+ amdgpu_amdkfd_gpu_reset(dev->kgd);
+ return;
+ }
+ break;
+ default:
+ break;
+ }
+ kfd_signal_event_interrupt(pasid, context_id0 & 0xffffff, 24);
+ } else if (source_id == SOC15_INTSRC_CP_BAD_OPCODE)
kfd_signal_hw_exception_event(pasid);
} else if (client_id == SOC15_IH_CLIENTID_SDMA0 ||
client_id == SOC15_IH_CLIENTID_SDMA1 ||
@@ -135,7 +251,7 @@ static void event_interrupt_wq_v9(struct kfd_dev *dev,
client_id == SOC15_IH_CLIENTID_SDMA6 ||
client_id == SOC15_IH_CLIENTID_SDMA7) {
if (source_id == SOC15_INTSRC_SDMA_TRAP)
- kfd_signal_event_interrupt(pasid, context_id & 0xfffffff, 28);
+ kfd_signal_event_interrupt(pasid, context_id0 & 0xfffffff, 28);
} else if (client_id == SOC15_IH_CLIENTID_VMC ||
client_id == SOC15_IH_CLIENTID_VMC1 ||
client_id == SOC15_IH_CLIENTID_UTCL2) {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
index 66bbca61e3ef..73f2257acc23 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.c
@@ -20,6 +20,10 @@
* OTHER DEALINGS IN THE SOFTWARE.
*/
+#include <linux/kconfig.h>
+
+#if IS_REACHABLE(CONFIG_AMD_IOMMU_V2)
+
#include <linux/printk.h>
#include <linux/device.h>
#include <linux/slab.h>
@@ -131,11 +135,11 @@ int kfd_iommu_bind_process_to_device(struct kfd_process_device *pdd)
*/
void kfd_iommu_unbind_process(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- if (pdd->bound == PDD_BOUND)
- amd_iommu_unbind_pasid(pdd->dev->pdev, p->pasid);
+ for (i = 0; i < p->n_pdds; i++)
+ if (p->pdds[i]->bound == PDD_BOUND)
+ amd_iommu_unbind_pasid(p->pdds[i]->dev->pdev, p->pasid);
}
/* Callback for process shutdown invoked by the IOMMU driver */
@@ -329,10 +333,6 @@ int kfd_iommu_resume(struct kfd_dev *kfd)
return 0;
}
-extern bool amd_iommu_pc_supported(void);
-extern u8 amd_iommu_pc_get_max_banks(u16 devid);
-extern u8 amd_iommu_pc_get_max_counters(u16 devid);
-
/** kfd_iommu_add_perf_counters - Add IOMMU performance counters to topology
*/
int kfd_iommu_add_perf_counters(struct kfd_topology_device *kdev)
@@ -355,3 +355,5 @@ int kfd_iommu_add_perf_counters(struct kfd_topology_device *kdev)
return 0;
}
+
+#endif
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.h b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.h
index dd23d9fdf6a8..afd420b01a0c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_iommu.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_iommu.h
@@ -23,7 +23,9 @@
#ifndef __KFD_IOMMU_H__
#define __KFD_IOMMU_H__
-#if defined(CONFIG_AMD_IOMMU_V2_MODULE) || defined(CONFIG_AMD_IOMMU_V2)
+#include <linux/kconfig.h>
+
+#if IS_REACHABLE(CONFIG_AMD_IOMMU_V2)
#define KFD_SUPPORT_IOMMU_V2
@@ -46,6 +48,9 @@ static inline int kfd_iommu_check_device(struct kfd_dev *kfd)
}
static inline int kfd_iommu_device_init(struct kfd_dev *kfd)
{
+#if IS_MODULE(CONFIG_AMD_IOMMU_V2)
+ WARN_ONCE(1, "iommu_v2 module is not usable by built-in KFD");
+#endif
return 0;
}
@@ -73,6 +78,6 @@ static inline int kfd_iommu_add_perf_counters(struct kfd_topology_device *kdev)
return 0;
}
-#endif /* defined(CONFIG_AMD_IOMMU_V2) */
+#endif /* IS_REACHABLE(CONFIG_AMD_IOMMU_V2) */
#endif /* __KFD_IOMMU_H__ */
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
index fbdb16418847..b5e2ea7550d4 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager.h
@@ -101,6 +101,7 @@ struct mqd_manager {
#if defined(CONFIG_DEBUG_FS)
int (*debugfs_show_mqd)(struct seq_file *m, void *data);
#endif
+ uint32_t (*read_doorbell_id)(void *mqd);
struct mutex mqd_mutex;
struct kfd_dev *dev;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
index 19f0fe547c57..064914e1e8d6 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_cik.c
@@ -226,6 +226,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
__update_mqd(mm, mqd, q, 1);
}
+static uint32_t read_doorbell_id(void *mqd)
+{
+ struct cik_mqd *m = (struct cik_mqd *)mqd;
+
+ return m->queue_doorbell_id0;
+}
+
static void update_mqd_hawaii(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
@@ -398,6 +405,7 @@ struct mqd_manager *mqd_manager_init_cik(enum KFD_MQD_TYPE type,
#if defined(CONFIG_DEBUG_FS)
mqd->debugfs_show_mqd = debugfs_show_mqd;
#endif
+ mqd->read_doorbell_id = read_doorbell_id;
break;
case KFD_MQD_TYPE_DIQ:
mqd->allocate_mqd = allocate_mqd;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
index 18e08d82d978..c7fb59ca597f 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v10.c
@@ -224,6 +224,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
q->is_active = QUEUE_IS_ACTIVE(*q);
}
+static uint32_t read_doorbell_id(void *mqd)
+{
+ struct v10_compute_mqd *m = (struct v10_compute_mqd *)mqd;
+
+ return m->queue_doorbell_id0;
+}
+
static int destroy_mqd(struct mqd_manager *mm, void *mqd,
enum kfd_preempt_type type,
unsigned int timeout, uint32_t pipe_id,
@@ -425,6 +432,7 @@ struct mqd_manager *mqd_manager_init_v10(enum KFD_MQD_TYPE type,
#if defined(CONFIG_DEBUG_FS)
mqd->debugfs_show_mqd = debugfs_show_mqd;
#endif
+ mqd->read_doorbell_id = read_doorbell_id;
pr_debug("%s@%i\n", __func__, __LINE__);
break;
case KFD_MQD_TYPE_DIQ:
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
index 3b6f5963180d..7f4e102ff4bd 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_v9.c
@@ -276,6 +276,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
}
+static uint32_t read_doorbell_id(void *mqd)
+{
+ struct v9_mqd *m = (struct v9_mqd *)mqd;
+
+ return m->queue_doorbell_id0;
+}
+
static int destroy_mqd(struct mqd_manager *mm, void *mqd,
enum kfd_preempt_type type,
unsigned int timeout, uint32_t pipe_id,
@@ -477,6 +484,7 @@ struct mqd_manager *mqd_manager_init_v9(enum KFD_MQD_TYPE type,
#if defined(CONFIG_DEBUG_FS)
mqd->debugfs_show_mqd = debugfs_show_mqd;
#endif
+ mqd->read_doorbell_id = read_doorbell_id;
break;
case KFD_MQD_TYPE_DIQ:
mqd->allocate_mqd = allocate_mqd;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
index 31799e5f3b3c..33dbd22d290f 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_mqd_manager_vi.c
@@ -243,6 +243,13 @@ static void update_mqd(struct mqd_manager *mm, void *mqd,
__update_mqd(mm, mqd, q, MTYPE_CC, 1);
}
+static uint32_t read_doorbell_id(void *mqd)
+{
+ struct vi_mqd *m = (struct vi_mqd *)mqd;
+
+ return m->queue_doorbell_id0;
+}
+
static void update_mqd_tonga(struct mqd_manager *mm, void *mqd,
struct queue_properties *q)
{
@@ -446,6 +453,7 @@ struct mqd_manager *mqd_manager_init_vi(enum KFD_MQD_TYPE type,
#if defined(CONFIG_DEBUG_FS)
mqd->debugfs_show_mqd = debugfs_show_mqd;
#endif
+ mqd->read_doorbell_id = read_doorbell_id;
break;
case KFD_MQD_TYPE_DIQ:
mqd->allocate_mqd = allocate_mqd;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
index 5d541e0cc8ca..e840dd581719 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager.c
@@ -242,6 +242,7 @@ int pm_init(struct packet_manager *pm, struct device_queue_manager *dqm)
case CHIP_RAVEN:
case CHIP_RENOIR:
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
case CHIP_NAVI10:
case CHIP_NAVI12:
case CHIP_NAVI14:
@@ -347,7 +348,7 @@ fail_create_runlist_ib:
}
int pm_send_query_status(struct packet_manager *pm, uint64_t fence_address,
- uint32_t fence_value)
+ uint64_t fence_value)
{
uint32_t *buffer, size;
int retval = 0;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
index dfaf771a42e6..e3ba0cd3b6fa 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_v9.c
@@ -283,7 +283,7 @@ static int pm_unmap_queues_v9(struct packet_manager *pm, uint32_t *buffer,
}
static int pm_query_status_v9(struct packet_manager *pm, uint32_t *buffer,
- uint64_t fence_address, uint32_t fence_value)
+ uint64_t fence_address, uint64_t fence_value)
{
struct pm4_mes_query_status *packet;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
index a852e0d7d804..08442e7d9944 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_packet_manager_vi.c
@@ -263,7 +263,7 @@ static int pm_unmap_queues_vi(struct packet_manager *pm, uint32_t *buffer,
}
static int pm_query_status_vi(struct packet_manager *pm, uint32_t *buffer,
- uint64_t fence_address, uint32_t fence_value)
+ uint64_t fence_address, uint64_t fence_value)
{
struct pm4_mes_query_status *packet;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index 09599efa41fc..0b6595f7acda 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -45,6 +45,7 @@
#include <linux/swap.h>
#include "amd_shared.h"
+#include "amdgpu.h"
#define KFD_MAX_RING_ENTRY_SIZE 8
@@ -169,6 +170,11 @@ extern bool hws_gws_support;
/* Queue preemption timeout in ms */
extern int queue_preemption_timeout_ms;
+/*
+ * Don't evict process queues on vm fault
+ */
+extern int amdgpu_no_queue_eviction_on_vm_fault;
+
/* Enable eviction debug messages */
extern bool debug_evictions;
@@ -644,12 +650,6 @@ enum kfd_pdd_bound {
/* Data that is per-process-per device. */
struct kfd_process_device {
- /*
- * List of all per-device data for a process.
- * Starts from kfd_process.per_device_data.
- */
- struct list_head per_device_list;
-
/* The device that owns this data. */
struct kfd_dev *dev;
@@ -766,10 +766,11 @@ struct kfd_process {
u32 pasid;
/*
- * List of kfd_process_device structures,
+ * Array of kfd_process_device pointers,
* one for each device the process is using.
*/
- struct list_head per_device_data;
+ struct kfd_process_device *pdds[MAX_GPU_INSTANCE];
+ uint32_t n_pdds;
struct process_queue_manager pqm;
@@ -867,14 +868,6 @@ void *kfd_process_device_translate_handle(struct kfd_process_device *p,
void kfd_process_device_remove_obj_handle(struct kfd_process_device *pdd,
int handle);
-/* Process device data iterator */
-struct kfd_process_device *kfd_get_first_process_device_data(
- struct kfd_process *p);
-struct kfd_process_device *kfd_get_next_process_device_data(
- struct kfd_process *p,
- struct kfd_process_device *pdd);
-bool kfd_has_process_device_data(struct kfd_process *p);
-
/* PASIDs */
int kfd_pasid_init(void);
void kfd_pasid_exit(void);
@@ -944,6 +937,10 @@ bool interrupt_is_wanted(struct kfd_dev *dev,
/* amdkfd Apertures */
int kfd_init_apertures(struct kfd_process *process);
+void kfd_process_set_trap_handler(struct qcm_process_device *qpd,
+ uint64_t tba_addr,
+ uint64_t tma_addr);
+
/* Queue Context Management */
int init_queue(struct queue **q, const struct queue_properties *properties);
void uninit_queue(struct queue *q);
@@ -1003,8 +1000,8 @@ int pqm_get_wave_state(struct process_queue_manager *pqm,
u32 *ctl_stack_used_size,
u32 *save_area_used_size);
-int amdkfd_fence_wait_timeout(unsigned int *fence_addr,
- unsigned int fence_value,
+int amdkfd_fence_wait_timeout(uint64_t *fence_addr,
+ uint64_t fence_value,
unsigned int timeout_ms);
/* Packet Manager */
@@ -1040,7 +1037,7 @@ struct packet_manager_funcs {
uint32_t filter_param, bool reset,
unsigned int sdma_engine);
int (*query_status)(struct packet_manager *pm, uint32_t *buffer,
- uint64_t fence_address, uint32_t fence_value);
+ uint64_t fence_address, uint64_t fence_value);
int (*release_mem)(uint64_t gpu_addr, uint32_t *buffer);
/* Packet sizes */
@@ -1062,7 +1059,7 @@ int pm_send_set_resources(struct packet_manager *pm,
struct scheduling_resources *res);
int pm_send_runlist(struct packet_manager *pm, struct list_head *dqm_queues);
int pm_send_query_status(struct packet_manager *pm, uint64_t fence_address,
- uint32_t fence_value);
+ uint64_t fence_value);
int pm_send_unmap_queue(struct packet_manager *pm, enum kfd_queue_type type,
enum kfd_unmap_queues_filter mode,
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 65803e153a22..d97e330a5022 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -505,7 +505,7 @@ static int kfd_sysfs_create_file(struct kfd_process *p, struct attribute *attr,
static int kfd_procfs_add_sysfs_stats(struct kfd_process *p)
{
int ret = 0;
- struct kfd_process_device *pdd;
+ int i;
char stats_dir_filename[MAX_SYSFS_FILENAME_LEN];
if (!p)
@@ -520,7 +520,8 @@ static int kfd_procfs_add_sysfs_stats(struct kfd_process *p)
* - proc/<pid>/stats_<gpuid>/evicted_ms
* - proc/<pid>/stats_<gpuid>/cu_occupancy
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
struct kobject *kobj_stats;
snprintf(stats_dir_filename, MAX_SYSFS_FILENAME_LEN,
@@ -571,7 +572,7 @@ err:
static int kfd_procfs_add_sysfs_files(struct kfd_process *p)
{
int ret = 0;
- struct kfd_process_device *pdd;
+ int i;
if (!p)
return -EINVAL;
@@ -584,7 +585,9 @@ static int kfd_procfs_add_sysfs_files(struct kfd_process *p)
* - proc/<pid>/vram_<gpuid>
* - proc/<pid>/sdma_<gpuid>
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
snprintf(pdd->vram_filename, MAX_SYSFS_FILENAME_LEN, "vram_%u",
pdd->dev->id);
ret = kfd_sysfs_create_file(p, &pdd->attr_vram, pdd->vram_filename);
@@ -775,10 +778,8 @@ struct kfd_process *kfd_create_process(struct file *filep)
goto out;
ret = kfd_process_init_cwsr_apu(process, filep);
- if (ret) {
- process = ERR_PTR(ret);
- goto out;
- }
+ if (ret)
+ goto out_destroy;
if (!procfs.kobj)
goto out;
@@ -826,6 +827,14 @@ out:
mutex_unlock(&kfd_processes_mutex);
return process;
+
+out_destroy:
+ hash_del_rcu(&process->kfd_processes);
+ mutex_unlock(&kfd_processes_mutex);
+ synchronize_srcu(&kfd_processes_srcu);
+ /* kfd_process_free_notifier will trigger the cleanup */
+ mmu_notifier_put(&process->mmu_notifier);
+ return ERR_PTR(ret);
}
struct kfd_process *kfd_get_process(const struct task_struct *thread)
@@ -875,21 +884,23 @@ void kfd_unref_process(struct kfd_process *p)
kref_put(&p->ref, kfd_process_ref_release);
}
+
static void kfd_process_device_free_bos(struct kfd_process_device *pdd)
{
struct kfd_process *p = pdd->process;
void *mem;
int id;
+ int i;
/*
* Remove all handles from idr and release appropriate
* local memory object
*/
idr_for_each_entry(&pdd->alloc_idr, mem, id) {
- struct kfd_process_device *peer_pdd;
- list_for_each_entry(peer_pdd, &p->per_device_data,
- per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *peer_pdd = p->pdds[i];
+
if (!peer_pdd->vm)
continue;
amdgpu_amdkfd_gpuvm_unmap_memory_from_gpu(
@@ -903,18 +914,19 @@ static void kfd_process_device_free_bos(struct kfd_process_device *pdd)
static void kfd_process_free_outstanding_kfd_bos(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- kfd_process_device_free_bos(pdd);
+ for (i = 0; i < p->n_pdds; i++)
+ kfd_process_device_free_bos(p->pdds[i]);
}
static void kfd_process_destroy_pdds(struct kfd_process *p)
{
- struct kfd_process_device *pdd, *temp;
+ int i;
+
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
- list_for_each_entry_safe(pdd, temp, &p->per_device_data,
- per_device_list) {
pr_debug("Releasing pdd (topology id %d) for process (pasid 0x%x)\n",
pdd->dev->id, p->pasid);
@@ -923,11 +935,6 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
pdd->dev->kgd, pdd->vm);
fput(pdd->drm_file);
}
- else if (pdd->vm)
- amdgpu_amdkfd_gpuvm_destroy_process_vm(
- pdd->dev->kgd, pdd->vm);
-
- list_del(&pdd->per_device_list);
if (pdd->qpd.cwsr_kaddr && !pdd->qpd.cwsr_base)
free_pages((unsigned long)pdd->qpd.cwsr_kaddr,
@@ -949,7 +956,9 @@ static void kfd_process_destroy_pdds(struct kfd_process *p)
}
kfree(pdd);
+ p->pdds[i] = NULL;
}
+ p->n_pdds = 0;
}
/* No process locking is needed in this function, because the process
@@ -961,7 +970,7 @@ static void kfd_process_wq_release(struct work_struct *work)
{
struct kfd_process *p = container_of(work, struct kfd_process,
release_work);
- struct kfd_process_device *pdd;
+ int i;
/* Remove the procfs files */
if (p->kobj) {
@@ -970,7 +979,9 @@ static void kfd_process_wq_release(struct work_struct *work)
kobject_put(p->kobj_queues);
p->kobj_queues = NULL;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
sysfs_remove_file(p->kobj, &pdd->attr_vram);
sysfs_remove_file(p->kobj, &pdd->attr_sdma);
sysfs_remove_file(p->kobj, &pdd->attr_evict);
@@ -1011,6 +1022,16 @@ static void kfd_process_ref_release(struct kref *ref)
queue_work(kfd_process_wq, &p->release_work);
}
+static struct mmu_notifier *kfd_process_alloc_notifier(struct mm_struct *mm)
+{
+ int idx = srcu_read_lock(&kfd_processes_srcu);
+ struct kfd_process *p = find_process_by_mm(mm);
+
+ srcu_read_unlock(&kfd_processes_srcu, idx);
+
+ return p ? &p->mmu_notifier : ERR_PTR(-ESRCH);
+}
+
static void kfd_process_free_notifier(struct mmu_notifier *mn)
{
kfd_unref_process(container_of(mn, struct kfd_process, mmu_notifier));
@@ -1020,7 +1041,7 @@ static void kfd_process_notifier_release(struct mmu_notifier *mn,
struct mm_struct *mm)
{
struct kfd_process *p;
- struct kfd_process_device *pdd = NULL;
+ int i;
/*
* The kfd_process structure can not be free because the
@@ -1044,8 +1065,8 @@ static void kfd_process_notifier_release(struct mmu_notifier *mn,
* pdd is in debug mode, we should first force unregistration,
* then we will be able to destroy the queues
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
- struct kfd_dev *dev = pdd->dev;
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_dev *dev = p->pdds[i]->dev;
mutex_lock(kfd_get_dbgmgr_mutex());
if (dev && dev->dbgmgr && dev->dbgmgr->pasid == p->pasid) {
@@ -1075,17 +1096,18 @@ static void kfd_process_notifier_release(struct mmu_notifier *mn,
static const struct mmu_notifier_ops kfd_process_mmu_notifier_ops = {
.release = kfd_process_notifier_release,
+ .alloc_notifier = kfd_process_alloc_notifier,
.free_notifier = kfd_process_free_notifier,
};
static int kfd_process_init_cwsr_apu(struct kfd_process *p, struct file *filep)
{
unsigned long offset;
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
- struct kfd_dev *dev = pdd->dev;
- struct qcm_process_device *qpd = &pdd->qpd;
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_dev *dev = p->pdds[i]->dev;
+ struct qcm_process_device *qpd = &p->pdds[i]->qpd;
if (!dev->cwsr_enabled || qpd->cwsr_kaddr || qpd->cwsr_base)
continue;
@@ -1145,6 +1167,25 @@ static int kfd_process_device_init_cwsr_dgpu(struct kfd_process_device *pdd)
return 0;
}
+void kfd_process_set_trap_handler(struct qcm_process_device *qpd,
+ uint64_t tba_addr,
+ uint64_t tma_addr)
+{
+ if (qpd->cwsr_kaddr) {
+ /* KFD trap handler is bound, record as second-level TBA/TMA
+ * in first-level TMA. First-level trap will jump to second.
+ */
+ uint64_t *tma =
+ (uint64_t *)(qpd->cwsr_kaddr + KFD_CWSR_TMA_OFFSET);
+ tma[0] = tba_addr;
+ tma[1] = tma_addr;
+ } else {
+ /* No trap handler bound, bind as first-level TBA/TMA. */
+ qpd->tba_addr = tba_addr;
+ qpd->tma_addr = tma_addr;
+ }
+}
+
/*
* On return the kfd_process is fully operational and will be freed when the
* mm is released
@@ -1152,6 +1193,7 @@ static int kfd_process_device_init_cwsr_dgpu(struct kfd_process_device *pdd)
static struct kfd_process *create_process(const struct task_struct *thread)
{
struct kfd_process *process;
+ struct mmu_notifier *mn;
int err = -ENOMEM;
process = kzalloc(sizeof(*process), GFP_KERNEL);
@@ -1162,7 +1204,7 @@ static struct kfd_process *create_process(const struct task_struct *thread)
mutex_init(&process->mutex);
process->mm = thread->mm;
process->lead_thread = thread->group_leader;
- INIT_LIST_HEAD(&process->per_device_data);
+ process->n_pdds = 0;
INIT_DELAYED_WORK(&process->eviction_work, evict_process_worker);
INIT_DELAYED_WORK(&process->restore_work, restore_process_worker);
process->last_restore_timestamp = get_jiffies_64();
@@ -1182,19 +1224,28 @@ static struct kfd_process *create_process(const struct task_struct *thread)
if (err != 0)
goto err_init_apertures;
- /* Must be last, have to use release destruction after this */
- process->mmu_notifier.ops = &kfd_process_mmu_notifier_ops;
- err = mmu_notifier_register(&process->mmu_notifier, process->mm);
- if (err)
+ /* alloc_notifier needs to find the process in the hash table */
+ hash_add_rcu(kfd_processes_table, &process->kfd_processes,
+ (uintptr_t)process->mm);
+
+ /* MMU notifier registration must be the last call that can fail
+ * because after this point we cannot unwind the process creation.
+ * After this point, mmu_notifier_put will trigger the cleanup by
+ * dropping the last process reference in the free_notifier.
+ */
+ mn = mmu_notifier_get(&kfd_process_mmu_notifier_ops, process->mm);
+ if (IS_ERR(mn)) {
+ err = PTR_ERR(mn);
goto err_register_notifier;
+ }
+ BUG_ON(mn != &process->mmu_notifier);
get_task_struct(process->lead_thread);
- hash_add_rcu(kfd_processes_table, &process->kfd_processes,
- (uintptr_t)process->mm);
return process;
err_register_notifier:
+ hash_del_rcu(&process->kfd_processes);
kfd_process_free_outstanding_kfd_bos(process);
kfd_process_destroy_pdds(process);
err_init_apertures:
@@ -1244,11 +1295,11 @@ static int init_doorbell_bitmap(struct qcm_process_device *qpd,
struct kfd_process_device *kfd_get_process_device_data(struct kfd_dev *dev,
struct kfd_process *p)
{
- struct kfd_process_device *pdd = NULL;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- if (pdd->dev == dev)
- return pdd;
+ for (i = 0; i < p->n_pdds; i++)
+ if (p->pdds[i]->dev == dev)
+ return p->pdds[i];
return NULL;
}
@@ -1258,6 +1309,8 @@ struct kfd_process_device *kfd_create_process_device_data(struct kfd_dev *dev,
{
struct kfd_process_device *pdd = NULL;
+ if (WARN_ON_ONCE(p->n_pdds >= MAX_GPU_INSTANCE))
+ return NULL;
pdd = kzalloc(sizeof(*pdd), GFP_KERNEL);
if (!pdd)
return NULL;
@@ -1286,7 +1339,7 @@ struct kfd_process_device *kfd_create_process_device_data(struct kfd_dev *dev,
pdd->vram_usage = 0;
pdd->sdma_past_activity_counter = 0;
atomic64_set(&pdd->evict_duration_counter, 0);
- list_add(&pdd->per_device_list, &p->per_device_data);
+ p->pdds[p->n_pdds++] = pdd;
/* Init idr used for memory handle translation */
idr_init(&pdd->alloc_idr);
@@ -1319,19 +1372,18 @@ int kfd_process_device_init_vm(struct kfd_process_device *pdd,
struct kfd_dev *dev;
int ret;
+ if (!drm_file)
+ return -EINVAL;
+
if (pdd->vm)
- return drm_file ? -EBUSY : 0;
+ return -EBUSY;
p = pdd->process;
dev = pdd->dev;
- if (drm_file)
- ret = amdgpu_amdkfd_gpuvm_acquire_process_vm(
- dev->kgd, drm_file, p->pasid,
- &pdd->vm, &p->kgd_process_info, &p->ef);
- else
- ret = amdgpu_amdkfd_gpuvm_create_process_vm(dev->kgd, p->pasid,
- &pdd->vm, &p->kgd_process_info, &p->ef);
+ ret = amdgpu_amdkfd_gpuvm_acquire_process_vm(
+ dev->kgd, drm_file, p->pasid,
+ &pdd->vm, &p->kgd_process_info, &p->ef);
if (ret) {
pr_err("Failed to create process VM object\n");
return ret;
@@ -1353,8 +1405,6 @@ int kfd_process_device_init_vm(struct kfd_process_device *pdd,
err_init_cwsr:
err_reserve_ib_mem:
kfd_process_device_free_bos(pdd);
- if (!drm_file)
- amdgpu_amdkfd_gpuvm_destroy_process_vm(dev->kgd, pdd->vm);
pdd->vm = NULL;
return ret;
@@ -1379,6 +1429,9 @@ struct kfd_process_device *kfd_bind_process_to_device(struct kfd_dev *dev,
return ERR_PTR(-ENOMEM);
}
+ if (!pdd->vm)
+ return ERR_PTR(-ENODEV);
+
/*
* signal runtime-pm system to auto resume and prevent
* further runtime suspend once device pdd is created until
@@ -1396,10 +1449,6 @@ struct kfd_process_device *kfd_bind_process_to_device(struct kfd_dev *dev,
if (err)
goto out;
- err = kfd_process_device_init_vm(pdd, NULL);
- if (err)
- goto out;
-
/*
* make sure that runtime_usage counter is incremented just once
* per pdd
@@ -1418,28 +1467,6 @@ out:
return ERR_PTR(err);
}
-struct kfd_process_device *kfd_get_first_process_device_data(
- struct kfd_process *p)
-{
- return list_first_entry(&p->per_device_data,
- struct kfd_process_device,
- per_device_list);
-}
-
-struct kfd_process_device *kfd_get_next_process_device_data(
- struct kfd_process *p,
- struct kfd_process_device *pdd)
-{
- if (list_is_last(&pdd->per_device_list, &p->per_device_data))
- return NULL;
- return list_next_entry(pdd, per_device_list);
-}
-
-bool kfd_has_process_device_data(struct kfd_process *p)
-{
- return !(list_empty(&p->per_device_data));
-}
-
/* Create specific handle mapped to mem from process local memory idr
* Assumes that the process lock is held.
*/
@@ -1515,11 +1542,13 @@ struct kfd_process *kfd_lookup_process_by_mm(const struct mm_struct *mm)
*/
int kfd_process_evict_queues(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
int r = 0;
+ int i;
unsigned int n_evicted = 0;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
r = pdd->dev->dqm->ops.evict_process_queues(pdd->dev->dqm,
&pdd->qpd);
if (r) {
@@ -1535,7 +1564,9 @@ fail:
/* To keep state consistent, roll back partial eviction by
* restoring queues
*/
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
+
if (n_evicted == 0)
break;
if (pdd->dev->dqm->ops.restore_process_queues(pdd->dev->dqm,
@@ -1551,10 +1582,12 @@ fail:
/* kfd_process_restore_queues - Restore all user queues of a process */
int kfd_process_restore_queues(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
int r, ret = 0;
+ int i;
+
+ for (i = 0; i < p->n_pdds; i++) {
+ struct kfd_process_device *pdd = p->pdds[i];
- list_for_each_entry(pdd, &p->per_device_data, per_device_list) {
r = pdd->dev->dqm->ops.restore_process_queues(pdd->dev->dqm,
&pdd->qpd);
if (r) {
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
index eb1635ac8988..95a6c36cea4c 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process_queue_manager.c
@@ -126,10 +126,10 @@ int pqm_set_gws(struct process_queue_manager *pqm, unsigned int qid,
void kfd_process_dequeue_from_all_devices(struct kfd_process *p)
{
- struct kfd_process_device *pdd;
+ int i;
- list_for_each_entry(pdd, &p->per_device_data, per_device_list)
- kfd_process_dequeue_from_device(pdd);
+ for (i = 0; i < p->n_pdds; i++)
+ kfd_process_dequeue_from_device(p->pdds[i]);
}
int pqm_init(struct process_queue_manager *pqm, struct kfd_process *p)
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c b/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
index 17d1736367ea..246522423559 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_smi_events.c
@@ -81,7 +81,7 @@ static ssize_t kfd_smi_ev_read(struct file *filep, char __user *user,
struct kfd_smi_client *client = filep->private_data;
unsigned char *buf;
- buf = kmalloc(MAX_KFIFO_SIZE * sizeof(*buf), GFP_KERNEL);
+ buf = kmalloc_array(MAX_KFIFO_SIZE, sizeof(*buf), GFP_KERNEL);
if (!buf)
return -ENOMEM;
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
index 0be72789ccbc..cdef608db4f4 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.c
@@ -1370,6 +1370,7 @@ int kfd_topology_add_device(struct kfd_dev *gpu)
case CHIP_RAVEN:
case CHIP_RENOIR:
case CHIP_ARCTURUS:
+ case CHIP_ALDEBARAN:
case CHIP_NAVI10:
case CHIP_NAVI12:
case CHIP_NAVI14:
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
index 416fd910e12e..b8b68087bd7a 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_topology.h
@@ -47,13 +47,14 @@
#define HSA_CAP_DOORBELL_TYPE_2_0 0x2
#define HSA_CAP_AQL_QUEUE_DOUBLE_MAP 0x00004000
-#define HSA_CAP_SRAM_EDCSUPPORTED 0x00080000
+#define HSA_CAP_RESERVED_WAS_SRAM_EDCSUPPORTED 0x00080000 /* Old buggy user mode depends on this being 0 */
#define HSA_CAP_MEM_EDCSUPPORTED 0x00100000
#define HSA_CAP_RASEVENTNOTIFY 0x00200000
#define HSA_CAP_ASIC_REVISION_MASK 0x03c00000
#define HSA_CAP_ASIC_REVISION_SHIFT 22
+#define HSA_CAP_SRAM_EDCSUPPORTED 0x04000000
-#define HSA_CAP_RESERVED 0xfc078000
+#define HSA_CAP_RESERVED 0xf80f8000
struct kfd_node_properties {
uint64_t hive_id;
diff --git a/drivers/gpu/drm/amd/display/Kconfig b/drivers/gpu/drm/amd/display/Kconfig
index e509a175ed17..7dffc04a557e 100644
--- a/drivers/gpu/drm/amd/display/Kconfig
+++ b/drivers/gpu/drm/amd/display/Kconfig
@@ -38,4 +38,18 @@ config DEBUG_KERNEL_DC
help
Choose this option if you want to hit kdgb_break in assert.
+config DRM_AMD_SECURE_DISPLAY
+ bool "Enable secure display support"
+ default n
+ depends on DEBUG_FS
+ depends on DRM_AMD_DC_DCN
+ help
+ Choose this option if you want to
+ support secure display
+
+ This option enables the calculation
+ of crc of specific region via debugfs.
+ Cooperate with specific DMCU FW.
+
+
endmenu
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
index 3e1fd1e7d09f..389eff96fcf6 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
@@ -34,6 +34,7 @@
#include "dc/inc/hw/dmcu.h"
#include "dc/inc/hw/abm.h"
#include "dc/dc_dmub_srv.h"
+#include "dc/dc_edid_parser.h"
#include "amdgpu_dm_trace.h"
#include "vid.h"
@@ -75,7 +76,6 @@
#include <drm/drm_edid.h>
#include <drm/drm_vblank.h>
#include <drm/drm_audio_component.h>
-#include <drm/drm_hdcp.h>
#if defined(CONFIG_DRM_AMD_DC_DCN)
#include "ivsrcid/dcn/irqsrcs_dcn_1_0.h"
@@ -121,7 +121,7 @@ MODULE_FIRMWARE(FIRMWARE_NAVI12_DMCU);
* DOC: overview
*
* The AMDgpu display manager, **amdgpu_dm** (or even simpler,
- * **dm**) sits between DRM and DC. It acts as a liason, converting DRM
+ * **dm**) sits between DRM and DC. It acts as a liaison, converting DRM
* requests into DC requests, and DC responses into DRM responses.
*
* The root control structure is &struct amdgpu_display_manager.
@@ -130,6 +130,7 @@ MODULE_FIRMWARE(FIRMWARE_NAVI12_DMCU);
/* basic init/fini API */
static int amdgpu_dm_init(struct amdgpu_device *adev);
static void amdgpu_dm_fini(struct amdgpu_device *adev);
+static bool is_freesync_video_mode(const struct drm_display_mode *mode, struct amdgpu_dm_connector *aconnector);
static enum drm_mode_subconnector get_subconnector_type(struct dc_link *link)
{
@@ -212,6 +213,9 @@ static bool amdgpu_dm_psr_disable_all(struct amdgpu_display_manager *dm);
static const struct drm_format_info *
amd_get_format_info(const struct drm_mode_fb_cmd2 *cmd);
+static bool
+is_timing_unchanged_for_freesync(struct drm_crtc_state *old_crtc_state,
+ struct drm_crtc_state *new_crtc_state);
/*
* dm_vblank_get_counter
*
@@ -335,6 +339,17 @@ static inline bool amdgpu_dm_vrr_active(struct dm_crtc_state *dm_state)
dm_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED;
}
+static inline bool is_dc_timing_adjust_needed(struct dm_crtc_state *old_state,
+ struct dm_crtc_state *new_state)
+{
+ if (new_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED)
+ return true;
+ else if (amdgpu_dm_vrr_active(old_state) != amdgpu_dm_vrr_active(new_state))
+ return true;
+ else
+ return false;
+}
+
/**
* dm_pflip_high_irq() - Handle pageflip interrupt
* @interrupt_params: ignored
@@ -357,14 +372,14 @@ static void dm_pflip_high_irq(void *interrupt_params)
/* IRQ could occur when in initial stage */
/* TODO work and BO cleanup */
if (amdgpu_crtc == NULL) {
- DRM_DEBUG_DRIVER("CRTC is null, returning.\n");
+ DC_LOG_PFLIP("CRTC is null, returning.\n");
return;
}
spin_lock_irqsave(&adev_to_drm(adev)->event_lock, flags);
if (amdgpu_crtc->pflip_status != AMDGPU_FLIP_SUBMITTED){
- DRM_DEBUG_DRIVER("amdgpu_crtc->pflip_status = %d !=AMDGPU_FLIP_SUBMITTED(%d) on crtc:%d[%p] \n",
+ DC_LOG_PFLIP("amdgpu_crtc->pflip_status = %d !=AMDGPU_FLIP_SUBMITTED(%d) on crtc:%d[%p] \n",
amdgpu_crtc->pflip_status,
AMDGPU_FLIP_SUBMITTED,
amdgpu_crtc->crtc_id,
@@ -435,9 +450,9 @@ static void dm_pflip_high_irq(void *interrupt_params)
amdgpu_crtc->pflip_status = AMDGPU_FLIP_NONE;
spin_unlock_irqrestore(&adev_to_drm(adev)->event_lock, flags);
- DRM_DEBUG_DRIVER("crtc:%d[%p], pflip_stat:AMDGPU_FLIP_NONE, vrr[%d]-fp %d\n",
- amdgpu_crtc->crtc_id, amdgpu_crtc,
- vrr_active, (int) !e);
+ DC_LOG_PFLIP("crtc:%d[%p], pflip_stat:AMDGPU_FLIP_NONE, vrr[%d]-fp %d\n",
+ amdgpu_crtc->crtc_id, amdgpu_crtc,
+ vrr_active, (int) !e);
}
static void dm_vupdate_high_irq(void *interrupt_params)
@@ -445,6 +460,9 @@ static void dm_vupdate_high_irq(void *interrupt_params)
struct common_irq_params *irq_params = interrupt_params;
struct amdgpu_device *adev = irq_params->adev;
struct amdgpu_crtc *acrtc;
+ struct drm_device *drm_dev;
+ struct drm_vblank_crtc *vblank;
+ ktime_t frame_duration_ns, previous_timestamp;
unsigned long flags;
int vrr_active;
@@ -452,8 +470,19 @@ static void dm_vupdate_high_irq(void *interrupt_params)
if (acrtc) {
vrr_active = amdgpu_dm_vrr_active_irq(acrtc);
+ drm_dev = acrtc->base.dev;
+ vblank = &drm_dev->vblank[acrtc->base.index];
+ previous_timestamp = atomic64_read(&irq_params->previous_timestamp);
+ frame_duration_ns = vblank->time - previous_timestamp;
+
+ if (frame_duration_ns > 0) {
+ trace_amdgpu_refresh_rate_track(acrtc->base.index,
+ frame_duration_ns,
+ ktime_divns(NSEC_PER_SEC, frame_duration_ns));
+ atomic64_set(&irq_params->previous_timestamp, vblank->time);
+ }
- DRM_DEBUG_VBL("crtc:%d, vupdate-vrr:%d\n",
+ DC_LOG_VBLANK("crtc:%d, vupdate-vrr:%d\n",
acrtc->crtc_id,
vrr_active);
@@ -506,7 +535,7 @@ static void dm_crtc_high_irq(void *interrupt_params)
vrr_active = amdgpu_dm_vrr_active_irq(acrtc);
- DRM_DEBUG_VBL("crtc:%d, vupdate-vrr:%d, planes:%d\n", acrtc->crtc_id,
+ DC_LOG_VBLANK("crtc:%d, vupdate-vrr:%d, planes:%d\n", acrtc->crtc_id,
vrr_active, acrtc->dm_irq_params.active_planes);
/**
@@ -566,6 +595,31 @@ static void dm_crtc_high_irq(void *interrupt_params)
spin_unlock_irqrestore(&adev_to_drm(adev)->event_lock, flags);
}
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+/**
+ * dm_dcn_vertical_interrupt0_high_irq() - Handles OTG Vertical interrupt0 for
+ * DCN generation ASICs
+ * @interrupt params - interrupt parameters
+ *
+ * Used to set crc window/read out crc value at vertical line 0 position
+ */
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+static void dm_dcn_vertical_interrupt0_high_irq(void *interrupt_params)
+{
+ struct common_irq_params *irq_params = interrupt_params;
+ struct amdgpu_device *adev = irq_params->adev;
+ struct amdgpu_crtc *acrtc;
+
+ acrtc = get_crtc_by_otg_inst(adev, irq_params->irq_src - IRQ_TYPE_VLINE0);
+
+ if (!acrtc)
+ return;
+
+ amdgpu_dm_crtc_handle_crc_window_irq(&acrtc->base);
+}
+#endif
+#endif
+
static int dm_set_clockgating_state(void *handle,
enum amd_clockgating_state state)
{
@@ -884,6 +938,32 @@ static int dm_dmub_hw_init(struct amdgpu_device *adev)
}
#if defined(CONFIG_DRM_AMD_DC_DCN)
+#define DMUB_TRACE_MAX_READ 64
+static void dm_dmub_trace_high_irq(void *interrupt_params)
+{
+ struct common_irq_params *irq_params = interrupt_params;
+ struct amdgpu_device *adev = irq_params->adev;
+ struct amdgpu_display_manager *dm = &adev->dm;
+ struct dmcub_trace_buf_entry entry = { 0 };
+ uint32_t count = 0;
+
+ do {
+ if (dc_dmub_srv_get_dmub_outbox0_msg(dm->dc, &entry)) {
+ trace_amdgpu_dmub_trace_high_irq(entry.trace_code, entry.tick_count,
+ entry.param0, entry.param1);
+
+ DRM_DEBUG_DRIVER("trace_code:%u, tick_count:%u, param0:%u, param1:%u\n",
+ entry.trace_code, entry.tick_count, entry.param0, entry.param1);
+ } else
+ break;
+
+ count++;
+
+ } while (count <= DMUB_TRACE_MAX_READ);
+
+ ASSERT(count <= DMUB_TRACE_MAX_READ);
+}
+
static void mmhub_read_system_context(struct amdgpu_device *adev, struct dc_phy_addr_space_config *pa_config)
{
uint64_t pt_base;
@@ -948,15 +1028,12 @@ static void event_mall_stutter(struct work_struct *work)
if (vblank_work->enable)
dm->active_vblank_irq_count++;
- else
+ else if(dm->active_vblank_irq_count)
dm->active_vblank_irq_count--;
+ dc_allow_idle_optimizations(dm->dc, dm->active_vblank_irq_count == 0);
- dc_allow_idle_optimizations(
- dm->dc, dm->active_vblank_irq_count == 0 ? true : false);
-
- DRM_DEBUG_DRIVER("Allow idle optimizations (MALL): %d\n", dm->active_vblank_irq_count == 0);
-
+ DRM_DEBUG_KMS("Allow idle optimizations (MALL): %d\n", dm->active_vblank_irq_count == 0);
mutex_unlock(&dm->dc_lock);
}
@@ -1060,6 +1137,7 @@ static int amdgpu_dm_init(struct amdgpu_device *adev)
init_data.flags.power_down_display_on_boot = true;
+ INIT_LIST_HEAD(&adev->dm.da_list);
/* Display Core create. */
adev->dm.dc = dc_create(&init_data);
@@ -1139,6 +1217,9 @@ static int amdgpu_dm_init(struct amdgpu_device *adev)
dc_init_callbacks(adev->dm.dc, &init_params);
}
#endif
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ adev->dm.crc_rd_wrk = amdgpu_dm_crtc_secure_display_create_work();
+#endif
if (amdgpu_dm_initialize_drm_device(adev)) {
DRM_ERROR(
"amdgpu: failed to initialize sw for display support.\n");
@@ -1182,6 +1263,13 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev)
amdgpu_dm_destroy_drm_device(&adev->dm);
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ if (adev->dm.crc_rd_wrk) {
+ flush_work(&adev->dm.crc_rd_wrk->notify_ta_work);
+ kfree(adev->dm.crc_rd_wrk);
+ adev->dm.crc_rd_wrk = NULL;
+ }
+#endif
#ifdef CONFIG_DRM_AMD_DC_HDCP
if (adev->dm.hdcp_workqueue) {
hdcp_destroy(&adev->dev->kobj, adev->dm.hdcp_workqueue);
@@ -1191,6 +1279,15 @@ static void amdgpu_dm_fini(struct amdgpu_device *adev)
if (adev->dm.dc)
dc_deinit_callbacks(adev->dm.dc);
#endif
+
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ if (adev->dm.vblank_workqueue) {
+ adev->dm.vblank_workqueue->dm = NULL;
+ kfree(adev->dm.vblank_workqueue);
+ adev->dm.vblank_workqueue = NULL;
+ }
+#endif
+
if (adev->dm.dc->ctx->dmub_srv) {
dc_dmub_srv_destroy(&adev->dm.dc->ctx->dmub_srv);
adev->dm.dc->ctx->dmub_srv = NULL;
@@ -1752,8 +1849,8 @@ static void dm_gpureset_toggle_interrupts(struct amdgpu_device *adev,
if (acrtc && state->stream_status[i].plane_count != 0) {
irq_source = IRQ_TYPE_PFLIP + acrtc->otg_inst;
rc = dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY;
- DRM_DEBUG("crtc %d - vupdate irq %sabling: r=%d\n",
- acrtc->crtc_id, enable ? "en" : "dis", rc);
+ DRM_DEBUG_VBL("crtc %d - vupdate irq %sabling: r=%d\n",
+ acrtc->crtc_id, enable ? "en" : "dis", rc);
if (rc)
DRM_WARN("Failed to %s pflip interrupts\n",
enable ? "enable" : "disable");
@@ -1847,6 +1944,9 @@ static int dm_suspend(void *handle)
return ret;
}
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+ amdgpu_dm_crtc_secure_display_suspend(adev);
+#endif
WARN_ON(adev->dm.cached_state);
adev->dm.cached_state = drm_atomic_helper_suspend(adev_to_drm(adev));
@@ -2171,6 +2271,10 @@ static int dm_resume(void *handle)
dm->cached_state = NULL;
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+ amdgpu_dm_crtc_secure_display_resume(adev);
+#endif
+
amdgpu_dm_irq_resume_late(adev);
amdgpu_dm_smu_write_watermarks_table(adev);
@@ -2267,6 +2371,11 @@ static void update_connector_ext_caps(struct amdgpu_dm_connector *aconnector)
caps->ext_caps->bits.hdr_aux_backlight_control == 1)
caps->aux_support = true;
+ if (amdgpu_backlight == 0)
+ caps->aux_support = false;
+ else if (amdgpu_backlight == 1)
+ caps->aux_support = true;
+
/* From the specification (CTA-861-G), for calculating the maximum
* luminance we need to use:
* Luminance = 50*2**(CV/32)
@@ -2443,11 +2552,14 @@ static void handle_hpd_irq(void *param)
struct drm_connector *connector = &aconnector->base;
struct drm_device *dev = connector->dev;
enum dc_connection_type new_connection_type = dc_connection_none;
-#ifdef CONFIG_DRM_AMD_DC_HDCP
struct amdgpu_device *adev = drm_to_adev(dev);
+#ifdef CONFIG_DRM_AMD_DC_HDCP
struct dm_connector_state *dm_con_state = to_dm_connector_state(connector->state);
#endif
+ if (adev->dm.disable_hpd_irq)
+ return;
+
/*
* In case of failure or MST no need to update connector status or notify the OS
* since (for MST case) MST does this in its own context.
@@ -2587,6 +2699,10 @@ static void handle_hpd_rx_irq(void *param)
memset(&hpd_irq_data, 0, sizeof(hpd_irq_data));
+ if (adev->dm.disable_hpd_irq)
+ return;
+
+
/*
* TODO:Temporary add mutex to protect hpd interrupt not have a gpio
* conflict, after implement i2c helper, this mutex should be
@@ -2902,6 +3018,16 @@ static int dcn10_register_irq_handlers(struct amdgpu_device *adev)
struct dc_interrupt_params int_params = {0};
int r;
int i;
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ static const unsigned int vrtl_int_srcid[] = {
+ DCN_1_0__SRCID__OTG1_VERTICAL_INTERRUPT0_CONTROL,
+ DCN_1_0__SRCID__OTG2_VERTICAL_INTERRUPT0_CONTROL,
+ DCN_1_0__SRCID__OTG3_VERTICAL_INTERRUPT0_CONTROL,
+ DCN_1_0__SRCID__OTG4_VERTICAL_INTERRUPT0_CONTROL,
+ DCN_1_0__SRCID__OTG5_VERTICAL_INTERRUPT0_CONTROL,
+ DCN_1_0__SRCID__OTG6_VERTICAL_INTERRUPT0_CONTROL
+ };
+#endif
int_params.requested_polarity = INTERRUPT_POLARITY_DEFAULT;
int_params.current_polarity = INTERRUPT_POLARITY_DEFAULT;
@@ -2942,6 +3068,37 @@ static int dcn10_register_irq_handlers(struct amdgpu_device *adev)
adev, &int_params, dm_crtc_high_irq, c_irq_params);
}
+ /* Use otg vertical line interrupt */
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ for (i = 0; i <= adev->mode_info.num_crtc - 1; i++) {
+ r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE,
+ vrtl_int_srcid[i], &adev->vline0_irq);
+
+ if (r) {
+ DRM_ERROR("Failed to add vline0 irq id!\n");
+ return r;
+ }
+
+ int_params.int_context = INTERRUPT_HIGH_IRQ_CONTEXT;
+ int_params.irq_source =
+ dc_interrupt_to_irq_source(dc, vrtl_int_srcid[i], 0);
+
+ if (int_params.irq_source == DC_IRQ_SOURCE_INVALID) {
+ DRM_ERROR("Failed to register vline0 irq %d!\n", vrtl_int_srcid[i]);
+ break;
+ }
+
+ c_irq_params = &adev->dm.vline0_params[int_params.irq_source
+ - DC_IRQ_SOURCE_DC1_VLINE0];
+
+ c_irq_params->adev = adev;
+ c_irq_params->irq_src = int_params.irq_source;
+
+ amdgpu_dm_irq_register_interrupt(adev, &int_params,
+ dm_dcn_vertical_interrupt0_high_irq, c_irq_params);
+ }
+#endif
+
/* Use VUPDATE_NO_LOCK interrupt on DCN, which seems to correspond to
* the regular VUPDATE interrupt on DCE. We want DC_IRQ_SOURCE_VUPDATEx
* to trigger at end of each vblank, regardless of state of the lock,
@@ -2994,6 +3151,28 @@ static int dcn10_register_irq_handlers(struct amdgpu_device *adev)
}
+ if (dc->ctx->dmub_srv) {
+ i = DCN_1_0__SRCID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT;
+ r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE, i, &adev->dmub_trace_irq);
+
+ if (r) {
+ DRM_ERROR("Failed to add dmub trace irq id!\n");
+ return r;
+ }
+
+ int_params.int_context = INTERRUPT_HIGH_IRQ_CONTEXT;
+ int_params.irq_source =
+ dc_interrupt_to_irq_source(dc, i, 0);
+
+ c_irq_params = &adev->dm.dmub_trace_params[0];
+
+ c_irq_params->adev = adev;
+ c_irq_params->irq_src = int_params.irq_source;
+
+ amdgpu_dm_irq_register_interrupt(adev, &int_params,
+ dm_dmub_trace_high_irq, c_irq_params);
+ }
+
/* HPD */
r = amdgpu_irq_add_id(adev, SOC15_IH_CLIENTID_DCE, DCN_1_0__SRCID__DC_HPD1_INT,
&adev->hpd_irq);
@@ -3185,19 +3364,6 @@ static void amdgpu_dm_update_backlight_caps(struct amdgpu_display_manager *dm)
#endif
}
-static int set_backlight_via_aux(struct dc_link *link, uint32_t brightness)
-{
- bool rc;
-
- if (!link)
- return 1;
-
- rc = dc_link_set_backlight_level_nits(link, true, brightness,
- AUX_BL_DEFAULT_TRANSITION_TIME_MS);
-
- return rc ? 0 : 1;
-}
-
static int get_brightness_range(const struct amdgpu_dm_backlight_caps *caps,
unsigned *min, unsigned *max)
{
@@ -3260,9 +3426,10 @@ static int amdgpu_dm_backlight_update_status(struct backlight_device *bd)
brightness = convert_brightness_from_user(&caps, bd->props.brightness);
// Change brightness based on AUX property
if (caps.aux_support)
- return set_backlight_via_aux(link, brightness);
-
- rc = dc_link_set_backlight_level(dm->backlight_link, brightness, 0);
+ rc = dc_link_set_backlight_level_nits(link, true, brightness,
+ AUX_BL_DEFAULT_TRANSITION_TIME_MS);
+ else
+ rc = dc_link_set_backlight_level(dm->backlight_link, brightness, 0);
return rc ? 0 : 1;
}
@@ -3270,11 +3437,27 @@ static int amdgpu_dm_backlight_update_status(struct backlight_device *bd)
static int amdgpu_dm_backlight_get_brightness(struct backlight_device *bd)
{
struct amdgpu_display_manager *dm = bl_get_data(bd);
- int ret = dc_link_get_backlight_level(dm->backlight_link);
+ struct amdgpu_dm_backlight_caps caps;
+
+ amdgpu_dm_update_backlight_caps(dm);
+ caps = dm->backlight_caps;
+
+ if (caps.aux_support) {
+ struct dc_link *link = (struct dc_link *)dm->backlight_link;
+ u32 avg, peak;
+ bool rc;
+
+ rc = dc_link_get_backlight_level_nits(link, &avg, &peak);
+ if (!rc)
+ return bd->props.brightness;
+ return convert_brightness_to_user(&caps, avg);
+ } else {
+ int ret = dc_link_get_backlight_level(dm->backlight_link);
- if (ret == DC_ERROR_UNEXPECTED)
- return bd->props.brightness;
- return convert_brightness_to_user(&dm->backlight_caps, ret);
+ if (ret == DC_ERROR_UNEXPECTED)
+ return bd->props.brightness;
+ return convert_brightness_to_user(&caps, ret);
+ }
}
static const struct backlight_ops amdgpu_dm_backlight_ops = {
@@ -3832,6 +4015,23 @@ static int fill_dc_scaling_info(const struct drm_plane_state *state,
scaling_info->src_rect.x = state->src_x >> 16;
scaling_info->src_rect.y = state->src_y >> 16;
+ /*
+ * For reasons we don't (yet) fully understand a non-zero
+ * src_y coordinate into an NV12 buffer can cause a
+ * system hang. To avoid hangs (and maybe be overly cautious)
+ * let's reject both non-zero src_x and src_y.
+ *
+ * We currently know of only one use-case to reproduce a
+ * scenario with non-zero src_x and src_y for NV12, which
+ * is to gesture the YouTube Android app into full screen
+ * on ChromeOS.
+ */
+ if (state->fb &&
+ state->fb->format->format == DRM_FORMAT_NV12 &&
+ (scaling_info->src_rect.x != 0 ||
+ scaling_info->src_rect.y != 0))
+ return -EINVAL;
+
scaling_info->src_rect.width = state->src_w >> 16;
if (scaling_info->src_rect.width == 0)
return -EINVAL;
@@ -4049,6 +4249,7 @@ static bool dm_plane_format_mod_supported(struct drm_plane *plane,
{
struct amdgpu_device *adev = drm_to_adev(plane->dev);
const struct drm_format_info *info = drm_format_info(format);
+ int i;
enum dm_micro_swizzle microtile = modifier_gfx9_swizzle_mode(modifier) & 3;
@@ -4056,17 +4257,21 @@ static bool dm_plane_format_mod_supported(struct drm_plane *plane,
return false;
/*
- * We always have to allow this modifier, because core DRM still
- * checks LINEAR support if userspace does not provide modifers.
+ * We always have to allow these modifiers:
+ * 1. Core DRM checks for LINEAR support if userspace does not provide modifiers.
+ * 2. Not passing any modifiers is the same as explicitly passing INVALID.
*/
- if (modifier == DRM_FORMAT_MOD_LINEAR)
+ if (modifier == DRM_FORMAT_MOD_LINEAR ||
+ modifier == DRM_FORMAT_MOD_INVALID) {
return true;
+ }
- /*
- * The arbitrary tiling support for multiplane formats has not been hooked
- * up.
- */
- if (info->num_planes > 1)
+ /* Check that the modifier is on the list of the plane's supported modifiers. */
+ for (i = 0; i < plane->modifier_count; i++) {
+ if (modifier == plane->modifiers[i])
+ break;
+ }
+ if (i == plane->modifier_count)
return false;
/*
@@ -4087,6 +4292,10 @@ static bool dm_plane_format_mod_supported(struct drm_plane *plane,
/* Per radeonsi comments 16/64 bpp are more complicated. */
if (info->cpp[0] != 4)
return false;
+ /* We support multi-planar formats, but not when combined with
+ * additional DCC metadata planes. */
+ if (info->num_planes > 1)
+ return false;
}
return true;
@@ -4287,7 +4496,7 @@ add_gfx10_3_modifiers(const struct amdgpu_device *adev,
AMD_FMT_MOD_SET(DCC_CONSTANT_ENCODE, 1) |
AMD_FMT_MOD_SET(DCC_INDEPENDENT_64B, 1) |
AMD_FMT_MOD_SET(DCC_INDEPENDENT_128B, 1) |
- AMD_FMT_MOD_SET(DCC_MAX_COMPRESSED_BLOCK, AMD_FMT_MOD_DCC_BLOCK_128B));
+ AMD_FMT_MOD_SET(DCC_MAX_COMPRESSED_BLOCK, AMD_FMT_MOD_DCC_BLOCK_64B));
add_modifier(mods, size, capacity, AMD_FMT_MOD |
AMD_FMT_MOD_SET(TILE, AMD_FMT_MOD_TILE_GFX9_64K_R_X) |
@@ -4299,7 +4508,7 @@ add_gfx10_3_modifiers(const struct amdgpu_device *adev,
AMD_FMT_MOD_SET(DCC_CONSTANT_ENCODE, 1) |
AMD_FMT_MOD_SET(DCC_INDEPENDENT_64B, 1) |
AMD_FMT_MOD_SET(DCC_INDEPENDENT_128B, 1) |
- AMD_FMT_MOD_SET(DCC_MAX_COMPRESSED_BLOCK, AMD_FMT_MOD_DCC_BLOCK_128B));
+ AMD_FMT_MOD_SET(DCC_MAX_COMPRESSED_BLOCK, AMD_FMT_MOD_DCC_BLOCK_64B));
add_modifier(mods, size, capacity, AMD_FMT_MOD |
AMD_FMT_MOD_SET(TILE, AMD_FMT_MOD_TILE_GFX9_64K_R_X) |
@@ -4574,7 +4783,6 @@ fill_dc_plane_info_and_addr(struct amdgpu_device *adev,
const struct drm_framebuffer *fb = plane_state->fb;
const struct amdgpu_framebuffer *afb =
to_amdgpu_framebuffer(plane_state->fb);
- struct drm_format_name_buf format_name;
int ret;
memset(plane_info, 0, sizeof(*plane_info));
@@ -4622,8 +4830,8 @@ fill_dc_plane_info_and_addr(struct amdgpu_device *adev,
break;
default:
DRM_ERROR(
- "Unsupported screen format %s\n",
- drm_get_format_name(fb->format->format, &format_name));
+ "Unsupported screen format %p4cc\n",
+ &fb->format->format);
return -EINVAL;
}
@@ -4716,6 +4924,7 @@ static int fill_dc_plane_attributes(struct amdgpu_device *adev,
dc_plane_state->global_alpha_value = plane_info.global_alpha_value;
dc_plane_state->dcc = plane_info.dcc;
dc_plane_state->layer_index = plane_info.layer_index; // Always returns 0
+ dc_plane_state->flip_int_enabled = true;
/*
* Always set input transfer function, since plane state is refreshed
@@ -4778,8 +4987,8 @@ static void update_stream_scaling_settings(const struct drm_display_mode *mode,
stream->src = src;
stream->dst = dst;
- DRM_DEBUG_DRIVER("Destination Rectangle x:%d y:%d width:%d height:%d\n",
- dst.x, dst.y, dst.width, dst.height);
+ DRM_DEBUG_KMS("Destination Rectangle x:%d y:%d width:%d height:%d\n",
+ dst.x, dst.y, dst.width, dst.height);
}
@@ -4992,19 +5201,28 @@ static void fill_stream_properties_from_drm_display_mode(
timing_out->hdmi_vic = hv_frame.vic;
}
- timing_out->h_addressable = mode_in->crtc_hdisplay;
- timing_out->h_total = mode_in->crtc_htotal;
- timing_out->h_sync_width =
- mode_in->crtc_hsync_end - mode_in->crtc_hsync_start;
- timing_out->h_front_porch =
- mode_in->crtc_hsync_start - mode_in->crtc_hdisplay;
- timing_out->v_total = mode_in->crtc_vtotal;
- timing_out->v_addressable = mode_in->crtc_vdisplay;
- timing_out->v_front_porch =
- mode_in->crtc_vsync_start - mode_in->crtc_vdisplay;
- timing_out->v_sync_width =
- mode_in->crtc_vsync_end - mode_in->crtc_vsync_start;
- timing_out->pix_clk_100hz = mode_in->crtc_clock * 10;
+ if (is_freesync_video_mode(mode_in, aconnector)) {
+ timing_out->h_addressable = mode_in->hdisplay;
+ timing_out->h_total = mode_in->htotal;
+ timing_out->h_sync_width = mode_in->hsync_end - mode_in->hsync_start;
+ timing_out->h_front_porch = mode_in->hsync_start - mode_in->hdisplay;
+ timing_out->v_total = mode_in->vtotal;
+ timing_out->v_addressable = mode_in->vdisplay;
+ timing_out->v_front_porch = mode_in->vsync_start - mode_in->vdisplay;
+ timing_out->v_sync_width = mode_in->vsync_end - mode_in->vsync_start;
+ timing_out->pix_clk_100hz = mode_in->clock * 10;
+ } else {
+ timing_out->h_addressable = mode_in->crtc_hdisplay;
+ timing_out->h_total = mode_in->crtc_htotal;
+ timing_out->h_sync_width = mode_in->crtc_hsync_end - mode_in->crtc_hsync_start;
+ timing_out->h_front_porch = mode_in->crtc_hsync_start - mode_in->crtc_hdisplay;
+ timing_out->v_total = mode_in->crtc_vtotal;
+ timing_out->v_addressable = mode_in->crtc_vdisplay;
+ timing_out->v_front_porch = mode_in->crtc_vsync_start - mode_in->crtc_vdisplay;
+ timing_out->v_sync_width = mode_in->crtc_vsync_end - mode_in->crtc_vsync_start;
+ timing_out->pix_clk_100hz = mode_in->crtc_clock * 10;
+ }
+
timing_out->aspect_ratio = get_aspect_ratio(mode_in);
stream->output_color_space = get_output_color_space(timing_out);
@@ -5123,9 +5341,14 @@ create_fake_sink(struct amdgpu_dm_connector *aconnector)
static void set_multisync_trigger_params(
struct dc_stream_state *stream)
{
+ struct dc_stream_state *master = NULL;
+
if (stream->triggered_crtc_reset.enabled) {
- stream->triggered_crtc_reset.event = CRTC_EVENT_VSYNC_RISING;
- stream->triggered_crtc_reset.delay = TRIGGER_DELAY_NEXT_LINE;
+ master = stream->triggered_crtc_reset.event_source;
+ stream->triggered_crtc_reset.event =
+ master->timing.flags.VSYNC_POSITIVE_POLARITY ?
+ CRTC_EVENT_VSYNC_RISING : CRTC_EVENT_VSYNC_FALLING;
+ stream->triggered_crtc_reset.delay = TRIGGER_DELAY_NEXT_PIXEL;
}
}
@@ -5155,6 +5378,7 @@ static void set_master_stream(struct dc_stream_state *stream_set[],
static void dm_enable_per_frame_crtc_master_sync(struct dc_state *context)
{
int i = 0;
+ struct dc_stream_state *stream;
if (context->stream_count < 2)
return;
@@ -5166,9 +5390,98 @@ static void dm_enable_per_frame_crtc_master_sync(struct dc_state *context)
* crtc_sync_master.multi_sync_enabled flag
* For now it's set to false
*/
- set_multisync_trigger_params(context->streams[i]);
}
+
set_master_stream(context->streams, context->stream_count);
+
+ for (i = 0; i < context->stream_count ; i++) {
+ stream = context->streams[i];
+
+ if (!stream)
+ continue;
+
+ set_multisync_trigger_params(stream);
+ }
+}
+
+static struct drm_display_mode *
+get_highest_refresh_rate_mode(struct amdgpu_dm_connector *aconnector,
+ bool use_probed_modes)
+{
+ struct drm_display_mode *m, *m_pref = NULL;
+ u16 current_refresh, highest_refresh;
+ struct list_head *list_head = use_probed_modes ?
+ &aconnector->base.probed_modes :
+ &aconnector->base.modes;
+
+ if (aconnector->freesync_vid_base.clock != 0)
+ return &aconnector->freesync_vid_base;
+
+ /* Find the preferred mode */
+ list_for_each_entry (m, list_head, head) {
+ if (m->type & DRM_MODE_TYPE_PREFERRED) {
+ m_pref = m;
+ break;
+ }
+ }
+
+ if (!m_pref) {
+ /* Probably an EDID with no preferred mode. Fallback to first entry */
+ m_pref = list_first_entry_or_null(
+ &aconnector->base.modes, struct drm_display_mode, head);
+ if (!m_pref) {
+ DRM_DEBUG_DRIVER("No preferred mode found in EDID\n");
+ return NULL;
+ }
+ }
+
+ highest_refresh = drm_mode_vrefresh(m_pref);
+
+ /*
+ * Find the mode with highest refresh rate with same resolution.
+ * For some monitors, preferred mode is not the mode with highest
+ * supported refresh rate.
+ */
+ list_for_each_entry (m, list_head, head) {
+ current_refresh = drm_mode_vrefresh(m);
+
+ if (m->hdisplay == m_pref->hdisplay &&
+ m->vdisplay == m_pref->vdisplay &&
+ highest_refresh < current_refresh) {
+ highest_refresh = current_refresh;
+ m_pref = m;
+ }
+ }
+
+ aconnector->freesync_vid_base = *m_pref;
+ return m_pref;
+}
+
+static bool is_freesync_video_mode(const struct drm_display_mode *mode,
+ struct amdgpu_dm_connector *aconnector)
+{
+ struct drm_display_mode *high_mode;
+ int timing_diff;
+
+ high_mode = get_highest_refresh_rate_mode(aconnector, false);
+ if (!high_mode || !mode)
+ return false;
+
+ timing_diff = high_mode->vtotal - mode->vtotal;
+
+ if (high_mode->clock == 0 || high_mode->clock != mode->clock ||
+ high_mode->hdisplay != mode->hdisplay ||
+ high_mode->vdisplay != mode->vdisplay ||
+ high_mode->hsync_start != mode->hsync_start ||
+ high_mode->hsync_end != mode->hsync_end ||
+ high_mode->htotal != mode->htotal ||
+ high_mode->hskew != mode->hskew ||
+ high_mode->vscan != mode->vscan ||
+ high_mode->vsync_start - mode->vsync_start != timing_diff ||
+ high_mode->vsync_end - mode->vsync_end != timing_diff)
+ return false;
+ else
+ return true;
}
static struct dc_stream_state *
@@ -5184,8 +5497,10 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector,
dm_state ? &dm_state->base : NULL;
struct dc_stream_state *stream = NULL;
struct drm_display_mode mode = *drm_mode;
+ struct drm_display_mode saved_mode;
+ struct drm_display_mode *freesync_mode = NULL;
bool native_mode_found = false;
- bool scale = dm_state ? (dm_state->scaling != RMX_OFF) : false;
+ bool recalculate_timing = dm_state ? (dm_state->scaling != RMX_OFF) : false;
int mode_refresh;
int preferred_refresh = 0;
#if defined(CONFIG_DRM_AMD_DC_DCN)
@@ -5193,6 +5508,9 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector,
uint32_t link_bandwidth_kbps;
#endif
struct dc_sink *sink = NULL;
+
+ memset(&saved_mode, 0, sizeof(saved_mode));
+
if (aconnector == NULL) {
DRM_ERROR("aconnector is NULL!\n");
return stream;
@@ -5245,25 +5563,38 @@ create_stream_for_sink(struct amdgpu_dm_connector *aconnector,
*/
DRM_DEBUG_DRIVER("No preferred mode found\n");
} else {
- decide_crtc_timing_for_drm_display_mode(
+ recalculate_timing |= amdgpu_freesync_vid_mode &&
+ is_freesync_video_mode(&mode, aconnector);
+ if (recalculate_timing) {
+ freesync_mode = get_highest_refresh_rate_mode(aconnector, false);
+ saved_mode = mode;
+ mode = *freesync_mode;
+ } else {
+ decide_crtc_timing_for_drm_display_mode(
&mode, preferred_mode,
dm_state ? (dm_state->scaling != RMX_OFF) : false);
+ }
+
preferred_refresh = drm_mode_vrefresh(preferred_mode);
}
- if (!dm_state)
+ if (recalculate_timing)
+ drm_mode_set_crtcinfo(&saved_mode, 0);
+ else if (!dm_state)
drm_mode_set_crtcinfo(&mode, 0);
- /*
+ /*
* If scaling is enabled and refresh rate didn't change
* we copy the vic and polarities of the old timings
*/
- if (!scale || mode_refresh != preferred_refresh)
- fill_stream_properties_from_drm_display_mode(stream,
- &mode, &aconnector->base, con_state, NULL, requested_bpc);
+ if (!recalculate_timing || mode_refresh != preferred_refresh)
+ fill_stream_properties_from_drm_display_mode(
+ stream, &mode, &aconnector->base, con_state, NULL,
+ requested_bpc);
else
- fill_stream_properties_from_drm_display_mode(stream,
- &mode, &aconnector->base, con_state, old_stream, requested_bpc);
+ fill_stream_properties_from_drm_display_mode(
+ stream, &mode, &aconnector->base, con_state, old_stream,
+ requested_bpc);
stream->timing.flags.DSC = 0;
@@ -5400,15 +5731,22 @@ dm_crtc_duplicate_state(struct drm_crtc *crtc)
state->abm_level = cur->abm_level;
state->vrr_supported = cur->vrr_supported;
state->freesync_config = cur->freesync_config;
- state->crc_src = cur->crc_src;
state->cm_has_degamma = cur->cm_has_degamma;
state->cm_is_degamma_srgb = cur->cm_is_degamma_srgb;
-
/* TODO Duplicate dc_stream after objects are stream object is flattened */
return &state->base;
}
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+static int amdgpu_dm_crtc_late_register(struct drm_crtc *crtc)
+{
+ crtc_debugfs_init(crtc);
+
+ return 0;
+}
+#endif
+
static inline int dm_set_vupdate_irq(struct drm_crtc *crtc, bool enable)
{
enum dc_irq_source irq_source;
@@ -5420,8 +5758,8 @@ static inline int dm_set_vupdate_irq(struct drm_crtc *crtc, bool enable)
rc = dc_interrupt_set(adev->dm.dc, irq_source, enable) ? 0 : -EBUSY;
- DRM_DEBUG_DRIVER("crtc %d - vupdate irq %sabling: r=%d\n",
- acrtc->crtc_id, enable ? "en" : "dis", rc);
+ DRM_DEBUG_VBL("crtc %d - vupdate irq %sabling: r=%d\n",
+ acrtc->crtc_id, enable ? "en" : "dis", rc);
return rc;
}
@@ -5494,6 +5832,9 @@ static const struct drm_crtc_funcs amdgpu_dm_crtc_funcs = {
.enable_vblank = dm_enable_vblank,
.disable_vblank = dm_disable_vblank,
.get_vblank_timestamp = drm_crtc_vblank_helper_get_vblank_timestamp,
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ .late_register = amdgpu_dm_crtc_late_register,
+#endif
};
static enum drm_connector_status
@@ -5856,6 +6197,15 @@ create_validate_stream_for_sink(struct amdgpu_dm_connector *aconnector,
} while (stream == NULL && requested_bpc >= 6);
+ if (dc_result == DC_FAIL_ENC_VALIDATE && !aconnector->force_yuv420_output) {
+ DRM_DEBUG_KMS("Retry forcing YCbCr420 encoding\n");
+
+ aconnector->force_yuv420_output = true;
+ stream = create_validate_stream_for_sink(aconnector, drm_mode,
+ dm_state, old_stream);
+ aconnector->force_yuv420_output = false;
+ }
+
return stream;
}
@@ -6358,7 +6708,7 @@ static int dm_plane_helper_prepare_fb(struct drm_plane *plane,
int r;
if (!new_state->fb) {
- DRM_DEBUG_DRIVER("No FB bound\n");
+ DRM_DEBUG_KMS("No FB bound\n");
return 0;
}
@@ -6479,13 +6829,17 @@ static int dm_plane_helper_check_state(struct drm_plane_state *state,
else if (state->crtc_y + state->crtc_h > new_crtc_state->mode.crtc_vdisplay)
viewport_height = new_crtc_state->mode.crtc_vdisplay - state->crtc_y;
- /* If completely outside of screen, viewport_width and/or viewport_height will be negative,
- * which is still OK to satisfy the condition below, thereby also covering these cases
- * (when plane is completely outside of screen).
- * x2 for width is because of pipe-split.
- */
- if (viewport_width < MIN_VIEWPORT_SIZE*2 || viewport_height < MIN_VIEWPORT_SIZE)
+ if (viewport_width < 0 || viewport_height < 0) {
+ DRM_DEBUG_ATOMIC("Plane completely outside of screen\n");
+ return -EINVAL;
+ } else if (viewport_width < MIN_VIEWPORT_SIZE*2) { /* x2 for width is because of pipe-split. */
+ DRM_DEBUG_ATOMIC("Viewport width %d smaller than %d\n", viewport_width, MIN_VIEWPORT_SIZE*2);
return -EINVAL;
+ } else if (viewport_height < MIN_VIEWPORT_SIZE) {
+ DRM_DEBUG_ATOMIC("Viewport height %d smaller than %d\n", viewport_height, MIN_VIEWPORT_SIZE);
+ return -EINVAL;
+ }
+
}
/* Get min/max allowed scaling factors from plane caps. */
@@ -6505,8 +6859,10 @@ static int dm_plane_helper_check_state(struct drm_plane_state *state,
}
static int dm_plane_atomic_check(struct drm_plane *plane,
- struct drm_plane_state *state)
+ struct drm_atomic_state *state)
{
+ struct drm_plane_state *new_plane_state = drm_atomic_get_new_plane_state(state,
+ plane);
struct amdgpu_device *adev = drm_to_adev(plane->dev);
struct dc *dc = adev->dm.dc;
struct dm_plane_state *dm_plane_state;
@@ -6514,23 +6870,24 @@ static int dm_plane_atomic_check(struct drm_plane *plane,
struct drm_crtc_state *new_crtc_state;
int ret;
- trace_amdgpu_dm_plane_atomic_check(state);
+ trace_amdgpu_dm_plane_atomic_check(new_plane_state);
- dm_plane_state = to_dm_plane_state(state);
+ dm_plane_state = to_dm_plane_state(new_plane_state);
if (!dm_plane_state->dc_state)
return 0;
new_crtc_state =
- drm_atomic_get_new_crtc_state(state->state, state->crtc);
+ drm_atomic_get_new_crtc_state(state,
+ new_plane_state->crtc);
if (!new_crtc_state)
return -EINVAL;
- ret = dm_plane_helper_check_state(state, new_crtc_state);
+ ret = dm_plane_helper_check_state(new_plane_state, new_crtc_state);
if (ret)
return ret;
- ret = fill_dc_scaling_info(state, &scaling_info);
+ ret = fill_dc_scaling_info(new_plane_state, &scaling_info);
if (ret)
return ret;
@@ -6541,7 +6898,7 @@ static int dm_plane_atomic_check(struct drm_plane *plane,
}
static int dm_plane_atomic_async_check(struct drm_plane *plane,
- struct drm_plane_state *new_plane_state)
+ struct drm_atomic_state *state)
{
/* Only support async updates on cursor planes. */
if (plane->type != DRM_PLANE_TYPE_CURSOR)
@@ -6551,10 +6908,12 @@ static int dm_plane_atomic_async_check(struct drm_plane *plane,
}
static void dm_plane_atomic_async_update(struct drm_plane *plane,
- struct drm_plane_state *new_state)
+ struct drm_atomic_state *state)
{
+ struct drm_plane_state *new_state = drm_atomic_get_new_plane_state(state,
+ plane);
struct drm_plane_state *old_state =
- drm_atomic_get_old_plane_state(new_state->state, plane);
+ drm_atomic_get_old_plane_state(state, plane);
trace_amdgpu_dm_atomic_update_cursor(new_state);
@@ -6961,11 +7320,118 @@ static void amdgpu_dm_connector_ddc_get_modes(struct drm_connector *connector,
*/
drm_mode_sort(&connector->probed_modes);
amdgpu_dm_get_native_mode(connector);
+
+ /* Freesync capabilities are reset by calling
+ * drm_add_edid_modes() and need to be
+ * restored here.
+ */
+ amdgpu_dm_update_freesync_caps(connector, edid);
} else {
amdgpu_dm_connector->num_modes = 0;
}
}
+static bool is_duplicate_mode(struct amdgpu_dm_connector *aconnector,
+ struct drm_display_mode *mode)
+{
+ struct drm_display_mode *m;
+
+ list_for_each_entry (m, &aconnector->base.probed_modes, head) {
+ if (drm_mode_equal(m, mode))
+ return true;
+ }
+
+ return false;
+}
+
+static uint add_fs_modes(struct amdgpu_dm_connector *aconnector)
+{
+ const struct drm_display_mode *m;
+ struct drm_display_mode *new_mode;
+ uint i;
+ uint32_t new_modes_count = 0;
+
+ /* Standard FPS values
+ *
+ * 23.976 - TV/NTSC
+ * 24 - Cinema
+ * 25 - TV/PAL
+ * 29.97 - TV/NTSC
+ * 30 - TV/NTSC
+ * 48 - Cinema HFR
+ * 50 - TV/PAL
+ * 60 - Commonly used
+ * 48,72,96 - Multiples of 24
+ */
+ const uint32_t common_rates[] = { 23976, 24000, 25000, 29970, 30000,
+ 48000, 50000, 60000, 72000, 96000 };
+
+ /*
+ * Find mode with highest refresh rate with the same resolution
+ * as the preferred mode. Some monitors report a preferred mode
+ * with lower resolution than the highest refresh rate supported.
+ */
+
+ m = get_highest_refresh_rate_mode(aconnector, true);
+ if (!m)
+ return 0;
+
+ for (i = 0; i < ARRAY_SIZE(common_rates); i++) {
+ uint64_t target_vtotal, target_vtotal_diff;
+ uint64_t num, den;
+
+ if (drm_mode_vrefresh(m) * 1000 < common_rates[i])
+ continue;
+
+ if (common_rates[i] < aconnector->min_vfreq * 1000 ||
+ common_rates[i] > aconnector->max_vfreq * 1000)
+ continue;
+
+ num = (unsigned long long)m->clock * 1000 * 1000;
+ den = common_rates[i] * (unsigned long long)m->htotal;
+ target_vtotal = div_u64(num, den);
+ target_vtotal_diff = target_vtotal - m->vtotal;
+
+ /* Check for illegal modes */
+ if (m->vsync_start + target_vtotal_diff < m->vdisplay ||
+ m->vsync_end + target_vtotal_diff < m->vsync_start ||
+ m->vtotal + target_vtotal_diff < m->vsync_end)
+ continue;
+
+ new_mode = drm_mode_duplicate(aconnector->base.dev, m);
+ if (!new_mode)
+ goto out;
+
+ new_mode->vtotal += (u16)target_vtotal_diff;
+ new_mode->vsync_start += (u16)target_vtotal_diff;
+ new_mode->vsync_end += (u16)target_vtotal_diff;
+ new_mode->type &= ~DRM_MODE_TYPE_PREFERRED;
+ new_mode->type |= DRM_MODE_TYPE_DRIVER;
+
+ if (!is_duplicate_mode(aconnector, new_mode)) {
+ drm_mode_probed_add(&aconnector->base, new_mode);
+ new_modes_count += 1;
+ } else
+ drm_mode_destroy(aconnector->base.dev, new_mode);
+ }
+ out:
+ return new_modes_count;
+}
+
+static void amdgpu_dm_connector_add_freesync_modes(struct drm_connector *connector,
+ struct edid *edid)
+{
+ struct amdgpu_dm_connector *amdgpu_dm_connector =
+ to_amdgpu_dm_connector(connector);
+
+ if (!(amdgpu_freesync_vid_mode && edid))
+ return;
+
+ if (amdgpu_dm_connector->max_vfreq - amdgpu_dm_connector->min_vfreq > 10)
+ amdgpu_dm_connector->num_modes +=
+ add_fs_modes(amdgpu_dm_connector);
+}
+
static int amdgpu_dm_connector_get_modes(struct drm_connector *connector)
{
struct amdgpu_dm_connector *amdgpu_dm_connector =
@@ -6981,6 +7447,7 @@ static int amdgpu_dm_connector_get_modes(struct drm_connector *connector)
} else {
amdgpu_dm_connector_ddc_get_modes(connector, edid);
amdgpu_dm_connector_add_common_modes(encoder, connector);
+ amdgpu_dm_connector_add_freesync_modes(connector, edid);
}
amdgpu_dm_fbc_init(connector);
@@ -7285,8 +7752,19 @@ static void manage_dm_interrupts(struct amdgpu_device *adev,
adev,
&adev->pageflip_irq,
irq_type);
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ amdgpu_irq_get(
+ adev,
+ &adev->vline0_irq,
+ irq_type);
+#endif
} else {
-
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ amdgpu_irq_put(
+ adev,
+ &adev->vline0_irq,
+ irq_type);
+#endif
amdgpu_irq_put(
adev,
&adev->pageflip_irq,
@@ -7410,10 +7888,6 @@ static int get_cursor_position(struct drm_plane *plane, struct drm_crtc *crtc,
int x, y;
int xorigin = 0, yorigin = 0;
- position->enable = false;
- position->x = 0;
- position->y = 0;
-
if (!crtc || !plane->state->fb)
return 0;
@@ -7460,18 +7934,18 @@ static void handle_cursor_update(struct drm_plane *plane,
struct dm_crtc_state *crtc_state = crtc ? to_dm_crtc_state(crtc->state) : NULL;
struct amdgpu_crtc *amdgpu_crtc = to_amdgpu_crtc(crtc);
uint64_t address = afb ? afb->address : 0;
- struct dc_cursor_position position;
+ struct dc_cursor_position position = {0};
struct dc_cursor_attributes attributes;
int ret;
if (!plane->state->fb && !old_plane_state->fb)
return;
- DRM_DEBUG_DRIVER("%s: crtc_id=%d with size %d to %d\n",
- __func__,
- amdgpu_crtc->crtc_id,
- plane->state->crtc_w,
- plane->state->crtc_h);
+ DC_LOG_CURSOR("%s: crtc_id=%d with size %d to %d\n",
+ __func__,
+ amdgpu_crtc->crtc_id,
+ plane->state->crtc_w,
+ plane->state->crtc_h);
ret = get_cursor_position(plane, crtc, &position);
if (ret)
@@ -7529,8 +8003,8 @@ static void prepare_flip_isr(struct amdgpu_crtc *acrtc)
/* Mark this event as consumed */
acrtc->base.state->event = NULL;
- DRM_DEBUG_DRIVER("crtc:%d, pflip_stat:AMDGPU_FLIP_SUBMITTED\n",
- acrtc->crtc_id);
+ DC_LOG_PFLIP("crtc:%d, pflip_stat:AMDGPU_FLIP_SUBMITTED\n",
+ acrtc->crtc_id);
}
static void update_freesync_state_on_stream(
@@ -7545,6 +8019,7 @@ static void update_freesync_state_on_stream(
struct amdgpu_device *adev = dm->adev;
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(new_crtc_state->base.crtc);
unsigned long flags;
+ bool pack_sdp_v1_3 = false;
if (!new_stream)
return;
@@ -7586,7 +8061,8 @@ static void update_freesync_state_on_stream(
&vrr_params,
PACKET_TYPE_VRR,
TRANSFER_FUNC_UNKNOWN,
- &vrr_infopacket);
+ &vrr_infopacket,
+ pack_sdp_v1_3);
new_crtc_state->freesync_timing_changed |=
(memcmp(&acrtc->dm_irq_params.vrr_params.adjust,
@@ -7640,9 +8116,22 @@ static void update_stream_irq_parameters(
if (new_crtc_state->vrr_supported &&
config.min_refresh_in_uhz &&
config.max_refresh_in_uhz) {
- config.state = new_crtc_state->base.vrr_enabled ?
- VRR_STATE_ACTIVE_VARIABLE :
- VRR_STATE_INACTIVE;
+ /*
+ * if freesync compatible mode was set, config.state will be set
+ * in atomic check
+ */
+ if (config.state == VRR_STATE_ACTIVE_FIXED && config.fixed_refresh_in_uhz &&
+ (!drm_atomic_crtc_needs_modeset(&new_crtc_state->base) ||
+ new_crtc_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED)) {
+ vrr_params.max_refresh_in_uhz = config.max_refresh_in_uhz;
+ vrr_params.min_refresh_in_uhz = config.min_refresh_in_uhz;
+ vrr_params.fixed_refresh_in_uhz = config.fixed_refresh_in_uhz;
+ vrr_params.state = VRR_STATE_ACTIVE_FIXED;
+ } else {
+ config.state = new_crtc_state->base.vrr_enabled ?
+ VRR_STATE_ACTIVE_VARIABLE :
+ VRR_STATE_INACTIVE;
+ }
} else {
config.state = VRR_STATE_UNSUPPORTED;
}
@@ -7821,7 +8310,7 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state,
&bundle->flip_addrs[planes_count].address,
afb->tmz_surface, false);
- DRM_DEBUG_DRIVER("plane: id=%d dcc_en=%d\n",
+ DRM_DEBUG_ATOMIC("plane: id=%d dcc_en=%d\n",
new_plane_state->plane->index,
bundle->plane_infos[planes_count].dcc.enable);
@@ -7855,7 +8344,7 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state,
dc_plane,
bundle->flip_addrs[planes_count].flip_timestamp_in_us);
- DRM_DEBUG_DRIVER("%s Flipping to hi: 0x%x, low: 0x%x\n",
+ DRM_DEBUG_ATOMIC("%s Flipping to hi: 0x%x, low: 0x%x\n",
__func__,
bundle->flip_addrs[planes_count].address.grph.addr.high_part,
bundle->flip_addrs[planes_count].address.grph.addr.low_part);
@@ -7963,8 +8452,7 @@ static void amdgpu_dm_commit_planes(struct drm_atomic_state *state,
* re-adjust the min/max bounds now that DC doesn't handle this
* as part of commit.
*/
- if (amdgpu_dm_vrr_active(dm_old_crtc_state) !=
- amdgpu_dm_vrr_active(acrtc_state)) {
+ if (is_dc_timing_adjust_needed(dm_old_crtc_state, acrtc_state)) {
spin_lock_irqsave(&pcrtc->dev->event_lock, flags);
dc_stream_adjust_vmin_vmax(
dm->dc, acrtc_state->stream,
@@ -8178,7 +8666,7 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
dm_new_crtc_state = to_dm_crtc_state(new_crtc_state);
dm_old_crtc_state = to_dm_crtc_state(old_crtc_state);
- DRM_DEBUG_DRIVER(
+ DRM_DEBUG_ATOMIC(
"amdgpu_crtc id:%d crtc_state_flags: enable:%d, active:%d, "
"planes_changed:%d, mode_changed:%d,active_changed:%d,"
"connectors_changed:%d\n",
@@ -8212,7 +8700,7 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
if (modeset_required(new_crtc_state, dm_new_crtc_state->stream, dm_old_crtc_state->stream)) {
- DRM_DEBUG_DRIVER("Atomic commit: SET crtc id %d: [%p]\n", acrtc->crtc_id, acrtc);
+ DRM_DEBUG_ATOMIC("Atomic commit: SET crtc id %d: [%p]\n", acrtc->crtc_id, acrtc);
if (!dm_new_crtc_state->stream) {
/*
@@ -8245,10 +8733,11 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
crtc->hwmode = new_crtc_state->mode;
mode_set_reset_required = true;
} else if (modereset_required(new_crtc_state)) {
- DRM_DEBUG_DRIVER("Atomic commit: RESET. crtc id %d:[%p]\n", acrtc->crtc_id, acrtc);
+ DRM_DEBUG_ATOMIC("Atomic commit: RESET. crtc id %d:[%p]\n", acrtc->crtc_id, acrtc);
/* i.e. reset mode */
if (dm_old_crtc_state->stream)
remove_stream(adev, acrtc, dm_old_crtc_state->stream);
+
mode_set_reset_required = true;
}
} /* for_each_crtc_in_state() */
@@ -8261,6 +8750,11 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
dm_enable_per_frame_crtc_master_sync(dc_state);
mutex_lock(&dm->dc_lock);
WARN_ON(!dc_commit_state(dm->dc, dc_state));
+#if defined(CONFIG_DRM_AMD_DC_DCN)
+ /* Allow idle optimization when vblank count is 0 for display off */
+ if (dm->active_vblank_irq_count == 0)
+ dc_allow_idle_optimizations(dm->dc,true);
+#endif
mutex_unlock(&dm->dc_lock);
}
@@ -8307,8 +8801,7 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
hdcp_update_display(
adev->dm.hdcp_workqueue, aconnector->dc_link->link_index, aconnector,
new_con_state->hdcp_content_type,
- new_con_state->content_protection == DRM_MODE_CONTENT_PROTECTION_DESIRED ? true
- : false);
+ new_con_state->content_protection == DRM_MODE_CONTENT_PROTECTION_DESIRED);
}
#endif
@@ -8418,7 +8911,10 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
*/
for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
-
+#ifdef CONFIG_DEBUG_FS
+ bool configure_crc = false;
+ enum amdgpu_dm_pipe_crc_source cur_crc_src;
+#endif
dm_new_crtc_state = to_dm_crtc_state(new_crtc_state);
if (new_crtc_state->active &&
@@ -8434,12 +8930,21 @@ static void amdgpu_dm_atomic_commit_tail(struct drm_atomic_state *state)
* settings for the stream.
*/
dm_new_crtc_state = to_dm_crtc_state(new_crtc_state);
+ spin_lock_irqsave(&adev_to_drm(adev)->event_lock, flags);
+ cur_crc_src = acrtc->dm_irq_params.crc_src;
+ spin_unlock_irqrestore(&adev_to_drm(adev)->event_lock, flags);
+
+ if (amdgpu_dm_is_valid_crc_source(cur_crc_src)) {
+ configure_crc = true;
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ if (amdgpu_dm_crc_window_is_activated(crtc))
+ configure_crc = false;
+#endif
+ }
- if (amdgpu_dm_is_valid_crc_source(dm_new_crtc_state->crc_src)) {
+ if (configure_crc)
amdgpu_dm_crtc_configure_crc_source(
- crtc, dm_new_crtc_state,
- dm_new_crtc_state->crc_src);
- }
+ crtc, dm_new_crtc_state, cur_crc_src);
#endif
}
}
@@ -8648,6 +9153,7 @@ static void get_freesync_config_for_crtc(
to_amdgpu_dm_connector(new_con_state->base.connector);
struct drm_display_mode *mode = &new_crtc_state->base.mode;
int vrefresh = drm_mode_vrefresh(mode);
+ bool fs_vid_mode = false;
new_crtc_state->vrr_supported = new_con_state->freesync_capable &&
vrefresh >= aconnector->min_vfreq &&
@@ -8655,17 +9161,24 @@ static void get_freesync_config_for_crtc(
if (new_crtc_state->vrr_supported) {
new_crtc_state->stream->ignore_msa_timing_param = true;
- config.state = new_crtc_state->base.vrr_enabled ?
- VRR_STATE_ACTIVE_VARIABLE :
- VRR_STATE_INACTIVE;
- config.min_refresh_in_uhz =
- aconnector->min_vfreq * 1000000;
- config.max_refresh_in_uhz =
- aconnector->max_vfreq * 1000000;
+ fs_vid_mode = new_crtc_state->freesync_config.state == VRR_STATE_ACTIVE_FIXED;
+
+ config.min_refresh_in_uhz = aconnector->min_vfreq * 1000000;
+ config.max_refresh_in_uhz = aconnector->max_vfreq * 1000000;
config.vsif_supported = true;
config.btr = true;
- }
+ if (fs_vid_mode) {
+ config.state = VRR_STATE_ACTIVE_FIXED;
+ config.fixed_refresh_in_uhz = new_crtc_state->freesync_config.fixed_refresh_in_uhz;
+ goto out;
+ } else if (new_crtc_state->base.vrr_enabled) {
+ config.state = VRR_STATE_ACTIVE_VARIABLE;
+ } else {
+ config.state = VRR_STATE_INACTIVE;
+ }
+ }
+out:
new_crtc_state->freesync_config = config;
}
@@ -8678,6 +9191,50 @@ static void reset_freesync_config_for_crtc(
sizeof(new_crtc_state->vrr_infopacket));
}
+static bool
+is_timing_unchanged_for_freesync(struct drm_crtc_state *old_crtc_state,
+ struct drm_crtc_state *new_crtc_state)
+{
+ struct drm_display_mode old_mode, new_mode;
+
+ if (!old_crtc_state || !new_crtc_state)
+ return false;
+
+ old_mode = old_crtc_state->mode;
+ new_mode = new_crtc_state->mode;
+
+ if (old_mode.clock == new_mode.clock &&
+ old_mode.hdisplay == new_mode.hdisplay &&
+ old_mode.vdisplay == new_mode.vdisplay &&
+ old_mode.htotal == new_mode.htotal &&
+ old_mode.vtotal != new_mode.vtotal &&
+ old_mode.hsync_start == new_mode.hsync_start &&
+ old_mode.vsync_start != new_mode.vsync_start &&
+ old_mode.hsync_end == new_mode.hsync_end &&
+ old_mode.vsync_end != new_mode.vsync_end &&
+ old_mode.hskew == new_mode.hskew &&
+ old_mode.vscan == new_mode.vscan &&
+ (old_mode.vsync_end - old_mode.vsync_start) ==
+ (new_mode.vsync_end - new_mode.vsync_start))
+ return true;
+
+ return false;
+}
+
+static void set_freesync_fixed_config(struct dm_crtc_state *dm_new_crtc_state) {
+ uint64_t num, den, res;
+ struct drm_crtc_state *new_crtc_state = &dm_new_crtc_state->base;
+
+ dm_new_crtc_state->freesync_config.state = VRR_STATE_ACTIVE_FIXED;
+
+ num = (unsigned long long)new_crtc_state->mode.clock * 1000 * 1000000;
+ den = (unsigned long long)new_crtc_state->mode.htotal *
+ (unsigned long long)new_crtc_state->mode.vtotal;
+
+ res = div_u64(num, den);
+ dm_new_crtc_state->freesync_config.fixed_refresh_in_uhz = res;
+}
+
static int dm_update_crtc_state(struct amdgpu_display_manager *dm,
struct drm_atomic_state *state,
struct drm_crtc *crtc,
@@ -8768,6 +9325,11 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm,
* TODO: Refactor this function to allow this check to work
* in all conditions.
*/
+ if (amdgpu_freesync_vid_mode &&
+ dm_new_crtc_state->stream &&
+ is_timing_unchanged_for_freesync(new_crtc_state, old_crtc_state))
+ goto skip_modeset;
+
if (dm_new_crtc_state->stream &&
dc_is_stream_unchanged(new_stream, dm_old_crtc_state->stream) &&
dc_is_stream_scaling_unchanged(new_stream, dm_old_crtc_state->stream)) {
@@ -8781,7 +9343,7 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm,
if (!drm_atomic_crtc_needs_modeset(new_crtc_state))
goto skip_modeset;
- DRM_DEBUG_DRIVER(
+ DRM_DEBUG_ATOMIC(
"amdgpu_crtc id:%d crtc_state_flags: enable:%d, active:%d, "
"planes_changed:%d, mode_changed:%d,active_changed:%d,"
"connectors_changed:%d\n",
@@ -8799,6 +9361,24 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm,
if (!dm_old_crtc_state->stream)
goto skip_modeset;
+ if (amdgpu_freesync_vid_mode && dm_new_crtc_state->stream &&
+ is_timing_unchanged_for_freesync(new_crtc_state,
+ old_crtc_state)) {
+ new_crtc_state->mode_changed = false;
+ DRM_DEBUG_DRIVER(
+ "Mode change not required for front porch change, "
+ "setting mode_changed to %d",
+ new_crtc_state->mode_changed);
+
+ set_freesync_fixed_config(dm_new_crtc_state);
+
+ goto skip_modeset;
+ } else if (amdgpu_freesync_vid_mode && aconnector &&
+ is_freesync_video_mode(&new_crtc_state->mode,
+ aconnector)) {
+ set_freesync_fixed_config(dm_new_crtc_state);
+ }
+
ret = dm_atomic_get_state(state, &dm_state);
if (ret)
goto fail;
@@ -8847,8 +9427,8 @@ static int dm_update_crtc_state(struct amdgpu_display_manager *dm,
dc_stream_retain(new_stream);
- DRM_DEBUG_DRIVER("Enabling DRM crtc: %d\n",
- crtc->base.id);
+ DRM_DEBUG_ATOMIC("Enabling DRM crtc: %d\n",
+ crtc->base.id);
if (dc_add_stream_to_ctx(
dm->dc,
@@ -9193,8 +9773,8 @@ static int dm_update_plane_state(struct dc *dc,
if (!dc_new_plane_state)
return -ENOMEM;
- DRM_DEBUG_DRIVER("Enabling DRM plane: %d on DRM crtc %d\n",
- plane->base.id, new_plane_crtc->base.id);
+ DRM_DEBUG_ATOMIC("Enabling DRM plane: %d on DRM crtc %d\n",
+ plane->base.id, new_plane_crtc->base.id);
ret = fill_dc_plane_attributes(
drm_to_adev(new_plane_crtc->dev),
@@ -9257,7 +9837,8 @@ static int dm_check_crtc_cursor(struct drm_atomic_state *state,
new_cursor_state = drm_atomic_get_new_plane_state(state, crtc->cursor);
new_primary_state = drm_atomic_get_new_plane_state(state, crtc->primary);
- if (!new_cursor_state || !new_primary_state || !new_cursor_state->fb) {
+ if (!new_cursor_state || !new_primary_state ||
+ !new_cursor_state->fb || !new_primary_state->fb) {
return 0;
}
@@ -9305,6 +9886,53 @@ static int add_affected_mst_dsc_crtcs(struct drm_atomic_state *state, struct drm
}
#endif
+static int validate_overlay(struct drm_atomic_state *state)
+{
+ int i;
+ struct drm_plane *plane;
+ struct drm_plane_state *old_plane_state, *new_plane_state;
+ struct drm_plane_state *primary_state, *overlay_state = NULL;
+
+ /* Check if primary plane is contained inside overlay */
+ for_each_oldnew_plane_in_state_reverse(state, plane, old_plane_state, new_plane_state, i) {
+ if (plane->type == DRM_PLANE_TYPE_OVERLAY) {
+ if (drm_atomic_plane_disabling(plane->state, new_plane_state))
+ return 0;
+
+ overlay_state = new_plane_state;
+ continue;
+ }
+ }
+
+ /* check if we're making changes to the overlay plane */
+ if (!overlay_state)
+ return 0;
+
+ /* check if overlay plane is enabled */
+ if (!overlay_state->crtc)
+ return 0;
+
+ /* find the primary plane for the CRTC that the overlay is enabled on */
+ primary_state = drm_atomic_get_plane_state(state, overlay_state->crtc->primary);
+ if (IS_ERR(primary_state))
+ return PTR_ERR(primary_state);
+
+ /* check if primary plane is enabled */
+ if (!primary_state->crtc)
+ return 0;
+
+ /* Perform the bounds check to ensure the overlay plane covers the primary */
+ if (primary_state->crtc_x < overlay_state->crtc_x ||
+ primary_state->crtc_y < overlay_state->crtc_y ||
+ primary_state->crtc_x + primary_state->crtc_w > overlay_state->crtc_x + overlay_state->crtc_w ||
+ primary_state->crtc_y + primary_state->crtc_h > overlay_state->crtc_y + overlay_state->crtc_h) {
+ DRM_DEBUG_ATOMIC("Overlay plane is enabled with hardware cursor but does not fully cover primary plane\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
/**
* amdgpu_dm_atomic_check() - Atomic check implementation for AMDgpu DM.
* @dev: The DRM device
@@ -9376,7 +10004,7 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
}
#if defined(CONFIG_DRM_AMD_DC_DCN)
- if (adev->asic_type >= CHIP_NAVI10) {
+ if (dc_resource_is_dsc_encoding_supported(dc)) {
for_each_oldnew_crtc_in_state(state, crtc, old_crtc_state, new_crtc_state, i) {
if (drm_atomic_crtc_needs_modeset(new_crtc_state)) {
ret = add_affected_mst_dsc_crtcs(state, crtc);
@@ -9479,6 +10107,10 @@ static int amdgpu_dm_atomic_check(struct drm_device *dev,
goto fail;
}
+ ret = validate_overlay(state);
+ if (ret)
+ goto fail;
+
/* Add new/modified planes */
for_each_oldnew_plane_in_state_reverse(state, plane, old_plane_state, new_plane_state, i) {
ret = dm_update_plane_state(dc, state, plane,
@@ -9682,11 +10314,85 @@ static bool is_dp_capable_without_timing_msa(struct dc *dc,
return capable;
}
+
+static bool parse_edid_cea(struct amdgpu_dm_connector *aconnector,
+ uint8_t *edid_ext, int len,
+ struct amdgpu_hdmi_vsdb_info *vsdb_info)
+{
+ int i;
+ struct amdgpu_device *adev = drm_to_adev(aconnector->base.dev);
+ struct dc *dc = adev->dm.dc;
+
+ /* send extension block to DMCU for parsing */
+ for (i = 0; i < len; i += 8) {
+ bool res;
+ int offset;
+
+ /* send 8 bytes a time */
+ if (!dc_edid_parser_send_cea(dc, i, len, &edid_ext[i], 8))
+ return false;
+
+ if (i+8 == len) {
+ /* EDID block sent completed, expect result */
+ int version, min_rate, max_rate;
+
+ res = dc_edid_parser_recv_amd_vsdb(dc, &version, &min_rate, &max_rate);
+ if (res) {
+ /* amd vsdb found */
+ vsdb_info->freesync_supported = 1;
+ vsdb_info->amd_vsdb_version = version;
+ vsdb_info->min_refresh_rate_hz = min_rate;
+ vsdb_info->max_refresh_rate_hz = max_rate;
+ return true;
+ }
+ /* not amd vsdb */
+ return false;
+ }
+
+ /* check for ack*/
+ res = dc_edid_parser_recv_cea_ack(dc, &offset);
+ if (!res)
+ return false;
+ }
+
+ return false;
+}
+
+static int parse_hdmi_amd_vsdb(struct amdgpu_dm_connector *aconnector,
+ struct edid *edid, struct amdgpu_hdmi_vsdb_info *vsdb_info)
+{
+ uint8_t *edid_ext = NULL;
+ int i;
+ bool valid_vsdb_found = false;
+
+ /*----- drm_find_cea_extension() -----*/
+ /* No EDID or EDID extensions */
+ if (edid == NULL || edid->extensions == 0)
+ return -ENODEV;
+
+ /* Find CEA extension */
+ for (i = 0; i < edid->extensions; i++) {
+ edid_ext = (uint8_t *)edid + EDID_LENGTH * (i + 1);
+ if (edid_ext[0] == CEA_EXT)
+ break;
+ }
+
+ if (i == edid->extensions)
+ return -ENODEV;
+
+ /*----- cea_db_offsets() -----*/
+ if (edid_ext[0] != CEA_EXT)
+ return -ENODEV;
+
+ valid_vsdb_found = parse_edid_cea(aconnector, edid_ext, EDID_LENGTH, vsdb_info);
+
+ return valid_vsdb_found ? i : -ENODEV;
+}
+
void amdgpu_dm_update_freesync_caps(struct drm_connector *connector,
struct edid *edid)
{
- int i;
- bool edid_check_required;
+ int i = 0;
struct detailed_timing *timing;
struct detailed_non_pixel *data;
struct detailed_data_monitor_range *range;
@@ -9697,6 +10403,7 @@ void amdgpu_dm_update_freesync_caps(struct drm_connector *connector,
struct drm_device *dev = connector->dev;
struct amdgpu_device *adev = drm_to_adev(dev);
bool freesync_capable = false;
+ struct amdgpu_hdmi_vsdb_info vsdb_info = {0};
if (!connector->state) {
DRM_ERROR("%s - Connector has no state", __func__);
@@ -9715,60 +10422,75 @@ void amdgpu_dm_update_freesync_caps(struct drm_connector *connector,
dm_con_state = to_dm_connector_state(connector->state);
- edid_check_required = false;
if (!amdgpu_dm_connector->dc_sink) {
DRM_ERROR("dc_sink NULL, could not add free_sync module.\n");
goto update;
}
if (!adev->dm.freesync_module)
goto update;
- /*
- * if edid non zero restrict freesync only for dp and edp
- */
- if (edid) {
- if (amdgpu_dm_connector->dc_sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT
- || amdgpu_dm_connector->dc_sink->sink_signal == SIGNAL_TYPE_EDP) {
+
+
+ if (amdgpu_dm_connector->dc_sink->sink_signal == SIGNAL_TYPE_DISPLAY_PORT
+ || amdgpu_dm_connector->dc_sink->sink_signal == SIGNAL_TYPE_EDP) {
+ bool edid_check_required = false;
+
+ if (edid) {
edid_check_required = is_dp_capable_without_timing_msa(
adev->dm.dc,
amdgpu_dm_connector);
}
- }
- if (edid_check_required == true && (edid->version > 1 ||
- (edid->version == 1 && edid->revision > 1))) {
- for (i = 0; i < 4; i++) {
- timing = &edid->detailed_timings[i];
- data = &timing->data.other_data;
- range = &data->data.range;
- /*
- * Check if monitor has continuous frequency mode
- */
- if (data->type != EDID_DETAIL_MONITOR_RANGE)
- continue;
- /*
- * Check for flag range limits only. If flag == 1 then
- * no additional timing information provided.
- * Default GTF, GTF Secondary curve and CVT are not
- * supported
- */
- if (range->flags != 1)
- continue;
+ if (edid_check_required == true && (edid->version > 1 ||
+ (edid->version == 1 && edid->revision > 1))) {
+ for (i = 0; i < 4; i++) {
- amdgpu_dm_connector->min_vfreq = range->min_vfreq;
- amdgpu_dm_connector->max_vfreq = range->max_vfreq;
- amdgpu_dm_connector->pixel_clock_mhz =
- range->pixel_clock_mhz * 10;
+ timing = &edid->detailed_timings[i];
+ data = &timing->data.other_data;
+ range = &data->data.range;
+ /*
+ * Check if monitor has continuous frequency mode
+ */
+ if (data->type != EDID_DETAIL_MONITOR_RANGE)
+ continue;
+ /*
+ * Check for flag range limits only. If flag == 1 then
+ * no additional timing information provided.
+ * Default GTF, GTF Secondary curve and CVT are not
+ * supported
+ */
+ if (range->flags != 1)
+ continue;
- connector->display_info.monitor_range.min_vfreq = range->min_vfreq;
- connector->display_info.monitor_range.max_vfreq = range->max_vfreq;
+ amdgpu_dm_connector->min_vfreq = range->min_vfreq;
+ amdgpu_dm_connector->max_vfreq = range->max_vfreq;
+ amdgpu_dm_connector->pixel_clock_mhz =
+ range->pixel_clock_mhz * 10;
- break;
- }
+ connector->display_info.monitor_range.min_vfreq = range->min_vfreq;
+ connector->display_info.monitor_range.max_vfreq = range->max_vfreq;
- if (amdgpu_dm_connector->max_vfreq -
- amdgpu_dm_connector->min_vfreq > 10) {
+ break;
+ }
+
+ if (amdgpu_dm_connector->max_vfreq -
+ amdgpu_dm_connector->min_vfreq > 10) {
- freesync_capable = true;
+ freesync_capable = true;
+ }
+ }
+ } else if (edid && amdgpu_dm_connector->dc_sink->sink_signal == SIGNAL_TYPE_HDMI_TYPE_A) {
+ i = parse_hdmi_amd_vsdb(amdgpu_dm_connector, edid, &vsdb_info);
+ if (i >= 0 && vsdb_info.freesync_supported) {
+ timing = &edid->detailed_timings[i];
+ data = &timing->data.other_data;
+
+ amdgpu_dm_connector->min_vfreq = vsdb_info.min_refresh_rate_hz;
+ amdgpu_dm_connector->max_vfreq = vsdb_info.max_refresh_rate_hz;
+ if (amdgpu_dm_connector->max_vfreq - amdgpu_dm_connector->min_vfreq > 10)
+ freesync_capable = true;
+
+ connector->display_info.monitor_range.min_vfreq = vsdb_info.min_refresh_rate_hz;
+ connector->display_info.monitor_range.max_vfreq = vsdb_info.max_refresh_rate_hz;
}
}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
index 8bfe901cf237..b2f2ccfc20bb 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.h
@@ -1,5 +1,5 @@
/*
- * Copyright 2015 Advanced Micro Devices, Inc.
+ * Copyright (C) 2015-2020 Advanced Micro Devices, Inc. All rights reserved.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -66,18 +66,7 @@ struct dc_plane_state;
struct common_irq_params {
struct amdgpu_device *adev;
enum dc_irq_source irq_src;
-};
-
-/**
- * struct irq_list_head - Linked-list for low context IRQ handlers.
- *
- * @head: The list_head within &struct handler_data
- * @work: A work_struct containing the deferred handler work
- */
-struct irq_list_head {
- struct list_head head;
- /* In case this interrupt needs post-processing, 'work' will be queued*/
- struct work_struct work;
+ atomic64_t previous_timestamp;
};
/**
@@ -145,6 +134,16 @@ struct amdgpu_dm_backlight_caps {
};
/**
+ * struct dal_allocation - Tracks mapped FB memory for SMU communication
+ */
+struct dal_allocation {
+ struct list_head list;
+ struct amdgpu_bo *bo;
+ void *cpu_ptr;
+ u64 gpu_addr;
+};
+
+/**
* struct amdgpu_display_manager - Central amdgpu display manager device
*
* @dc: Display Core control structure
@@ -257,12 +256,12 @@ struct amdgpu_display_manager {
*/
struct mutex audio_lock;
+#if defined(CONFIG_DRM_AMD_DC_DCN)
/**
- * @vblank_work_lock:
+ * @vblank_lock:
*
* Guards access to deferred vblank work state.
*/
-#if defined(CONFIG_DRM_AMD_DC_DCN)
spinlock_t vblank_lock;
#endif
@@ -293,7 +292,7 @@ struct amdgpu_display_manager {
* Note that handlers are called in the same order as they were
* registered (FIFO).
*/
- struct irq_list_head irq_handler_list_low_tab[DAL_IRQ_SOURCES_NUMBER];
+ struct list_head irq_handler_list_low_tab[DAL_IRQ_SOURCES_NUMBER];
/**
* @irq_handler_list_high_tab:
@@ -324,6 +323,15 @@ struct amdgpu_display_manager {
vblank_params[DC_IRQ_SOURCE_VBLANK6 - DC_IRQ_SOURCE_VBLANK1 + 1];
/**
+ * @vline0_params:
+ *
+ * OTG vertical interrupt0 IRQ parameters, passed to registered
+ * handlers when triggered.
+ */
+ struct common_irq_params
+ vline0_params[DC_IRQ_SOURCE_DC6_VLINE0 - DC_IRQ_SOURCE_DC1_VLINE0 + 1];
+
+ /**
* @vupdate_params:
*
* Vertical update IRQ parameters, passed to registered handlers when
@@ -332,6 +340,15 @@ struct amdgpu_display_manager {
struct common_irq_params
vupdate_params[DC_IRQ_SOURCE_VUPDATE6 - DC_IRQ_SOURCE_VUPDATE1 + 1];
+ /**
+ * @dmub_trace_params:
+ *
+ * DMUB trace event IRQ parameters, passed to registered handlers when
+ * triggered.
+ */
+ struct common_irq_params
+ dmub_trace_params[1];
+
spinlock_t irq_handler_list_table_lock;
struct backlight_device *backlight_dev;
@@ -345,6 +362,11 @@ struct amdgpu_display_manager {
#endif
#if defined(CONFIG_DRM_AMD_DC_DCN)
+ /**
+ * @vblank_workqueue:
+ *
+ * amdgpu workqueue during vblank
+ */
struct vblank_workqueue *vblank_workqueue;
#endif
@@ -363,12 +385,23 @@ struct amdgpu_display_manager {
*/
const struct gpu_info_soc_bounding_box_v1_0 *soc_bounding_box;
+#if defined(CONFIG_DRM_AMD_DC_DCN)
/**
* @active_vblank_irq_count:
*
* number of currently active vblank irqs
*/
uint32_t active_vblank_irq_count;
+#endif
+
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ /**
+ * @crc_rd_wrk:
+ *
+ * Work to be executed in a separate thread to communicate with PSP.
+ */
+ struct crc_rd_work *crc_rd_wrk;
+#endif
/**
* @mst_encoders:
@@ -377,6 +410,14 @@ struct amdgpu_display_manager {
*/
struct amdgpu_encoder mst_encoders[AMDGPU_DM_MAX_CRTC];
bool force_timing_sync;
+ bool disable_hpd_irq;
+ bool dmcub_trace_event_en;
+ /**
+ * @da_list:
+ *
+ * DAL fb memory allocation list, for communication with SMU.
+ */
+ struct list_head da_list;
};
enum dsc_clock_force_state {
@@ -440,6 +481,8 @@ struct amdgpu_dm_connector {
#endif
bool force_yuv420_output;
struct dsc_preferred_settings dsc_settings;
+ /* Cached display modes */
+ struct drm_display_mode freesync_vid_base;
};
#define to_amdgpu_dm_connector(x) container_of(x, struct amdgpu_dm_connector, base)
@@ -462,7 +505,6 @@ struct dm_crtc_state {
int active_planes;
int crc_skip_count;
- enum amdgpu_dm_pipe_crc_source crc_src;
bool freesync_timing_changed;
bool freesync_vrr_info_changed;
@@ -501,6 +543,14 @@ struct dm_connector_state {
uint64_t pbn;
};
+struct amdgpu_hdmi_vsdb_info {
+ unsigned int amd_vsdb_version; /* VSDB version, should be used to determine which VSIF to send */
+ bool freesync_supported; /* FreeSync Supported */
+ unsigned int min_refresh_rate_hz; /* FreeSync Minimum Refresh Rate in Hz */
+ unsigned int max_refresh_rate_hz; /* FreeSync Maximum Refresh Rate in Hz */
+};
+
+
#define to_dm_connector_state(x)\
container_of((x), struct dm_connector_state, base)
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c
index 66cb8730586b..5cd788b20c21 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.c
@@ -29,6 +29,7 @@
#include "amdgpu.h"
#include "amdgpu_dm.h"
#include "dc.h"
+#include "amdgpu_securedisplay.h"
static const char *const pipe_crc_sources[] = {
"none",
@@ -81,6 +82,73 @@ const char *const *amdgpu_dm_crtc_get_crc_sources(struct drm_crtc *crtc,
return pipe_crc_sources;
}
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+static void amdgpu_dm_set_crc_window_default(struct drm_crtc *crtc)
+{
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ acrtc->dm_irq_params.crc_window.x_start = 0;
+ acrtc->dm_irq_params.crc_window.y_start = 0;
+ acrtc->dm_irq_params.crc_window.x_end = 0;
+ acrtc->dm_irq_params.crc_window.y_end = 0;
+ acrtc->dm_irq_params.crc_window.activated = false;
+ acrtc->dm_irq_params.crc_window.update_win = false;
+ acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ spin_unlock_irq(&drm_dev->event_lock);
+}
+
+static void amdgpu_dm_crtc_notify_ta_to_read(struct work_struct *work)
+{
+ struct crc_rd_work *crc_rd_wrk;
+ struct amdgpu_device *adev;
+ struct psp_context *psp;
+ struct securedisplay_cmd *securedisplay_cmd;
+ struct drm_crtc *crtc;
+ uint8_t phy_id;
+ int ret;
+
+ crc_rd_wrk = container_of(work, struct crc_rd_work, notify_ta_work);
+ spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock);
+ crtc = crc_rd_wrk->crtc;
+
+ if (!crtc) {
+ spin_unlock_irq(&crc_rd_wrk->crc_rd_work_lock);
+ return;
+ }
+
+ adev = drm_to_adev(crtc->dev);
+ psp = &adev->psp;
+ phy_id = crc_rd_wrk->phy_inst;
+ spin_unlock_irq(&crc_rd_wrk->crc_rd_work_lock);
+
+ psp_prep_securedisplay_cmd_buf(psp, &securedisplay_cmd,
+ TA_SECUREDISPLAY_COMMAND__SEND_ROI_CRC);
+ securedisplay_cmd->securedisplay_in_message.send_roi_crc.phy_id =
+ phy_id;
+ ret = psp_securedisplay_invoke(psp, TA_SECUREDISPLAY_COMMAND__SEND_ROI_CRC);
+ if (!ret) {
+ if (securedisplay_cmd->status != TA_SECUREDISPLAY_STATUS__SUCCESS) {
+ psp_securedisplay_parse_resp_status(psp, securedisplay_cmd->status);
+ }
+ }
+}
+
+bool amdgpu_dm_crc_window_is_activated(struct drm_crtc *crtc)
+{
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+ bool ret = false;
+
+ spin_lock_irq(&drm_dev->event_lock);
+ ret = acrtc->dm_irq_params.crc_window.activated;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return ret;
+}
+#endif
+
int
amdgpu_dm_crtc_verify_crc_source(struct drm_crtc *crtc, const char *src_name,
size_t *values_cnt)
@@ -114,6 +182,20 @@ int amdgpu_dm_crtc_configure_crc_source(struct drm_crtc *crtc,
/* Enable CRTC CRC generation if necessary. */
if (dm_is_crc_source_crtc(source) || source == AMDGPU_DM_PIPE_CRC_SOURCE_NONE) {
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ if (!enable) {
+ if (adev->dm.crc_rd_wrk) {
+ flush_work(&adev->dm.crc_rd_wrk->notify_ta_work);
+ spin_lock_irq(&adev->dm.crc_rd_wrk->crc_rd_work_lock);
+ if (adev->dm.crc_rd_wrk->crtc == crtc) {
+ dc_stream_stop_dmcu_crc_win_update(stream_state->ctx->dc,
+ dm_crtc_state->stream);
+ adev->dm.crc_rd_wrk->crtc = NULL;
+ }
+ spin_unlock_irq(&adev->dm.crc_rd_wrk->crc_rd_work_lock);
+ }
+ }
+#endif
if (!dc_stream_configure_crc(stream_state->ctx->dc,
stream_state, NULL, enable, enable)) {
ret = -EINVAL;
@@ -142,8 +224,11 @@ unlock:
int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name)
{
enum amdgpu_dm_pipe_crc_source source = dm_parse_crc_source(src_name);
+ enum amdgpu_dm_pipe_crc_source cur_crc_src;
struct drm_crtc_commit *commit;
struct dm_crtc_state *crtc_state;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
struct drm_dp_aux *aux = NULL;
bool enable = false;
bool enabled = false;
@@ -182,6 +267,9 @@ int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name)
enable = amdgpu_dm_is_valid_crc_source(source);
crtc_state = to_dm_crtc_state(crtc->state);
+ spin_lock_irq(&drm_dev->event_lock);
+ cur_crc_src = acrtc->dm_irq_params.crc_src;
+ spin_unlock_irq(&drm_dev->event_lock);
/*
* USER REQ SRC | CURRENT SRC | BEHAVIOR
@@ -198,7 +286,7 @@ int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name)
*/
if (dm_is_crc_source_dprx(source) ||
(source == AMDGPU_DM_PIPE_CRC_SOURCE_NONE &&
- dm_is_crc_source_dprx(crtc_state->crc_src))) {
+ dm_is_crc_source_dprx(cur_crc_src))) {
struct amdgpu_dm_connector *aconn = NULL;
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
@@ -219,7 +307,7 @@ int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name)
goto cleanup;
}
- aux = &aconn->dm_dp_aux.aux;
+ aux = (aconn->port) ? &aconn->port->aux : &aconn->dm_dp_aux.aux;
if (!aux) {
DRM_DEBUG_DRIVER("No dp aux for amd connector\n");
@@ -228,6 +316,10 @@ int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name)
}
}
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ amdgpu_dm_set_crc_window_default(crtc);
+#endif
+
if (amdgpu_dm_crtc_configure_crc_source(crtc, crtc_state, source)) {
ret = -EINVAL;
goto cleanup;
@@ -237,7 +329,7 @@ int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name)
* Reading the CRC requires the vblank interrupt handler to be
* enabled. Keep a reference until CRC capture stops.
*/
- enabled = amdgpu_dm_is_valid_crc_source(crtc_state->crc_src);
+ enabled = amdgpu_dm_is_valid_crc_source(cur_crc_src);
if (!enabled && enable) {
ret = drm_crtc_vblank_get(crtc);
if (ret)
@@ -261,7 +353,9 @@ int amdgpu_dm_crtc_set_crc_source(struct drm_crtc *crtc, const char *src_name)
}
}
- crtc_state->crc_src = source;
+ spin_lock_irq(&drm_dev->event_lock);
+ acrtc->dm_irq_params.crc_src = source;
+ spin_unlock_irq(&drm_dev->event_lock);
/* Reset crc_skipped on dm state */
crtc_state->crc_skip_count = 0;
@@ -286,16 +380,26 @@ void amdgpu_dm_crtc_handle_crc_irq(struct drm_crtc *crtc)
{
struct dm_crtc_state *crtc_state;
struct dc_stream_state *stream_state;
+ struct drm_device *drm_dev = NULL;
+ enum amdgpu_dm_pipe_crc_source cur_crc_src;
+ struct amdgpu_crtc *acrtc = NULL;
uint32_t crcs[3];
+ unsigned long flags;
if (crtc == NULL)
return;
crtc_state = to_dm_crtc_state(crtc->state);
stream_state = crtc_state->stream;
+ acrtc = to_amdgpu_crtc(crtc);
+ drm_dev = crtc->dev;
+
+ spin_lock_irqsave(&drm_dev->event_lock, flags);
+ cur_crc_src = acrtc->dm_irq_params.crc_src;
+ spin_unlock_irqrestore(&drm_dev->event_lock, flags);
/* Early return if CRC capture is not enabled. */
- if (!amdgpu_dm_is_valid_crc_source(crtc_state->crc_src))
+ if (!amdgpu_dm_is_valid_crc_source(cur_crc_src))
return;
/*
@@ -309,7 +413,7 @@ void amdgpu_dm_crtc_handle_crc_irq(struct drm_crtc *crtc)
return;
}
- if (dm_is_crc_source_crtc(crtc_state->crc_src)) {
+ if (dm_is_crc_source_crtc(cur_crc_src)) {
if (!dc_stream_get_crc(stream_state->ctx->dc, stream_state,
&crcs[0], &crcs[1], &crcs[2]))
return;
@@ -318,3 +422,182 @@ void amdgpu_dm_crtc_handle_crc_irq(struct drm_crtc *crtc)
drm_crtc_accurate_vblank_count(crtc), crcs);
}
}
+
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+void amdgpu_dm_crtc_handle_crc_window_irq(struct drm_crtc *crtc)
+{
+ struct dc_stream_state *stream_state;
+ struct drm_device *drm_dev = NULL;
+ enum amdgpu_dm_pipe_crc_source cur_crc_src;
+ struct amdgpu_crtc *acrtc = NULL;
+ struct amdgpu_device *adev = NULL;
+ struct crc_rd_work *crc_rd_wrk = NULL;
+ struct crc_params *crc_window = NULL, tmp_window;
+ unsigned long flags1, flags2;
+ struct crtc_position position;
+ uint32_t v_blank;
+ uint32_t v_back_porch;
+ uint32_t crc_window_latch_up_line;
+ struct dc_crtc_timing *timing_out;
+
+ if (crtc == NULL)
+ return;
+
+ acrtc = to_amdgpu_crtc(crtc);
+ adev = drm_to_adev(crtc->dev);
+ drm_dev = crtc->dev;
+
+ spin_lock_irqsave(&drm_dev->event_lock, flags1);
+ stream_state = acrtc->dm_irq_params.stream;
+ cur_crc_src = acrtc->dm_irq_params.crc_src;
+ timing_out = &stream_state->timing;
+
+ /* Early return if CRC capture is not enabled. */
+ if (!amdgpu_dm_is_valid_crc_source(cur_crc_src))
+ goto cleanup;
+
+ if (dm_is_crc_source_crtc(cur_crc_src)) {
+ if (acrtc->dm_irq_params.crc_window.activated) {
+ if (acrtc->dm_irq_params.crc_window.update_win) {
+ if (acrtc->dm_irq_params.crc_window.skip_frame_cnt) {
+ acrtc->dm_irq_params.crc_window.skip_frame_cnt -= 1;
+ goto cleanup;
+ }
+ crc_window = &tmp_window;
+
+ tmp_window.windowa_x_start =
+ acrtc->dm_irq_params.crc_window.x_start;
+ tmp_window.windowa_y_start =
+ acrtc->dm_irq_params.crc_window.y_start;
+ tmp_window.windowa_x_end =
+ acrtc->dm_irq_params.crc_window.x_end;
+ tmp_window.windowa_y_end =
+ acrtc->dm_irq_params.crc_window.y_end;
+ tmp_window.windowb_x_start =
+ acrtc->dm_irq_params.crc_window.x_start;
+ tmp_window.windowb_y_start =
+ acrtc->dm_irq_params.crc_window.y_start;
+ tmp_window.windowb_x_end =
+ acrtc->dm_irq_params.crc_window.x_end;
+ tmp_window.windowb_y_end =
+ acrtc->dm_irq_params.crc_window.y_end;
+
+ dc_stream_forward_dmcu_crc_window(stream_state->ctx->dc,
+ stream_state, crc_window);
+
+ acrtc->dm_irq_params.crc_window.update_win = false;
+
+ dc_stream_get_crtc_position(stream_state->ctx->dc, &stream_state, 1,
+ &position.vertical_count,
+ &position.nominal_vcount);
+
+ v_blank = timing_out->v_total - timing_out->v_border_top -
+ timing_out->v_addressable - timing_out->v_border_bottom;
+
+ v_back_porch = v_blank - timing_out->v_front_porch -
+ timing_out->v_sync_width;
+
+ crc_window_latch_up_line = v_back_porch + timing_out->v_sync_width;
+
+ /* take 3 lines margin*/
+ if ((position.vertical_count + 3) >= crc_window_latch_up_line)
+ acrtc->dm_irq_params.crc_window.skip_frame_cnt = 1;
+ else
+ acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ } else {
+ if (acrtc->dm_irq_params.crc_window.skip_frame_cnt == 0) {
+ if (adev->dm.crc_rd_wrk) {
+ crc_rd_wrk = adev->dm.crc_rd_wrk;
+ spin_lock_irqsave(&crc_rd_wrk->crc_rd_work_lock, flags2);
+ crc_rd_wrk->phy_inst =
+ stream_state->link->link_enc_hw_inst;
+ spin_unlock_irqrestore(&crc_rd_wrk->crc_rd_work_lock, flags2);
+ schedule_work(&crc_rd_wrk->notify_ta_work);
+ }
+ } else {
+ acrtc->dm_irq_params.crc_window.skip_frame_cnt -= 1;
+ }
+ }
+ }
+ }
+
+cleanup:
+ spin_unlock_irqrestore(&drm_dev->event_lock, flags1);
+}
+
+void amdgpu_dm_crtc_secure_display_resume(struct amdgpu_device *adev)
+{
+ struct drm_crtc *crtc;
+ enum amdgpu_dm_pipe_crc_source cur_crc_src;
+ struct crc_rd_work *crc_rd_wrk = adev->dm.crc_rd_wrk;
+ struct crc_window_parm cur_crc_window;
+ struct amdgpu_crtc *acrtc = NULL;
+
+ drm_for_each_crtc(crtc, &adev->ddev) {
+ acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&adev_to_drm(adev)->event_lock);
+ cur_crc_src = acrtc->dm_irq_params.crc_src;
+ cur_crc_window = acrtc->dm_irq_params.crc_window;
+ spin_unlock_irq(&adev_to_drm(adev)->event_lock);
+
+ if (amdgpu_dm_is_valid_crc_source(cur_crc_src)) {
+ amdgpu_dm_crtc_set_crc_source(crtc,
+ pipe_crc_sources[cur_crc_src]);
+ spin_lock_irq(&adev_to_drm(adev)->event_lock);
+ acrtc->dm_irq_params.crc_window = cur_crc_window;
+ if (acrtc->dm_irq_params.crc_window.activated) {
+ acrtc->dm_irq_params.crc_window.update_win = true;
+ acrtc->dm_irq_params.crc_window.skip_frame_cnt = 1;
+ spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock);
+ crc_rd_wrk->crtc = crtc;
+ spin_unlock_irq(&crc_rd_wrk->crc_rd_work_lock);
+ }
+ spin_unlock_irq(&adev_to_drm(adev)->event_lock);
+ }
+ }
+}
+
+void amdgpu_dm_crtc_secure_display_suspend(struct amdgpu_device *adev)
+{
+ struct drm_crtc *crtc;
+ struct crc_window_parm cur_crc_window;
+ enum amdgpu_dm_pipe_crc_source cur_crc_src;
+ struct amdgpu_crtc *acrtc = NULL;
+
+ drm_for_each_crtc(crtc, &adev->ddev) {
+ acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&adev_to_drm(adev)->event_lock);
+ cur_crc_src = acrtc->dm_irq_params.crc_src;
+ cur_crc_window = acrtc->dm_irq_params.crc_window;
+ cur_crc_window.update_win = false;
+ spin_unlock_irq(&adev_to_drm(adev)->event_lock);
+
+ if (amdgpu_dm_is_valid_crc_source(cur_crc_src)) {
+ amdgpu_dm_crtc_set_crc_source(crtc, NULL);
+ spin_lock_irq(&adev_to_drm(adev)->event_lock);
+ /* For resume to set back crc source*/
+ acrtc->dm_irq_params.crc_src = cur_crc_src;
+ acrtc->dm_irq_params.crc_window = cur_crc_window;
+ spin_unlock_irq(&adev_to_drm(adev)->event_lock);
+ }
+ }
+
+}
+
+struct crc_rd_work *amdgpu_dm_crtc_secure_display_create_work(void)
+{
+ struct crc_rd_work *crc_rd_wrk = NULL;
+
+ crc_rd_wrk = kzalloc(sizeof(*crc_rd_wrk), GFP_KERNEL);
+
+ if (!crc_rd_wrk)
+ return NULL;
+
+ spin_lock_init(&crc_rd_wrk->crc_rd_work_lock);
+ INIT_WORK(&crc_rd_wrk->notify_ta_work, amdgpu_dm_crtc_notify_ta_to_read);
+
+ return crc_rd_wrk;
+}
+#endif
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h
index f7d731797d3f..737e701fb0f0 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crc.h
@@ -39,6 +39,29 @@ enum amdgpu_dm_pipe_crc_source {
AMDGPU_DM_PIPE_CRC_SOURCE_INVALID = -1,
};
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+struct crc_window_parm {
+ uint16_t x_start;
+ uint16_t y_start;
+ uint16_t x_end;
+ uint16_t y_end;
+ /* CRC windwo is activated or not*/
+ bool activated;
+ /* Update crc window during vertical blank or not */
+ bool update_win;
+ /* skip reading/writing for few frames */
+ int skip_frame_cnt;
+};
+
+struct crc_rd_work {
+ struct work_struct notify_ta_work;
+ /* To protect crc_rd_work carried fields*/
+ spinlock_t crc_rd_work_lock;
+ struct drm_crtc *crtc;
+ uint8_t phy_inst;
+};
+#endif
+
static inline bool amdgpu_dm_is_valid_crc_source(enum amdgpu_dm_pipe_crc_source source)
{
return (source > AMDGPU_DM_PIPE_CRC_SOURCE_NONE) &&
@@ -64,4 +87,18 @@ void amdgpu_dm_crtc_handle_crc_irq(struct drm_crtc *crtc);
#define amdgpu_dm_crtc_handle_crc_irq(x)
#endif
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+bool amdgpu_dm_crc_window_is_activated(struct drm_crtc *crtc);
+void amdgpu_dm_crtc_handle_crc_window_irq(struct drm_crtc *crtc);
+struct crc_rd_work *amdgpu_dm_crtc_secure_display_create_work(void);
+void amdgpu_dm_crtc_secure_display_resume(struct amdgpu_device *adev);
+void amdgpu_dm_crtc_secure_display_suspend(struct amdgpu_device *adev);
+#else
+#define amdgpu_dm_crc_window_is_activated(x)
+#define amdgpu_dm_crtc_handle_crc_window_irq(x)
+#define amdgpu_dm_crtc_secure_display_create_work()
+#define amdgpu_dm_crtc_secure_display_resume(x)
+#define amdgpu_dm_crtc_secure_display_suspend(x)
+#endif
+
#endif /* AMD_DAL_DEV_AMDGPU_DM_AMDGPU_DM_CRC_H_ */
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
index 360952129b6d..1b6b15708b96 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.c
@@ -25,8 +25,6 @@
#include <linux/uaccess.h>
-#include <drm/drm_debugfs.h>
-
#include "dc.h"
#include "amdgpu.h"
#include "amdgpu_dm.h"
@@ -36,6 +34,8 @@
#include "resource.h"
#include "dsc.h"
#include "dc_link_dp.h"
+#include "link_hwss.h"
+#include "dc/dc_dmub_srv.h"
struct dmub_debugfs_trace_header {
uint32_t entry_count;
@@ -150,7 +150,7 @@ static int parse_write_buffer_into_params(char *wr_buf, uint32_t wr_buf_size,
*
* --- to get dp configuration
*
- * cat link_settings
+ * cat /sys/kernel/debug/dri/0/DP-x/link_settings
*
* It will list current, verified, reported, preferred dp configuration.
* current -- for current video mode
@@ -163,7 +163,7 @@ static int parse_write_buffer_into_params(char *wr_buf, uint32_t wr_buf_size,
* echo <lane_count> <link_rate> > link_settings
*
* for example, to force to 2 lane, 2.7GHz,
- * echo 4 0xa > link_settings
+ * echo 4 0xa > /sys/kernel/debug/dri/0/DP-x/link_settings
*
* spread_spectrum could not be changed dynamically.
*
@@ -171,7 +171,7 @@ static int parse_write_buffer_into_params(char *wr_buf, uint32_t wr_buf_size,
* done. please check link settings after force operation to see if HW get
* programming.
*
- * cat link_settings
+ * cat /sys/kernel/debug/dri/0/DP-x/link_settings
*
* check current and preferred settings.
*
@@ -247,7 +247,6 @@ static ssize_t dp_link_settings_write(struct file *f, const char __user *buf,
{
struct amdgpu_dm_connector *connector = file_inode(f)->i_private;
struct dc_link *link = connector->dc_link;
- struct dc *dc = (struct dc *)link->dc;
struct dc_link_settings prefer_link_settings;
char *wr_buf = NULL;
const uint32_t wr_buf_size = 40;
@@ -255,7 +254,7 @@ static ssize_t dp_link_settings_write(struct file *f, const char __user *buf,
int max_param_num = 2;
uint8_t param_nums = 0;
long param[2];
- bool valid_input = false;
+ bool valid_input = true;
if (size == 0)
return -EINVAL;
@@ -282,9 +281,9 @@ static ssize_t dp_link_settings_write(struct file *f, const char __user *buf,
case LANE_COUNT_ONE:
case LANE_COUNT_TWO:
case LANE_COUNT_FOUR:
- valid_input = true;
break;
default:
+ valid_input = false;
break;
}
@@ -294,9 +293,9 @@ static ssize_t dp_link_settings_write(struct file *f, const char __user *buf,
case LINK_RATE_RBR2:
case LINK_RATE_HIGH2:
case LINK_RATE_HIGH3:
- valid_input = true;
break;
default:
+ valid_input = false;
break;
}
@@ -310,10 +309,11 @@ static ssize_t dp_link_settings_write(struct file *f, const char __user *buf,
* spread spectrum will not be changed
*/
prefer_link_settings.link_spread = link->cur_link_settings.link_spread;
+ prefer_link_settings.use_link_rate_set = false;
prefer_link_settings.lane_count = param[0];
prefer_link_settings.link_rate = param[1];
- dc_link_set_preferred_link_settings(dc, &prefer_link_settings, link);
+ dp_retrain_link_dp_test(link, &prefer_link_settings, false);
kfree(wr_buf);
return size;
@@ -400,6 +400,70 @@ static ssize_t dp_phy_settings_read(struct file *f, char __user *buf,
return result;
}
+static int dp_lttpr_status_show(struct seq_file *m, void *d)
+{
+ char *data;
+ struct amdgpu_dm_connector *connector = file_inode(m->file)->i_private;
+ struct dc_link *link = connector->dc_link;
+ uint32_t read_size = 1;
+ uint8_t repeater_count = 0;
+
+ data = kzalloc(read_size, GFP_KERNEL);
+ if (!data)
+ return 0;
+
+ dm_helpers_dp_read_dpcd(link->ctx, link, 0xF0002, data, read_size);
+
+ switch ((uint8_t)*data) {
+ case 0x80:
+ repeater_count = 1;
+ break;
+ case 0x40:
+ repeater_count = 2;
+ break;
+ case 0x20:
+ repeater_count = 3;
+ break;
+ case 0x10:
+ repeater_count = 4;
+ break;
+ case 0x8:
+ repeater_count = 5;
+ break;
+ case 0x4:
+ repeater_count = 6;
+ break;
+ case 0x2:
+ repeater_count = 7;
+ break;
+ case 0x1:
+ repeater_count = 8;
+ break;
+ case 0x0:
+ repeater_count = 0;
+ break;
+ default:
+ repeater_count = (uint8_t)*data;
+ break;
+ }
+
+ seq_printf(m, "phy repeater count: %d\n", repeater_count);
+
+ dm_helpers_dp_read_dpcd(link->ctx, link, 0xF0003, data, read_size);
+
+ if ((uint8_t)*data == 0x55)
+ seq_printf(m, "phy repeater mode: transparent\n");
+ else if ((uint8_t)*data == 0xAA)
+ seq_printf(m, "phy repeater mode: non-transparent\n");
+ else if ((uint8_t)*data == 0x00)
+ seq_printf(m, "phy repeater mode: non lttpr\n");
+ else
+ seq_printf(m, "phy repeater mode: read error\n");
+
+ kfree(data);
+ return 0;
+}
+
static ssize_t dp_phy_settings_write(struct file *f, const char __user *buf,
size_t size, loff_t *pos)
{
@@ -2154,10 +2218,154 @@ static ssize_t dp_dsc_slice_bpg_offset_read(struct file *f, char __user *buf,
return result;
}
+
+/*
+ * function description: Read max_requested_bpc property from the connector
+ *
+ * Access it with the following command:
+ *
+ * cat /sys/kernel/debug/dri/0/DP-X/max_bpc
+ *
+ */
+static ssize_t dp_max_bpc_read(struct file *f, char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_dm_connector *aconnector = file_inode(f)->i_private;
+ struct drm_connector *connector = &aconnector->base;
+ struct drm_device *dev = connector->dev;
+ struct dm_connector_state *state;
+ ssize_t result = 0;
+ char *rd_buf = NULL;
+ char *rd_buf_ptr = NULL;
+ const uint32_t rd_buf_size = 10;
+ int r;
+
+ rd_buf = kcalloc(rd_buf_size, sizeof(char), GFP_KERNEL);
+
+ if (!rd_buf)
+ return -ENOMEM;
+
+ mutex_lock(&dev->mode_config.mutex);
+ drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+
+ if (connector->state == NULL)
+ goto unlock;
+
+ state = to_dm_connector_state(connector->state);
+
+ rd_buf_ptr = rd_buf;
+ snprintf(rd_buf_ptr, rd_buf_size,
+ "%u\n",
+ state->base.max_requested_bpc);
+
+ while (size) {
+ if (*pos >= rd_buf_size)
+ break;
+
+ r = put_user(*(rd_buf + result), buf);
+ if (r) {
+ result = r; /* r = -EFAULT */
+ goto unlock;
+ }
+ buf += 1;
+ size -= 1;
+ *pos += 1;
+ result += 1;
+ }
+unlock:
+ drm_modeset_unlock(&dev->mode_config.connection_mutex);
+ mutex_unlock(&dev->mode_config.mutex);
+ kfree(rd_buf);
+ return result;
+}
+
+
+/*
+ * function description: Set max_requested_bpc property on the connector
+ *
+ * This function will not force the input BPC on connector, it will only
+ * change the max value. This is equivalent to setting max_bpc through
+ * xrandr.
+ *
+ * The BPC value written must be >= 6 and <= 16. Values outside of this
+ * range will result in errors.
+ *
+ * BPC values:
+ * 0x6 - 6 BPC
+ * 0x8 - 8 BPC
+ * 0xa - 10 BPC
+ * 0xc - 12 BPC
+ * 0x10 - 16 BPC
+ *
+ * Write the max_bpc in the following way:
+ *
+ * echo 0x6 > /sys/kernel/debug/dri/0/DP-X/max_bpc
+ *
+ */
+static ssize_t dp_max_bpc_write(struct file *f, const char __user *buf,
+ size_t size, loff_t *pos)
+{
+ struct amdgpu_dm_connector *aconnector = file_inode(f)->i_private;
+ struct drm_connector *connector = &aconnector->base;
+ struct dm_connector_state *state;
+ struct drm_device *dev = connector->dev;
+ char *wr_buf = NULL;
+ uint32_t wr_buf_size = 42;
+ int max_param_num = 1;
+ long param[1] = {0};
+ uint8_t param_nums = 0;
+
+ if (size == 0)
+ return -EINVAL;
+
+ wr_buf = kcalloc(wr_buf_size, sizeof(char), GFP_KERNEL);
+
+ if (!wr_buf) {
+ DRM_DEBUG_DRIVER("no memory to allocate write buffer\n");
+ return -ENOSPC;
+ }
+
+ if (parse_write_buffer_into_params(wr_buf, size,
+ (long *)param, buf,
+ max_param_num,
+ &param_nums)) {
+ kfree(wr_buf);
+ return -EINVAL;
+ }
+
+ if (param_nums <= 0) {
+ DRM_DEBUG_DRIVER("user data not be read\n");
+ kfree(wr_buf);
+ return -EINVAL;
+ }
+
+ if (param[0] < 6 || param[0] > 16) {
+ DRM_DEBUG_DRIVER("bad max_bpc value\n");
+ kfree(wr_buf);
+ return -EINVAL;
+ }
+
+ mutex_lock(&dev->mode_config.mutex);
+ drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
+
+ if (connector->state == NULL)
+ goto unlock;
+
+ state = to_dm_connector_state(connector->state);
+ state->base.max_requested_bpc = param[0];
+unlock:
+ drm_modeset_unlock(&dev->mode_config.connection_mutex);
+ mutex_unlock(&dev->mode_config.mutex);
+
+ kfree(wr_buf);
+ return size;
+}
+
DEFINE_SHOW_ATTRIBUTE(dp_dsc_fec_support);
DEFINE_SHOW_ATTRIBUTE(dmub_fw_state);
DEFINE_SHOW_ATTRIBUTE(dmub_tracebuffer);
DEFINE_SHOW_ATTRIBUTE(output_bpc);
+DEFINE_SHOW_ATTRIBUTE(dp_lttpr_status);
#ifdef CONFIG_DRM_AMD_DC_HDCP
DEFINE_SHOW_ATTRIBUTE(hdcp_sink_capability);
#endif
@@ -2265,12 +2473,20 @@ static const struct file_operations dp_dpcd_data_debugfs_fops = {
.llseek = default_llseek
};
+static const struct file_operations dp_max_bpc_debugfs_fops = {
+ .owner = THIS_MODULE,
+ .read = dp_max_bpc_read,
+ .write = dp_max_bpc_write,
+ .llseek = default_llseek
+};
+
static const struct {
char *name;
const struct file_operations *fops;
} dp_debugfs_entries[] = {
{"link_settings", &dp_link_settings_debugfs_fops},
{"phy_settings", &dp_phy_settings_debugfs_fop},
+ {"lttpr_status", &dp_lttpr_status_fops},
{"test_pattern", &dp_phy_test_pattern_fops},
#ifdef CONFIG_DRM_AMD_DC_HDCP
{"hdcp_sink_capability", &hdcp_sink_capability_fops},
@@ -2287,7 +2503,8 @@ static const struct {
{"dsc_pic_height", &dp_dsc_pic_height_debugfs_fops},
{"dsc_chunk_size", &dp_dsc_chunk_size_debugfs_fops},
{"dsc_slice_bpg", &dp_dsc_slice_bpg_offset_debugfs_fops},
- {"dp_dsc_fec_support", &dp_dsc_fec_support_fops}
+ {"dp_dsc_fec_support", &dp_dsc_fec_support_fops},
+ {"max_bpc", &dp_max_bpc_debugfs_fops}
};
#ifdef CONFIG_DRM_AMD_DC_HDCP
@@ -2341,9 +2558,51 @@ static int psr_get(void *data, u64 *val)
return 0;
}
+/*
+ * Set dmcub trace event IRQ enable or disable.
+ * Usage to enable dmcub trace event IRQ: echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_dmcub_trace_event_en
+ * Usage to disable dmcub trace event IRQ: echo 0 > /sys/kernel/debug/dri/0/amdgpu_dm_dmcub_trace_event_en
+ */
+static int dmcub_trace_event_state_set(void *data, u64 val)
+{
+ struct amdgpu_device *adev = data;
+
+ if (val == 1 || val == 0) {
+ dc_dmub_trace_event_control(adev->dm.dc, val);
+ adev->dm.dmcub_trace_event_en = (bool)val;
+ } else
+ return 0;
+
+ return 0;
+}
+
+/*
+ * The interface doesn't need get function, so it will return the
+ * value of zero
+ * Usage: cat /sys/kernel/debug/dri/0/amdgpu_dm_dmcub_trace_event_en
+ */
+static int dmcub_trace_event_state_get(void *data, u64 *val)
+{
+ struct amdgpu_device *adev = data;
+
+ *val = adev->dm.dmcub_trace_event_en;
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(dmcub_trace_event_state_fops, dmcub_trace_event_state_get,
+ dmcub_trace_event_state_set, "%llu\n");
DEFINE_DEBUGFS_ATTRIBUTE(psr_fops, psr_get, NULL, "%llu\n");
+static const struct {
+ char *name;
+ const struct file_operations *fops;
+} connector_debugfs_entries[] = {
+ {"force_yuv420_output", &force_yuv420_output_fops},
+ {"output_bpc", &output_bpc_fops},
+ {"trigger_hotplug", &trigger_hotplug_debugfs_fops}
+};
+
void connector_debugfs_init(struct amdgpu_dm_connector *connector)
{
int i;
@@ -2360,14 +2619,11 @@ void connector_debugfs_init(struct amdgpu_dm_connector *connector)
if (connector->base.connector_type == DRM_MODE_CONNECTOR_eDP)
debugfs_create_file_unsafe("psr_state", 0444, dir, connector, &psr_fops);
- debugfs_create_file_unsafe("force_yuv420_output", 0644, dir, connector,
- &force_yuv420_output_fops);
-
- debugfs_create_file("output_bpc", 0644, dir, connector,
- &output_bpc_fops);
-
- debugfs_create_file("trigger_hotplug", 0644, dir, connector,
- &trigger_hotplug_debugfs_fops);
+ for (i = 0; i < ARRAY_SIZE(connector_debugfs_entries); i++) {
+ debugfs_create_file(connector_debugfs_entries[i].name,
+ 0644, dir, connector,
+ connector_debugfs_entries[i].fops);
+ }
connector->debugfs_dpcd_address = 0;
connector->debugfs_dpcd_size = 0;
@@ -2383,6 +2639,225 @@ void connector_debugfs_init(struct amdgpu_dm_connector *connector)
#endif
}
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+/*
+ * Set crc window coordinate x start
+ */
+static int crc_win_x_start_set(void *data, u64 val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ acrtc->dm_irq_params.crc_window.x_start = (uint16_t) val;
+ acrtc->dm_irq_params.crc_window.update_win = false;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+/*
+ * Get crc window coordinate x start
+ */
+static int crc_win_x_start_get(void *data, u64 *val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ *val = acrtc->dm_irq_params.crc_window.x_start;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(crc_win_x_start_fops, crc_win_x_start_get,
+ crc_win_x_start_set, "%llu\n");
+
+
+/*
+ * Set crc window coordinate y start
+ */
+static int crc_win_y_start_set(void *data, u64 val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ acrtc->dm_irq_params.crc_window.y_start = (uint16_t) val;
+ acrtc->dm_irq_params.crc_window.update_win = false;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+/*
+ * Get crc window coordinate y start
+ */
+static int crc_win_y_start_get(void *data, u64 *val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ *val = acrtc->dm_irq_params.crc_window.y_start;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(crc_win_y_start_fops, crc_win_y_start_get,
+ crc_win_y_start_set, "%llu\n");
+
+/*
+ * Set crc window coordinate x end
+ */
+static int crc_win_x_end_set(void *data, u64 val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ acrtc->dm_irq_params.crc_window.x_end = (uint16_t) val;
+ acrtc->dm_irq_params.crc_window.update_win = false;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+/*
+ * Get crc window coordinate x end
+ */
+static int crc_win_x_end_get(void *data, u64 *val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ *val = acrtc->dm_irq_params.crc_window.x_end;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(crc_win_x_end_fops, crc_win_x_end_get,
+ crc_win_x_end_set, "%llu\n");
+
+/*
+ * Set crc window coordinate y end
+ */
+static int crc_win_y_end_set(void *data, u64 val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ acrtc->dm_irq_params.crc_window.y_end = (uint16_t) val;
+ acrtc->dm_irq_params.crc_window.update_win = false;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+/*
+ * Get crc window coordinate y end
+ */
+static int crc_win_y_end_get(void *data, u64 *val)
+{
+ struct drm_crtc *crtc = data;
+ struct drm_device *drm_dev = crtc->dev;
+ struct amdgpu_crtc *acrtc = to_amdgpu_crtc(crtc);
+
+ spin_lock_irq(&drm_dev->event_lock);
+ *val = acrtc->dm_irq_params.crc_window.y_end;
+ spin_unlock_irq(&drm_dev->event_lock);
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(crc_win_y_end_fops, crc_win_y_end_get,
+ crc_win_y_end_set, "%llu\n");
+/*
+ * Trigger to commit crc window
+ */
+static int crc_win_update_set(void *data, u64 val)
+{
+ struct drm_crtc *new_crtc = data;
+ struct drm_crtc *old_crtc = NULL;
+ struct amdgpu_crtc *new_acrtc, *old_acrtc;
+ struct amdgpu_device *adev = drm_to_adev(new_crtc->dev);
+ struct crc_rd_work *crc_rd_wrk = adev->dm.crc_rd_wrk;
+
+ if (val) {
+ spin_lock_irq(&adev_to_drm(adev)->event_lock);
+ spin_lock_irq(&crc_rd_wrk->crc_rd_work_lock);
+ if (crc_rd_wrk && crc_rd_wrk->crtc) {
+ old_crtc = crc_rd_wrk->crtc;
+ old_acrtc = to_amdgpu_crtc(old_crtc);
+ }
+ new_acrtc = to_amdgpu_crtc(new_crtc);
+
+ if (old_crtc && old_crtc != new_crtc) {
+ old_acrtc->dm_irq_params.crc_window.activated = false;
+ old_acrtc->dm_irq_params.crc_window.update_win = false;
+ old_acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+
+ new_acrtc->dm_irq_params.crc_window.activated = true;
+ new_acrtc->dm_irq_params.crc_window.update_win = true;
+ new_acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ crc_rd_wrk->crtc = new_crtc;
+ } else {
+ new_acrtc->dm_irq_params.crc_window.activated = true;
+ new_acrtc->dm_irq_params.crc_window.update_win = true;
+ new_acrtc->dm_irq_params.crc_window.skip_frame_cnt = 0;
+ crc_rd_wrk->crtc = new_crtc;
+ }
+ spin_unlock_irq(&crc_rd_wrk->crc_rd_work_lock);
+ spin_unlock_irq(&adev_to_drm(adev)->event_lock);
+ }
+
+ return 0;
+}
+
+/*
+ * Get crc window update flag
+ */
+static int crc_win_update_get(void *data, u64 *val)
+{
+ *val = 0;
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(crc_win_update_fops, crc_win_update_get,
+ crc_win_update_set, "%llu\n");
+
+void crtc_debugfs_init(struct drm_crtc *crtc)
+{
+ struct dentry *dir = debugfs_lookup("crc", crtc->debugfs_entry);
+
+ if (!dir)
+ return;
+
+ debugfs_create_file_unsafe("crc_win_x_start", 0644, dir, crtc,
+ &crc_win_x_start_fops);
+ debugfs_create_file_unsafe("crc_win_y_start", 0644, dir, crtc,
+ &crc_win_y_start_fops);
+ debugfs_create_file_unsafe("crc_win_x_end", 0644, dir, crtc,
+ &crc_win_x_end_fops);
+ debugfs_create_file_unsafe("crc_win_y_end", 0644, dir, crtc,
+ &crc_win_y_end_fops);
+ debugfs_create_file_unsafe("crc_win_update", 0644, dir, crtc,
+ &crc_win_update_fops);
+
+}
+#endif
/*
* Writes DTN log state to the user supplied buffer.
* Example usage: cat /sys/kernel/debug/dri/0/amdgpu_dm_dtn_log
@@ -2450,11 +2925,9 @@ static ssize_t dtn_log_write(
* As written to display, taking ABM and backlight lut into account.
* Ranges from 0x0 to 0x10000 (= 100% PWM)
*/
-static int current_backlight_read(struct seq_file *m, void *data)
+static int current_backlight_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
struct amdgpu_display_manager *dm = &adev->dm;
unsigned int backlight = dc_link_get_backlight_level(dm->backlight_link);
@@ -2468,11 +2941,9 @@ static int current_backlight_read(struct seq_file *m, void *data)
* As written to display, taking ABM and backlight lut into account.
* Ranges from 0x0 to 0x10000 (= 100% PWM)
*/
-static int target_backlight_read(struct seq_file *m, void *data)
+static int target_backlight_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
- struct amdgpu_device *adev = drm_to_adev(dev);
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
struct amdgpu_display_manager *dm = &adev->dm;
unsigned int backlight = dc_link_get_target_backlight_pwm(dm->backlight_link);
@@ -2481,10 +2952,10 @@ static int target_backlight_read(struct seq_file *m, void *data)
return 0;
}
-static int mst_topo(struct seq_file *m, void *unused)
+static int mst_topo_show(struct seq_file *m, void *unused)
{
- struct drm_info_node *node = (struct drm_info_node *)m->private;
- struct drm_device *dev = node->minor->dev;
+ struct amdgpu_device *adev = (struct amdgpu_device *)m->private;
+ struct drm_device *dev = adev_to_drm(adev);
struct drm_connector *connector;
struct drm_connector_list_iter conn_iter;
struct amdgpu_dm_connector *aconnector;
@@ -2496,6 +2967,10 @@ static int mst_topo(struct seq_file *m, void *unused)
aconnector = to_amdgpu_dm_connector(connector);
+ /* Ensure we're only dumping the topology of a root mst node */
+ if (!aconnector->mst_mgr.mst_state)
+ continue;
+
seq_printf(m, "\nMST topology for connector %d\n", aconnector->connector_id);
drm_dp_mst_dump_topology(m, &aconnector->mst_mgr);
}
@@ -2504,14 +2979,74 @@ static int mst_topo(struct seq_file *m, void *unused)
return 0;
}
-static const struct drm_info_list amdgpu_dm_debugfs_list[] = {
- {"amdgpu_current_backlight_pwm", &current_backlight_read},
- {"amdgpu_target_backlight_pwm", &target_backlight_read},
- {"amdgpu_mst_topology", &mst_topo},
-};
+/*
+ * Sets trigger hpd for MST topologies.
+ * All connected connectors will be rediscovered and re started as needed if val of 1 is sent.
+ * All topologies will be disconnected if val of 0 is set .
+ * Usage to enable topologies: echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_trigger_hpd_mst
+ * Usage to disable topologies: echo 0 > /sys/kernel/debug/dri/0/amdgpu_dm_trigger_hpd_mst
+ */
+static int trigger_hpd_mst_set(void *data, u64 val)
+{
+ struct amdgpu_device *adev = data;
+ struct drm_device *dev = adev_to_drm(adev);
+ struct drm_connector_list_iter iter;
+ struct amdgpu_dm_connector *aconnector;
+ struct drm_connector *connector;
+ struct dc_link *link = NULL;
+
+ if (val == 1) {
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter) {
+ aconnector = to_amdgpu_dm_connector(connector);
+ if (aconnector->dc_link->type == dc_connection_mst_branch &&
+ aconnector->mst_mgr.aux) {
+ dc_link_detect(aconnector->dc_link, DETECT_REASON_HPD);
+ drm_dp_mst_topology_mgr_set_mst(&aconnector->mst_mgr, true);
+ }
+ }
+ } else if (val == 0) {
+ drm_connector_list_iter_begin(dev, &iter);
+ drm_for_each_connector_iter(connector, &iter) {
+ aconnector = to_amdgpu_dm_connector(connector);
+ if (!aconnector->dc_link)
+ continue;
+
+ if (!aconnector->mst_port)
+ continue;
+
+ link = aconnector->dc_link;
+ dp_receiver_power_ctrl(link, false);
+ drm_dp_mst_topology_mgr_set_mst(&aconnector->mst_port->mst_mgr, false);
+ link->mst_stream_alloc_table.stream_count = 0;
+ memset(link->mst_stream_alloc_table.stream_allocations, 0,
+ sizeof(link->mst_stream_alloc_table.stream_allocations));
+ }
+ } else {
+ return 0;
+ }
+ drm_kms_helper_hotplug_event(dev);
+
+ return 0;
+}
/*
- * Sets the force_timing_sync debug optino from the given string.
+ * The interface doesn't need get function, so it will return the
+ * value of zero
+ * Usage: cat /sys/kernel/debug/dri/0/amdgpu_dm_trigger_hpd_mst
+ */
+static int trigger_hpd_mst_get(void *data, u64 *val)
+{
+ *val = 0;
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(trigger_hpd_mst_ops, trigger_hpd_mst_get,
+ trigger_hpd_mst_set, "%llu\n");
+
+
+/*
+ * Sets the force_timing_sync debug option from the given string.
* All connected displays will be force synchronized immediately.
* Usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_dm_force_timing_sync
*/
@@ -2542,6 +3077,37 @@ static int force_timing_sync_get(void *data, u64 *val)
DEFINE_DEBUGFS_ATTRIBUTE(force_timing_sync_ops, force_timing_sync_get,
force_timing_sync_set, "%llu\n");
+
+/*
+ * Disables all HPD and HPD RX interrupt handling in the
+ * driver when set to 1. Default is 0.
+ */
+static int disable_hpd_set(void *data, u64 val)
+{
+ struct amdgpu_device *adev = data;
+
+ adev->dm.disable_hpd_irq = (bool)val;
+
+ return 0;
+}
+
+
+/*
+ * Returns 1 if HPD and HPRX interrupt handling is disabled,
+ * 0 otherwise.
+ */
+static int disable_hpd_get(void *data, u64 *val)
+{
+ struct amdgpu_device *adev = data;
+
+ *val = adev->dm.disable_hpd_irq;
+
+ return 0;
+}
+
+DEFINE_DEBUGFS_ATTRIBUTE(disable_hpd_ops, disable_hpd_get,
+ disable_hpd_set, "%llu\n");
+
/*
* Sets the DC visual confirm debug option from the given string.
* Example usage: echo 1 > /sys/kernel/debug/dri/0/amdgpu_visual_confirm
@@ -2568,10 +3134,71 @@ static int visual_confirm_get(void *data, u64 *val)
return 0;
}
+DEFINE_SHOW_ATTRIBUTE(current_backlight);
+DEFINE_SHOW_ATTRIBUTE(target_backlight);
+DEFINE_SHOW_ATTRIBUTE(mst_topo);
DEFINE_DEBUGFS_ATTRIBUTE(visual_confirm_fops, visual_confirm_get,
visual_confirm_set, "%llu\n");
-int dtn_debugfs_init(struct amdgpu_device *adev)
+/*
+ * Dumps the DCC_EN bit for each pipe.
+ * Example usage: cat /sys/kernel/debug/dri/0/amdgpu_dm_dcc_en
+ */
+static ssize_t dcc_en_bits_read(
+ struct file *f,
+ char __user *buf,
+ size_t size,
+ loff_t *pos)
+{
+ struct amdgpu_device *adev = file_inode(f)->i_private;
+ struct dc *dc = adev->dm.dc;
+ char *rd_buf = NULL;
+ const uint32_t rd_buf_size = 32;
+ uint32_t result = 0;
+ int offset = 0;
+ int num_pipes = dc->res_pool->pipe_count;
+ int *dcc_en_bits;
+ int i, r;
+
+ dcc_en_bits = kcalloc(num_pipes, sizeof(int), GFP_KERNEL);
+ if (!dcc_en_bits)
+ return -ENOMEM;
+
+ if (!dc->hwss.get_dcc_en_bits) {
+ kfree(dcc_en_bits);
+ return 0;
+ }
+
+ dc->hwss.get_dcc_en_bits(dc, dcc_en_bits);
+
+ rd_buf = kcalloc(rd_buf_size, sizeof(char), GFP_KERNEL);
+ if (!rd_buf)
+ return -ENOMEM;
+
+ for (i = 0; i < num_pipes; i++)
+ offset += snprintf(rd_buf + offset, rd_buf_size - offset,
+ "%d ", dcc_en_bits[i]);
+ rd_buf[strlen(rd_buf)] = '\n';
+
+ kfree(dcc_en_bits);
+
+ while (size) {
+ if (*pos >= rd_buf_size)
+ break;
+ r = put_user(*(rd_buf + result), buf);
+ if (r)
+ return r; /* r = -EFAULT */
+ buf += 1;
+ size -= 1;
+ *pos += 1;
+ result += 1;
+ }
+
+ kfree(rd_buf);
+ return result;
+}
+
+void dtn_debugfs_init(struct amdgpu_device *adev)
{
static const struct file_operations dtn_log_fops = {
.owner = THIS_MODULE,
@@ -2579,16 +3206,21 @@ int dtn_debugfs_init(struct amdgpu_device *adev)
.write = dtn_log_write,
.llseek = default_llseek
};
+ static const struct file_operations dcc_en_bits_fops = {
+ .owner = THIS_MODULE,
+ .read = dcc_en_bits_read,
+ .llseek = default_llseek
+ };
struct drm_minor *minor = adev_to_drm(adev)->primary;
struct dentry *root = minor->debugfs_root;
- int ret;
-
- ret = amdgpu_debugfs_add_files(adev, amdgpu_dm_debugfs_list,
- ARRAY_SIZE(amdgpu_dm_debugfs_list));
- if (ret)
- return ret;
+ debugfs_create_file("amdgpu_current_backlight_pwm", 0444,
+ root, adev, &current_backlight_fops);
+ debugfs_create_file("amdgpu_target_backlight_pwm", 0444,
+ root, adev, &target_backlight_fops);
+ debugfs_create_file("amdgpu_mst_topology", 0444, root,
+ adev, &mst_topo_fops);
debugfs_create_file("amdgpu_dm_dtn_log", 0644, root, adev,
&dtn_log_fops);
@@ -2604,5 +3236,16 @@ int dtn_debugfs_init(struct amdgpu_device *adev)
debugfs_create_file_unsafe("amdgpu_dm_force_timing_sync", 0644, root,
adev, &force_timing_sync_ops);
- return 0;
+ debugfs_create_file_unsafe("amdgpu_dm_dmcub_trace_event_en", 0644, root,
+ adev, &dmcub_trace_event_state_fops);
+
+ debugfs_create_file_unsafe("amdgpu_dm_trigger_hpd_mst", 0644, root,
+ adev, &trigger_hpd_mst_ops);
+
+ debugfs_create_file_unsafe("amdgpu_dm_dcc_en", 0644, root, adev,
+ &dcc_en_bits_fops);
+
+ debugfs_create_file_unsafe("amdgpu_dm_disable_hpd", 0644, root, adev,
+ &disable_hpd_ops);
+
}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.h
index 5e5b2b2afa31..3366cb644053 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_debugfs.h
@@ -30,6 +30,9 @@
#include "amdgpu_dm.h"
void connector_debugfs_init(struct amdgpu_dm_connector *connector);
-int dtn_debugfs_init(struct amdgpu_device *adev);
+void dtn_debugfs_init(struct amdgpu_device *adev);
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+void crtc_debugfs_init(struct drm_crtc *crtc);
+#endif
#endif
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
index 0cdbfcd475ec..666796a0067c 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_hdcp.c
@@ -191,7 +191,7 @@ void hdcp_update_display(struct hdcp_workqueue *hdcp_work,
psp_set_srm(hdcp_work->hdcp.config.psp.handle, hdcp_work->srm, hdcp_work->srm_size,
&hdcp_work->srm_version);
- display->adjust.disable = 0;
+ display->adjust.disable = MOD_HDCP_DISPLAY_NOT_DISABLE;
if (content_type == DRM_MODE_HDCP_CONTENT_TYPE0) {
hdcp_w->link.adjust.hdcp1.disable = 0;
hdcp_w->link.adjust.hdcp2.force_type = MOD_HDCP_FORCE_TYPE_0;
@@ -203,7 +203,7 @@ void hdcp_update_display(struct hdcp_workqueue *hdcp_work,
schedule_delayed_work(&hdcp_w->property_validate_dwork,
msecs_to_jiffies(DRM_HDCP_CHECK_PERIOD_MS));
} else {
- display->adjust.disable = 1;
+ display->adjust.disable = MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION;
hdcp_w->encryption_status = MOD_HDCP_ENCRYPTION_STATUS_HDCP_OFF;
cancel_delayed_work(&hdcp_w->property_validate_dwork);
}
@@ -434,6 +434,7 @@ static void update_config(void *handle, struct cp_psp_stream_config *config)
int link_index = aconnector->dc_link->link_index;
struct mod_hdcp_display *display = &hdcp_work[link_index].display;
struct mod_hdcp_link *link = &hdcp_work[link_index].link;
+ struct drm_connector_state *conn_state;
if (config->dpms_off) {
hdcp_remove_display(hdcp_work, link_index, aconnector);
@@ -456,11 +457,16 @@ static void update_config(void *handle, struct cp_psp_stream_config *config)
link->dp.rev = aconnector->dc_link->dpcd_caps.dpcd_rev.raw;
link->dp.assr_enabled = config->assr_enabled;
link->dp.mst_enabled = config->mst_enabled;
- display->adjust.disable = 1;
+ display->adjust.disable = MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION;
link->adjust.auth_delay = 3;
link->adjust.hdcp1.disable = 0;
+ conn_state = aconnector->base.state;
- hdcp_update_display(hdcp_work, link_index, aconnector, DRM_MODE_HDCP_CONTENT_TYPE0, false);
+ pr_debug("[HDCP_DM] display %d, CP %d, type %d\n", aconnector->base.index,
+ (!!aconnector->base.state) ? aconnector->base.state->content_protection : -1,
+ (!!aconnector->base.state) ? aconnector->base.state->hdcp_content_type : -1);
+
+ hdcp_update_display(hdcp_work, link_index, aconnector, conn_state->hdcp_content_type, false);
}
@@ -644,6 +650,7 @@ struct hdcp_workqueue *hdcp_create_workqueue(struct amdgpu_device *adev, struct
/* File created at /sys/class/drm/card0/device/hdcp_srm*/
hdcp_work[0].attr = data_attr;
+ sysfs_bin_attr_init(&hdcp_work[0].attr);
if (sysfs_create_bin_file(&adev->dev->kobj, &hdcp_work[0].attr))
DRM_WARN("Failed to create device file hdcp_srm");
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
index 5750818db8f6..e8b325a828c1 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_helpers.c
@@ -652,8 +652,31 @@ void *dm_helpers_allocate_gpu_mem(
size_t size,
long long *addr)
{
- // TODO
- return NULL;
+ struct amdgpu_device *adev = ctx->driver_context;
+ struct dal_allocation *da;
+ u32 domain = (type == DC_MEM_ALLOC_TYPE_GART) ?
+ AMDGPU_GEM_DOMAIN_GTT : AMDGPU_GEM_DOMAIN_VRAM;
+ int ret;
+
+ da = kzalloc(sizeof(struct dal_allocation), GFP_KERNEL);
+ if (!da)
+ return NULL;
+
+ ret = amdgpu_bo_create_kernel(adev, size, PAGE_SIZE,
+ domain, &da->bo,
+ &da->gpu_addr, &da->cpu_ptr);
+
+ *addr = da->gpu_addr;
+
+ if (ret) {
+ kfree(da);
+ return NULL;
+ }
+
+ /* add da to list in dm */
+ list_add(&da->list, &adev->dm.da_list);
+
+ return da->cpu_ptr;
}
void dm_helpers_free_gpu_mem(
@@ -661,5 +684,55 @@ void dm_helpers_free_gpu_mem(
enum dc_gpu_mem_alloc_type type,
void *pvMem)
{
- // TODO
+ struct amdgpu_device *adev = ctx->driver_context;
+ struct dal_allocation *da;
+
+ /* walk the da list in DM */
+ list_for_each_entry(da, &adev->dm.da_list, list) {
+ if (pvMem == da->cpu_ptr) {
+ amdgpu_bo_free_kernel(&da->bo, &da->gpu_addr, &da->cpu_ptr);
+ list_del(&da->list);
+ kfree(da);
+ break;
+ }
+ }
+}
+
+bool dm_helpers_dmub_outbox0_interrupt_control(struct dc_context *ctx, bool enable)
+{
+ enum dc_irq_source irq_source;
+ bool ret;
+
+ irq_source = DC_IRQ_SOURCE_DMCUB_OUTBOX0;
+
+ ret = dc_interrupt_set(ctx->dc, irq_source, enable);
+
+ DRM_DEBUG_DRIVER("Dmub trace irq %sabling: r=%d\n",
+ enable ? "en" : "dis", ret);
+ return ret;
+}
+
+void dm_helpers_mst_enable_stream_features(const struct dc_stream_state *stream)
+{
+ /* TODO: virtual DPCD */
+ struct dc_link *link = stream->link;
+ union down_spread_ctrl old_downspread;
+ union down_spread_ctrl new_downspread;
+
+ if (link->aux_access_disabled)
+ return;
+
+ if (!dm_helpers_dp_read_dpcd(link->ctx, link, DP_DOWNSPREAD_CTRL,
+ &old_downspread.raw,
+ sizeof(old_downspread)))
+ return;
+
+ new_downspread.raw = old_downspread.raw;
+ new_downspread.bits.IGNORE_MSA_TIMING_PARAM =
+ (stream->ignore_msa_timing_param) ? 1 : 0;
+
+ if (new_downspread.raw != old_downspread.raw)
+ dm_helpers_dp_write_dpcd(link->ctx, link, DP_DOWNSPREAD_CTRL,
+ &new_downspread.raw,
+ sizeof(new_downspread));
}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
index e0000c180ed1..b3ed7e777720 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq.c
@@ -73,6 +73,7 @@
* @handler_arg: Argument passed to the handler when triggered
* @dm: DM which this handler belongs to
* @irq_source: DC interrupt source that this handler is registered for
+ * @work: work struct
*/
struct amdgpu_dm_irq_handler_data {
struct list_head list;
@@ -82,6 +83,7 @@ struct amdgpu_dm_irq_handler_data {
struct amdgpu_display_manager *dm;
/* DAL irq source which registered for this interrupt. */
enum dc_irq_source irq_source;
+ struct work_struct work;
};
#define DM_IRQ_TABLE_LOCK(adev, flags) \
@@ -111,20 +113,10 @@ static void init_handler_common_data(struct amdgpu_dm_irq_handler_data *hcd,
*/
static void dm_irq_work_func(struct work_struct *work)
{
- struct irq_list_head *irq_list_head =
- container_of(work, struct irq_list_head, work);
- struct list_head *handler_list = &irq_list_head->head;
- struct amdgpu_dm_irq_handler_data *handler_data;
-
- list_for_each_entry(handler_data, handler_list, list) {
- DRM_DEBUG_KMS("DM_IRQ: work_func: for dal_src=%d\n",
- handler_data->irq_source);
+ struct amdgpu_dm_irq_handler_data *handler_data =
+ container_of(work, struct amdgpu_dm_irq_handler_data, work);
- DRM_DEBUG_KMS("DM_IRQ: schedule_work: for dal_src=%d\n",
- handler_data->irq_source);
-
- handler_data->handler(handler_data->handler_arg);
- }
+ handler_data->handler(handler_data->handler_arg);
/* Call a DAL subcomponent which registered for interrupt notification
* at INTERRUPT_LOW_IRQ_CONTEXT.
@@ -156,7 +148,7 @@ static struct list_head *remove_irq_handler(struct amdgpu_device *adev,
break;
case INTERRUPT_LOW_IRQ_CONTEXT:
default:
- hnd_list = &adev->dm.irq_handler_list_low_tab[irq_source].head;
+ hnd_list = &adev->dm.irq_handler_list_low_tab[irq_source];
break;
}
@@ -193,6 +185,55 @@ static struct list_head *remove_irq_handler(struct amdgpu_device *adev,
return hnd_list;
}
+/**
+ * unregister_all_irq_handlers() - Cleans up handlers from the DM IRQ table
+ * @adev: The base driver device containing the DM device
+ *
+ * Go through low and high context IRQ tables and deallocate handlers.
+ */
+static void unregister_all_irq_handlers(struct amdgpu_device *adev)
+{
+ struct list_head *hnd_list_low;
+ struct list_head *hnd_list_high;
+ struct list_head *entry, *tmp;
+ struct amdgpu_dm_irq_handler_data *handler;
+ unsigned long irq_table_flags;
+ int i;
+
+ DM_IRQ_TABLE_LOCK(adev, irq_table_flags);
+
+ for (i = 0; i < DAL_IRQ_SOURCES_NUMBER; i++) {
+ hnd_list_low = &adev->dm.irq_handler_list_low_tab[i];
+ hnd_list_high = &adev->dm.irq_handler_list_high_tab[i];
+
+ list_for_each_safe(entry, tmp, hnd_list_low) {
+
+ handler = list_entry(entry, struct amdgpu_dm_irq_handler_data,
+ list);
+
+ if (handler == NULL || handler->handler == NULL)
+ continue;
+
+ list_del(&handler->list);
+ kfree(handler);
+ }
+
+ list_for_each_safe(entry, tmp, hnd_list_high) {
+
+ handler = list_entry(entry, struct amdgpu_dm_irq_handler_data,
+ list);
+
+ if (handler == NULL || handler->handler == NULL)
+ continue;
+
+ list_del(&handler->list);
+ kfree(handler);
+ }
+ }
+
+ DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
+}
+
static bool
validate_irq_registration_params(struct dc_interrupt_params *int_params,
void (*ih)(void *))
@@ -290,7 +331,8 @@ void *amdgpu_dm_irq_register_interrupt(struct amdgpu_device *adev,
break;
case INTERRUPT_LOW_IRQ_CONTEXT:
default:
- hnd_list = &adev->dm.irq_handler_list_low_tab[irq_source].head;
+ hnd_list = &adev->dm.irq_handler_list_low_tab[irq_source];
+ INIT_WORK(&handler_data->work, dm_irq_work_func);
break;
}
@@ -372,7 +414,7 @@ void amdgpu_dm_irq_unregister_interrupt(struct amdgpu_device *adev,
int amdgpu_dm_irq_init(struct amdgpu_device *adev)
{
int src;
- struct irq_list_head *lh;
+ struct list_head *lh;
DRM_DEBUG_KMS("DM_IRQ\n");
@@ -381,9 +423,7 @@ int amdgpu_dm_irq_init(struct amdgpu_device *adev)
for (src = 0; src < DAL_IRQ_SOURCES_NUMBER; src++) {
/* low context handler list init */
lh = &adev->dm.irq_handler_list_low_tab[src];
- INIT_LIST_HEAD(&lh->head);
- INIT_WORK(&lh->work, dm_irq_work_func);
-
+ INIT_LIST_HEAD(lh);
/* high context handler init */
INIT_LIST_HEAD(&adev->dm.irq_handler_list_high_tab[src]);
}
@@ -400,8 +440,11 @@ int amdgpu_dm_irq_init(struct amdgpu_device *adev)
void amdgpu_dm_irq_fini(struct amdgpu_device *adev)
{
int src;
- struct irq_list_head *lh;
+ struct list_head *lh;
+ struct list_head *entry, *tmp;
+ struct amdgpu_dm_irq_handler_data *handler;
unsigned long irq_table_flags;
+
DRM_DEBUG_KMS("DM_IRQ: releasing resources.\n");
for (src = 0; src < DAL_IRQ_SOURCES_NUMBER; src++) {
DM_IRQ_TABLE_LOCK(adev, irq_table_flags);
@@ -410,8 +453,19 @@ void amdgpu_dm_irq_fini(struct amdgpu_device *adev)
* (because no code can schedule a new one). */
lh = &adev->dm.irq_handler_list_low_tab[src];
DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
- flush_work(&lh->work);
+
+ if (!list_empty(lh)) {
+ list_for_each_safe(entry, tmp, lh) {
+ handler = list_entry(
+ entry,
+ struct amdgpu_dm_irq_handler_data,
+ list);
+ flush_work(&handler->work);
+ }
+ }
}
+ /* Deallocate handlers from the table. */
+ unregister_all_irq_handlers(adev);
}
int amdgpu_dm_irq_suspend(struct amdgpu_device *adev)
@@ -420,6 +474,8 @@ int amdgpu_dm_irq_suspend(struct amdgpu_device *adev)
struct list_head *hnd_list_h;
struct list_head *hnd_list_l;
unsigned long irq_table_flags;
+ struct list_head *entry, *tmp;
+ struct amdgpu_dm_irq_handler_data *handler;
DM_IRQ_TABLE_LOCK(adev, irq_table_flags);
@@ -430,14 +486,22 @@ int amdgpu_dm_irq_suspend(struct amdgpu_device *adev)
* will be disabled from manage_dm_interrupts on disable CRTC.
*/
for (src = DC_IRQ_SOURCE_HPD1; src <= DC_IRQ_SOURCE_HPD6RX; src++) {
- hnd_list_l = &adev->dm.irq_handler_list_low_tab[src].head;
+ hnd_list_l = &adev->dm.irq_handler_list_low_tab[src];
hnd_list_h = &adev->dm.irq_handler_list_high_tab[src];
if (!list_empty(hnd_list_l) || !list_empty(hnd_list_h))
dc_interrupt_set(adev->dm.dc, src, false);
DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
- flush_work(&adev->dm.irq_handler_list_low_tab[src].work);
+ if (!list_empty(hnd_list_l)) {
+ list_for_each_safe (entry, tmp, hnd_list_l) {
+ handler = list_entry(
+ entry,
+ struct amdgpu_dm_irq_handler_data,
+ list);
+ flush_work(&handler->work);
+ }
+ }
DM_IRQ_TABLE_LOCK(adev, irq_table_flags);
}
@@ -457,7 +521,7 @@ int amdgpu_dm_irq_resume_early(struct amdgpu_device *adev)
/* re-enable short pulse interrupts HW interrupt */
for (src = DC_IRQ_SOURCE_HPD1RX; src <= DC_IRQ_SOURCE_HPD6RX; src++) {
- hnd_list_l = &adev->dm.irq_handler_list_low_tab[src].head;
+ hnd_list_l = &adev->dm.irq_handler_list_low_tab[src];
hnd_list_h = &adev->dm.irq_handler_list_high_tab[src];
if (!list_empty(hnd_list_l) || !list_empty(hnd_list_h))
dc_interrupt_set(adev->dm.dc, src, true);
@@ -483,7 +547,7 @@ int amdgpu_dm_irq_resume_late(struct amdgpu_device *adev)
* will be enabled from manage_dm_interrupts on enable CRTC.
*/
for (src = DC_IRQ_SOURCE_HPD1; src <= DC_IRQ_SOURCE_HPD6; src++) {
- hnd_list_l = &adev->dm.irq_handler_list_low_tab[src].head;
+ hnd_list_l = &adev->dm.irq_handler_list_low_tab[src];
hnd_list_h = &adev->dm.irq_handler_list_high_tab[src];
if (!list_empty(hnd_list_l) || !list_empty(hnd_list_h))
dc_interrupt_set(adev->dm.dc, src, true);
@@ -500,22 +564,51 @@ int amdgpu_dm_irq_resume_late(struct amdgpu_device *adev)
static void amdgpu_dm_irq_schedule_work(struct amdgpu_device *adev,
enum dc_irq_source irq_source)
{
- unsigned long irq_table_flags;
- struct work_struct *work = NULL;
+ struct list_head *handler_list = &adev->dm.irq_handler_list_low_tab[irq_source];
+ struct amdgpu_dm_irq_handler_data *handler_data;
+ bool work_queued = false;
- DM_IRQ_TABLE_LOCK(adev, irq_table_flags);
+ if (list_empty(handler_list))
+ return;
- if (!list_empty(&adev->dm.irq_handler_list_low_tab[irq_source].head))
- work = &adev->dm.irq_handler_list_low_tab[irq_source].work;
+ list_for_each_entry (handler_data, handler_list, list) {
+ if (queue_work(system_highpri_wq, &handler_data->work)) {
+ work_queued = true;
+ break;
+ }
+ }
- DM_IRQ_TABLE_UNLOCK(adev, irq_table_flags);
+ if (!work_queued) {
+ struct amdgpu_dm_irq_handler_data *handler_data_add;
+ /*get the amdgpu_dm_irq_handler_data of first item pointed by handler_list*/
+ handler_data = container_of(handler_list->next, struct amdgpu_dm_irq_handler_data, list);
- if (work) {
- if (!schedule_work(work))
- DRM_INFO("amdgpu_dm_irq_schedule_work FAILED src %d\n",
- irq_source);
- }
+ /*allocate a new amdgpu_dm_irq_handler_data*/
+ handler_data_add = kzalloc(sizeof(*handler_data), GFP_KERNEL);
+ if (!handler_data_add) {
+ DRM_ERROR("DM_IRQ: failed to allocate irq handler!\n");
+ return;
+ }
+ /*copy new amdgpu_dm_irq_handler_data members from handler_data*/
+ handler_data_add->handler = handler_data->handler;
+ handler_data_add->handler_arg = handler_data->handler_arg;
+ handler_data_add->dm = handler_data->dm;
+ handler_data_add->irq_source = irq_source;
+
+ list_add_tail(&handler_data_add->list, handler_list);
+
+ INIT_WORK(&handler_data_add->work, dm_irq_work_func);
+
+ if (queue_work(system_highpri_wq, &handler_data_add->work))
+ DRM_DEBUG("Queued work for handling interrupt from "
+ "display for IRQ source %d\n",
+ irq_source);
+ else
+ DRM_ERROR("Failed to queue work for handling interrupt "
+ "from display for IRQ source %d\n",
+ irq_source);
+ }
}
/*
@@ -690,6 +783,18 @@ static int amdgpu_dm_set_vupdate_irq_state(struct amdgpu_device *adev,
__func__);
}
+static int amdgpu_dm_set_dmub_trace_irq_state(struct amdgpu_device *adev,
+ struct amdgpu_irq_src *source,
+ unsigned int type,
+ enum amdgpu_interrupt_state state)
+{
+ enum dc_irq_source irq_source = DC_IRQ_SOURCE_DMCUB_OUTBOX0;
+ bool st = (state == AMDGPU_IRQ_STATE_ENABLE);
+
+ dc_interrupt_set(adev->dm.dc, irq_source, st);
+ return 0;
+}
+
static const struct amdgpu_irq_src_funcs dm_crtc_irq_funcs = {
.set = amdgpu_dm_set_crtc_irq_state,
.process = amdgpu_dm_irq_handler,
@@ -705,6 +810,11 @@ static const struct amdgpu_irq_src_funcs dm_vupdate_irq_funcs = {
.process = amdgpu_dm_irq_handler,
};
+static const struct amdgpu_irq_src_funcs dm_dmub_trace_irq_funcs = {
+ .set = amdgpu_dm_set_dmub_trace_irq_state,
+ .process = amdgpu_dm_irq_handler,
+};
+
static const struct amdgpu_irq_src_funcs dm_pageflip_irq_funcs = {
.set = amdgpu_dm_set_pflip_irq_state,
.process = amdgpu_dm_irq_handler,
@@ -727,6 +837,9 @@ void amdgpu_dm_set_irq_funcs(struct amdgpu_device *adev)
adev->vupdate_irq.num_types = adev->mode_info.num_crtc;
adev->vupdate_irq.funcs = &dm_vupdate_irq_funcs;
+ adev->dmub_trace_irq.num_types = 1;
+ adev->dmub_trace_irq.funcs = &dm_dmub_trace_irq_funcs;
+
adev->pageflip_irq.num_types = adev->mode_info.num_crtc;
adev->pageflip_irq.funcs = &dm_pageflip_irq_funcs;
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h
index 45825a34f8eb..f3b93ba69a27 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_irq_params.h
@@ -26,12 +26,21 @@
#ifndef __AMDGPU_DM_IRQ_PARAMS_H__
#define __AMDGPU_DM_IRQ_PARAMS_H__
+#include "amdgpu_dm_crc.h"
+
struct dm_irq_params {
u32 last_flip_vblank;
struct mod_vrr_params vrr_params;
struct dc_stream_state *stream;
int active_planes;
struct mod_freesync_config freesync_config;
+
+#ifdef CONFIG_DEBUG_FS
+ enum amdgpu_dm_pipe_crc_source crc_src;
+#ifdef CONFIG_DRM_AMD_SECURE_DISPLAY
+ struct crc_window_parm crc_window;
+#endif
+#endif
};
#endif /* __AMDGPU_DM_IRQ_PARAMS_H__ */
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
index 41b09ab22233..9b221db526dc 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
@@ -38,6 +38,7 @@
#include "dc_link_ddc.h"
#include "i2caux_interface.h"
+#include "dmub_cmd.h"
#if defined(CONFIG_DEBUG_FS)
#include "amdgpu_dm_debugfs.h"
#endif
@@ -51,7 +52,7 @@ static ssize_t dm_dp_aux_transfer(struct drm_dp_aux *aux,
{
ssize_t result = 0;
struct aux_payload payload;
- enum aux_channel_operation_result operation_result;
+ enum aux_return_code_type operation_result;
if (WARN_ON(msg->size > 16))
return -E2BIG;
@@ -73,17 +74,19 @@ static ssize_t dm_dp_aux_transfer(struct drm_dp_aux *aux,
if (result < 0)
switch (operation_result) {
- case AUX_CHANNEL_OPERATION_SUCCEEDED:
+ case AUX_RET_SUCCESS:
break;
- case AUX_CHANNEL_OPERATION_FAILED_HPD_DISCON:
- case AUX_CHANNEL_OPERATION_FAILED_REASON_UNKNOWN:
+ case AUX_RET_ERROR_HPD_DISCON:
+ case AUX_RET_ERROR_UNKNOWN:
+ case AUX_RET_ERROR_INVALID_OPERATION:
+ case AUX_RET_ERROR_PROTOCOL_ERROR:
result = -EIO;
break;
- case AUX_CHANNEL_OPERATION_FAILED_INVALID_REPLY:
- case AUX_CHANNEL_OPERATION_FAILED_ENGINE_ACQUIRE:
+ case AUX_RET_ERROR_INVALID_REPLY:
+ case AUX_RET_ERROR_ENGINE_ACQUIRE:
result = -EBUSY;
break;
- case AUX_CHANNEL_OPERATION_FAILED_TIMEOUT:
+ case AUX_RET_ERROR_TIMEOUT:
result = -ETIMEDOUT;
break;
}
@@ -226,6 +229,11 @@ static int dm_dp_mst_get_modes(struct drm_connector *connector)
(aconnector->edid->extensions + 1) * EDID_LENGTH,
&init_params);
+ if (!dc_sink) {
+ DRM_ERROR("Unable to add a remote sink\n");
+ return 0;
+ }
+
dc_sink->priv = aconnector;
/* dc_link_add_remote_sink returns a new reference */
aconnector->dc_sink = dc_sink;
@@ -742,8 +750,8 @@ static bool compute_mst_dsc_configs_for_link(struct drm_atomic_state *state,
if (!dc_dsc_compute_bandwidth_range(
stream->sink->ctx->dc->res_pool->dscs[0],
stream->sink->ctx->dc->debug.dsc_min_slice_height_override,
- dsc_policy.min_target_bpp,
- dsc_policy.max_target_bpp,
+ dsc_policy.min_target_bpp * 16,
+ dsc_policy.max_target_bpp * 16,
&stream->sink->dsc_caps.dsc_dec_caps,
&stream->timing, &params[count].bw_range))
params[count].bw_range.stream_kbps = dc_bandwidth_in_kbps_from_timing(&stream->timing);
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c
index 607ec0999445..eba270121698 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_pp_smu.c
@@ -32,15 +32,12 @@
#include "amdgpu_dm_irq.h"
#include "amdgpu_pm.h"
#include "dm_pp_smu.h"
-#include "amdgpu_smu.h"
-
bool dm_pp_apply_display_requirements(
const struct dc_context *ctx,
const struct dm_pp_display_configuration *pp_display_cfg)
{
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
int i;
if (adev->pm.dpm_enabled) {
@@ -106,9 +103,6 @@ bool dm_pp_apply_display_requirements(
adev->powerplay.pp_funcs->display_configuration_change(
adev->powerplay.pp_handle,
&adev->pm.pm_display_cfg);
- else if (adev->smu.ppt_funcs)
- smu_display_configuration_change(smu,
- &adev->pm.pm_display_cfg);
amdgpu_pm_compute_clocks(adev);
}
@@ -148,36 +142,6 @@ static void get_default_clock_levels(
}
}
-static enum smu_clk_type dc_to_smu_clock_type(
- enum dm_pp_clock_type dm_pp_clk_type)
-{
- enum smu_clk_type smu_clk_type = SMU_CLK_COUNT;
-
- switch (dm_pp_clk_type) {
- case DM_PP_CLOCK_TYPE_DISPLAY_CLK:
- smu_clk_type = SMU_DISPCLK;
- break;
- case DM_PP_CLOCK_TYPE_ENGINE_CLK:
- smu_clk_type = SMU_GFXCLK;
- break;
- case DM_PP_CLOCK_TYPE_MEMORY_CLK:
- smu_clk_type = SMU_MCLK;
- break;
- case DM_PP_CLOCK_TYPE_DCEFCLK:
- smu_clk_type = SMU_DCEFCLK;
- break;
- case DM_PP_CLOCK_TYPE_SOCCLK:
- smu_clk_type = SMU_SOCCLK;
- break;
- default:
- DRM_ERROR("DM_PPLIB: invalid clock type: %d!\n",
- dm_pp_clk_type);
- break;
- }
-
- return smu_clk_type;
-}
-
static enum amd_pp_clock_type dc_to_pp_clock_type(
enum dm_pp_clock_type dm_pp_clk_type)
{
@@ -417,14 +381,8 @@ bool dm_pp_get_clock_levels_by_type_with_latency(
&pp_clks);
if (ret)
return false;
- } else if (adev->smu.ppt_funcs && adev->smu.ppt_funcs->get_clock_by_type_with_latency) {
- if (smu_get_clock_by_type_with_latency(&adev->smu,
- dc_to_smu_clock_type(clk_type),
- &pp_clks))
- return false;
}
-
pp_to_dc_clock_levels_with_latency(&pp_clks, clk_level_info, clk_type);
return true;
@@ -502,10 +460,6 @@ bool dm_pp_apply_clock_for_voltage_request(
ret = adev->powerplay.pp_funcs->display_clock_voltage_request(
adev->powerplay.pp_handle,
&pp_clock_request);
- else if (adev->smu.ppt_funcs &&
- adev->smu.ppt_funcs->display_clock_voltage_request)
- ret = smu_display_clock_voltage_request(&adev->smu,
- &pp_clock_request);
if (ret)
return false;
return true;
@@ -655,8 +609,11 @@ static enum pp_smu_status pp_nv_set_wm_ranges(struct pp_smu *pp,
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- smu_set_watermarks_for_clock_ranges(&adev->smu, ranges);
+ if (pp_funcs && pp_funcs->set_watermarks_for_clocks_ranges)
+ pp_funcs->set_watermarks_for_clocks_ranges(pp_handle, ranges);
return PP_SMU_RESULT_OK;
}
@@ -665,13 +622,14 @@ static enum pp_smu_status pp_nv_set_display_count(struct pp_smu *pp, int count)
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->set_active_display_count)
return PP_SMU_RESULT_UNSUPPORTED;
/* 0: successful or smu.ppt_funcs->set_display_count = NULL; 1: fail */
- if (smu_set_display_count(smu, count))
+ if (pp_funcs->set_active_display_count(pp_handle, count))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
@@ -682,13 +640,14 @@ pp_nv_set_min_deep_sleep_dcfclk(struct pp_smu *pp, int mhz)
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->set_min_deep_sleep_dcefclk)
return PP_SMU_RESULT_UNSUPPORTED;
/* 0: successful or smu.ppt_funcs->set_deep_sleep_dcefclk = NULL;1: fail */
- if (smu_set_deep_sleep_dcefclk(smu, mhz))
+ if (pp_funcs->set_min_deep_sleep_dcefclk(pp_handle, mhz))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
@@ -699,10 +658,11 @@ static enum pp_smu_status pp_nv_set_hard_min_dcefclk_by_freq(
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
struct pp_display_clock_request clock_req;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->display_clock_voltage_request)
return PP_SMU_RESULT_UNSUPPORTED;
clock_req.clock_type = amd_pp_dcef_clock;
@@ -711,7 +671,7 @@ static enum pp_smu_status pp_nv_set_hard_min_dcefclk_by_freq(
/* 0: successful or smu.ppt_funcs->display_clock_voltage_request = NULL
* 1: fail
*/
- if (smu_display_clock_voltage_request(smu, &clock_req))
+ if (pp_funcs->display_clock_voltage_request(pp_handle, &clock_req))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
@@ -722,10 +682,11 @@ pp_nv_set_hard_min_uclk_by_freq(struct pp_smu *pp, int mhz)
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
struct pp_display_clock_request clock_req;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->display_clock_voltage_request)
return PP_SMU_RESULT_UNSUPPORTED;
clock_req.clock_type = amd_pp_mem_clock;
@@ -734,7 +695,7 @@ pp_nv_set_hard_min_uclk_by_freq(struct pp_smu *pp, int mhz)
/* 0: successful or smu.ppt_funcs->display_clock_voltage_request = NULL
* 1: fail
*/
- if (smu_display_clock_voltage_request(smu, &clock_req))
+ if (pp_funcs->display_clock_voltage_request(pp_handle, &clock_req))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
@@ -745,10 +706,14 @@ static enum pp_smu_status pp_nv_set_pstate_handshake_support(
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (smu_display_disable_memory_clock_switch(smu, !pstate_handshake_supported))
- return PP_SMU_RESULT_FAIL;
+ if (pp_funcs && pp_funcs->display_disable_memory_clock_switch) {
+ if (pp_funcs->display_disable_memory_clock_switch(pp_handle,
+ !pstate_handshake_supported))
+ return PP_SMU_RESULT_FAIL;
+ }
return PP_SMU_RESULT_OK;
}
@@ -758,10 +723,11 @@ static enum pp_smu_status pp_nv_set_voltage_by_freq(struct pp_smu *pp,
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
struct pp_display_clock_request clock_req;
- if (!smu->ppt_funcs)
+ if (!pp_funcs || !pp_funcs->display_clock_voltage_request)
return PP_SMU_RESULT_UNSUPPORTED;
switch (clock_id) {
@@ -782,7 +748,7 @@ static enum pp_smu_status pp_nv_set_voltage_by_freq(struct pp_smu *pp,
/* 0: successful or smu.ppt_funcs->display_clock_voltage_request = NULL
* 1: fail
*/
- if (smu_display_clock_voltage_request(smu, &clock_req))
+ if (pp_funcs->display_clock_voltage_request(pp_handle, &clock_req))
return PP_SMU_RESULT_FAIL;
return PP_SMU_RESULT_OK;
@@ -793,15 +759,13 @@ static enum pp_smu_status pp_nv_get_maximum_sustainable_clocks(
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
-
- if (!smu->ppt_funcs)
- return PP_SMU_RESULT_UNSUPPORTED;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs->get_max_sustainable_clocks_by_dc)
+ if (!pp_funcs || !pp_funcs->get_max_sustainable_clocks_by_dc)
return PP_SMU_RESULT_UNSUPPORTED;
- if (!smu_get_max_sustainable_clocks_by_dc(smu, max_clocks))
+ if (!pp_funcs->get_max_sustainable_clocks_by_dc(pp_handle, max_clocks))
return PP_SMU_RESULT_OK;
return PP_SMU_RESULT_FAIL;
@@ -812,16 +776,15 @@ static enum pp_smu_status pp_nv_get_uclk_dpm_states(struct pp_smu *pp,
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
-
- if (!smu->ppt_funcs)
- return PP_SMU_RESULT_UNSUPPORTED;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs->get_uclk_dpm_states)
+ if (!pp_funcs || !pp_funcs->get_uclk_dpm_states)
return PP_SMU_RESULT_UNSUPPORTED;
- if (!smu_get_uclk_dpm_states(smu,
- clock_values_in_khz, num_states))
+ if (!pp_funcs->get_uclk_dpm_states(pp_handle,
+ clock_values_in_khz,
+ num_states))
return PP_SMU_RESULT_OK;
return PP_SMU_RESULT_FAIL;
@@ -832,15 +795,13 @@ static enum pp_smu_status pp_rn_get_dpm_clock_table(
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
- struct smu_context *smu = &adev->smu;
-
- if (!smu->ppt_funcs)
- return PP_SMU_RESULT_UNSUPPORTED;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- if (!smu->ppt_funcs->get_dpm_clock_table)
+ if (!pp_funcs || !pp_funcs->get_dpm_clock_table)
return PP_SMU_RESULT_UNSUPPORTED;
- if (!smu_get_dpm_clock_table(smu, clock_table))
+ if (!pp_funcs->get_dpm_clock_table(pp_handle, clock_table))
return PP_SMU_RESULT_OK;
return PP_SMU_RESULT_FAIL;
@@ -851,8 +812,11 @@ static enum pp_smu_status pp_rn_set_wm_ranges(struct pp_smu *pp,
{
const struct dc_context *ctx = pp->dm;
struct amdgpu_device *adev = ctx->driver_context;
+ void *pp_handle = adev->powerplay.pp_handle;
+ const struct amd_pm_funcs *pp_funcs = adev->powerplay.pp_funcs;
- smu_set_watermarks_for_clock_ranges(&adev->smu, ranges);
+ if (pp_funcs && pp_funcs->set_watermarks_for_clocks_ranges)
+ pp_funcs->set_watermarks_for_clocks_ranges(pp_handle, ranges);
return PP_SMU_RESULT_OK;
}
diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h
index 86960476823c..46a33f64cf8e 100644
--- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h
+++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_trace.h
@@ -597,6 +597,46 @@ TRACE_EVENT(amdgpu_dm_dce_clocks_state,
)
);
+TRACE_EVENT(amdgpu_dmub_trace_high_irq,
+ TP_PROTO(uint32_t trace_code, uint32_t tick_count, uint32_t param0,
+ uint32_t param1),
+ TP_ARGS(trace_code, tick_count, param0, param1),
+ TP_STRUCT__entry(
+ __field(uint32_t, trace_code)
+ __field(uint32_t, tick_count)
+ __field(uint32_t, param0)
+ __field(uint32_t, param1)
+ ),
+ TP_fast_assign(
+ __entry->trace_code = trace_code;
+ __entry->tick_count = tick_count;
+ __entry->param0 = param0;
+ __entry->param1 = param1;
+ ),
+ TP_printk("trace_code=%u tick_count=%u param0=%u param1=%u",
+ __entry->trace_code, __entry->tick_count,
+ __entry->param0, __entry->param1)
+);
+
+TRACE_EVENT(amdgpu_refresh_rate_track,
+ TP_PROTO(int crtc_index, ktime_t refresh_rate_ns, uint32_t refresh_rate_hz),
+ TP_ARGS(crtc_index, refresh_rate_ns, refresh_rate_hz),
+ TP_STRUCT__entry(
+ __field(int, crtc_index)
+ __field(ktime_t, refresh_rate_ns)
+ __field(uint32_t, refresh_rate_hz)
+ ),
+ TP_fast_assign(
+ __entry->crtc_index = crtc_index;
+ __entry->refresh_rate_ns = refresh_rate_ns;
+ __entry->refresh_rate_hz = refresh_rate_hz;
+ ),
+ TP_printk("crtc_index=%d refresh_rate=%dHz (%lld)",
+ __entry->crtc_index,
+ __entry->refresh_rate_hz,
+ __entry->refresh_rate_ns)
+);
+
#endif /* _AMDGPU_DM_TRACE_H_ */
#undef TRACE_INCLUDE_PATH
diff --git a/drivers/gpu/drm/amd/display/dc/Makefile b/drivers/gpu/drm/amd/display/dc/Makefile
index 5bf2f2375b40..f33847299bca 100644
--- a/drivers/gpu/drm/amd/display/dc/Makefile
+++ b/drivers/gpu/drm/amd/display/dc/Makefile
@@ -55,7 +55,8 @@ AMD_DC = $(addsuffix /Makefile, $(addprefix $(FULL_AMD_DISPLAY_PATH)/dc/,$(DC_LI
include $(AMD_DC)
DISPLAY_CORE = dc.o dc_link.o dc_resource.o dc_hw_sequencer.o dc_sink.o \
-dc_surface.o dc_link_hwss.o dc_link_dp.o dc_link_ddc.o dc_debug.o dc_stream.o
+dc_surface.o dc_link_hwss.o dc_link_dp.o dc_link_ddc.o dc_debug.o dc_stream.o \
+dc_link_enc_cfg.o
ifdef CONFIG_DRM_AMD_DC_DCN
DISPLAY_CORE += dc_vm_helper.o
diff --git a/drivers/gpu/drm/amd/display/dc/basics/dc_common.c b/drivers/gpu/drm/amd/display/dc/basics/dc_common.c
index ad04ef98e652..b2fc4f8e6482 100644
--- a/drivers/gpu/drm/amd/display/dc/basics/dc_common.c
+++ b/drivers/gpu/drm/amd/display/dc/basics/dc_common.c
@@ -49,24 +49,20 @@ bool is_rgb_cspace(enum dc_color_space output_color_space)
}
}
-bool is_child_pipe_tree_visible(struct pipe_ctx *pipe_ctx)
+bool is_lower_pipe_tree_visible(struct pipe_ctx *pipe_ctx)
{
if (pipe_ctx->plane_state && pipe_ctx->plane_state->visible)
return true;
- if (pipe_ctx->bottom_pipe && is_child_pipe_tree_visible(pipe_ctx->bottom_pipe))
- return true;
- if (pipe_ctx->next_odm_pipe && is_child_pipe_tree_visible(pipe_ctx->next_odm_pipe))
+ if (pipe_ctx->bottom_pipe && is_lower_pipe_tree_visible(pipe_ctx->bottom_pipe))
return true;
return false;
}
-bool is_parent_pipe_tree_visible(struct pipe_ctx *pipe_ctx)
+bool is_upper_pipe_tree_visible(struct pipe_ctx *pipe_ctx)
{
if (pipe_ctx->plane_state && pipe_ctx->plane_state->visible)
return true;
- if (pipe_ctx->top_pipe && is_parent_pipe_tree_visible(pipe_ctx->top_pipe))
- return true;
- if (pipe_ctx->prev_odm_pipe && is_parent_pipe_tree_visible(pipe_ctx->prev_odm_pipe))
+ if (pipe_ctx->top_pipe && is_upper_pipe_tree_visible(pipe_ctx->top_pipe))
return true;
return false;
}
@@ -75,13 +71,9 @@ bool is_pipe_tree_visible(struct pipe_ctx *pipe_ctx)
{
if (pipe_ctx->plane_state && pipe_ctx->plane_state->visible)
return true;
- if (pipe_ctx->top_pipe && is_parent_pipe_tree_visible(pipe_ctx->top_pipe))
- return true;
- if (pipe_ctx->bottom_pipe && is_child_pipe_tree_visible(pipe_ctx->bottom_pipe))
- return true;
- if (pipe_ctx->prev_odm_pipe && is_parent_pipe_tree_visible(pipe_ctx->prev_odm_pipe))
+ if (pipe_ctx->top_pipe && is_upper_pipe_tree_visible(pipe_ctx->top_pipe))
return true;
- if (pipe_ctx->next_odm_pipe && is_child_pipe_tree_visible(pipe_ctx->next_odm_pipe))
+ if (pipe_ctx->bottom_pipe && is_lower_pipe_tree_visible(pipe_ctx->bottom_pipe))
return true;
return false;
}
diff --git a/drivers/gpu/drm/amd/display/dc/basics/dc_common.h b/drivers/gpu/drm/amd/display/dc/basics/dc_common.h
index b061497480b8..7c0cbf47e8ce 100644
--- a/drivers/gpu/drm/amd/display/dc/basics/dc_common.h
+++ b/drivers/gpu/drm/amd/display/dc/basics/dc_common.h
@@ -30,9 +30,9 @@
bool is_rgb_cspace(enum dc_color_space output_color_space);
-bool is_child_pipe_tree_visible(struct pipe_ctx *pipe_ctx);
+bool is_lower_pipe_tree_visible(struct pipe_ctx *pipe_ctx);
-bool is_parent_pipe_tree_visible(struct pipe_ctx *pipe_ctx);
+bool is_upper_pipe_tree_visible(struct pipe_ctx *pipe_ctx);
bool is_pipe_tree_visible(struct pipe_ctx *pipe_ctx);
diff --git a/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c b/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
index 9f9fda3118d1..d79f4fe06c47 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/bios_parser2.c
@@ -916,6 +916,192 @@ static enum bp_result bios_parser_get_soc_bb_info(
return result;
}
+static enum bp_result get_disp_caps_v4_1(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_1 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_1,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result get_disp_caps_v4_2(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_2 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_2,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result get_disp_caps_v4_3(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_3 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_3,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result get_disp_caps_v4_4(
+ struct bios_parser *bp,
+ uint8_t *dce_caps)
+{
+ enum bp_result result = BP_RESULT_OK;
+ struct atom_display_controller_info_v4_4 *disp_cntl_tbl = NULL;
+
+ if (!dce_caps)
+ return BP_RESULT_BADINPUT;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_BADBIOSTABLE;
+
+ disp_cntl_tbl = GET_IMAGE(struct atom_display_controller_info_v4_4,
+ DATA_TABLES(dce_info));
+
+ if (!disp_cntl_tbl)
+ return BP_RESULT_BADBIOSTABLE;
+
+ *dce_caps = disp_cntl_tbl->display_caps;
+
+ return result;
+}
+
+static enum bp_result bios_parser_get_lttpr_interop(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps)
+{
+ struct bios_parser *bp = BP_FROM_DCB(dcb);
+ enum bp_result result = BP_RESULT_UNSUPPORTED;
+ struct atom_common_table_header *header;
+ struct atom_data_revision tbl_revision;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_UNSUPPORTED;
+
+ header = GET_IMAGE(struct atom_common_table_header,
+ DATA_TABLES(dce_info));
+ get_atom_data_table_revision(header, &tbl_revision);
+ switch (tbl_revision.major) {
+ case 4:
+ switch (tbl_revision.minor) {
+ case 1:
+ result = get_disp_caps_v4_1(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ case 2:
+ result = get_disp_caps_v4_2(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ case 3:
+ result = get_disp_caps_v4_3(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ case 4:
+ result = get_disp_caps_v4_4(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_VBIOS_LTTPR_TRANSPARENT_ENABLE);
+ break;
+ default:
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+
+ return result;
+}
+
+static enum bp_result bios_parser_get_lttpr_caps(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps)
+{
+ struct bios_parser *bp = BP_FROM_DCB(dcb);
+ enum bp_result result = BP_RESULT_UNSUPPORTED;
+ struct atom_common_table_header *header;
+ struct atom_data_revision tbl_revision;
+
+ if (!DATA_TABLES(dce_info))
+ return BP_RESULT_UNSUPPORTED;
+
+ header = GET_IMAGE(struct atom_common_table_header,
+ DATA_TABLES(dce_info));
+ get_atom_data_table_revision(header, &tbl_revision);
+ switch (tbl_revision.major) {
+ case 4:
+ switch (tbl_revision.minor) {
+ case 1:
+ result = get_disp_caps_v4_1(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ case 2:
+ result = get_disp_caps_v4_2(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ case 3:
+ result = get_disp_caps_v4_3(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ case 4:
+ result = get_disp_caps_v4_4(bp, dce_caps);
+ *dce_caps = !!(*dce_caps & DCE_INFO_CAPS_LTTPR_SUPPORT_ENABLE);
+ break;
+ default:
+ break;
+ }
+ break;
+ default:
+ break;
+ }
+
+ return result;
+}
+
static enum bp_result get_embedded_panel_info_v2_1(
struct bios_parser *bp,
struct embedded_panel_info *info)
@@ -1180,14 +1366,15 @@ static enum bp_result bios_parser_enable_disp_power_gating(
static enum bp_result bios_parser_enable_lvtma_control(
struct dc_bios *dcb,
- uint8_t uc_pwr_on)
+ uint8_t uc_pwr_on,
+ uint8_t panel_instance)
{
struct bios_parser *bp = BP_FROM_DCB(dcb);
if (!bp->cmd_tbl.enable_lvtma_control)
return BP_RESULT_FAILURE;
- return bp->cmd_tbl.enable_lvtma_control(bp, uc_pwr_on);
+ return bp->cmd_tbl.enable_lvtma_control(bp, uc_pwr_on, panel_instance);
}
static bool bios_parser_is_accelerated_mode(
@@ -2530,6 +2717,10 @@ static const struct dc_vbios_funcs vbios_funcs = {
.get_soc_bb_info = bios_parser_get_soc_bb_info,
.get_disp_connector_caps_info = bios_parser_get_disp_connector_caps_info,
+
+ .get_lttpr_caps = bios_parser_get_lttpr_caps,
+
+ .get_lttpr_interop = bios_parser_get_lttpr_interop,
};
static bool bios_parser2_construct(
diff --git a/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.c b/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.c
index fce46ab54c54..53d7513b5083 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.c
@@ -64,9 +64,10 @@ bool bios_is_accelerated_mode(
void bios_set_scratch_acc_mode_change(
- struct dc_bios *bios)
+ struct dc_bios *bios,
+ uint32_t state)
{
- REG_UPDATE(BIOS_SCRATCH_6, S6_ACC_MODE, 1);
+ REG_UPDATE(BIOS_SCRATCH_6, S6_ACC_MODE, state);
}
diff --git a/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.h b/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.h
index 75a29e68fb27..e1b4a40a353d 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.h
+++ b/drivers/gpu/drm/amd/display/dc/bios/bios_parser_helper.h
@@ -32,7 +32,7 @@ uint8_t *bios_get_image(struct dc_bios *bp, uint32_t offset,
uint32_t size);
bool bios_is_accelerated_mode(struct dc_bios *bios);
-void bios_set_scratch_acc_mode_change(struct dc_bios *bios);
+void bios_set_scratch_acc_mode_change(struct dc_bios *bios, uint32_t state);
void bios_set_scratch_critical_state(struct dc_bios *bios, bool state);
uint32_t bios_get_vga_enabled_displays(struct dc_bios *bios);
diff --git a/drivers/gpu/drm/amd/display/dc/bios/command_table.c b/drivers/gpu/drm/amd/display/dc/bios/command_table.c
index afc10b954ffa..ad13e4e36d77 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/command_table.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/command_table.c
@@ -1531,6 +1531,27 @@ static enum bp_result adjust_display_pll_v2(
params.ucEncodeMode =
(uint8_t)bp->cmd_helper->encoder_mode_bp_to_atom(
bp_params->signal_type, false);
+
+ if (EXEC_BIOS_CMD_TABLE(AdjustDisplayPll, params)) {
+ /* Convert output pixel clock back 10KHz-->KHz: multiply
+ * original pixel clock in KHz by ratio
+ * [output pxlClk/input pxlClk] */
+ uint64_t pixel_clk_10_khz_out =
+ (uint64_t)le16_to_cpu(params.usPixelClock);
+ uint64_t pixel_clk = (uint64_t)bp_params->pixel_clock;
+
+ if (pixel_clock_10KHz_in != 0) {
+ bp_params->adjusted_pixel_clock =
+ div_u64(pixel_clk * pixel_clk_10_khz_out,
+ pixel_clock_10KHz_in);
+ } else {
+ bp_params->adjusted_pixel_clock = 0;
+ BREAK_TO_DEBUGGER();
+ }
+
+ result = BP_RESULT_OK;
+ }
+
return result;
}
diff --git a/drivers/gpu/drm/amd/display/dc/bios/command_table2.c b/drivers/gpu/drm/amd/display/dc/bios/command_table2.c
index 25bdf1c38e0a..f1f672a997d7 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/command_table2.c
+++ b/drivers/gpu/drm/amd/display/dc/bios/command_table2.c
@@ -218,6 +218,10 @@ static enum bp_result transmitter_control_v1_6(
struct bios_parser *bp,
struct bp_transmitter_control *cntl);
+static enum bp_result transmitter_control_v1_7(
+ struct bios_parser *bp,
+ struct bp_transmitter_control *cntl);
+
static enum bp_result transmitter_control_fallback(
struct bios_parser *bp,
struct bp_transmitter_control *cntl);
@@ -233,6 +237,9 @@ static void init_transmitter_control(struct bios_parser *bp)
case 6:
bp->cmd_tbl.transmitter_control = transmitter_control_v1_6;
break;
+ case 7:
+ bp->cmd_tbl.transmitter_control = transmitter_control_v1_7;
+ break;
default:
dm_output_to_console("Don't have transmitter_control for v%d\n", crev);
bp->cmd_tbl.transmitter_control = transmitter_control_fallback;
@@ -304,13 +311,76 @@ static enum bp_result transmitter_control_v1_6(
return result;
}
+static void transmitter_control_dmcub_v1_7(
+ struct dc_dmub_srv *dmcub,
+ struct dmub_dig_transmitter_control_data_v1_7 *dig)
+{
+ union dmub_rb_cmd cmd;
+
+ memset(&cmd, 0, sizeof(cmd));
+
+ cmd.dig1_transmitter_control.header.type = DMUB_CMD__VBIOS;
+ cmd.dig1_transmitter_control.header.sub_type =
+ DMUB_CMD__VBIOS_DIG1_TRANSMITTER_CONTROL;
+ cmd.dig1_transmitter_control.header.payload_bytes =
+ sizeof(cmd.dig1_transmitter_control) -
+ sizeof(cmd.dig1_transmitter_control.header);
+ cmd.dig1_transmitter_control.transmitter_control.dig_v1_7 = *dig;
+
+ dc_dmub_srv_cmd_queue(dmcub, &cmd);
+ dc_dmub_srv_cmd_execute(dmcub);
+ dc_dmub_srv_wait_idle(dmcub);
+}
+
+static enum bp_result transmitter_control_v1_7(
+ struct bios_parser *bp,
+ struct bp_transmitter_control *cntl)
+{
+ enum bp_result result = BP_RESULT_FAILURE;
+ const struct command_table_helper *cmd = bp->cmd_helper;
+ struct dmub_dig_transmitter_control_data_v1_7 dig_v1_7 = {0};
+
+ dig_v1_7.phyid = cmd->phy_id_to_atom(cntl->transmitter);
+ dig_v1_7.action = (uint8_t)cntl->action;
+
+ if (cntl->action == TRANSMITTER_CONTROL_SET_VOLTAGE_AND_PREEMPASIS)
+ dig_v1_7.mode_laneset.dplaneset = (uint8_t)cntl->lane_settings;
+ else
+ dig_v1_7.mode_laneset.digmode =
+ cmd->signal_type_to_atom_dig_mode(cntl->signal);
+
+ dig_v1_7.lanenum = (uint8_t)cntl->lanes_number;
+ dig_v1_7.hpdsel = cmd->hpd_sel_to_atom(cntl->hpd_sel);
+ dig_v1_7.digfe_sel = cmd->dig_encoder_sel_to_atom(cntl->engine_id);
+ dig_v1_7.connobj_id = (uint8_t)cntl->connector_obj_id.id;
+ dig_v1_7.symclk_units.symclk_10khz = cntl->pixel_clock/10;
+
+ if (cntl->action == TRANSMITTER_CONTROL_ENABLE ||
+ cntl->action == TRANSMITTER_CONTROL_ACTIAVATE ||
+ cntl->action == TRANSMITTER_CONTROL_DEACTIVATE) {
+ DC_LOG_BIOS("%s:dig_v1_7.symclk_units.symclk_10khz = %d\n",
+ __func__, dig_v1_7.symclk_units.symclk_10khz);
+ }
+
+ if (bp->base.ctx->dc->ctx->dmub_srv &&
+ bp->base.ctx->dc->debug.dmub_command_table) {
+ transmitter_control_dmcub_v1_7(bp->base.ctx->dmub_srv, &dig_v1_7);
+ return BP_RESULT_OK;
+ }
+
+/*color_depth not used any more, driver has deep color factor in the Phyclk*/
+ if (EXEC_BIOS_CMD_TABLE(dig1transmittercontrol, dig_v1_7))
+ result = BP_RESULT_OK;
+ return result;
+}
+
static enum bp_result transmitter_control_fallback(
struct bios_parser *bp,
struct bp_transmitter_control *cntl)
{
if (bp->base.ctx->dc->ctx->dmub_srv &&
bp->base.ctx->dc->debug.dmub_command_table) {
- return transmitter_control_v1_6(bp, cntl);
+ return transmitter_control_v1_7(bp, cntl);
}
return BP_RESULT_FAILURE;
@@ -911,7 +981,8 @@ static unsigned int get_smu_clock_info_v3_1(struct bios_parser *bp, uint8_t id)
static enum bp_result enable_lvtma_control(
struct bios_parser *bp,
- uint8_t uc_pwr_on);
+ uint8_t uc_pwr_on,
+ uint8_t panel_instance);
static void init_enable_lvtma_control(struct bios_parser *bp)
{
@@ -922,19 +993,21 @@ static void init_enable_lvtma_control(struct bios_parser *bp)
static void enable_lvtma_control_dmcub(
struct dc_dmub_srv *dmcub,
- uint8_t uc_pwr_on)
+ uint8_t uc_pwr_on,
+ uint8_t panel_instance)
{
union dmub_rb_cmd cmd;
memset(&cmd, 0, sizeof(cmd));
- cmd.cmd_common.header.type = DMUB_CMD__VBIOS;
- cmd.cmd_common.header.sub_type =
+ cmd.lvtma_control.header.type = DMUB_CMD__VBIOS;
+ cmd.lvtma_control.header.sub_type =
DMUB_CMD__VBIOS_LVTMA_CONTROL;
- cmd.cmd_common.cmd_buffer[0] =
+ cmd.lvtma_control.data.uc_pwr_action =
uc_pwr_on;
-
+ cmd.lvtma_control.data.panel_inst =
+ panel_instance;
dc_dmub_srv_cmd_queue(dmcub, &cmd);
dc_dmub_srv_cmd_execute(dmcub);
dc_dmub_srv_wait_idle(dmcub);
@@ -943,14 +1016,16 @@ static void enable_lvtma_control_dmcub(
static enum bp_result enable_lvtma_control(
struct bios_parser *bp,
- uint8_t uc_pwr_on)
+ uint8_t uc_pwr_on,
+ uint8_t panel_instance)
{
enum bp_result result = BP_RESULT_FAILURE;
if (bp->base.ctx->dc->ctx->dmub_srv &&
bp->base.ctx->dc->debug.dmub_command_table) {
enable_lvtma_control_dmcub(bp->base.ctx->dmub_srv,
- uc_pwr_on);
+ uc_pwr_on,
+ panel_instance);
return BP_RESULT_OK;
}
return result;
diff --git a/drivers/gpu/drm/amd/display/dc/bios/command_table2.h b/drivers/gpu/drm/amd/display/dc/bios/command_table2.h
index 7bdce013cde5..be060b4b87db 100644
--- a/drivers/gpu/drm/amd/display/dc/bios/command_table2.h
+++ b/drivers/gpu/drm/amd/display/dc/bios/command_table2.h
@@ -95,7 +95,8 @@ struct cmd_tbl {
unsigned int (*get_smu_clock_info)(
struct bios_parser *bp, uint8_t id);
enum bp_result (*enable_lvtma_control)(struct bios_parser *bp,
- uint8_t uc_pwr_on);
+ uint8_t uc_pwr_on,
+ uint8_t panel_instance);
};
void dal_firmware_parser_init_cmd_tbl(struct bios_parser *bp);
diff --git a/drivers/gpu/drm/amd/display/dc/calcs/dce_calcs.c b/drivers/gpu/drm/amd/display/dc/calcs/dce_calcs.c
index e633f8a51edb..1244fcb0f446 100644
--- a/drivers/gpu/drm/amd/display/dc/calcs/dce_calcs.c
+++ b/drivers/gpu/drm/amd/display/dc/calcs/dce_calcs.c
@@ -98,16 +98,16 @@ static void calculate_bandwidth(
int32_t num_cursor_lines;
int32_t i, j, k;
- struct bw_fixed yclk[3];
- struct bw_fixed sclk[8];
+ struct bw_fixed *yclk;
+ struct bw_fixed *sclk;
bool d0_underlay_enable;
bool d1_underlay_enable;
bool fbc_enabled;
bool lpt_enabled;
enum bw_defines sclk_message;
enum bw_defines yclk_message;
- enum bw_defines tiling_mode[maximum_number_of_surfaces];
- enum bw_defines surface_type[maximum_number_of_surfaces];
+ enum bw_defines *tiling_mode;
+ enum bw_defines *surface_type;
enum bw_defines voltage;
enum bw_defines pipe_check;
enum bw_defines hsr_check;
@@ -122,6 +122,22 @@ static void calculate_bandwidth(
int32_t number_of_displays_enabled_with_margin = 0;
int32_t number_of_aligned_displays_with_no_margin = 0;
+ yclk = kcalloc(3, sizeof(*yclk), GFP_KERNEL);
+ if (!yclk)
+ return;
+
+ sclk = kcalloc(8, sizeof(*sclk), GFP_KERNEL);
+ if (!sclk)
+ goto free_yclk;
+
+ tiling_mode = kcalloc(maximum_number_of_surfaces, sizeof(*tiling_mode), GFP_KERNEL);
+ if (!tiling_mode)
+ goto free_sclk;
+
+ surface_type = kcalloc(maximum_number_of_surfaces, sizeof(*surface_type), GFP_KERNEL);
+ if (!surface_type)
+ goto free_tiling_mode;
+
yclk[low] = vbios->low_yclk;
yclk[mid] = vbios->mid_yclk;
yclk[high] = vbios->high_yclk;
@@ -2013,6 +2029,14 @@ static void calculate_bandwidth(
}
}
}
+
+ kfree(surface_type);
+free_tiling_mode:
+ kfree(tiling_mode);
+free_yclk:
+ kfree(yclk);
+free_sclk:
+ kfree(sclk);
}
/*******************************************************************************
@@ -2022,707 +2046,719 @@ void bw_calcs_init(struct bw_calcs_dceip *bw_dceip,
struct bw_calcs_vbios *bw_vbios,
struct hw_asic_id asic_id)
{
- struct bw_calcs_dceip dceip = { 0 };
- struct bw_calcs_vbios vbios = { 0 };
+ struct bw_calcs_dceip *dceip;
+ struct bw_calcs_vbios *vbios;
enum bw_calcs_version version = bw_calcs_version_from_asic_id(asic_id);
- dceip.version = version;
+ dceip = kzalloc(sizeof(*dceip), GFP_KERNEL);
+ if (!dceip)
+ return;
+
+ vbios = kzalloc(sizeof(*vbios), GFP_KERNEL);
+ if (!vbios) {
+ kfree(dceip);
+ return;
+ }
+
+ dceip->version = version;
switch (version) {
case BW_CALCS_VERSION_CARRIZO:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 64;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(1600);
- vbios.mid_yclk = bw_int_to_fixed(1600);
- vbios.low_yclk = bw_frc_to_fixed(66666, 100);
- vbios.low_sclk = bw_int_to_fixed(200);
- vbios.mid1_sclk = bw_int_to_fixed(300);
- vbios.mid2_sclk = bw_int_to_fixed(300);
- vbios.mid3_sclk = bw_int_to_fixed(300);
- vbios.mid4_sclk = bw_int_to_fixed(300);
- vbios.mid5_sclk = bw_int_to_fixed(300);
- vbios.mid6_sclk = bw_int_to_fixed(300);
- vbios.high_sclk = bw_frc_to_fixed(62609, 100);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(352);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(467);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(643);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(50);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
- vbios.stutter_self_refresh_exit_latency = bw_frc_to_fixed(153, 10);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_frc_to_fixed(19649, 1000);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 3;
- dceip.number_of_underlay_pipes = 1;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = false;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 64;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(1600);
+ vbios->mid_yclk = bw_int_to_fixed(1600);
+ vbios->low_yclk = bw_frc_to_fixed(66666, 100);
+ vbios->low_sclk = bw_int_to_fixed(200);
+ vbios->mid1_sclk = bw_int_to_fixed(300);
+ vbios->mid2_sclk = bw_int_to_fixed(300);
+ vbios->mid3_sclk = bw_int_to_fixed(300);
+ vbios->mid4_sclk = bw_int_to_fixed(300);
+ vbios->mid5_sclk = bw_int_to_fixed(300);
+ vbios->mid6_sclk = bw_int_to_fixed(300);
+ vbios->high_sclk = bw_frc_to_fixed(62609, 100);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(352);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(467);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(643);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(50);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->stutter_self_refresh_exit_latency = bw_frc_to_fixed(153, 10);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_frc_to_fixed(19649, 1000);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 3;
+ dceip->number_of_underlay_pipes = 1;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = false;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 2;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(82176);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = false;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 2;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(82176);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = false;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(0);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0); /* todo: this is a bug*/
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0); /* todo: this is a bug*/
break;
case BW_CALCS_VERSION_POLARIS10:
/* TODO: Treat VEGAM the same as P10 for now
* Need to tune the para for VEGAM if needed */
case BW_CALCS_VERSION_VEGAM:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 32;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(6000);
- vbios.mid_yclk = bw_int_to_fixed(3200);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(300);
- vbios.mid1_sclk = bw_int_to_fixed(400);
- vbios.mid2_sclk = bw_int_to_fixed(500);
- vbios.mid3_sclk = bw_int_to_fixed(600);
- vbios.mid4_sclk = bw_int_to_fixed(700);
- vbios.mid5_sclk = bw_int_to_fixed(800);
- vbios.mid6_sclk = bw_int_to_fixed(974);
- vbios.high_sclk = bw_int_to_fixed(1154);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(459);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(654);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1108);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_int_to_fixed(45);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 6;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 32;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(6000);
+ vbios->mid_yclk = bw_int_to_fixed(3200);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(300);
+ vbios->mid1_sclk = bw_int_to_fixed(400);
+ vbios->mid2_sclk = bw_int_to_fixed(500);
+ vbios->mid3_sclk = bw_int_to_fixed(600);
+ vbios->mid4_sclk = bw_int_to_fixed(700);
+ vbios->mid5_sclk = bw_int_to_fixed(800);
+ vbios->mid6_sclk = bw_int_to_fixed(974);
+ vbios->high_sclk = bw_int_to_fixed(1154);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(459);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(654);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1108);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(45);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 6;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_POLARIS11:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 32;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(6000);
- vbios.mid_yclk = bw_int_to_fixed(3200);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(300);
- vbios.mid1_sclk = bw_int_to_fixed(400);
- vbios.mid2_sclk = bw_int_to_fixed(500);
- vbios.mid3_sclk = bw_int_to_fixed(600);
- vbios.mid4_sclk = bw_int_to_fixed(700);
- vbios.mid5_sclk = bw_int_to_fixed(800);
- vbios.mid6_sclk = bw_int_to_fixed(974);
- vbios.high_sclk = bw_int_to_fixed(1154);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(459);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(654);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1108);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- if (vbios.number_of_dram_channels == 2) // 64-bit
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 32;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(6000);
+ vbios->mid_yclk = bw_int_to_fixed(3200);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(300);
+ vbios->mid1_sclk = bw_int_to_fixed(400);
+ vbios->mid2_sclk = bw_int_to_fixed(500);
+ vbios->mid3_sclk = bw_int_to_fixed(600);
+ vbios->mid4_sclk = bw_int_to_fixed(700);
+ vbios->mid5_sclk = bw_int_to_fixed(800);
+ vbios->mid6_sclk = bw_int_to_fixed(974);
+ vbios->high_sclk = bw_int_to_fixed(1154);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(459);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(654);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1108);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ if (vbios->number_of_dram_channels == 2) // 64-bit
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
else
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_int_to_fixed(45);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 5;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(45);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 5;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_POLARIS12:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 32;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(6000);
- vbios.mid_yclk = bw_int_to_fixed(3200);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(678);
- vbios.mid1_sclk = bw_int_to_fixed(864);
- vbios.mid2_sclk = bw_int_to_fixed(900);
- vbios.mid3_sclk = bw_int_to_fixed(920);
- vbios.mid4_sclk = bw_int_to_fixed(940);
- vbios.mid5_sclk = bw_int_to_fixed(960);
- vbios.mid6_sclk = bw_int_to_fixed(980);
- vbios.high_sclk = bw_int_to_fixed(1049);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(459);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(654);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1108);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- if (vbios.number_of_dram_channels == 2) // 64-bit
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 32;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(6000);
+ vbios->mid_yclk = bw_int_to_fixed(3200);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(678);
+ vbios->mid1_sclk = bw_int_to_fixed(864);
+ vbios->mid2_sclk = bw_int_to_fixed(900);
+ vbios->mid3_sclk = bw_int_to_fixed(920);
+ vbios->mid4_sclk = bw_int_to_fixed(940);
+ vbios->mid5_sclk = bw_int_to_fixed(960);
+ vbios->mid6_sclk = bw_int_to_fixed(980);
+ vbios->high_sclk = bw_int_to_fixed(1049);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(459);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(654);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1108);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ if (vbios->number_of_dram_channels == 2) // 64-bit
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
else
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_int_to_fixed(250);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = false;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 5;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = true;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_int_to_fixed(5);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(250);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = false;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 5;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = true;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_STONEY:
- vbios.memory_type = bw_def_gddr5;
- vbios.dram_channel_width_in_bits = 64;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 8;
- vbios.high_yclk = bw_int_to_fixed(1866);
- vbios.mid_yclk = bw_int_to_fixed(1866);
- vbios.low_yclk = bw_int_to_fixed(1333);
- vbios.low_sclk = bw_int_to_fixed(200);
- vbios.mid1_sclk = bw_int_to_fixed(600);
- vbios.mid2_sclk = bw_int_to_fixed(600);
- vbios.mid3_sclk = bw_int_to_fixed(600);
- vbios.mid4_sclk = bw_int_to_fixed(600);
- vbios.mid5_sclk = bw_int_to_fixed(600);
- vbios.mid6_sclk = bw_int_to_fixed(600);
- vbios.high_sclk = bw_int_to_fixed(800);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(352);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(467);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(643);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(50);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(4);
- vbios.stutter_self_refresh_exit_latency = bw_frc_to_fixed(158, 10);
- vbios.stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
- vbios.nbp_state_change_latency = bw_frc_to_fixed(2008, 100);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = true;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(768);
- dceip.dmif_pipe_en_fbc_chunk_tracker = false;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 2;
- dceip.number_of_underlay_pipes = 1;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = false;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_gddr5;
+ vbios->dram_channel_width_in_bits = 64;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 8;
+ vbios->high_yclk = bw_int_to_fixed(1866);
+ vbios->mid_yclk = bw_int_to_fixed(1866);
+ vbios->low_yclk = bw_int_to_fixed(1333);
+ vbios->low_sclk = bw_int_to_fixed(200);
+ vbios->mid1_sclk = bw_int_to_fixed(600);
+ vbios->mid2_sclk = bw_int_to_fixed(600);
+ vbios->mid3_sclk = bw_int_to_fixed(600);
+ vbios->mid4_sclk = bw_int_to_fixed(600);
+ vbios->mid5_sclk = bw_int_to_fixed(600);
+ vbios->mid6_sclk = bw_int_to_fixed(600);
+ vbios->high_sclk = bw_int_to_fixed(800);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(352);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(467);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(643);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(50);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(4);
+ vbios->stutter_self_refresh_exit_latency = bw_frc_to_fixed(158, 10);
+ vbios->stutter_self_refresh_entry_latency = bw_int_to_fixed(0);
+ vbios->nbp_state_change_latency = bw_frc_to_fixed(2008, 100);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = true;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 256;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(768);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = false;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 2;
+ dceip->number_of_underlay_pipes = 1;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = false;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 2;
- dceip.graphics_dmif_size = 12288;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = true;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(82176);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = false;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 2;
+ dceip->graphics_dmif_size = 12288;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = true;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(82176);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = false;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(0);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
case BW_CALCS_VERSION_VEGA10:
- vbios.memory_type = bw_def_hbm;
- vbios.dram_channel_width_in_bits = 128;
- vbios.number_of_dram_channels = asic_id.vram_width / vbios.dram_channel_width_in_bits;
- vbios.number_of_dram_banks = 16;
- vbios.high_yclk = bw_int_to_fixed(2400);
- vbios.mid_yclk = bw_int_to_fixed(1700);
- vbios.low_yclk = bw_int_to_fixed(1000);
- vbios.low_sclk = bw_int_to_fixed(300);
- vbios.mid1_sclk = bw_int_to_fixed(350);
- vbios.mid2_sclk = bw_int_to_fixed(400);
- vbios.mid3_sclk = bw_int_to_fixed(500);
- vbios.mid4_sclk = bw_int_to_fixed(600);
- vbios.mid5_sclk = bw_int_to_fixed(700);
- vbios.mid6_sclk = bw_int_to_fixed(760);
- vbios.high_sclk = bw_int_to_fixed(776);
- vbios.low_voltage_max_dispclk = bw_int_to_fixed(460);
- vbios.mid_voltage_max_dispclk = bw_int_to_fixed(670);
- vbios.high_voltage_max_dispclk = bw_int_to_fixed(1133);
- vbios.low_voltage_max_phyclk = bw_int_to_fixed(540);
- vbios.mid_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.high_voltage_max_phyclk = bw_int_to_fixed(810);
- vbios.data_return_bus_width = bw_int_to_fixed(32);
- vbios.trc = bw_int_to_fixed(48);
- vbios.dmifmc_urgent_latency = bw_int_to_fixed(3);
- vbios.stutter_self_refresh_exit_latency = bw_frc_to_fixed(75, 10);
- vbios.stutter_self_refresh_entry_latency = bw_frc_to_fixed(19, 10);
- vbios.nbp_state_change_latency = bw_int_to_fixed(39);
- vbios.mcifwrmc_urgent_latency = bw_int_to_fixed(10);
- vbios.scatter_gather_enable = false;
- vbios.down_spread_percentage = bw_frc_to_fixed(5, 10);
- vbios.cursor_width = 32;
- vbios.average_compression_rate = 4;
- vbios.number_of_request_slots_gmc_reserves_for_dmif_per_channel = 8;
- vbios.blackout_duration = bw_int_to_fixed(0); /* us */
- vbios.maximum_blackout_recovery_time = bw_int_to_fixed(0);
-
- dceip.max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
- dceip.max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
- dceip.percent_of_ideal_port_bw_received_after_urgent_latency = 100;
- dceip.large_cursor = false;
- dceip.dmif_request_buffer_size = bw_int_to_fixed(2304);
- dceip.dmif_pipe_en_fbc_chunk_tracker = true;
- dceip.cursor_max_outstanding_group_num = 1;
- dceip.lines_interleaved_into_lb = 2;
- dceip.chunk_width = 256;
- dceip.number_of_graphics_pipes = 6;
- dceip.number_of_underlay_pipes = 0;
- dceip.low_power_tiling_mode = 0;
- dceip.display_write_back_supported = true;
- dceip.argb_compression_support = true;
- dceip.underlay_vscaler_efficiency6_bit_per_component =
+ vbios->memory_type = bw_def_hbm;
+ vbios->dram_channel_width_in_bits = 128;
+ vbios->number_of_dram_channels = asic_id.vram_width / vbios->dram_channel_width_in_bits;
+ vbios->number_of_dram_banks = 16;
+ vbios->high_yclk = bw_int_to_fixed(2400);
+ vbios->mid_yclk = bw_int_to_fixed(1700);
+ vbios->low_yclk = bw_int_to_fixed(1000);
+ vbios->low_sclk = bw_int_to_fixed(300);
+ vbios->mid1_sclk = bw_int_to_fixed(350);
+ vbios->mid2_sclk = bw_int_to_fixed(400);
+ vbios->mid3_sclk = bw_int_to_fixed(500);
+ vbios->mid4_sclk = bw_int_to_fixed(600);
+ vbios->mid5_sclk = bw_int_to_fixed(700);
+ vbios->mid6_sclk = bw_int_to_fixed(760);
+ vbios->high_sclk = bw_int_to_fixed(776);
+ vbios->low_voltage_max_dispclk = bw_int_to_fixed(460);
+ vbios->mid_voltage_max_dispclk = bw_int_to_fixed(670);
+ vbios->high_voltage_max_dispclk = bw_int_to_fixed(1133);
+ vbios->low_voltage_max_phyclk = bw_int_to_fixed(540);
+ vbios->mid_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->high_voltage_max_phyclk = bw_int_to_fixed(810);
+ vbios->data_return_bus_width = bw_int_to_fixed(32);
+ vbios->trc = bw_int_to_fixed(48);
+ vbios->dmifmc_urgent_latency = bw_int_to_fixed(3);
+ vbios->stutter_self_refresh_exit_latency = bw_frc_to_fixed(75, 10);
+ vbios->stutter_self_refresh_entry_latency = bw_frc_to_fixed(19, 10);
+ vbios->nbp_state_change_latency = bw_int_to_fixed(39);
+ vbios->mcifwrmc_urgent_latency = bw_int_to_fixed(10);
+ vbios->scatter_gather_enable = false;
+ vbios->down_spread_percentage = bw_frc_to_fixed(5, 10);
+ vbios->cursor_width = 32;
+ vbios->average_compression_rate = 4;
+ vbios->number_of_request_slots_gmc_reserves_for_dmif_per_channel = 8;
+ vbios->blackout_duration = bw_int_to_fixed(0); /* us */
+ vbios->maximum_blackout_recovery_time = bw_int_to_fixed(0);
+
+ dceip->max_average_percent_of_ideal_port_bw_display_can_use_in_normal_system_operation = 100;
+ dceip->max_average_percent_of_ideal_drambw_display_can_use_in_normal_system_operation = 100;
+ dceip->percent_of_ideal_port_bw_received_after_urgent_latency = 100;
+ dceip->large_cursor = false;
+ dceip->dmif_request_buffer_size = bw_int_to_fixed(2304);
+ dceip->dmif_pipe_en_fbc_chunk_tracker = true;
+ dceip->cursor_max_outstanding_group_num = 1;
+ dceip->lines_interleaved_into_lb = 2;
+ dceip->chunk_width = 256;
+ dceip->number_of_graphics_pipes = 6;
+ dceip->number_of_underlay_pipes = 0;
+ dceip->low_power_tiling_mode = 0;
+ dceip->display_write_back_supported = true;
+ dceip->argb_compression_support = true;
+ dceip->underlay_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35556, 10000);
- dceip.underlay_vscaler_efficiency8_bit_per_component =
+ dceip->underlay_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.underlay_vscaler_efficiency10_bit_per_component =
+ dceip->underlay_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.underlay_vscaler_efficiency12_bit_per_component =
+ dceip->underlay_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.graphics_vscaler_efficiency6_bit_per_component =
+ dceip->graphics_vscaler_efficiency6_bit_per_component =
bw_frc_to_fixed(35, 10);
- dceip.graphics_vscaler_efficiency8_bit_per_component =
+ dceip->graphics_vscaler_efficiency8_bit_per_component =
bw_frc_to_fixed(34286, 10000);
- dceip.graphics_vscaler_efficiency10_bit_per_component =
+ dceip->graphics_vscaler_efficiency10_bit_per_component =
bw_frc_to_fixed(32, 10);
- dceip.graphics_vscaler_efficiency12_bit_per_component =
+ dceip->graphics_vscaler_efficiency12_bit_per_component =
bw_int_to_fixed(3);
- dceip.alpha_vscaler_efficiency = bw_int_to_fixed(3);
- dceip.max_dmif_buffer_allocated = 4;
- dceip.graphics_dmif_size = 24576;
- dceip.underlay_luma_dmif_size = 19456;
- dceip.underlay_chroma_dmif_size = 23552;
- dceip.pre_downscaler_enabled = true;
- dceip.underlay_downscale_prefetch_enabled = false;
- dceip.lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
- dceip.lb_size_per_component444 = bw_int_to_fixed(245952);
- dceip.graphics_lb_nodownscaling_multi_line_prefetching = true;
- dceip.stutter_and_dram_clock_state_change_gated_before_cursor =
+ dceip->alpha_vscaler_efficiency = bw_int_to_fixed(3);
+ dceip->max_dmif_buffer_allocated = 4;
+ dceip->graphics_dmif_size = 24576;
+ dceip->underlay_luma_dmif_size = 19456;
+ dceip->underlay_chroma_dmif_size = 23552;
+ dceip->pre_downscaler_enabled = true;
+ dceip->underlay_downscale_prefetch_enabled = false;
+ dceip->lb_write_pixels_per_dispclk = bw_int_to_fixed(1);
+ dceip->lb_size_per_component444 = bw_int_to_fixed(245952);
+ dceip->graphics_lb_nodownscaling_multi_line_prefetching = true;
+ dceip->stutter_and_dram_clock_state_change_gated_before_cursor =
bw_int_to_fixed(1);
- dceip.underlay420_luma_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay420_luma_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.underlay420_chroma_lb_size_per_component =
+ dceip->underlay420_chroma_lb_size_per_component =
bw_int_to_fixed(164352);
- dceip.underlay422_lb_size_per_component = bw_int_to_fixed(
+ dceip->underlay422_lb_size_per_component = bw_int_to_fixed(
82176);
- dceip.cursor_chunk_width = bw_int_to_fixed(64);
- dceip.cursor_dcp_buffer_lines = bw_int_to_fixed(4);
- dceip.underlay_maximum_width_efficient_for_tiling =
+ dceip->cursor_chunk_width = bw_int_to_fixed(64);
+ dceip->cursor_dcp_buffer_lines = bw_int_to_fixed(4);
+ dceip->underlay_maximum_width_efficient_for_tiling =
bw_int_to_fixed(1920);
- dceip.underlay_maximum_height_efficient_for_tiling =
+ dceip->underlay_maximum_height_efficient_for_tiling =
bw_int_to_fixed(1080);
- dceip.peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_multiple_displays_or_single_rotated_display =
bw_frc_to_fixed(3, 10);
- dceip.peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
+ dceip->peak_pte_request_to_eviction_ratio_limiting_single_display_no_rotation =
bw_int_to_fixed(25);
- dceip.minimum_outstanding_pte_request_limit = bw_int_to_fixed(
+ dceip->minimum_outstanding_pte_request_limit = bw_int_to_fixed(
2);
- dceip.maximum_total_outstanding_pte_requests_allowed_by_saw =
+ dceip->maximum_total_outstanding_pte_requests_allowed_by_saw =
bw_int_to_fixed(128);
- dceip.limit_excessive_outstanding_dmif_requests = true;
- dceip.linear_mode_line_request_alternation_slice =
+ dceip->limit_excessive_outstanding_dmif_requests = true;
+ dceip->linear_mode_line_request_alternation_slice =
bw_int_to_fixed(64);
- dceip.scatter_gather_lines_of_pte_prefetching_in_linear_mode =
+ dceip->scatter_gather_lines_of_pte_prefetching_in_linear_mode =
32;
- dceip.display_write_back420_luma_mcifwr_buffer_size = 12288;
- dceip.display_write_back420_chroma_mcifwr_buffer_size = 8192;
- dceip.request_efficiency = bw_frc_to_fixed(8, 10);
- dceip.dispclk_per_request = bw_int_to_fixed(2);
- dceip.dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
- dceip.display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
- dceip.scatter_gather_pte_request_rows_in_tiling_mode = 2;
- dceip.mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
+ dceip->display_write_back420_luma_mcifwr_buffer_size = 12288;
+ dceip->display_write_back420_chroma_mcifwr_buffer_size = 8192;
+ dceip->request_efficiency = bw_frc_to_fixed(8, 10);
+ dceip->dispclk_per_request = bw_int_to_fixed(2);
+ dceip->dispclk_ramping_factor = bw_frc_to_fixed(105, 100);
+ dceip->display_pipe_throughput_factor = bw_frc_to_fixed(105, 100);
+ dceip->scatter_gather_pte_request_rows_in_tiling_mode = 2;
+ dceip->mcifwr_all_surfaces_burst_time = bw_int_to_fixed(0);
break;
default:
break;
}
- *bw_dceip = dceip;
- *bw_vbios = vbios;
+ *bw_dceip = *dceip;
+ *bw_vbios = *vbios;
+ kfree(dceip);
+ kfree(vbios);
}
/*
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c
index 995ffbbf64e7..7d6c68c5dea9 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/clk_mgr.c
@@ -87,12 +87,16 @@ int clk_mgr_helper_get_active_plane_cnt(
void clk_mgr_exit_optimized_pwr_state(const struct dc *dc, struct clk_mgr *clk_mgr)
{
- struct dc_link *edp_link = get_edp_link(dc);
+ struct dc_link *edp_links[MAX_NUM_EDP];
+ struct dc_link *edp_link = NULL;
+ int edp_num;
+ get_edp_links(dc, edp_links, &edp_num);
if (dc->hwss.exit_optimized_pwr_state)
dc->hwss.exit_optimized_pwr_state(dc, dc->current_state);
- if (edp_link) {
+ if (edp_num) {
+ edp_link = edp_links[0];
clk_mgr->psr_allow_active_cache = edp_link->psr_settings.psr_allow_active;
dc_link_set_psr_allow_active(edp_link, false, false, false);
}
@@ -101,11 +105,16 @@ void clk_mgr_exit_optimized_pwr_state(const struct dc *dc, struct clk_mgr *clk_m
void clk_mgr_optimize_pwr_state(const struct dc *dc, struct clk_mgr *clk_mgr)
{
- struct dc_link *edp_link = get_edp_link(dc);
+ struct dc_link *edp_links[MAX_NUM_EDP];
+ struct dc_link *edp_link = NULL;
+ int edp_num;
- if (edp_link)
+ get_edp_links(dc, edp_links, &edp_num);
+ if (edp_num) {
+ edp_link = edp_links[0];
dc_link_set_psr_allow_active(edp_link,
clk_mgr->psr_allow_active_cache, false, false);
+ }
if (dc->hwss.optimize_pwr_state)
dc->hwss.optimize_pwr_state(dc, dc->current_state);
@@ -116,87 +125,136 @@ struct clk_mgr *dc_clk_mgr_create(struct dc_context *ctx, struct pp_smu_funcs *p
{
struct hw_asic_id asic_id = ctx->asic_id;
- struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
-
- if (clk_mgr == NULL) {
- BREAK_TO_DEBUGGER();
- return NULL;
- }
-
switch (asic_id.chip_family) {
#if defined(CONFIG_DRM_AMD_DC_SI)
- case FAMILY_SI:
+ case FAMILY_SI: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
dce60_clk_mgr_construct(ctx, clk_mgr);
- break;
+ dce_clk_mgr_construct(ctx, clk_mgr);
+ return &clk_mgr->base;
+ }
#endif
case FAMILY_CI:
- case FAMILY_KV:
+ case FAMILY_KV: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
dce_clk_mgr_construct(ctx, clk_mgr);
- break;
- case FAMILY_CZ:
+ return &clk_mgr->base;
+ }
+ case FAMILY_CZ: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
dce110_clk_mgr_construct(ctx, clk_mgr);
- break;
- case FAMILY_VI:
+ return &clk_mgr->base;
+ }
+ case FAMILY_VI: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
if (ASIC_REV_IS_TONGA_P(asic_id.hw_internal_rev) ||
ASIC_REV_IS_FIJI_P(asic_id.hw_internal_rev)) {
dce_clk_mgr_construct(ctx, clk_mgr);
- break;
+ return &clk_mgr->base;
}
if (ASIC_REV_IS_POLARIS10_P(asic_id.hw_internal_rev) ||
ASIC_REV_IS_POLARIS11_M(asic_id.hw_internal_rev) ||
ASIC_REV_IS_POLARIS12_V(asic_id.hw_internal_rev)) {
dce112_clk_mgr_construct(ctx, clk_mgr);
- break;
+ return &clk_mgr->base;
}
if (ASIC_REV_IS_VEGAM(asic_id.hw_internal_rev)) {
dce112_clk_mgr_construct(ctx, clk_mgr);
- break;
+ return &clk_mgr->base;
+ }
+ return &clk_mgr->base;
+ }
+ case FAMILY_AI: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
}
- break;
- case FAMILY_AI:
if (ASICREV_IS_VEGA20_P(asic_id.hw_internal_rev))
dce121_clk_mgr_construct(ctx, clk_mgr);
else
dce120_clk_mgr_construct(ctx, clk_mgr);
- break;
-
+ return &clk_mgr->base;
+ }
#if defined(CONFIG_DRM_AMD_DC_DCN)
- case FAMILY_RV:
+ case FAMILY_RV: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
+
if (ASICREV_IS_RENOIR(asic_id.hw_internal_rev)) {
rn_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_GREEN_SARDINE(asic_id.hw_internal_rev)) {
rn_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_RAVEN2(asic_id.hw_internal_rev)) {
rv2_clk_mgr_construct(ctx, clk_mgr, pp_smu);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_RAVEN(asic_id.hw_internal_rev) ||
ASICREV_IS_PICASSO(asic_id.hw_internal_rev)) {
rv1_clk_mgr_construct(ctx, clk_mgr, pp_smu);
- break;
+ return &clk_mgr->base;
}
- break;
+ return &clk_mgr->base;
+ }
+ case FAMILY_NV: {
+ struct clk_mgr_internal *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
- case FAMILY_NV:
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
if (ASICREV_IS_SIENNA_CICHLID_P(asic_id.hw_internal_rev)) {
dcn3_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
if (ASICREV_IS_DIMGREY_CAVEFISH_P(asic_id.hw_internal_rev)) {
dcn3_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
+ return &clk_mgr->base;
}
dcn20_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
- break;
-
+ return &clk_mgr->base;
+ }
case FAMILY_VGH:
- if (ASICREV_IS_VANGOGH(asic_id.hw_internal_rev))
+ if (ASICREV_IS_VANGOGH(asic_id.hw_internal_rev)) {
+ struct clk_mgr_vgh *clk_mgr = kzalloc(sizeof(*clk_mgr), GFP_KERNEL);
+
+ if (clk_mgr == NULL) {
+ BREAK_TO_DEBUGGER();
+ return NULL;
+ }
vg_clk_mgr_construct(ctx, clk_mgr, pp_smu, dccg);
+ return &clk_mgr->base.base;
+ }
break;
#endif
default:
@@ -204,7 +262,7 @@ struct clk_mgr *dc_clk_mgr_create(struct dc_context *ctx, struct pp_smu_funcs *p
break;
}
- return &clk_mgr->base;
+ return NULL;
}
void dc_destroy_clk_mgr(struct clk_mgr *clk_mgr_base)
@@ -217,6 +275,9 @@ void dc_destroy_clk_mgr(struct clk_mgr *clk_mgr_base)
if (ASICREV_IS_SIENNA_CICHLID_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
dcn3_clk_mgr_destroy(clk_mgr);
}
+ if (ASICREV_IS_DIMGREY_CAVEFISH_P(clk_mgr_base->ctx->asic_id.hw_internal_rev)) {
+ dcn3_clk_mgr_destroy(clk_mgr);
+ }
break;
case FAMILY_VGH:
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c
index ec9dc265cde0..372d53b5a34d 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn20/dcn20_clk_mgr.c
@@ -361,7 +361,7 @@ void dcn2_read_clocks_from_hw_dentist(struct clk_mgr *clk_mgr_base)
REG_GET(DENTIST_DISPCLK_CNTL, DENTIST_DPPCLK_WDIVIDER, &dppclk_wdivider);
disp_divider = dentist_get_divider_from_did(dispclk_wdivider);
- dpp_divider = dentist_get_divider_from_did(dispclk_wdivider);
+ dpp_divider = dentist_get_divider_from_did(dppclk_wdivider);
if (disp_divider && dpp_divider) {
/* Calculate the current DFS clock, in kHz.*/
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c
index 01b1853b7750..a06e86853bb9 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn21/rn_clk_mgr.c
@@ -128,7 +128,7 @@ void rn_update_clocks(struct clk_mgr *clk_mgr_base,
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
struct dc_clocks *new_clocks = &context->bw_ctx.bw.dcn.clk;
struct dc *dc = clk_mgr_base->ctx->dc;
- int display_count;
+ int display_count, i;
bool update_dppclk = false;
bool update_dispclk = false;
bool dpp_clock_lowered = false;
@@ -210,6 +210,14 @@ void rn_update_clocks(struct clk_mgr *clk_mgr_base,
clk_mgr_base->clks.dppclk_khz,
safe_to_lower);
+ for (i = 0; i < context->stream_count; i++) {
+ if (context->streams[i]->signal == SIGNAL_TYPE_EDP &&
+ context->streams[i]->apply_seamless_boot_optimization) {
+ dc_wait_for_vblank(dc, context->streams[i]);
+ break;
+ }
+ }
+
clk_mgr_base->clks.actual_dppclk_khz =
rn_vbios_smu_set_dppclk(clk_mgr, clk_mgr_base->clks.dppclk_khz);
@@ -761,6 +769,43 @@ static struct wm_table ddr4_wm_table_rn = {
}
};
+static struct wm_table ddr4_1R_wm_table_rn = {
+ .entries = {
+ {
+ .wm_inst = WM_A,
+ .wm_type = WM_TYPE_PSTATE_CHG,
+ .pstate_latency_us = 11.72,
+ .sr_exit_time_us = 13.90,
+ .sr_enter_plus_exit_time_us = 14.80,
+ .valid = true,
+ },
+ {
+ .wm_inst = WM_B,
+ .wm_type = WM_TYPE_PSTATE_CHG,
+ .pstate_latency_us = 11.72,
+ .sr_exit_time_us = 13.90,
+ .sr_enter_plus_exit_time_us = 14.80,
+ .valid = true,
+ },
+ {
+ .wm_inst = WM_C,
+ .wm_type = WM_TYPE_PSTATE_CHG,
+ .pstate_latency_us = 11.72,
+ .sr_exit_time_us = 13.90,
+ .sr_enter_plus_exit_time_us = 14.80,
+ .valid = true,
+ },
+ {
+ .wm_inst = WM_D,
+ .wm_type = WM_TYPE_PSTATE_CHG,
+ .pstate_latency_us = 11.72,
+ .sr_exit_time_us = 13.90,
+ .sr_enter_plus_exit_time_us = 14.80,
+ .valid = true,
+ },
+ }
+};
+
static struct wm_table lpddr4_wm_table_rn = {
.entries = {
{
@@ -797,7 +842,18 @@ static struct wm_table lpddr4_wm_table_rn = {
},
}
};
+static unsigned int find_socclk_for_voltage(struct dpm_clocks *clock_table, unsigned int voltage)
+{
+ int i;
+ for (i = 0; i < PP_SMU_NUM_SOCCLK_DPM_LEVELS; i++) {
+ if (clock_table->SocClocks[i].Vol == voltage)
+ return clock_table->SocClocks[i].Freq;
+ }
+
+ ASSERT(0);
+ return 0;
+}
static unsigned int find_dcfclk_for_voltage(struct dpm_clocks *clock_table, unsigned int voltage)
{
int i;
@@ -841,6 +897,8 @@ static void rn_clk_mgr_helper_populate_bw_params(struct clk_bw_params *bw_params
bw_params->clk_table.entries[i].memclk_mhz = clock_table->MemClocks[j].Freq;
bw_params->clk_table.entries[i].voltage = clock_table->FClocks[j].Vol;
bw_params->clk_table.entries[i].dcfclk_mhz = find_dcfclk_for_voltage(clock_table, clock_table->FClocks[j].Vol);
+ bw_params->clk_table.entries[i].socclk_mhz = find_socclk_for_voltage(clock_table,
+ bw_params->clk_table.entries[i].voltage);
}
bw_params->vram_type = bios_info->memory_type;
@@ -932,8 +990,12 @@ void rn_clk_mgr_construct(
} else {
if (is_green_sardine)
rn_bw_params.wm_table = ddr4_wm_table_gs;
- else
- rn_bw_params.wm_table = ddr4_wm_table_rn;
+ else {
+ if (ctx->dc->config.is_single_rank_dimm)
+ rn_bw_params.wm_table = ddr4_1R_wm_table_rn;
+ else
+ rn_bw_params.wm_table = ddr4_wm_table_rn;
+ }
}
/* Saved clocks configured at boot for debug purposes */
rn_dump_clk_registers(&clk_mgr->base.boot_snapshot, &clk_mgr->base, &log_info);
@@ -951,6 +1013,9 @@ void rn_clk_mgr_construct(
if (status == PP_SMU_RESULT_OK &&
ctx->dc_bios && ctx->dc_bios->integrated_info) {
rn_clk_mgr_helper_populate_bw_params (clk_mgr->base.bw_params, &clock_table, ctx->dc_bios->integrated_info);
+ /* treat memory config as single channel if memory is asymmetrics. */
+ if (ctx->dc->config.is_asymmetric_memory)
+ clk_mgr->base.bw_params->num_channels = 1;
}
}
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c
index c7e5a64e06af..652fa89fae5f 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn30/dcn30_clk_mgr.c
@@ -252,6 +252,7 @@ static void dcn3_update_clocks(struct clk_mgr *clk_mgr_base,
bool force_reset = false;
bool update_uclk = false;
bool p_state_change_support;
+ int total_plane_count;
if (dc->work_arounds.skip_clock_update || !clk_mgr->smu_present)
return;
@@ -292,7 +293,8 @@ static void dcn3_update_clocks(struct clk_mgr *clk_mgr_base,
clk_mgr_base->clks.socclk_khz = new_clocks->socclk_khz;
clk_mgr_base->clks.prev_p_state_change_support = clk_mgr_base->clks.p_state_change_support;
- p_state_change_support = new_clocks->p_state_change_support || (display_count == 0);
+ total_plane_count = clk_mgr_helper_get_active_plane_cnt(dc, context);
+ p_state_change_support = new_clocks->p_state_change_support || (total_plane_count == 0);
if (should_update_pstate_support(safe_to_lower, p_state_change_support, clk_mgr_base->clks.p_state_change_support)) {
clk_mgr_base->clks.p_state_change_support = p_state_change_support;
@@ -430,6 +432,12 @@ static void dcn3_get_memclk_states_from_smu(struct clk_mgr *clk_mgr_base)
clk_mgr->base.ctx->dc, clk_mgr_base->bw_params);
}
+static bool dcn3_is_smu_present(struct clk_mgr *clk_mgr_base)
+{
+ struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
+ return clk_mgr->smu_present;
+}
+
static bool dcn3_are_clock_states_equal(struct dc_clocks *a,
struct dc_clocks *b)
{
@@ -492,6 +500,7 @@ static struct clk_mgr_funcs dcn3_funcs = {
.are_clock_states_equal = dcn3_are_clock_states_equal,
.enable_pme_wa = dcn3_enable_pme_wa,
.notify_link_rate_change = dcn30_notify_link_rate_change,
+ .is_smu_present = dcn3_is_smu_present
};
static void dcn3_init_clocks_fpga(struct clk_mgr *clk_mgr)
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c
index 68942bbc7472..07774fa2c2cf 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/dcn301_smu.c
@@ -113,10 +113,13 @@ int dcn301_smu_send_msg_with_param(
int dcn301_smu_get_smu_version(struct clk_mgr_internal *clk_mgr)
{
- return dcn301_smu_send_msg_with_param(
- clk_mgr,
- VBIOSSMC_MSG_GetSmuVersion,
- 0);
+ int smu_version = dcn301_smu_send_msg_with_param(clk_mgr,
+ VBIOSSMC_MSG_GetSmuVersion,
+ 0);
+
+ DC_LOG_DEBUG("%s %x\n", __func__, smu_version);
+
+ return smu_version;
}
@@ -124,6 +127,8 @@ int dcn301_smu_set_dispclk(struct clk_mgr_internal *clk_mgr, int requested_dispc
{
int actual_dispclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_dispclk_khz);
+
/* Unit of SMU msg parameter is Mhz */
actual_dispclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
@@ -137,6 +142,8 @@ int dcn301_smu_set_dprefclk(struct clk_mgr_internal *clk_mgr)
{
int actual_dprefclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s %d\n", __func__, clk_mgr->base.dprefclk_khz / 1000);
+
actual_dprefclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDprefclkFreq,
@@ -151,6 +158,8 @@ int dcn301_smu_set_hard_min_dcfclk(struct clk_mgr_internal *clk_mgr, int request
{
int actual_dcfclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_dcfclk_khz);
+
actual_dcfclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetHardMinDcfclkByFreq,
@@ -163,6 +172,8 @@ int dcn301_smu_set_min_deep_sleep_dcfclk(struct clk_mgr_internal *clk_mgr, int r
{
int actual_min_ds_dcfclk_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_min_ds_dcfclk_khz);
+
actual_min_ds_dcfclk_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetMinDeepSleepDcfclk,
@@ -175,6 +186,8 @@ int dcn301_smu_set_dppclk(struct clk_mgr_internal *clk_mgr, int requested_dpp_kh
{
int actual_dppclk_set_mhz = -1;
+ DC_LOG_DEBUG("%s(%d)\n", __func__, requested_dpp_khz);
+
actual_dppclk_set_mhz = dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDppclkFreq,
@@ -187,6 +200,8 @@ void dcn301_smu_set_display_idle_optimization(struct clk_mgr_internal *clk_mgr,
{
//TODO: Work with smu team to define optimization options.
+ DC_LOG_DEBUG("%s(%x)\n", __func__, idle_info);
+
dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDisplayIdleOptimizations,
@@ -202,6 +217,8 @@ void dcn301_smu_enable_phy_refclk_pwrdwn(struct clk_mgr_internal *clk_mgr, bool
idle_info.idle_info.phy_ref_clk_off = 1;
}
+ DC_LOG_DEBUG("%s(%d)\n", __func__, enable);
+
dcn301_smu_send_msg_with_param(
clk_mgr,
VBIOSSMC_MSG_SetDisplayIdleOptimizations,
@@ -218,12 +235,16 @@ void dcn301_smu_enable_pme_wa(struct clk_mgr_internal *clk_mgr)
void dcn301_smu_set_dram_addr_high(struct clk_mgr_internal *clk_mgr, uint32_t addr_high)
{
+ DC_LOG_DEBUG("%s(%x)\n", __func__, addr_high);
+
dcn301_smu_send_msg_with_param(clk_mgr,
VBIOSSMC_MSG_SetVbiosDramAddrHigh, addr_high);
}
void dcn301_smu_set_dram_addr_low(struct clk_mgr_internal *clk_mgr, uint32_t addr_low)
{
+ DC_LOG_DEBUG("%s(%x)\n", __func__, addr_low);
+
dcn301_smu_send_msg_with_param(clk_mgr,
VBIOSSMC_MSG_SetVbiosDramAddrLow, addr_low);
}
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c
index aadb801447a7..c636b589d69d 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.c
@@ -32,9 +32,8 @@
// For dcn20_update_clocks_update_dpp_dto
#include "dcn20/dcn20_clk_mgr.h"
-
-
#include "vg_clk_mgr.h"
+#include "dcn301_smu.h"
#include "reg_helper.h"
#include "core_types.h"
#include "dm_helpers.h"
@@ -50,11 +49,14 @@
/* Macros */
+#define TO_CLK_MGR_VGH(clk_mgr)\
+ container_of(clk_mgr, struct clk_mgr_vgh, base)
+
#define REG(reg_name) \
(CLK_BASE.instance[0].segment[mm ## reg_name ## _BASE_IDX] + mm ## reg_name)
/* TODO: evaluate how to lower or disable all dcn clocks in screen off case */
-int vg_get_active_display_cnt_wa(
+static int vg_get_active_display_cnt_wa(
struct dc *dc,
struct dc_state *context)
{
@@ -134,13 +136,13 @@ void vg_update_clocks(struct clk_mgr *clk_mgr_base,
}
}
- if (should_set_clock(safe_to_lower, new_clocks->dcfclk_khz, clk_mgr_base->clks.dcfclk_khz)) {
+ if (should_set_clock(safe_to_lower, new_clocks->dcfclk_khz, clk_mgr_base->clks.dcfclk_khz) && !dc->debug.disable_min_fclk) {
clk_mgr_base->clks.dcfclk_khz = new_clocks->dcfclk_khz;
dcn301_smu_set_hard_min_dcfclk(clk_mgr, clk_mgr_base->clks.dcfclk_khz);
}
if (should_set_clock(safe_to_lower,
- new_clocks->dcfclk_deep_sleep_khz, clk_mgr_base->clks.dcfclk_deep_sleep_khz)) {
+ new_clocks->dcfclk_deep_sleep_khz, clk_mgr_base->clks.dcfclk_deep_sleep_khz) && !dc->debug.disable_min_fclk) {
clk_mgr_base->clks.dcfclk_deep_sleep_khz = new_clocks->dcfclk_deep_sleep_khz;
dcn301_smu_set_min_deep_sleep_dcfclk(clk_mgr, clk_mgr_base->clks.dcfclk_deep_sleep_khz);
}
@@ -377,7 +379,7 @@ void vg_get_clk_states(struct clk_mgr *clk_mgr_base, struct clk_states *s)
s->dprefclk_khz = sb.dprefclk * 1000;
}
-void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base)
+static void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
@@ -449,15 +451,16 @@ static void vg_build_watermark_ranges(struct clk_bw_params *bw_params, struct wa
}
-void vg_notify_wm_ranges(struct clk_mgr *clk_mgr_base)
+static void vg_notify_wm_ranges(struct clk_mgr *clk_mgr_base)
{
struct clk_mgr_internal *clk_mgr = TO_CLK_MGR_INTERNAL(clk_mgr_base);
- struct watermarks *table = clk_mgr_base->smu_wm_set.wm_set;
+ struct clk_mgr_vgh *clk_mgr_vgh = TO_CLK_MGR_VGH(clk_mgr);
+ struct watermarks *table = clk_mgr_vgh->smu_wm_set.wm_set;
if (!clk_mgr->smu_ver)
return;
- if (!table || clk_mgr_base->smu_wm_set.mc_address.quad_part == 0)
+ if (!table || clk_mgr_vgh->smu_wm_set.mc_address.quad_part == 0)
return;
memset(table, 0, sizeof(*table));
@@ -465,9 +468,9 @@ void vg_notify_wm_ranges(struct clk_mgr *clk_mgr_base)
vg_build_watermark_ranges(clk_mgr_base->bw_params, table);
dcn301_smu_set_dram_addr_high(clk_mgr,
- clk_mgr_base->smu_wm_set.mc_address.high_part);
+ clk_mgr_vgh->smu_wm_set.mc_address.high_part);
dcn301_smu_set_dram_addr_low(clk_mgr,
- clk_mgr_base->smu_wm_set.mc_address.low_part);
+ clk_mgr_vgh->smu_wm_set.mc_address.low_part);
dcn301_smu_transfer_wm_table_dram_2_smu(clk_mgr);
}
@@ -625,7 +628,7 @@ static unsigned int find_dcfclk_for_voltage(const struct vg_dpm_clocks *clock_ta
return 0;
}
-void vg_clk_mgr_helper_populate_bw_params(
+static void vg_clk_mgr_helper_populate_bw_params(
struct clk_mgr_internal *clk_mgr,
struct integrated_info *bios_info,
const struct vg_dpm_clocks *clock_table)
@@ -703,7 +706,7 @@ static struct vg_dpm_clocks dummy_clocks = {
static struct watermarks dummy_wms = { 0 };
-void vg_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
+static void vg_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
struct smu_dpm_clks *smu_dpm_clks)
{
struct vg_dpm_clocks *table = smu_dpm_clks->dpm_clks;
@@ -725,39 +728,39 @@ void vg_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
void vg_clk_mgr_construct(
struct dc_context *ctx,
- struct clk_mgr_internal *clk_mgr,
+ struct clk_mgr_vgh *clk_mgr,
struct pp_smu_funcs *pp_smu,
struct dccg *dccg)
{
struct smu_dpm_clks smu_dpm_clks = { 0 };
- clk_mgr->base.ctx = ctx;
- clk_mgr->base.funcs = &vg_funcs;
+ clk_mgr->base.base.ctx = ctx;
+ clk_mgr->base.base.funcs = &vg_funcs;
- clk_mgr->pp_smu = pp_smu;
+ clk_mgr->base.pp_smu = pp_smu;
- clk_mgr->dccg = dccg;
- clk_mgr->dfs_bypass_disp_clk = 0;
+ clk_mgr->base.dccg = dccg;
+ clk_mgr->base.dfs_bypass_disp_clk = 0;
- clk_mgr->dprefclk_ss_percentage = 0;
- clk_mgr->dprefclk_ss_divider = 1000;
- clk_mgr->ss_on_dprefclk = false;
- clk_mgr->dfs_ref_freq_khz = 48000;
+ clk_mgr->base.dprefclk_ss_percentage = 0;
+ clk_mgr->base.dprefclk_ss_divider = 1000;
+ clk_mgr->base.ss_on_dprefclk = false;
+ clk_mgr->base.dfs_ref_freq_khz = 48000;
- clk_mgr->base.smu_wm_set.wm_set = (struct watermarks *)dm_helpers_allocate_gpu_mem(
- clk_mgr->base.ctx,
+ clk_mgr->smu_wm_set.wm_set = (struct watermarks *)dm_helpers_allocate_gpu_mem(
+ clk_mgr->base.base.ctx,
DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
sizeof(struct watermarks),
- &clk_mgr->base.smu_wm_set.mc_address.quad_part);
+ &clk_mgr->smu_wm_set.mc_address.quad_part);
- if (clk_mgr->base.smu_wm_set.wm_set == 0) {
- clk_mgr->base.smu_wm_set.wm_set = &dummy_wms;
- clk_mgr->base.smu_wm_set.mc_address.quad_part = 0;
+ if (clk_mgr->smu_wm_set.wm_set == 0) {
+ clk_mgr->smu_wm_set.wm_set = &dummy_wms;
+ clk_mgr->smu_wm_set.mc_address.quad_part = 0;
}
- ASSERT(clk_mgr->base.smu_wm_set.wm_set);
+ ASSERT(clk_mgr->smu_wm_set.wm_set);
smu_dpm_clks.dpm_clks = (struct vg_dpm_clocks *)dm_helpers_allocate_gpu_mem(
- clk_mgr->base.ctx,
+ clk_mgr->base.base.ctx,
DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
sizeof(struct vg_dpm_clocks),
&smu_dpm_clks.mc_address.quad_part);
@@ -771,21 +774,21 @@ void vg_clk_mgr_construct(
if (IS_FPGA_MAXIMUS_DC(ctx->dce_environment)) {
vg_funcs.update_clocks = dcn2_update_clocks_fpga;
- clk_mgr->base.dentist_vco_freq_khz = 3600000;
+ clk_mgr->base.base.dentist_vco_freq_khz = 3600000;
} else {
struct clk_log_info log_info = {0};
- clk_mgr->smu_ver = dcn301_smu_get_smu_version(clk_mgr);
+ clk_mgr->base.smu_ver = dcn301_smu_get_smu_version(&clk_mgr->base);
- if (clk_mgr->smu_ver)
- clk_mgr->smu_present = true;
+ if (clk_mgr->base.smu_ver)
+ clk_mgr->base.smu_present = true;
/* TODO: Check we get what we expect during bringup */
- clk_mgr->base.dentist_vco_freq_khz = get_vco_frequency_from_reg(clk_mgr);
+ clk_mgr->base.base.dentist_vco_freq_khz = get_vco_frequency_from_reg(&clk_mgr->base);
/* in case we don't get a value from the register, use default */
- if (clk_mgr->base.dentist_vco_freq_khz == 0)
- clk_mgr->base.dentist_vco_freq_khz = 3600000;
+ if (clk_mgr->base.base.dentist_vco_freq_khz == 0)
+ clk_mgr->base.base.dentist_vco_freq_khz = 3600000;
if (ctx->dc_bios->integrated_info->memory_type == LpDdr5MemType) {
vg_bw_params.wm_table = lpddr5_wm_table;
@@ -793,36 +796,38 @@ void vg_clk_mgr_construct(
vg_bw_params.wm_table = ddr4_wm_table;
}
/* Saved clocks configured at boot for debug purposes */
- vg_dump_clk_registers(&clk_mgr->base.boot_snapshot, &clk_mgr->base, &log_info);
+ vg_dump_clk_registers(&clk_mgr->base.base.boot_snapshot, &clk_mgr->base.base, &log_info);
}
- clk_mgr->base.dprefclk_khz = 600000;
- dce_clock_read_ss_info(clk_mgr);
+ clk_mgr->base.base.dprefclk_khz = 600000;
+ dce_clock_read_ss_info(&clk_mgr->base);
- clk_mgr->base.bw_params = &vg_bw_params;
+ clk_mgr->base.base.bw_params = &vg_bw_params;
- vg_get_dpm_table_from_smu(clk_mgr, &smu_dpm_clks);
+ vg_get_dpm_table_from_smu(&clk_mgr->base, &smu_dpm_clks);
if (ctx->dc_bios && ctx->dc_bios->integrated_info) {
vg_clk_mgr_helper_populate_bw_params(
- clk_mgr,
+ &clk_mgr->base,
ctx->dc_bios->integrated_info,
smu_dpm_clks.dpm_clks);
}
if (smu_dpm_clks.dpm_clks && smu_dpm_clks.mc_address.quad_part != 0)
- dm_helpers_free_gpu_mem(clk_mgr->base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
+ dm_helpers_free_gpu_mem(clk_mgr->base.base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
smu_dpm_clks.dpm_clks);
/*
- if (!IS_FPGA_MAXIMUS_DC(ctx->dce_environment) && clk_mgr->smu_ver) {
+ if (!IS_FPGA_MAXIMUS_DC(ctx->dce_environment) && clk_mgr->base.smu_ver) {
enable powerfeatures when displaycount goes to 0
dcn301_smu_enable_phy_refclk_pwrdwn(clk_mgr, !debug->disable_48mhz_pwrdwn);
}
*/
}
-void vg_clk_mgr_destroy(struct clk_mgr_internal *clk_mgr)
+void vg_clk_mgr_destroy(struct clk_mgr_internal *clk_mgr_int)
{
- if (clk_mgr->base.smu_wm_set.wm_set && clk_mgr->base.smu_wm_set.mc_address.quad_part != 0)
- dm_helpers_free_gpu_mem(clk_mgr->base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
- clk_mgr->base.smu_wm_set.wm_set);
+ struct clk_mgr_vgh *clk_mgr = TO_CLK_MGR_VGH(clk_mgr_int);
+
+ if (clk_mgr->smu_wm_set.wm_set && clk_mgr->smu_wm_set.mc_address.quad_part != 0)
+ dm_helpers_free_gpu_mem(clk_mgr_int->base.ctx, DC_MEM_ALLOC_TYPE_FRAME_BUFFER,
+ clk_mgr->smu_wm_set.wm_set);
}
diff --git a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.h b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.h
index b5115b3123a1..7255477307f1 100644
--- a/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.h
+++ b/drivers/gpu/drm/amd/display/dc/clk_mgr/dcn301/vg_clk_mgr.h
@@ -25,29 +25,25 @@
#ifndef __VG_CLK_MGR_H__
#define __VG_CLK_MGR_H__
+#include "clk_mgr_internal.h"
-int vg_get_active_display_cnt_wa(
- struct dc *dc,
- struct dc_state *context);
+struct watermarks;
-void vg_enable_pme_wa(struct clk_mgr *clk_mgr_base);
+struct smu_watermark_set {
+ struct watermarks *wm_set;
+ union large_integer mc_address;
+};
+
+struct clk_mgr_vgh {
+ struct clk_mgr_internal base;
+ struct smu_watermark_set smu_wm_set;
+};
void vg_clk_mgr_construct(struct dc_context *ctx,
- struct clk_mgr_internal *clk_mgr,
+ struct clk_mgr_vgh *clk_mgr,
struct pp_smu_funcs *pp_smu,
struct dccg *dccg);
void vg_clk_mgr_destroy(struct clk_mgr_internal *clk_mgr);
-#include "dcn301_smu.h"
-void vg_notify_wm_ranges(struct clk_mgr *clk_mgr_base);
-
-void vg_get_dpm_table_from_smu(struct clk_mgr_internal *clk_mgr,
- struct smu_dpm_clks *smu_dpm_clks);
-
-void vg_clk_mgr_helper_populate_bw_params(
- struct clk_mgr_internal *clk_mgr,
- struct integrated_info *bios_info,
- const struct vg_dpm_clocks *clock_table);
-
#endif //__VG_CLK_MGR_H__
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc.c b/drivers/gpu/drm/amd/display/dc/core/dc.c
index 8f8a13c7cf73..4713f09bcbf1 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc.c
@@ -41,6 +41,7 @@
#include "dc_bios_types.h"
#include "bios_parser_interface.h"
+#include "bios/bios_parser_helper.h"
#include "include/irq_service_interface.h"
#include "transform.h"
#include "dmcu.h"
@@ -48,10 +49,13 @@
#include "timing_generator.h"
#include "abm.h"
#include "virtual/virtual_link_encoder.h"
+#include "hubp.h"
#include "link_hwss.h"
#include "link_encoder.h"
+#include "link_enc_cfg.h"
+#include "dc_link.h"
#include "dc_link_ddc.h"
#include "dm_helpers.h"
#include "mem_input.h"
@@ -68,6 +72,7 @@
#include "dmub/dmub_srv.h"
+#include "i2caux_interface.h"
#include "dce/dmub_hw_lock_mgr.h"
#include "dc_trace.h"
@@ -163,6 +168,18 @@ static uint32_t get_num_of_internal_disp(struct dc_link **links, uint32_t num_li
return count;
}
+static int get_seamless_boot_stream_count(struct dc_state *ctx)
+{
+ uint8_t i;
+ uint8_t seamless_boot_stream_count = 0;
+
+ for (i = 0; i < ctx->stream_count; i++)
+ if (ctx->streams[i]->apply_seamless_boot_optimization)
+ seamless_boot_stream_count++;
+
+ return seamless_boot_stream_count;
+}
+
static bool create_links(
struct dc *dc,
uint32_t num_virtual_links)
@@ -290,7 +307,10 @@ bool dc_stream_adjust_vmin_vmax(struct dc *dc,
int i = 0;
bool ret = false;
- stream->adjust = *adjust;
+ stream->adjust.v_total_max = adjust->v_total_max;
+ stream->adjust.v_total_mid = adjust->v_total_mid;
+ stream->adjust.v_total_mid_frame_num = adjust->v_total_mid_frame_num;
+ stream->adjust.v_total_min = adjust->v_total_min;
for (i = 0; i < MAX_PIPES; i++) {
struct pipe_ctx *pipe = &dc->current_state->res_ctx.pipe_ctx[i];
@@ -298,10 +318,7 @@ bool dc_stream_adjust_vmin_vmax(struct dc *dc,
if (pipe->stream == stream && pipe->stream_res.tg) {
dc->hwss.set_drr(&pipe,
1,
- adjust->v_total_min,
- adjust->v_total_max,
- adjust->v_total_mid,
- adjust->v_total_mid_frame_num);
+ *adjust);
ret = true;
}
@@ -334,6 +351,88 @@ bool dc_stream_get_crtc_position(struct dc *dc,
return ret;
}
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+bool dc_stream_forward_dmcu_crc_window(struct dc *dc, struct dc_stream_state *stream,
+ struct crc_params *crc_window)
+{
+ int i;
+ struct dmcu *dmcu = dc->res_pool->dmcu;
+ struct pipe_ctx *pipe;
+ struct crc_region tmp_win, *crc_win;
+ struct otg_phy_mux mapping_tmp, *mux_mapping;
+
+ /*crc window can't be null*/
+ if (!crc_window)
+ return false;
+
+ if ((dmcu != NULL && dmcu->funcs->is_dmcu_initialized(dmcu))) {
+ crc_win = &tmp_win;
+ mux_mapping = &mapping_tmp;
+ /*set crc window*/
+ tmp_win.x_start = crc_window->windowa_x_start;
+ tmp_win.y_start = crc_window->windowa_y_start;
+ tmp_win.x_end = crc_window->windowa_x_end;
+ tmp_win.y_end = crc_window->windowa_y_end;
+
+ for (i = 0; i < MAX_PIPES; i++) {
+ pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+ if (pipe->stream == stream && !pipe->top_pipe && !pipe->prev_odm_pipe)
+ break;
+ }
+
+ /* Stream not found */
+ if (i == MAX_PIPES)
+ return false;
+
+
+ /*set mux routing info*/
+ mapping_tmp.phy_output_num = stream->link->link_enc_hw_inst;
+ mapping_tmp.otg_output_num = pipe->stream_res.tg->inst;
+
+ dmcu->funcs->forward_crc_window(dmcu, crc_win, mux_mapping);
+ } else {
+ DC_LOG_DC("dmcu is not initialized");
+ return false;
+ }
+
+ return true;
+}
+
+bool dc_stream_stop_dmcu_crc_win_update(struct dc *dc, struct dc_stream_state *stream)
+{
+ int i;
+ struct dmcu *dmcu = dc->res_pool->dmcu;
+ struct pipe_ctx *pipe;
+ struct otg_phy_mux mapping_tmp, *mux_mapping;
+
+ if ((dmcu != NULL && dmcu->funcs->is_dmcu_initialized(dmcu))) {
+ mux_mapping = &mapping_tmp;
+
+ for (i = 0; i < MAX_PIPES; i++) {
+ pipe = &dc->current_state->res_ctx.pipe_ctx[i];
+ if (pipe->stream == stream && !pipe->top_pipe && !pipe->prev_odm_pipe)
+ break;
+ }
+
+ /* Stream not found */
+ if (i == MAX_PIPES)
+ return false;
+
+
+ /*set mux routing info*/
+ mapping_tmp.phy_output_num = stream->link->link_enc_hw_inst;
+ mapping_tmp.otg_output_num = pipe->stream_res.tg->inst;
+
+ dmcu->funcs->stop_crc_win_update(dmcu, mux_mapping);
+ } else {
+ DC_LOG_DC("dmcu is not initialized");
+ return false;
+ }
+
+ return true;
+}
+#endif
+
/**
* dc_stream_configure_crc() - Configure CRC capture for the given stream.
* @dc: DC Object
@@ -774,6 +873,9 @@ static bool dc_construct(struct dc *dc,
if (!create_links(dc, init_params->num_virtual_links))
goto fail;
+ /* Initialise DIG link encoder resource tracking variables. */
+ link_enc_cfg_init(dc, dc->current_state);
+
return true;
fail:
@@ -970,7 +1072,6 @@ struct dc *dc_create(const struct dc_init_data *init_params)
full_pipe_count,
dc->res_pool->stream_enc_count);
- dc->optimize_seamless_boot_streams = 0;
dc->caps.max_links = dc->link_count;
dc->caps.max_audios = dc->res_pool->audio_count;
dc->caps.linear_pitch_alignment = 64;
@@ -1000,22 +1101,25 @@ destruct_dc:
static void detect_edp_presence(struct dc *dc)
{
- struct dc_link *edp_link = get_edp_link(dc);
- bool edp_sink_present = true;
+ struct dc_link *edp_links[MAX_NUM_EDP];
+ struct dc_link *edp_link = NULL;
+ enum dc_connection_type type;
+ int i;
+ int edp_num;
- if (!edp_link)
+ get_edp_links(dc, edp_links, &edp_num);
+ if (!edp_num)
return;
- if (dc->config.edp_not_connected) {
- edp_sink_present = false;
- } else {
- enum dc_connection_type type;
- dc_link_detect_sink(edp_link, &type);
- if (type == dc_connection_none)
- edp_sink_present = false;
+ for (i = 0; i < edp_num; i++) {
+ edp_link = edp_links[i];
+ if (dc->config.edp_not_connected) {
+ edp_link->edp_sink_present = false;
+ } else {
+ dc_link_detect_sink(edp_link, &type);
+ edp_link->edp_sink_present = (type != dc_connection_none);
+ }
}
-
- edp_link->edp_sink_present = edp_sink_present;
}
void dc_hardware_init(struct dc *dc)
@@ -1091,6 +1195,7 @@ static void program_timing_sync(
for (i = 0; i < pipe_count; i++) {
int group_size = 1;
+ enum timing_synchronization_type sync_type = NOT_SYNCHRONIZABLE;
struct pipe_ctx *pipe_set[MAX_PIPES];
if (!unsynced_pipes[i])
@@ -1105,10 +1210,22 @@ static void program_timing_sync(
for (j = i + 1; j < pipe_count; j++) {
if (!unsynced_pipes[j])
continue;
-
- if (resource_are_streams_timing_synchronizable(
+ if (sync_type != TIMING_SYNCHRONIZABLE &&
+ dc->hwss.enable_vblanks_synchronization &&
+ unsynced_pipes[j]->stream_res.tg->funcs->align_vblanks &&
+ resource_are_vblanks_synchronizable(
+ unsynced_pipes[j]->stream,
+ pipe_set[0]->stream)) {
+ sync_type = VBLANK_SYNCHRONIZABLE;
+ pipe_set[group_size] = unsynced_pipes[j];
+ unsynced_pipes[j] = NULL;
+ group_size++;
+ } else
+ if (sync_type != VBLANK_SYNCHRONIZABLE &&
+ resource_are_streams_timing_synchronizable(
unsynced_pipes[j]->stream,
pipe_set[0]->stream)) {
+ sync_type = TIMING_SYNCHRONIZABLE;
pipe_set[group_size] = unsynced_pipes[j];
unsynced_pipes[j] = NULL;
group_size++;
@@ -1134,7 +1251,6 @@ static void program_timing_sync(
}
}
-
for (k = 0; k < group_size; k++) {
struct dc_stream_status *status = dc_stream_get_status_from_state(ctx, pipe_set[k]->stream);
@@ -1164,8 +1280,14 @@ static void program_timing_sync(
}
if (group_size > 1) {
- dc->hwss.enable_timing_synchronization(
- dc, group_index, group_size, pipe_set);
+ if (sync_type == TIMING_SYNCHRONIZABLE) {
+ dc->hwss.enable_timing_synchronization(
+ dc, group_index, group_size, pipe_set);
+ } else
+ if (sync_type == VBLANK_SYNCHRONIZABLE) {
+ dc->hwss.enable_vblanks_synchronization(
+ dc, group_index, group_size, pipe_set);
+ }
group_index++;
}
num_group++;
@@ -1201,10 +1323,10 @@ bool dc_validate_seamless_boot_timing(const struct dc *dc,
struct dc_link *link = sink->link;
unsigned int i, enc_inst, tg_inst = 0;
- // Seamless port only support single DP and EDP so far
- if (sink->sink_signal != SIGNAL_TYPE_DISPLAY_PORT &&
- sink->sink_signal != SIGNAL_TYPE_EDP)
+ /* Support seamless boot on EDP displays only */
+ if (sink->sink_signal != SIGNAL_TYPE_EDP) {
return false;
+ }
/* Check for enabled DIG to identify enabled display */
if (!link->link_enc->funcs->is_dig_enabled(link->link_enc))
@@ -1277,6 +1399,10 @@ bool dc_validate_seamless_boot_timing(const struct dc *dc,
if (crtc_timing->v_sync_width != hw_crtc_timing.v_sync_width)
return false;
+ /* block DSC for now, as VBIOS does not currently support DSC timings */
+ if (crtc_timing->flags.DSC)
+ return false;
+
if (dc_is_dp_signal(link->connector_signal)) {
unsigned int pix_clk_100hz;
@@ -1307,6 +1433,11 @@ bool dc_validate_seamless_boot_timing(const struct dc *dc,
return false;
}
+ if (is_edp_ilr_optimization_required(link, crtc_timing)) {
+ DC_LOG_EVENT_LINK_TRAINING("Seamless boot disabled to optimize eDP link rate\n");
+ return false;
+ }
+
return true;
}
@@ -1377,11 +1508,7 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c
dc->hwss.enable_accelerated_mode(dc, context);
}
- for (i = 0; i < context->stream_count; i++)
- if (context->streams[i]->apply_seamless_boot_optimization)
- dc->optimize_seamless_boot_streams++;
-
- if (context->stream_count > dc->optimize_seamless_boot_streams ||
+ if (context->stream_count > get_seamless_boot_stream_count(context) ||
context->stream_count == 0)
dc->hwss.prepare_bandwidth(dc, context);
@@ -1464,7 +1591,7 @@ static enum dc_status dc_commit_state_no_check(struct dc *dc, struct dc_state *c
dc_enable_stereo(dc, context, dc_streams, context->stream_count);
- if (context->stream_count > dc->optimize_seamless_boot_streams ||
+ if (context->stream_count > get_seamless_boot_stream_count(context) ||
context->stream_count == 0) {
/* Must wait for no flips to be pending before doing optimize bw */
wait_for_no_pipes_pending(dc, context);
@@ -1578,7 +1705,7 @@ void dc_post_update_surfaces_to_stream(struct dc *dc)
int i;
struct dc_state *context = dc->current_state;
- if ((!dc->optimized_required) || dc->optimize_seamless_boot_streams > 0)
+ if ((!dc->optimized_required) || get_seamless_boot_stream_count(context) > 0)
return;
post_surface_trace(dc);
@@ -1978,6 +2105,10 @@ static enum surface_update_type check_update_surfaces_for_stream(
if (stream_status == NULL || stream_status->plane_count != surface_count)
overall_type = UPDATE_TYPE_FULL;
+ if (stream_update && stream_update->pending_test_pattern) {
+ overall_type = UPDATE_TYPE_FULL;
+ }
+
/* some stream updates require passive update */
if (stream_update) {
union stream_update_flags *su_flags = &stream_update->stream->update_flags;
@@ -2324,7 +2455,6 @@ static void commit_planes_do_stream_update(struct dc *dc,
struct dc_state *context)
{
int j;
- bool should_program_abm;
// Stream updates
for (j = 0; j < dc->res_pool->pipe_count; j++) {
@@ -2375,6 +2505,14 @@ static void commit_planes_do_stream_update(struct dc *dc,
}
}
+
+ /* Full fe update*/
+ if (update_type == UPDATE_TYPE_FAST)
+ continue;
+
+ if (stream_update->dsc_config)
+ dp_update_dsc_config(pipe_ctx);
+
if (stream_update->pending_test_pattern) {
dc_link_dp_set_test_pattern(stream->link,
stream->test_pattern.type,
@@ -2384,13 +2522,6 @@ static void commit_planes_do_stream_update(struct dc *dc,
stream->test_pattern.cust_pattern_size);
}
- /* Full fe update*/
- if (update_type == UPDATE_TYPE_FAST)
- continue;
-
- if (stream_update->dsc_config)
- dp_update_dsc_config(pipe_ctx);
-
if (stream_update->dpms_off) {
if (*stream_update->dpms_off) {
core_link_disable_stream(pipe_ctx);
@@ -2398,9 +2529,10 @@ static void commit_planes_do_stream_update(struct dc *dc,
if (pipe_ctx->stream_res.audio && !dc->debug.az_endpoint_mute_only)
pipe_ctx->stream_res.audio->funcs->az_disable(pipe_ctx->stream_res.audio);
- dc->hwss.optimize_bandwidth(dc, dc->current_state);
+ dc->optimized_required = true;
+
} else {
- if (dc->optimize_seamless_boot_streams == 0)
+ if (get_seamless_boot_stream_count(context) == 0)
dc->hwss.prepare_bandwidth(dc, dc->current_state);
core_link_enable_stream(dc->current_state, pipe_ctx);
@@ -2408,7 +2540,7 @@ static void commit_planes_do_stream_update(struct dc *dc,
}
if (stream_update->abm_level && pipe_ctx->stream_res.abm) {
- should_program_abm = true;
+ bool should_program_abm = true;
// if otg funcs defined check if blanked before programming
if (pipe_ctx->stream_res.tg->funcs->is_blanked)
@@ -2439,7 +2571,7 @@ static void commit_planes_for_stream(struct dc *dc,
int i, j;
struct pipe_ctx *top_pipe_to_program = NULL;
- if (dc->optimize_seamless_boot_streams > 0 && surface_count > 0) {
+ if (get_seamless_boot_stream_count(context) > 0 && surface_count > 0) {
/* Optimize seamless boot flag keeps clocks and watermarks high until
* first flip. After first flip, optimization is required to lower
* bandwidth. Important to note that it is expected UEFI will
@@ -2448,9 +2580,8 @@ static void commit_planes_for_stream(struct dc *dc,
*/
if (stream->apply_seamless_boot_optimization) {
stream->apply_seamless_boot_optimization = false;
- dc->optimize_seamless_boot_streams--;
- if (dc->optimize_seamless_boot_streams == 0)
+ if (get_seamless_boot_stream_count(context) == 0)
dc->optimized_required = true;
}
}
@@ -2460,7 +2591,7 @@ static void commit_planes_for_stream(struct dc *dc,
dc_allow_idle_optimizations(dc, false);
#endif
- if (dc->optimize_seamless_boot_streams == 0)
+ if (get_seamless_boot_stream_count(context) == 0)
dc->hwss.prepare_bandwidth(dc, context);
context_clock_trace(dc, context);
@@ -2477,6 +2608,17 @@ static void commit_planes_for_stream(struct dc *dc,
}
}
+#ifdef CONFIG_DRM_AMD_DC_DCN
+ if (stream->test_pattern.type != DP_TEST_PATTERN_VIDEO_MODE) {
+ struct pipe_ctx *mpcc_pipe;
+ struct pipe_ctx *odm_pipe;
+
+ for (mpcc_pipe = top_pipe_to_program; mpcc_pipe; mpcc_pipe = mpcc_pipe->bottom_pipe)
+ for (odm_pipe = mpcc_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe)
+ odm_pipe->ttu_regs.min_ttu_vblank = MAX_TTU;
+ }
+#endif
+
if ((update_type != UPDATE_TYPE_FAST) && stream->update_flags.bits.dsc_changed)
if (top_pipe_to_program->stream_res.tg->funcs->lock_doublebuffer_enable) {
if (should_use_dmub_lock(stream->link)) {
@@ -2545,6 +2687,10 @@ static void commit_planes_for_stream(struct dc *dc,
plane_state->triplebuffer_flips = true;
}
}
+ if (update_type == UPDATE_TYPE_FULL) {
+ /* force vsync flip when reconfiguring pipes to prevent underflow */
+ plane_state->flip_immediate = false;
+ }
}
}
@@ -2683,9 +2829,13 @@ static void commit_planes_for_stream(struct dc *dc,
for (j = 0; j < dc->res_pool->pipe_count; j++) {
struct pipe_ctx *pipe_ctx = &context->res_ctx.pipe_ctx[j];
+ if (!pipe_ctx->plane_state)
+ continue;
+
if (pipe_ctx->bottom_pipe || pipe_ctx->next_odm_pipe ||
!pipe_ctx->stream || pipe_ctx->stream != stream ||
- !pipe_ctx->plane_state->update_flags.bits.addr_update)
+ !pipe_ctx->plane_state->update_flags.bits.addr_update ||
+ pipe_ctx->plane_state->skip_manual_trigger)
continue;
if (pipe_ctx->stream_res.tg->funcs->program_manual_trigger)
@@ -2868,6 +3018,9 @@ void dc_set_power_state(
struct kref refcount;
struct display_mode_lib *dml;
+ if (!dc->current_state)
+ return;
+
switch (power_state) {
case DC_ACPI_CM_POWER_STATE_D0:
dc_resource_state_construct(dc, dc->current_state);
@@ -3066,6 +3219,19 @@ void dc_link_remove_remote_sink(struct dc_link *link, struct dc_sink *sink)
}
}
+void dc_wait_for_vblank(struct dc *dc, struct dc_stream_state *stream)
+{
+ int i;
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++)
+ if (dc->current_state->res_ctx.pipe_ctx[i].stream == stream) {
+ struct timing_generator *tg =
+ dc->current_state->res_ctx.pipe_ctx[i].stream_res.tg;
+ tg->funcs->wait_for_state(tg, CRTC_STATE_VBLANK);
+ break;
+ }
+}
+
void get_clock_requirements_for_state(struct dc_state *state, struct AsicStateEx *info)
{
info->displayClock = (unsigned int)state->bw_ctx.bw.dcn.clk.dispclk_khz;
@@ -3121,6 +3287,10 @@ void dc_allow_idle_optimizations(struct dc *dc, bool allow)
if (dc->debug.disable_idle_power_optimizations)
return;
+ if (dc->clk_mgr->funcs->is_smu_present)
+ if (!dc->clk_mgr->funcs->is_smu_present(dc->clk_mgr))
+ return;
+
if (allow == dc->idle_optimizations_allowed)
return;
@@ -3176,3 +3346,113 @@ void dc_hardware_release(struct dc *dc)
dc->hwss.hardware_release(dc);
}
#endif
+
+/**
+ *****************************************************************************
+ * Function: dc_enable_dmub_notifications
+ *
+ * @brief
+ * Returns whether dmub notification can be enabled
+ *
+ * @param
+ * [in] dc: dc structure
+ *
+ * @return
+ * True to enable dmub notifications, False otherwise
+ *****************************************************************************
+ */
+bool dc_enable_dmub_notifications(struct dc *dc)
+{
+ /* dmub aux needs dmub notifications to be enabled */
+ return dc->debug.enable_dmub_aux_for_legacy_ddc;
+}
+
+/**
+ *****************************************************************************
+ * Function: dc_process_dmub_aux_transfer_async
+ *
+ * @brief
+ * Submits aux command to dmub via inbox message
+ * Sets port index appropriately for legacy DDC
+ *
+ * @param
+ * [in] dc: dc structure
+ * [in] link_index: link index
+ * [in] payload: aux payload
+ *
+ * @return
+ * True if successful, False if failure
+ *****************************************************************************
+ */
+bool dc_process_dmub_aux_transfer_async(struct dc *dc,
+ uint32_t link_index,
+ struct aux_payload *payload)
+{
+ uint8_t action;
+ union dmub_rb_cmd cmd = {0};
+ struct dc_dmub_srv *dmub_srv = dc->ctx->dmub_srv;
+
+ ASSERT(payload->length <= 16);
+
+ cmd.dp_aux_access.header.type = DMUB_CMD__DP_AUX_ACCESS;
+ cmd.dp_aux_access.header.payload_bytes = 0;
+ cmd.dp_aux_access.aux_control.type = AUX_CHANNEL_LEGACY_DDC;
+ cmd.dp_aux_access.aux_control.instance = dc->links[link_index]->ddc_hw_inst;
+ cmd.dp_aux_access.aux_control.sw_crc_enabled = 0;
+ cmd.dp_aux_access.aux_control.timeout = 0;
+ cmd.dp_aux_access.aux_control.dpaux.address = payload->address;
+ cmd.dp_aux_access.aux_control.dpaux.is_i2c_over_aux = payload->i2c_over_aux;
+ cmd.dp_aux_access.aux_control.dpaux.length = payload->length;
+
+ /* set aux action */
+ if (payload->i2c_over_aux) {
+ if (payload->write) {
+ if (payload->mot)
+ action = DP_AUX_REQ_ACTION_I2C_WRITE_MOT;
+ else
+ action = DP_AUX_REQ_ACTION_I2C_WRITE;
+ } else {
+ if (payload->mot)
+ action = DP_AUX_REQ_ACTION_I2C_READ_MOT;
+ else
+ action = DP_AUX_REQ_ACTION_I2C_READ;
+ }
+ } else {
+ if (payload->write)
+ action = DP_AUX_REQ_ACTION_DPCD_WRITE;
+ else
+ action = DP_AUX_REQ_ACTION_DPCD_READ;
+ }
+
+ cmd.dp_aux_access.aux_control.dpaux.action = action;
+
+ if (payload->length && payload->write) {
+ memcpy(cmd.dp_aux_access.aux_control.dpaux.data,
+ payload->data,
+ payload->length
+ );
+ }
+
+ dc_dmub_srv_cmd_queue(dmub_srv, &cmd);
+ dc_dmub_srv_cmd_execute(dmub_srv);
+ dc_dmub_srv_wait_idle(dmub_srv);
+
+ return true;
+}
+
+/**
+ *****************************************************************************
+ * Function: dc_disable_accelerated_mode
+ *
+ * @brief
+ * disable accelerated mode
+ *
+ * @param
+ * [in] dc: dc structure
+ *
+ *****************************************************************************
+ */
+void dc_disable_accelerated_mode(struct dc *dc)
+{
+ bios_set_scratch_acc_mode_change(dc->ctx->dc_bios, 0);
+}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link.c b/drivers/gpu/drm/amd/display/dc/core/dc_link.c
index fa5059f71727..f4374d83662a 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link.c
@@ -91,8 +91,17 @@ static void dc_link_destruct(struct dc_link *link)
if (link->panel_cntl)
link->panel_cntl->funcs->destroy(&link->panel_cntl);
- if (link->link_enc)
+ if (link->link_enc) {
+ /* Update link encoder resource tracking variables. These are used for
+ * the dynamic assignment of link encoders to streams. Virtual links
+ * are not assigned encoder resources on creation.
+ */
+ if (link->link_id.id != CONNECTOR_ID_VIRTUAL) {
+ link->dc->res_pool->link_encoders[link->eng_id - ENGINE_ID_DIGA] = NULL;
+ link->dc->res_pool->dig_link_enc_count--;
+ }
link->link_enc->funcs->destroy(&link->link_enc);
+ }
if (link->local_sink)
dc_sink_release(link->local_sink);
@@ -1401,6 +1410,8 @@ static bool dc_link_construct(struct dc_link *link,
link->link_id =
bios->funcs->get_connector_id(bios, init_params->connector_index);
+ link->ep_type = DISPLAY_ENDPOINT_PHY;
+
DC_LOG_DC("BIOS object table - link_id: %d", link->link_id.id);
if (bios->funcs->get_disp_connector_caps_info) {
@@ -1500,10 +1511,12 @@ static bool dc_link_construct(struct dc_link *link,
(link->link_id.id == CONNECTOR_ID_EDP ||
link->link_id.id == CONNECTOR_ID_LVDS)) {
panel_cntl_init_data.ctx = dc_ctx;
- panel_cntl_init_data.inst = 0;
+ panel_cntl_init_data.inst =
+ panel_cntl_init_data.ctx->dc_edp_id_count;
link->panel_cntl =
link->dc->res_pool->funcs->panel_cntl_create(
&panel_cntl_init_data);
+ panel_cntl_init_data.ctx->dc_edp_id_count++;
if (link->panel_cntl == NULL) {
DC_ERROR("Failed to create link panel_cntl!\n");
@@ -1532,6 +1545,13 @@ static bool dc_link_construct(struct dc_link *link,
DC_LOG_DC("BIOS object table - DP_IS_USB_C: %d", link->link_enc->features.flags.bits.DP_IS_USB_C);
+ /* Update link encoder tracking variables. These are used for the dynamic
+ * assignment of link encoders to streams.
+ */
+ link->eng_id = link->link_enc->preferred_engine;
+ link->dc->res_pool->link_encoders[link->eng_id - ENGINE_ID_DIGA] = link->link_enc;
+ link->dc->res_pool->dig_link_enc_count++;
+
link->link_enc_hw_inst = link->link_enc->transmitter;
for (i = 0; i < 4; i++) {
@@ -1603,6 +1623,7 @@ static bool dc_link_construct(struct dc_link *link,
link->psr_settings.psr_version = DC_PSR_VERSION_UNSUPPORTED;
DC_LOG_DC("BIOS object table - %s finished successfully.\n", __func__);
+ kfree(info);
return true;
device_tag_fail:
link->link_enc->funcs->destroy(&link->link_enc);
@@ -1658,21 +1679,27 @@ void link_destroy(struct dc_link **link)
static void enable_stream_features(struct pipe_ctx *pipe_ctx)
{
struct dc_stream_state *stream = pipe_ctx->stream;
- struct dc_link *link = stream->link;
- union down_spread_ctrl old_downspread;
- union down_spread_ctrl new_downspread;
- core_link_read_dpcd(link, DP_DOWNSPREAD_CTRL,
- &old_downspread.raw, sizeof(old_downspread));
+ if (pipe_ctx->stream->signal != SIGNAL_TYPE_DISPLAY_PORT_MST) {
+ struct dc_link *link = stream->link;
+ union down_spread_ctrl old_downspread;
+ union down_spread_ctrl new_downspread;
+
+ core_link_read_dpcd(link, DP_DOWNSPREAD_CTRL,
+ &old_downspread.raw, sizeof(old_downspread));
- new_downspread.raw = old_downspread.raw;
+ new_downspread.raw = old_downspread.raw;
- new_downspread.bits.IGNORE_MSA_TIMING_PARAM =
- (stream->ignore_msa_timing_param) ? 1 : 0;
+ new_downspread.bits.IGNORE_MSA_TIMING_PARAM =
+ (stream->ignore_msa_timing_param) ? 1 : 0;
- if (new_downspread.raw != old_downspread.raw) {
- core_link_write_dpcd(link, DP_DOWNSPREAD_CTRL,
- &new_downspread.raw, sizeof(new_downspread));
+ if (new_downspread.raw != old_downspread.raw) {
+ core_link_write_dpcd(link, DP_DOWNSPREAD_CTRL,
+ &new_downspread.raw, sizeof(new_downspread));
+ }
+
+ } else {
+ dm_helpers_mst_enable_stream_features(stream);
}
}
@@ -2602,7 +2629,6 @@ bool dc_link_set_backlight_level(const struct dc_link *link,
if (pipe_ctx->plane_state == NULL)
frame_ramp = 0;
} else {
- ASSERT(false);
return false;
}
@@ -2793,12 +2819,9 @@ bool dc_link_setup_psr(struct dc_link *link,
psr_context->psr_level.u32all = 0;
-#if defined(CONFIG_DRM_AMD_DC_DCN)
/*skip power down the single pipe since it blocks the cstate*/
- if ((link->ctx->asic_id.chip_family == FAMILY_RV) &&
- ASICREV_IS_RAVEN(link->ctx->asic_id.hw_internal_rev))
+ if (link->ctx->asic_id.chip_family >= FAMILY_RV)
psr_context->psr_level.bits.SKIP_CRTC_DISABLE = true;
-#endif
/* SMU will perform additional powerdown sequence.
* For unsupported ASICs, set psr_level flag to skip PSR
@@ -2871,8 +2894,8 @@ static struct fixed31_32 get_pbn_per_slot(struct dc_stream_state *stream)
static struct fixed31_32 get_pbn_from_bw_in_kbps(uint64_t kbps)
{
struct fixed31_32 peak_kbps;
- uint32_t numerator;
- uint32_t denominator;
+ uint32_t numerator = 0;
+ uint32_t denominator = 1;
/*
* margin 5300ppm + 300ppm ~ 0.6% as per spec, factor is 1.006
@@ -3119,50 +3142,6 @@ static enum dc_status deallocate_mst_payload(struct pipe_ctx *pipe_ctx)
return DC_OK;
}
-enum dc_status dc_link_reallocate_mst_payload(struct dc_link *link)
-{
- int i;
- struct pipe_ctx *pipe_ctx;
-
- // Clear all of MST payload then reallocate
- for (i = 0; i < MAX_PIPES; i++) {
- pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i];
-
- /* driver enable split pipe for external monitors
- * we have to check pipe_ctx is split pipe or not
- * If it's split pipe, driver using top pipe to
- * reaallocate.
- */
- if (!pipe_ctx || pipe_ctx->top_pipe)
- continue;
-
- if (pipe_ctx->stream && pipe_ctx->stream->link == link &&
- pipe_ctx->stream->dpms_off == false &&
- pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) {
- deallocate_mst_payload(pipe_ctx);
- }
- }
-
- for (i = 0; i < MAX_PIPES; i++) {
- pipe_ctx = &link->dc->current_state->res_ctx.pipe_ctx[i];
-
- if (!pipe_ctx || pipe_ctx->top_pipe)
- continue;
-
- if (pipe_ctx->stream && pipe_ctx->stream->link == link &&
- pipe_ctx->stream->dpms_off == false &&
- pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST) {
- /* enable/disable PHY will clear connection between BE and FE
- * need to restore it.
- */
- link->link_enc->funcs->connect_dig_be_to_fe(link->link_enc,
- pipe_ctx->stream_res.stream_enc->id, true);
- dc_link_allocate_mst_payload(pipe_ctx);
- }
- }
-
- return DC_OK;
-}
#if defined(CONFIG_DRM_AMD_DC_HDCP)
static void update_psp_stream_config(struct pipe_ctx *pipe_ctx, bool dpms_off)
@@ -3258,6 +3237,16 @@ void core_link_enable_stream(
/* Do not touch link on seamless boot optimization. */
if (pipe_ctx->stream->apply_seamless_boot_optimization) {
pipe_ctx->stream->dpms_off = false;
+
+ /* Still enable stream features & audio on seamless boot for DP external displays */
+ if (pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT) {
+ enable_stream_features(pipe_ctx);
+ if (pipe_ctx->stream_res.audio != NULL) {
+ pipe_ctx->stream_res.stream_enc->funcs->dp_audio_enable(pipe_ctx->stream_res.stream_enc);
+ dc->hwss.enable_audio_stream(pipe_ctx);
+ }
+ }
+
#if defined(CONFIG_DRM_AMD_DC_HDCP)
update_psp_stream_config(pipe_ctx, false);
#endif
@@ -3266,7 +3255,8 @@ void core_link_enable_stream(
/* eDP lit up by bios already, no need to enable again. */
if (pipe_ctx->stream->signal == SIGNAL_TYPE_EDP &&
- apply_edp_fast_boot_optimization) {
+ apply_edp_fast_boot_optimization &&
+ !pipe_ctx->stream->timing.flags.DSC) {
pipe_ctx->stream->dpms_off = false;
#if defined(CONFIG_DRM_AMD_DC_HDCP)
update_psp_stream_config(pipe_ctx, false);
@@ -3328,8 +3318,10 @@ void core_link_enable_stream(
/* Set DPS PPS SDP (AKA "info frames") */
if (pipe_ctx->stream->timing.flags.DSC) {
if (dc_is_dp_signal(pipe_ctx->stream->signal) ||
- dc_is_virtual_signal(pipe_ctx->stream->signal))
+ dc_is_virtual_signal(pipe_ctx->stream->signal)) {
+ dp_set_dsc_on_rx(pipe_ctx, true);
dp_set_dsc_pps_sdp(pipe_ctx, true);
+ }
}
if (pipe_ctx->stream->signal == SIGNAL_TYPE_DISPLAY_PORT_MST)
@@ -3481,15 +3473,12 @@ uint32_t dc_bandwidth_in_kbps_from_timing(
{
uint32_t bits_per_channel = 0;
uint32_t kbps;
- struct fixed31_32 link_bw_kbps;
+#if defined(CONFIG_DRM_AMD_DC_DCN)
if (timing->flags.DSC) {
- link_bw_kbps = dc_fixpt_from_int(timing->pix_clk_100hz);
- link_bw_kbps = dc_fixpt_div_int(link_bw_kbps, 160);
- link_bw_kbps = dc_fixpt_mul_int(link_bw_kbps, timing->dsc_cfg.bits_per_pixel);
- kbps = dc_fixpt_ceil(link_bw_kbps);
- return kbps;
+ return dc_dsc_stream_bandwidth_in_kbps(timing->pix_clk_100hz, timing->dsc_cfg.bits_per_pixel);
}
+#endif
switch (timing->display_color_depth) {
case COLOR_DEPTH_666:
@@ -3727,7 +3716,8 @@ bool dc_link_should_enable_fec(const struct dc_link *link)
if ((link->connector_signal != SIGNAL_TYPE_DISPLAY_PORT_MST &&
link->local_sink &&
link->local_sink->edid_caps.panel_patch.disable_fec) ||
- link->connector_signal == SIGNAL_TYPE_EDP) // Disable FEC for eDP
+ (link->connector_signal == SIGNAL_TYPE_EDP &&
+ link->dc->debug.force_enable_edp_fec == false)) // Disable FEC for eDP
is_fec_disable = true;
if (dc_link_is_fec_supported(link) && !link->dc->debug.disable_fec && !is_fec_disable)
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
index ae6484ab567b..64414c51312d 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_ddc.c
@@ -36,6 +36,7 @@
#include "core_types.h"
#include "dc_link_ddc.h"
#include "dce/dce_aux.h"
+#include "dmub/inc/dmub_cmd.h"
#define DC_LOGGER_INIT(logger)
@@ -558,7 +559,7 @@ bool dal_ddc_service_query_ddc_data(
/* should not set mot (middle of transaction) to 0
* if there are pending read payloads
*/
- payload.mot = read_size == 0 ? false : true;
+ payload.mot = !(read_size == 0);
payload.length = write_size;
payload.data = write_buf;
@@ -655,7 +656,7 @@ bool dal_ddc_submit_aux_command(struct ddc_service *ddc,
*/
int dc_link_aux_transfer_raw(struct ddc_service *ddc,
struct aux_payload *payload,
- enum aux_channel_operation_result *operation_result)
+ enum aux_return_code_type *operation_result)
{
return dce_aux_transfer_raw(ddc, payload, operation_result);
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
index c1391bfb7a9b..3ff3d9e90983 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_dp.c
@@ -284,7 +284,7 @@ static uint8_t dc_dp_initialize_scrambling_data_symbols(
static inline bool is_repeater(struct dc_link *link, uint32_t offset)
{
- return (link->lttpr_non_transparent_mode && offset != 0);
+ return (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) && (offset != 0);
}
static void dpcd_set_lt_pattern_and_lane_settings(
@@ -1072,7 +1072,7 @@ static enum link_training_result perform_clock_recovery_sequence(
/* 3. wait receiver to lock-on*/
wait_time_microsec = lt_settings->cr_pattern_time;
- if (link->lttpr_non_transparent_mode)
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT)
wait_time_microsec = TRAINING_AUX_RD_INTERVAL;
wait_for_training_aux_rd_interval(
@@ -1098,11 +1098,13 @@ static enum link_training_result perform_clock_recovery_sequence(
if (is_max_vs_reached(lt_settings))
break;
- /* 7. same voltage*/
- /* Note: VS same for all lanes,
- * so comparing first lane is sufficient*/
- if (lt_settings->lane_settings[0].VOLTAGE_SWING ==
+ /* 7. same lane settings*/
+ /* Note: settings are the same for all lanes,
+ * so comparing first lane is sufficient*/
+ if ((lt_settings->lane_settings[0].VOLTAGE_SWING ==
req_settings.lane_settings[0].VOLTAGE_SWING)
+ && (lt_settings->lane_settings[0].PRE_EMPHASIS ==
+ req_settings.lane_settings[0].PRE_EMPHASIS))
retries_cr++;
else
retries_cr = 0;
@@ -1130,11 +1132,6 @@ static inline enum link_training_result perform_link_training_int(
enum link_training_result status)
{
union lane_count_set lane_count_set = { {0} };
- union dpcd_training_pattern dpcd_pattern = { {0} };
-
- /* 3. set training not in progress*/
- dpcd_pattern.v1_4.TRAINING_PATTERN_SET = DPCD_TRAINING_PATTERN_VIDEOIDLE;
- dpcd_set_training_pattern(link, dpcd_pattern);
/* 4. mainlink output idle pattern*/
dp_set_hw_test_pattern(link, DP_TEST_PATTERN_VIDEO_MODE, NULL, 0);
@@ -1324,7 +1321,17 @@ static uint8_t convert_to_count(uint8_t lttpr_repeater_count)
return 0; // invalid value
}
-static void configure_lttpr_mode(struct dc_link *link)
+static void configure_lttpr_mode_transparent(struct dc_link *link)
+{
+ uint8_t repeater_mode = DP_PHY_REPEATER_MODE_TRANSPARENT;
+
+ core_link_write_dpcd(link,
+ DP_PHY_REPEATER_MODE,
+ (uint8_t *)&repeater_mode,
+ sizeof(repeater_mode));
+}
+
+static void configure_lttpr_mode_non_transparent(struct dc_link *link)
{
/* aux timeout is already set to extended */
/* RESET/SET lttpr mode to enable non transparent mode */
@@ -1344,7 +1351,7 @@ static void configure_lttpr_mode(struct dc_link *link)
link->dpcd_caps.lttpr_caps.mode = repeater_mode;
}
- if (link->lttpr_non_transparent_mode) {
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) {
DC_LOG_HW_LINK_TRAINING("%s\n Set LTTPR to Non Transparent Mode\n", __func__);
@@ -1548,6 +1555,7 @@ enum link_training_result dc_link_dp_perform_link_training(
{
enum link_training_result status = LINK_TRAINING_SUCCESS;
struct link_training_settings lt_settings;
+ union dpcd_training_pattern dpcd_pattern = { { 0 } };
bool fec_enable;
uint8_t repeater_cnt;
@@ -1560,8 +1568,10 @@ enum link_training_result dc_link_dp_perform_link_training(
&lt_settings);
/* Configure lttpr mode */
- if (link->lttpr_non_transparent_mode)
- configure_lttpr_mode(link);
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT)
+ configure_lttpr_mode_non_transparent(link);
+ else if (link->lttpr_mode == LTTPR_MODE_TRANSPARENT)
+ configure_lttpr_mode_transparent(link);
if (link->ctx->dc->work_arounds.lt_early_cr_pattern)
start_clock_recovery_pattern_early(link, &lt_settings, DPRX);
@@ -1576,7 +1586,7 @@ enum link_training_result dc_link_dp_perform_link_training(
dp_set_fec_ready(link, fec_enable);
- if (link->lttpr_non_transparent_mode) {
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) {
/* 2. perform link training (set link training done
* to false is done as well)
@@ -1610,6 +1620,9 @@ enum link_training_result dc_link_dp_perform_link_training(
}
}
+ /* 3. set training not in progress*/
+ dpcd_pattern.v1_4.TRAINING_PATTERN_SET = DPCD_TRAINING_PATTERN_VIDEOIDLE;
+ dpcd_set_training_pattern(link, dpcd_pattern);
if ((status == LINK_TRAINING_SUCCESS) || !skip_video_pattern) {
status = perform_link_training_int(link,
&lt_settings,
@@ -1633,6 +1646,42 @@ enum link_training_result dc_link_dp_perform_link_training(
return status;
}
+static enum dp_panel_mode try_enable_assr(struct dc_stream_state *stream)
+{
+ struct dc_link *link = stream->link;
+ enum dp_panel_mode panel_mode = dp_get_panel_mode(link);
+#ifdef CONFIG_DRM_AMD_DC_HDCP
+ struct cp_psp *cp_psp = &stream->ctx->cp_psp;
+#endif
+
+ /* ASSR must be supported on the panel */
+ if (panel_mode == DP_PANEL_MODE_DEFAULT)
+ return panel_mode;
+
+ /* eDP or internal DP only */
+ if (link->connector_signal != SIGNAL_TYPE_EDP &&
+ !(link->connector_signal == SIGNAL_TYPE_DISPLAY_PORT &&
+ link->is_internal_display))
+ return DP_PANEL_MODE_DEFAULT;
+
+#ifdef CONFIG_DRM_AMD_DC_HDCP
+ if (cp_psp && cp_psp->funcs.enable_assr) {
+ if (!cp_psp->funcs.enable_assr(cp_psp->handle, link)) {
+ /* since eDP implies ASSR on, change panel
+ * mode to disable ASSR
+ */
+ panel_mode = DP_PANEL_MODE_DEFAULT;
+ }
+ } else
+ panel_mode = DP_PANEL_MODE_DEFAULT;
+
+#else
+ /* turn off ASSR if the implementation is not compiled in */
+ panel_mode = DP_PANEL_MODE_DEFAULT;
+#endif
+ return panel_mode;
+}
+
bool perform_link_training_with_retries(
const struct dc_link_settings *link_setting,
bool skip_video_pattern,
@@ -1644,7 +1693,7 @@ bool perform_link_training_with_retries(
uint8_t delay_between_attempts = LINK_TRAINING_RETRY_DELAY;
struct dc_stream_state *stream = pipe_ctx->stream;
struct dc_link *link = stream->link;
- enum dp_panel_mode panel_mode = dp_get_panel_mode(link);
+ enum dp_panel_mode panel_mode;
/* We need to do this before the link training to ensure the idle pattern in SST
* mode will be sent right after the link training
@@ -1669,16 +1718,25 @@ bool perform_link_training_with_retries(
msleep(delay_dp_power_up_in_ms);
}
+ panel_mode = try_enable_assr(stream);
dp_set_panel_mode(link, panel_mode);
+ DC_LOG_DETECTION_DP_CAPS("Link: %d ASSR enabled: %d\n",
+ link->link_index,
+ panel_mode != DP_PANEL_MODE_DEFAULT);
if (link->aux_access_disabled) {
dc_link_dp_perform_link_training_skip_aux(link, link_setting);
return true;
- } else if (dc_link_dp_perform_link_training(
- link,
- link_setting,
- skip_video_pattern) == LINK_TRAINING_SUCCESS)
- return true;
+ } else {
+ enum link_training_result status = LINK_TRAINING_CR_FAIL_LANE0;
+
+ status = dc_link_dp_perform_link_training(
+ link,
+ link_setting,
+ skip_video_pattern);
+ if (status == LINK_TRAINING_SUCCESS)
+ return true;
+ }
/* latest link training still fail, skip delay and keep PHY on
*/
@@ -1857,7 +1915,7 @@ static struct dc_link_settings get_max_link_cap(struct dc_link *link)
* account for lttpr repeaters cap
* notes: repeaters do not snoop in the DPRX Capabilities addresses (3.6.3).
*/
- if (link->lttpr_non_transparent_mode) {
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) {
if (link->dpcd_caps.lttpr_caps.max_lane_count < max_link_cap.lane_count)
max_link_cap.lane_count = link->dpcd_caps.lttpr_caps.max_lane_count;
@@ -2015,7 +2073,7 @@ bool dp_verify_link_cap(
max_link_cap = get_max_link_cap(link);
/* Grant extended timeout request */
- if (link->lttpr_non_transparent_mode && link->dpcd_caps.lttpr_caps.max_ext_timeout > 0) {
+ if ((link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) && (link->dpcd_caps.lttpr_caps.max_ext_timeout > 0)) {
uint8_t grant = link->dpcd_caps.lttpr_caps.max_ext_timeout & 0x80;
core_link_write_dpcd(link, DP_PHY_REPEATER_EXTENDED_WAIT_TIMEOUT, &grant, sizeof(grant));
@@ -2431,7 +2489,7 @@ static bool decide_dp_link_settings(struct dc_link *link, struct dc_link_setting
return false;
}
-static bool decide_edp_link_settings(struct dc_link *link, struct dc_link_settings *link_setting, uint32_t req_bw)
+bool decide_edp_link_settings(struct dc_link *link, struct dc_link_settings *link_setting, uint32_t req_bw)
{
struct dc_link_settings initial_link_setting;
struct dc_link_settings current_link_setting;
@@ -2766,10 +2824,27 @@ static void dp_test_send_link_test_pattern(struct dc_link *link)
enum dp_test_pattern test_pattern;
enum dp_test_pattern_color_space test_pattern_color_space =
DP_TEST_PATTERN_COLOR_SPACE_UNDEFINED;
+ enum dc_color_depth requestColorDepth = COLOR_DEPTH_UNDEFINED;
+ struct pipe_ctx *pipes = link->dc->current_state->res_ctx.pipe_ctx;
+ struct pipe_ctx *pipe_ctx = NULL;
+ int i;
memset(&dpcd_test_pattern, 0, sizeof(dpcd_test_pattern));
memset(&dpcd_test_params, 0, sizeof(dpcd_test_params));
+ for (i = 0; i < MAX_PIPES; i++) {
+ if (pipes[i].stream == NULL)
+ continue;
+
+ if (pipes[i].stream->link == link && !pipes[i].top_pipe && !pipes[i].prev_odm_pipe) {
+ pipe_ctx = &pipes[i];
+ break;
+ }
+ }
+
+ if (pipe_ctx == NULL)
+ return;
+
/* get link test pattern and pattern parameters */
core_link_read_dpcd(
link,
@@ -2807,6 +2882,33 @@ static void dp_test_send_link_test_pattern(struct dc_link *link)
DP_TEST_PATTERN_COLOR_SPACE_YCBCR709 :
DP_TEST_PATTERN_COLOR_SPACE_YCBCR601;
+ switch (dpcd_test_params.bits.BPC) {
+ case 0: // 6 bits
+ requestColorDepth = COLOR_DEPTH_666;
+ break;
+ case 1: // 8 bits
+ requestColorDepth = COLOR_DEPTH_888;
+ break;
+ case 2: // 10 bits
+ requestColorDepth = COLOR_DEPTH_101010;
+ break;
+ case 3: // 12 bits
+ requestColorDepth = COLOR_DEPTH_121212;
+ break;
+ default:
+ break;
+ }
+
+ if (requestColorDepth != COLOR_DEPTH_UNDEFINED
+ && pipe_ctx->stream->timing.display_color_depth != requestColorDepth) {
+ DC_LOG_DEBUG("%s: original bpc %d, changing to %d\n",
+ __func__,
+ pipe_ctx->stream->timing.display_color_depth,
+ requestColorDepth);
+ pipe_ctx->stream->timing.display_color_depth = requestColorDepth;
+ dp_update_dsc_config(pipe_ctx);
+ }
+
dc_link_dp_set_test_pattern(
link,
test_pattern,
@@ -3353,6 +3455,9 @@ static bool retrieve_link_cap(struct dc_link *link)
struct dp_sink_hw_fw_revision dp_hw_fw_revision;
bool is_lttpr_present = false;
const uint32_t post_oui_delay = 30; // 30ms
+ bool vbios_lttpr_enable = false;
+ bool vbios_lttpr_interop = false;
+ struct dc_bios *bios = link->dc->ctx->dc_bios;
memset(dpcd_data, '\0', sizeof(dpcd_data));
memset(lttpr_dpcd_data, '\0', sizeof(lttpr_dpcd_data));
@@ -3400,13 +3505,45 @@ static bool retrieve_link_cap(struct dc_link *link)
return false;
}
- if (link->dc->caps.extended_aux_timeout_support &&
- link->dc->config.allow_lttpr_non_transparent_mode) {
+ /* Query BIOS to determine if LTTPR functionality is forced on by system */
+ if (bios->funcs->get_lttpr_caps) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_lttpr_enable = 0;
+
+ bp_query_result = bios->funcs->get_lttpr_caps(bios, &is_vbios_lttpr_enable);
+ vbios_lttpr_enable = (bp_query_result == BP_RESULT_OK) && !!is_vbios_lttpr_enable;
+ }
+
+ if (bios->funcs->get_lttpr_interop) {
+ enum bp_result bp_query_result;
+ uint8_t is_vbios_interop_enabled = 0;
+
+ bp_query_result = bios->funcs->get_lttpr_interop(bios, &is_vbios_interop_enabled);
+ vbios_lttpr_interop = (bp_query_result == BP_RESULT_OK) && !!is_vbios_interop_enabled;
+ }
+
+ /*
+ * Logic to determine LTTPR mode
+ */
+ link->lttpr_mode = LTTPR_MODE_NON_LTTPR;
+ if (vbios_lttpr_enable && vbios_lttpr_interop)
+ link->lttpr_mode = LTTPR_MODE_NON_TRANSPARENT;
+ else if (!vbios_lttpr_enable && vbios_lttpr_interop) {
+ if (link->dc->config.allow_lttpr_non_transparent_mode)
+ link->lttpr_mode = LTTPR_MODE_NON_TRANSPARENT;
+ else
+ link->lttpr_mode = LTTPR_MODE_TRANSPARENT;
+ } else if (!vbios_lttpr_enable && !vbios_lttpr_interop) {
+ if (!link->dc->config.allow_lttpr_non_transparent_mode
+ || !link->dc->caps.extended_aux_timeout_support)
+ link->lttpr_mode = LTTPR_MODE_NON_LTTPR;
+ else
+ link->lttpr_mode = LTTPR_MODE_NON_TRANSPARENT;
+ }
+
+ if (link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT || link->lttpr_mode == LTTPR_MODE_TRANSPARENT) {
/* By reading LTTPR capability, RX assumes that we will enable
- * LTTPR non transparent if LTTPR is present.
- * Therefore, only query LTTPR capability when both LTTPR
- * extended aux timeout and
- * non transparent mode is supported by hardware
+ * LTTPR extended aux timeout if LTTPR is present.
*/
status = core_link_read_dpcd(
link,
@@ -3444,11 +3581,10 @@ static bool retrieve_link_cap(struct dc_link *link)
link->dpcd_caps.lttpr_caps.revision.raw >= 0x14);
if (is_lttpr_present)
CONN_DATA_DETECT(link, lttpr_dpcd_data, sizeof(lttpr_dpcd_data), "LTTPR Caps: ");
+ else
+ link->lttpr_mode = LTTPR_MODE_NON_LTTPR;
}
- /* decide lttpr non transparent mode */
- link->lttpr_non_transparent_mode = is_lttpr_present;
-
if (!is_lttpr_present)
dc_link_aux_try_to_configure_timeout(link->ddc, LINK_AUX_DEFAULT_TIMEOUT_PERIOD);
@@ -3757,7 +3893,7 @@ void detect_edp_sink_caps(struct dc_link *link)
memset(supported_link_rates, 0, sizeof(supported_link_rates));
if (link->dpcd_caps.dpcd_rev.raw >= DPCD_REV_14 &&
- (link->dc->config.optimize_edp_link_rate ||
+ (link->dc->debug.optimize_edp_link_rate ||
link->reported_link_cap.link_rate == LINK_RATE_UNKNOWN)) {
// Read DPCD 00010h - 0001Fh 16 bytes at one shot
core_link_read_dpcd(link, DP_SUPPORTED_LINK_RATES,
@@ -4265,7 +4401,7 @@ void dp_set_panel_mode(struct dc_link *link, enum dp_panel_mode panel_mode)
if (edp_config_set.bits.PANEL_MODE_EDP
!= panel_mode_edp) {
- enum dc_status result = DC_ERROR_UNEXPECTED;
+ enum dc_status result;
edp_config_set.bits.PANEL_MODE_EDP =
panel_mode_edp;
@@ -4583,3 +4719,51 @@ bool dc_link_set_default_brightness_aux(struct dc_link *link)
}
return false;
}
+
+bool is_edp_ilr_optimization_required(struct dc_link *link, struct dc_crtc_timing *crtc_timing)
+{
+ struct dc_link_settings link_setting;
+ uint8_t link_bw_set;
+ uint8_t link_rate_set;
+ uint32_t req_bw;
+ union lane_count_set lane_count_set = { {0} };
+
+ ASSERT(link || crtc_timing); // invalid input
+
+ if (link->dpcd_caps.edp_supported_link_rates_count == 0 ||
+ !link->dc->debug.optimize_edp_link_rate)
+ return false;
+
+
+ // Read DPCD 00100h to find if standard link rates are set
+ core_link_read_dpcd(link, DP_LINK_BW_SET,
+ &link_bw_set, sizeof(link_bw_set));
+
+ if (link_bw_set) {
+ DC_LOG_EVENT_LINK_TRAINING("eDP ILR: Optimization required, VBIOS used link_bw_set\n");
+ return true;
+ }
+
+ // Read DPCD 00115h to find the edp link rate set used
+ core_link_read_dpcd(link, DP_LINK_RATE_SET,
+ &link_rate_set, sizeof(link_rate_set));
+
+ // Read DPCD 00101h to find out the number of lanes currently set
+ core_link_read_dpcd(link, DP_LANE_COUNT_SET,
+ &lane_count_set.raw, sizeof(lane_count_set));
+
+ req_bw = dc_bandwidth_in_kbps_from_timing(crtc_timing);
+
+ decide_edp_link_settings(link, &link_setting, req_bw);
+
+ if (link->dpcd_caps.edp_supported_link_rates[link_rate_set] != link_setting.link_rate ||
+ lane_count_set.bits.LANE_COUNT_SET != link_setting.lane_count) {
+ DC_LOG_EVENT_LINK_TRAINING("eDP ILR: Optimization required, VBIOS link_rate_set not optimal\n");
+ return true;
+ }
+
+ DC_LOG_EVENT_LINK_TRAINING("eDP ILR: No optimization required, VBIOS set optimal link_rate_set\n");
+ return false;
+}
+
+
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c
new file mode 100644
index 000000000000..1361b87d86d7
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_enc_cfg.c
@@ -0,0 +1,303 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#include "link_enc_cfg.h"
+#include "resource.h"
+#include "dc_link_dp.h"
+
+/* Check whether stream is supported by DIG link encoders. */
+static bool is_dig_link_enc_stream(struct dc_stream_state *stream)
+{
+ bool is_dig_stream = false;
+ struct link_encoder *link_enc = NULL;
+ int i;
+
+ /* Loop over created link encoder objects. */
+ for (i = 0; i < stream->ctx->dc->res_pool->res_cap->num_dig_link_enc; i++) {
+ link_enc = stream->ctx->dc->res_pool->link_encoders[i];
+
+ if (link_enc &&
+ ((uint32_t)stream->signal & link_enc->output_signals)) {
+ if (dc_is_dp_signal(stream->signal)) {
+ /* DIGs do not support DP2.0 streams with 128b/132b encoding. */
+ struct dc_link_settings link_settings = {0};
+
+ decide_link_settings(stream, &link_settings);
+ if ((link_settings.link_rate >= LINK_RATE_LOW) &&
+ link_settings.link_rate <= LINK_RATE_HIGH3) {
+ is_dig_stream = true;
+ break;
+ }
+ } else {
+ is_dig_stream = true;
+ break;
+ }
+ }
+ }
+
+ return is_dig_stream;
+}
+
+/* Update DIG link encoder resource tracking variables in dc_state. */
+static void update_link_enc_assignment(
+ struct dc_state *state,
+ struct dc_stream_state *stream,
+ enum engine_id eng_id,
+ bool add_enc)
+{
+ int eng_idx;
+ int stream_idx;
+ int i;
+
+ if (eng_id != ENGINE_ID_UNKNOWN) {
+ eng_idx = eng_id - ENGINE_ID_DIGA;
+ stream_idx = -1;
+
+ /* Index of stream in dc_state used to update correct entry in
+ * link_enc_assignments table.
+ */
+ for (i = 0; i < state->stream_count; i++) {
+ if (stream == state->streams[i]) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ /* Update link encoder assignments table, link encoder availability
+ * pool and link encoder assigned to stream in state.
+ * Add/remove encoder resource to/from stream.
+ */
+ if (stream_idx != -1) {
+ if (add_enc) {
+ state->res_ctx.link_enc_assignments[stream_idx] = (struct link_enc_assignment){
+ .valid = true,
+ .ep_id = (struct display_endpoint_id) {
+ .link_id = stream->link->link_id,
+ .ep_type = stream->link->ep_type},
+ .eng_id = eng_id};
+ state->res_ctx.link_enc_avail[eng_idx] = ENGINE_ID_UNKNOWN;
+ stream->link_enc = stream->ctx->dc->res_pool->link_encoders[eng_idx];
+ } else {
+ state->res_ctx.link_enc_assignments[stream_idx].valid = false;
+ state->res_ctx.link_enc_avail[eng_idx] = eng_id;
+ stream->link_enc = NULL;
+ }
+ } else {
+ dm_output_to_console("%s: Stream not found in dc_state.\n", __func__);
+ }
+ }
+}
+
+/* Return first available DIG link encoder. */
+static enum engine_id find_first_avail_link_enc(
+ struct dc_context *ctx,
+ struct dc_state *state)
+{
+ enum engine_id eng_id = ENGINE_ID_UNKNOWN;
+ int i;
+
+ for (i = 0; i < ctx->dc->res_pool->res_cap->num_dig_link_enc; i++) {
+ eng_id = state->res_ctx.link_enc_avail[i];
+ if (eng_id != ENGINE_ID_UNKNOWN)
+ break;
+ }
+
+ return eng_id;
+}
+
+/* Return stream using DIG link encoder resource. NULL if unused. */
+static struct dc_stream_state *get_stream_using_link_enc(
+ struct dc_state *state,
+ enum engine_id eng_id)
+{
+ struct dc_stream_state *stream = NULL;
+ int stream_idx = -1;
+ int i;
+
+ for (i = 0; i < state->stream_count; i++) {
+ struct link_enc_assignment assignment = state->res_ctx.link_enc_assignments[i];
+
+ if (assignment.valid && (assignment.eng_id == eng_id)) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ if (stream_idx != -1)
+ stream = state->streams[stream_idx];
+ else
+ dm_output_to_console("%s: No stream using DIG(%d).\n", __func__, eng_id);
+
+ return stream;
+}
+
+void link_enc_cfg_init(
+ struct dc *dc,
+ struct dc_state *state)
+{
+ int i;
+
+ for (i = 0; i < dc->res_pool->res_cap->num_dig_link_enc; i++) {
+ if (dc->res_pool->link_encoders[i])
+ state->res_ctx.link_enc_avail[i] = (enum engine_id) i;
+ else
+ state->res_ctx.link_enc_avail[i] = ENGINE_ID_UNKNOWN;
+ }
+}
+
+void link_enc_cfg_link_encs_assign(
+ struct dc *dc,
+ struct dc_state *state,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count)
+{
+ enum engine_id eng_id = ENGINE_ID_UNKNOWN;
+ int i;
+
+ /* Release DIG link encoder resources before running assignment algorithm. */
+ for (i = 0; i < stream_count; i++)
+ dc->res_pool->funcs->link_enc_unassign(state, streams[i]);
+
+ /* (a) Assign DIG link encoders to physical (unmappable) endpoints first. */
+ for (i = 0; i < stream_count; i++) {
+ struct dc_stream_state *stream = streams[i];
+
+ /* Skip stream if not supported by DIG link encoder. */
+ if (!is_dig_link_enc_stream(stream))
+ continue;
+
+ /* Physical endpoints have a fixed mapping to DIG link encoders. */
+ if (!stream->link->is_dig_mapping_flexible) {
+ eng_id = stream->link->eng_id;
+ update_link_enc_assignment(state, stream, eng_id, true);
+ }
+ }
+
+ /* (b) Then assign encoders to mappable endpoints. */
+ eng_id = ENGINE_ID_UNKNOWN;
+
+ for (i = 0; i < stream_count; i++) {
+ struct dc_stream_state *stream = streams[i];
+
+ /* Skip stream if not supported by DIG link encoder. */
+ if (!is_dig_link_enc_stream(stream))
+ continue;
+
+ /* Mappable endpoints have a flexible mapping to DIG link encoders. */
+ if (stream->link->is_dig_mapping_flexible) {
+ eng_id = find_first_avail_link_enc(stream->ctx, state);
+ update_link_enc_assignment(state, stream, eng_id, true);
+ }
+ }
+}
+
+void link_enc_cfg_link_enc_unassign(
+ struct dc_state *state,
+ struct dc_stream_state *stream)
+{
+ enum engine_id eng_id = ENGINE_ID_UNKNOWN;
+
+ /* Only DIG link encoders. */
+ if (!is_dig_link_enc_stream(stream))
+ return;
+
+ if (stream->link_enc)
+ eng_id = stream->link_enc->preferred_engine;
+
+ update_link_enc_assignment(state, stream, eng_id, false);
+}
+
+bool link_enc_cfg_is_transmitter_mappable(
+ struct dc_state *state,
+ struct link_encoder *link_enc)
+{
+ bool is_mappable = false;
+ enum engine_id eng_id = link_enc->preferred_engine;
+ struct dc_stream_state *stream = get_stream_using_link_enc(state, eng_id);
+
+ if (stream)
+ is_mappable = stream->link->is_dig_mapping_flexible;
+
+ return is_mappable;
+}
+
+struct dc_link *link_enc_cfg_get_link_using_link_enc(
+ struct dc_state *state,
+ enum engine_id eng_id)
+{
+ struct dc_link *link = NULL;
+ int stream_idx = -1;
+ int i;
+
+ for (i = 0; i < state->stream_count; i++) {
+ struct link_enc_assignment assignment = state->res_ctx.link_enc_assignments[i];
+
+ if (assignment.valid && (assignment.eng_id == eng_id)) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ if (stream_idx != -1)
+ link = state->streams[stream_idx]->link;
+ else
+ dm_output_to_console("%s: No link using DIG(%d).\n", __func__, eng_id);
+
+ return link;
+}
+
+struct link_encoder *link_enc_cfg_get_link_enc_used_by_link(
+ struct dc_state *state,
+ struct dc_link *link)
+{
+ struct link_encoder *link_enc = NULL;
+ struct display_endpoint_id ep_id;
+ int stream_idx = -1;
+ int i;
+
+ ep_id = (struct display_endpoint_id) {
+ .link_id = link->link_id,
+ .ep_type = link->ep_type};
+
+ for (i = 0; i < state->stream_count; i++) {
+ struct link_enc_assignment assignment = state->res_ctx.link_enc_assignments[i];
+
+ if (assignment.valid &&
+ assignment.ep_id.link_id.id == ep_id.link_id.id &&
+ assignment.ep_id.link_id.enum_id == ep_id.link_id.enum_id &&
+ assignment.ep_id.link_id.type == ep_id.link_id.type &&
+ assignment.ep_id.ep_type == ep_id.ep_type) {
+ stream_idx = i;
+ break;
+ }
+ }
+
+ if (stream_idx != -1)
+ link_enc = state->streams[stream_idx]->link_enc;
+ else
+ dm_output_to_console("%s: No link encoder used by link(%d).\n", __func__, link->link_index);
+
+ return link_enc;
+}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c b/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c
index 124ce215fca5..b426f878fb99 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_link_hwss.c
@@ -14,6 +14,7 @@
#include "dpcd_defs.h"
#include "dsc.h"
#include "resource.h"
+#include "link_enc_cfg.h"
#include "clk_mgr.h"
static uint8_t convert_to_count(uint8_t lttpr_repeater_count)
@@ -95,7 +96,7 @@ void dp_enable_link_phy(
enum clock_source_id clock_source,
const struct dc_link_settings *link_settings)
{
- struct link_encoder *link_enc = link->link_enc;
+ struct link_encoder *link_enc;
struct dc *dc = link->ctx->dc;
struct dmcu *dmcu = dc->res_pool->dmcu;
@@ -105,6 +106,13 @@ void dp_enable_link_phy(
link->dc->res_pool->dp_clock_source;
unsigned int i;
+ /* Link should always be assigned encoder when en-/disabling. */
+ if (link->is_dig_mapping_flexible && dc->res_pool->funcs->link_encs_assign)
+ link_enc = link_enc_cfg_get_link_enc_used_by_link(link->dc->current_state, link);
+ else
+ link_enc = link->link_enc;
+ ASSERT(link_enc);
+
if (link->connector_signal == SIGNAL_TYPE_EDP) {
link->dc->hwss.edp_power_control(link, true);
link->dc->hwss.edp_wait_for_hpd_ready(link, true);
@@ -227,6 +235,14 @@ void dp_disable_link_phy(struct dc_link *link, enum signal_type signal)
{
struct dc *dc = link->ctx->dc;
struct dmcu *dmcu = dc->res_pool->dmcu;
+ struct link_encoder *link_enc;
+
+ /* Link should always be assigned encoder when en-/disabling. */
+ if (link->is_dig_mapping_flexible && dc->res_pool->funcs->link_encs_assign)
+ link_enc = link_enc_cfg_get_link_enc_used_by_link(link->dc->current_state, link);
+ else
+ link_enc = link->link_enc;
+ ASSERT(link_enc);
if (!link->wa_flags.dp_keep_receiver_powered)
dp_receiver_power_ctrl(link, false);
@@ -234,13 +250,13 @@ void dp_disable_link_phy(struct dc_link *link, enum signal_type signal)
if (signal == SIGNAL_TYPE_EDP) {
if (link->dc->hwss.edp_backlight_control)
link->dc->hwss.edp_backlight_control(link, false);
- link->link_enc->funcs->disable_output(link->link_enc, signal);
+ link_enc->funcs->disable_output(link_enc, signal);
link->dc->hwss.edp_power_control(link, false);
} else {
if (dmcu != NULL && dmcu->funcs->lock_phy)
dmcu->funcs->lock_phy(dmcu);
- link->link_enc->funcs->disable_output(link->link_enc, signal);
+ link_enc->funcs->disable_output(link_enc, signal);
if (dmcu != NULL && dmcu->funcs->unlock_phy)
dmcu->funcs->unlock_phy(dmcu);
@@ -302,7 +318,7 @@ void dp_set_hw_lane_settings(
{
struct link_encoder *encoder = link->link_enc;
- if (link->lttpr_non_transparent_mode && !is_immediate_downstream(link, offset))
+ if ((link->lttpr_mode == LTTPR_MODE_NON_TRANSPARENT) && !is_immediate_downstream(link, offset))
return;
/* call Encoder to set lane settings */
@@ -415,7 +431,7 @@ static void dsc_optc_config_log(struct display_stream_compressor *dsc,
DC_LOG_DSC("\tslice_width %d", config->slice_width);
}
-static bool dp_set_dsc_on_rx(struct pipe_ctx *pipe_ctx, bool enable)
+bool dp_set_dsc_on_rx(struct pipe_ctx *pipe_ctx, bool enable)
{
struct dc *dc = pipe_ctx->stream->ctx->dc;
struct dc_stream_state *stream = pipe_ctx->stream;
@@ -525,7 +541,7 @@ bool dp_set_dsc_enable(struct pipe_ctx *pipe_ctx, bool enable)
goto out;
if (enable) {
- if (dp_set_dsc_on_rx(pipe_ctx, true)) {
+ {
dp_set_dsc_on_stream(pipe_ctx, true);
result = true;
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
index 0c26c2ade782..8cb937c046aa 100644
--- a/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_resource.c
@@ -417,6 +417,49 @@ int resource_get_clock_source_reference(
return -1;
}
+bool resource_are_vblanks_synchronizable(
+ struct dc_stream_state *stream1,
+ struct dc_stream_state *stream2)
+{
+ uint32_t base60_refresh_rates[] = {10, 20, 5};
+ uint8_t i;
+ uint8_t rr_count = sizeof(base60_refresh_rates)/sizeof(base60_refresh_rates[0]);
+ uint64_t frame_time_diff;
+
+ if (stream1->ctx->dc->config.vblank_alignment_dto_params &&
+ stream1->ctx->dc->config.vblank_alignment_max_frame_time_diff > 0 &&
+ dc_is_dp_signal(stream1->signal) &&
+ dc_is_dp_signal(stream2->signal) &&
+ false == stream1->has_non_synchronizable_pclk &&
+ false == stream2->has_non_synchronizable_pclk &&
+ stream1->timing.flags.VBLANK_SYNCHRONIZABLE &&
+ stream2->timing.flags.VBLANK_SYNCHRONIZABLE) {
+ /* disable refresh rates higher than 60Hz for now */
+ if (stream1->timing.pix_clk_100hz*100/stream1->timing.h_total/
+ stream1->timing.v_total > 60)
+ return false;
+ if (stream2->timing.pix_clk_100hz*100/stream2->timing.h_total/
+ stream2->timing.v_total > 60)
+ return false;
+ frame_time_diff = (uint64_t)10000 *
+ stream1->timing.h_total *
+ stream1->timing.v_total *
+ stream2->timing.pix_clk_100hz;
+ frame_time_diff = div_u64(frame_time_diff, stream1->timing.pix_clk_100hz);
+ frame_time_diff = div_u64(frame_time_diff, stream2->timing.h_total);
+ frame_time_diff = div_u64(frame_time_diff, stream2->timing.v_total);
+ for (i = 0; i < rr_count; i++) {
+ int64_t diff = (int64_t)div_u64(frame_time_diff * base60_refresh_rates[i], 10) - 10000;
+
+ if (diff < 0)
+ diff = -diff;
+ if (diff < stream1->ctx->dc->config.vblank_alignment_max_frame_time_diff)
+ return true;
+ }
+ }
+ return false;
+}
+
bool resource_are_streams_timing_synchronizable(
struct dc_stream_state *stream1,
struct dc_stream_state *stream2)
@@ -1887,6 +1930,9 @@ enum dc_status dc_remove_stream_from_ctx(
dc->res_pool,
del_pipe->stream_res.stream_enc,
false);
+ /* Release link encoder from stream in new dc_state. */
+ if (dc->res_pool->funcs->link_enc_unassign)
+ dc->res_pool->funcs->link_enc_unassign(new_ctx, del_pipe->stream);
if (del_pipe->stream_res.audio)
update_audio_usage(
@@ -2460,26 +2506,31 @@ static void set_avi_info_frame(
hdmi_info.bits.ITC = itc_value;
}
+ if (stream->qs_bit == 1) {
+ if (color_space == COLOR_SPACE_SRGB ||
+ color_space == COLOR_SPACE_2020_RGB_FULLRANGE)
+ hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_FULL_RANGE;
+ else if (color_space == COLOR_SPACE_SRGB_LIMITED ||
+ color_space == COLOR_SPACE_2020_RGB_LIMITEDRANGE)
+ hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_LIMITED_RANGE;
+ else
+ hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_DEFAULT_RANGE;
+ } else
+ hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_DEFAULT_RANGE;
+
/* TODO : We should handle YCC quantization */
/* but we do not have matrix calculation */
- if (stream->qs_bit == 1 &&
- stream->qy_bit == 1) {
+ if (stream->qy_bit == 1) {
if (color_space == COLOR_SPACE_SRGB ||
- color_space == COLOR_SPACE_2020_RGB_FULLRANGE) {
- hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_FULL_RANGE;
+ color_space == COLOR_SPACE_2020_RGB_FULLRANGE)
hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
- } else if (color_space == COLOR_SPACE_SRGB_LIMITED ||
- color_space == COLOR_SPACE_2020_RGB_LIMITEDRANGE) {
- hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_LIMITED_RANGE;
+ else if (color_space == COLOR_SPACE_SRGB_LIMITED ||
+ color_space == COLOR_SPACE_2020_RGB_LIMITEDRANGE)
hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
- } else {
- hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_DEFAULT_RANGE;
+ else
hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
- }
- } else {
- hdmi_info.bits.Q0_Q1 = RGB_QUANTIZATION_DEFAULT_RANGE;
- hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
- }
+ } else
+ hdmi_info.bits.YQ0_YQ1 = YYC_QUANTIZATION_LIMITED_RANGE;
///VIC
format = stream->timing.timing_3d_format;
@@ -2799,6 +2850,10 @@ bool pipe_need_reprogram(
if (pipe_ctx_old->stream_res.dsc != pipe_ctx->stream_res.dsc)
return true;
+ /* DIG link encoder resource assignment for stream changed. */
+ if (pipe_ctx_old->stream->link_enc != pipe_ctx->stream->link_enc)
+ return true;
+
return false;
}
diff --git a/drivers/gpu/drm/amd/display/dc/core/dc_stat.c b/drivers/gpu/drm/amd/display/dc/core/dc_stat.c
new file mode 100644
index 000000000000..31761f3595a6
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/core/dc_stat.c
@@ -0,0 +1,64 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ */
+
+#include "dc/dc_stat.h"
+#include "dmub/dmub_srv_stat.h"
+#include "dc_dmub_srv.h"
+
+/**
+ * DOC: DC STAT Interface
+ *
+ * These interfaces are called without acquiring DAL and DC locks.
+ * Hence, there is limitations on whese interfaces can access. Only
+ * variables exclusively defined for these interfaces can be modified.
+ */
+
+/**
+ *****************************************************************************
+ * Function: dc_stat_get_dmub_notification
+ *
+ * @brief
+ * Calls dmub layer to retrieve dmub notification
+ *
+ * @param
+ * [in] dc: dc structure
+ * [in] notify: dmub notification structure
+ *
+ * @return
+ * None
+ *****************************************************************************
+ */
+void dc_stat_get_dmub_notification(const struct dc *dc, struct dmub_notification *notify)
+{
+ /**
+ * This function is called without dal and dc locks, so
+ * we shall not modify any dc, dc_dmub_srv or dmub variables
+ * except variables exclusively accessed by this function
+ */
+ struct dmub_srv *dmub = dc->ctx->dmub_srv->dmub;
+ enum dmub_status status;
+
+ status = dmub_srv_stat_get_notification(dmub, notify);
+ ASSERT(status == DMUB_STATUS_OK);
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dc.h b/drivers/gpu/drm/amd/display/dc/dc.h
index 4eee3a55fa30..100d434f7a03 100644
--- a/drivers/gpu/drm/amd/display/dc/dc.h
+++ b/drivers/gpu/drm/amd/display/dc/dc.h
@@ -42,13 +42,17 @@
#include "inc/hw/dmcu.h"
#include "dml/display_mode_lib.h"
-#define DC_VER "3.2.122"
+/* forward declaration */
+struct aux_payload;
+
+#define DC_VER "3.2.132"
#define MAX_SURFACES 3
#define MAX_PLANES 6
#define MAX_STREAMS 6
#define MAX_SINKS_PER_LINK 4
#define MIN_VIEWPORT_SIZE 12
+#define MAX_NUM_EDP 2
/*******************************************************************************
* Display Core Interfaces
@@ -151,6 +155,8 @@ struct dc_caps {
uint32_t max_links;
uint32_t max_audios;
uint32_t max_slave_planes;
+ uint32_t max_slave_yuv_planes;
+ uint32_t max_slave_rgb_planes;
uint32_t max_planes;
uint32_t max_downscale_ratio;
uint32_t i2c_speed_in_khz;
@@ -287,7 +293,6 @@ struct dc_config {
bool gpu_vm_support;
bool disable_disp_pll_sharing;
bool fbc_support;
- bool optimize_edp_link_rate;
bool disable_fractional_pwm;
bool allow_seamless_boot_optimization;
bool power_down_display_on_boot;
@@ -301,6 +306,10 @@ struct dc_config {
#if defined(CONFIG_DRM_AMD_DC_DCN)
bool clamp_min_dcfclk;
#endif
+ uint64_t vblank_alignment_dto_params;
+ uint8_t vblank_alignment_max_frame_time_diff;
+ bool is_asymmetric_memory;
+ bool is_single_rank_dimm;
};
enum visual_confirm {
@@ -452,6 +461,7 @@ struct dc_debug_options {
enum pipe_split_policy pipe_split_policy;
bool force_single_disp_pipe_split;
bool voltage_align_fclk;
+ bool disable_min_fclk;
bool disable_dfs_bypass;
bool disable_dpp_power_gate;
@@ -528,6 +538,15 @@ struct dc_debug_options {
bool disable_dsc;
bool enable_dram_clock_change_one_display_vactive;
union mem_low_power_enable_options enable_mem_low_power;
+ bool force_vblank_alignment;
+
+ /* Enable dmub aux for legacy ddc */
+ bool enable_dmub_aux_for_legacy_ddc;
+ bool optimize_edp_link_rate; /* eDP ILR */
+ /* force enable edp FEC */
+ bool force_enable_edp_fec;
+ /* FEC/PSR1 sequence enable delay in 100us */
+ uint8_t fec_enable_delay_in100us;
};
struct dc_debug_data {
@@ -628,7 +647,6 @@ struct dc {
#endif
/* Require to maintain clocks and bandwidth for UEFI enabled HW */
- int optimize_seamless_boot_streams;
/* FBC compressor */
struct compressor *fbc_compressor;
@@ -701,6 +719,7 @@ void dc_init_callbacks(struct dc *dc,
void dc_deinit_callbacks(struct dc *dc);
void dc_destroy(struct dc **dc);
+void dc_wait_for_vblank(struct dc *dc, struct dc_stream_state *stream);
/*******************************************************************************
* Surface Interfaces
******************************************************************************/
@@ -887,6 +906,9 @@ struct dc_plane_state {
int layer_index;
union surface_update_flags update_flags;
+ bool flip_int_enabled;
+ bool skip_manual_trigger;
+
/* private to DC core */
struct dc_plane_status status;
struct dc_context *ctx;
@@ -1291,8 +1313,20 @@ void dc_hardware_release(struct dc *dc);
bool dc_set_psr_allow_active(struct dc *dc, bool enable);
+bool dc_enable_dmub_notifications(struct dc *dc);
+
+bool dc_process_dmub_aux_transfer_async(struct dc *dc,
+ uint32_t link_index,
+ struct aux_payload *payload);
+
/*******************************************************************************
* DSC Interfaces
******************************************************************************/
#include "dc_dsc.h"
+
+/*******************************************************************************
+ * Disable acc mode Interfaces
+ ******************************************************************************/
+void dc_disable_accelerated_mode(struct dc *dc);
+
#endif /* DC_INTERFACE_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dc_bios_types.h b/drivers/gpu/drm/amd/display/dc/dc_bios_types.h
index 509d23fdd3c9..67abda44eb1f 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_bios_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_bios_types.h
@@ -139,7 +139,8 @@ struct dc_vbios_funcs {
enum bp_result (*enable_lvtma_control)(
struct dc_bios *bios,
- uint8_t uc_pwr_on);
+ uint8_t uc_pwr_on,
+ uint8_t panel_instance);
enum bp_result (*get_soc_bb_info)(
struct dc_bios *dcb,
@@ -149,6 +150,12 @@ struct dc_vbios_funcs {
struct dc_bios *dcb,
struct graphics_object_id object_id,
struct bp_disp_connector_caps_info *info);
+ enum bp_result (*get_lttpr_caps)(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps);
+ enum bp_result (*get_lttpr_interop)(
+ struct dc_bios *dcb,
+ uint8_t *dce_caps);
};
struct bios_registers {
diff --git a/drivers/gpu/drm/amd/display/dc/dc_ddc_types.h b/drivers/gpu/drm/amd/display/dc/dc_ddc_types.h
index 4f8f576d5fcf..7769bd099a5a 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_ddc_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_ddc_types.h
@@ -44,16 +44,6 @@ enum i2caux_transaction_action {
I2CAUX_TRANSACTION_ACTION_DP_READ = 0x90
};
-enum aux_channel_operation_result {
- AUX_CHANNEL_OPERATION_SUCCEEDED,
- AUX_CHANNEL_OPERATION_FAILED_REASON_UNKNOWN,
- AUX_CHANNEL_OPERATION_FAILED_INVALID_REPLY,
- AUX_CHANNEL_OPERATION_FAILED_TIMEOUT,
- AUX_CHANNEL_OPERATION_FAILED_HPD_DISCON,
- AUX_CHANNEL_OPERATION_FAILED_ENGINE_ACQUIRE
-};
-
-
struct aux_request_transaction_data {
enum aux_transaction_type type;
enum i2caux_transaction_action action;
diff --git a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
index b98754811977..6b72af2b3f4c 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
+++ b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.c
@@ -26,6 +26,10 @@
#include "dc.h"
#include "dc_dmub_srv.h"
#include "../dmub/dmub_srv.h"
+#include "dm_helpers.h"
+
+#define CTX dc_dmub_srv->ctx
+#define DC_LOGGER CTX->logger
static void dc_dmub_srv_construct(struct dc_dmub_srv *dc_srv, struct dc *dc,
struct dmub_srv *dmub)
@@ -106,6 +110,25 @@ void dc_dmub_srv_wait_idle(struct dc_dmub_srv *dc_dmub_srv)
DC_ERROR("Error waiting for DMUB idle: status=%d\n", status);
}
+bool dc_dmub_srv_cmd_with_reply_data(struct dc_dmub_srv *dc_dmub_srv, union dmub_rb_cmd *cmd)
+{
+ struct dmub_srv *dmub;
+ enum dmub_status status;
+
+ if (!dc_dmub_srv || !dc_dmub_srv->dmub)
+ return false;
+
+ dmub = dc_dmub_srv->dmub;
+
+ status = dmub_srv_cmd_with_reply_data(dmub, cmd);
+ if (status != DMUB_STATUS_OK) {
+ DC_LOG_DEBUG("No reply for DMUB command: status=%d\n", status);
+ return false;
+ }
+
+ return true;
+}
+
void dc_dmub_srv_wait_phy_init(struct dc_dmub_srv *dc_dmub_srv)
{
struct dmub_srv *dmub = dc_dmub_srv->dmub;
@@ -148,3 +171,14 @@ bool dc_dmub_srv_notify_stream_mask(struct dc_dmub_srv *dc_dmub_srv,
dmub, DMUB_GPINT__IDLE_OPT_NOTIFY_STREAM_MASK,
stream_mask, timeout) == DMUB_STATUS_OK;
}
+
+bool dc_dmub_srv_get_dmub_outbox0_msg(const struct dc *dc, struct dmcub_trace_buf_entry *entry)
+{
+ struct dmub_srv *dmub = dc->ctx->dmub_srv->dmub;
+ return dmub_srv_get_outbox0_msg(dmub, entry);
+}
+
+void dc_dmub_trace_event_control(struct dc *dc, bool enable)
+{
+ dm_helpers_dmub_outbox0_interrupt_control(dc->ctx, enable);
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h
index bb4ab61887e4..338f776990db 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_dmub_srv.h
@@ -30,6 +30,7 @@
#include "dmub/dmub_srv.h"
struct dmub_srv;
+struct dc;
struct dc_reg_helper_state {
bool gather_in_progress;
@@ -56,6 +57,13 @@ void dc_dmub_srv_wait_idle(struct dc_dmub_srv *dc_dmub_srv);
void dc_dmub_srv_wait_phy_init(struct dc_dmub_srv *dc_dmub_srv);
+bool dc_dmub_srv_cmd_with_reply_data(struct dc_dmub_srv *dc_dmub_srv, union dmub_rb_cmd *cmd);
+
bool dc_dmub_srv_notify_stream_mask(struct dc_dmub_srv *dc_dmub_srv,
unsigned int stream_mask);
+
+bool dc_dmub_srv_get_dmub_outbox0_msg(const struct dc *dc, struct dmcub_trace_buf_entry *entry);
+
+void dc_dmub_trace_event_control(struct dc *dc, bool enable);
+
#endif /* _DMUB_DC_SRV_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dc_dsc.h b/drivers/gpu/drm/amd/display/dc/dc_dsc.h
index ec55b77727d5..c51d2d961b7a 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_dsc.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_dsc.h
@@ -51,6 +51,7 @@ struct dc_dsc_policy {
int min_slice_height; // Must not be less than 8
uint32_t max_target_bpp;
uint32_t min_target_bpp;
+ uint32_t preferred_bpp_x16;
bool enable_dsc_when_not_needed;
};
@@ -62,8 +63,8 @@ bool dc_dsc_parse_dsc_dpcd(const struct dc *dc,
bool dc_dsc_compute_bandwidth_range(
const struct display_stream_compressor *dsc,
uint32_t dsc_min_slice_height_override,
- uint32_t min_bpp,
- uint32_t max_bpp,
+ uint32_t min_bpp_x16,
+ uint32_t max_bpp_x16,
const struct dsc_dec_dpcd_caps *dsc_sink_caps,
const struct dc_crtc_timing *timing,
struct dc_dsc_bw_range *range);
@@ -77,12 +78,16 @@ bool dc_dsc_compute_config(
const struct dc_crtc_timing *timing,
struct dc_dsc_config *dsc_cfg);
+uint32_t dc_dsc_stream_bandwidth_in_kbps(uint32_t pix_clk_100hz, uint32_t bpp_x16);
+
void dc_dsc_get_policy_for_timing(const struct dc_crtc_timing *timing,
- uint32_t max_target_bpp_limit_override,
+ uint32_t max_target_bpp_limit_override_x16,
struct dc_dsc_policy *policy);
void dc_dsc_policy_set_max_target_bpp_limit(uint32_t limit);
void dc_dsc_policy_set_enable_dsc_when_not_needed(bool enable);
+uint32_t dc_dsc_stream_bandwidth_in_kbps(uint32_t pix_clk_100hz, uint32_t bpp_x16);
+
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/dc_hw_types.h b/drivers/gpu/drm/amd/display/dc/dc_hw_types.h
index b41e6367b15e..bcec019efa6f 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_hw_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_hw_types.h
@@ -705,6 +705,7 @@ struct dc_crtc_timing_flags {
#ifndef TRIM_FSFT
uint32_t FAST_TRANSPORT: 1;
#endif
+ uint32_t VBLANK_SYNCHRONIZABLE: 1;
};
enum dc_timing_3d_format {
@@ -769,6 +770,7 @@ struct dc_crtc_timing {
#endif
struct dc_crtc_timing_flags flags;
+ uint32_t dsc_fixed_bits_per_pixel_x16; /* DSC target bitrate in 1/16 of bpp (e.g. 128 -> 8bpp) */
struct dc_dsc_config dsc_cfg;
};
diff --git a/drivers/gpu/drm/amd/display/dc/dc_link.h b/drivers/gpu/drm/amd/display/dc/dc_link.h
index e189f16bc026..054bab45ee17 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_link.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_link.h
@@ -35,6 +35,13 @@ enum dc_link_fec_state {
dc_link_fec_ready,
dc_link_fec_enabled
};
+
+enum lttpr_mode {
+ LTTPR_MODE_NON_LTTPR,
+ LTTPR_MODE_TRANSPARENT,
+ LTTPR_MODE_NON_TRANSPARENT,
+};
+
struct dc_link_status {
bool link_active;
struct dpcd_caps *dpcd_caps;
@@ -100,9 +107,13 @@ struct dc_link {
bool link_state_valid;
bool aux_access_disabled;
bool sync_lt_in_progress;
- bool lttpr_non_transparent_mode;
+ enum lttpr_mode lttpr_mode;
bool is_internal_display;
+ /* TODO: Rename. Flag an endpoint as having a programmable mapping to a
+ * DIG encoder. */
+ bool is_dig_mapping_flexible;
+
bool edp_sink_present;
/* caps is the same as reported_link_cap. link_traing use
@@ -121,6 +132,11 @@ struct dc_link {
uint8_t hpd_src;
uint8_t link_enc_hw_inst;
+ /* DIG link encoder ID. Used as index in link encoder resource pool.
+ * For links with fixed mapping to DIG, this is not changed after dc_link
+ * object creation.
+ */
+ enum engine_id eng_id;
bool test_pattern_enabled;
union compliance_test_state compliance_test_state;
@@ -140,6 +156,11 @@ struct dc_link {
struct panel_cntl *panel_cntl;
struct link_encoder *link_enc;
struct graphics_object_id link_id;
+ /* Endpoint type distinguishes display endpoints which do not have entries
+ * in the BIOS connector table from those that do. Helps when tracking link
+ * encoder to display endpoint assignments.
+ */
+ enum display_endpoint_type ep_type;
union ddi_channel_mapping ddi_channel_mapping;
struct connector_device_tag_info device_tag;
struct dpcd_caps dpcd_caps;
@@ -183,16 +204,21 @@ static inline struct dc_link *dc_get_link_at_index(struct dc *dc, uint32_t link_
return dc->links[link_index];
}
-static inline struct dc_link *get_edp_link(const struct dc *dc)
+static inline void get_edp_links(const struct dc *dc,
+ struct dc_link **edp_links,
+ int *edp_num)
{
int i;
- // report any eDP links, even unconnected DDI's
+ *edp_num = 0;
for (i = 0; i < dc->link_count; i++) {
- if (dc->links[i]->connector_signal == SIGNAL_TYPE_EDP)
- return dc->links[i];
+ // report any eDP links, even unconnected DDI's
+ if (dc->links[i]->connector_signal == SIGNAL_TYPE_EDP) {
+ edp_links[*edp_num] = dc->links[i];
+ if (++(*edp_num) == MAX_NUM_EDP)
+ return;
+ }
}
- return NULL;
}
/* Set backlight level of an embedded panel (eDP, LVDS).
@@ -250,7 +276,6 @@ enum dc_detect_reason {
bool dc_link_detect(struct dc_link *dc_link, enum dc_detect_reason reason);
bool dc_link_get_hpd_state(struct dc_link *dc_link);
enum dc_status dc_link_allocate_mst_payload(struct pipe_ctx *pipe_ctx);
-enum dc_status dc_link_reallocate_mst_payload(struct dc_link *link);
/* Notify DC about DP RX Interrupt (aka Short Pulse Interrupt).
* Return:
diff --git a/drivers/gpu/drm/amd/display/dc/dc_stat.h b/drivers/gpu/drm/amd/display/dc/dc_stat.h
new file mode 100644
index 000000000000..2a000ba54ddb
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/dc_stat.h
@@ -0,0 +1,42 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef _DC_STAT_H_
+#define _DC_STAT_H_
+
+/**
+ * DOC: DC STAT Interface
+ *
+ * These interfaces are called without acquiring DAL and DC locks.
+ * Hence, there is limitations on whese interfaces can access. Only
+ * variables exclusively defined for these interfaces can be modified.
+ */
+
+#include "dc.h"
+#include "dmub/dmub_srv.h"
+
+void dc_stat_get_dmub_notification(const struct dc *dc, struct dmub_notification *notify);
+
+#endif /* _DC_STAT_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dc_stream.h b/drivers/gpu/drm/amd/display/dc/dc_stream.h
index 80b67b860091..13dae7238a58 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_stream.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_stream.h
@@ -144,6 +144,10 @@ struct dc_stream_state {
struct dc_sink *sink;
struct dc_link *link;
+ /* For dynamic link encoder assignment, update the link encoder assigned to
+ * a stream via the volatile dc_state rather than the static dc_link.
+ */
+ struct link_encoder *link_enc;
struct dc_panel_patch sink_patches;
union display_content_support content_support;
struct dc_crtc_timing timing;
@@ -234,10 +238,12 @@ struct dc_stream_state {
bool apply_seamless_boot_optimization;
uint32_t stream_id;
- bool is_dsc_enabled;
struct test_pattern test_pattern;
union stream_update_flags update_flags;
+
+ bool has_non_synchronizable_pclk;
+ bool vblank_synchronized;
};
#define ABM_LEVEL_IMMEDIATE_DISABLE 255
@@ -271,6 +277,7 @@ struct dc_stream_update {
struct dc_dsc_config *dsc_config;
struct dc_transfer_func *func_shaper;
struct dc_3dlut *lut3d_func;
+
struct test_pattern *pending_test_pattern;
};
@@ -461,6 +468,13 @@ bool dc_stream_get_crtc_position(struct dc *dc,
unsigned int *v_pos,
unsigned int *nom_v_pos);
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+bool dc_stream_forward_dmcu_crc_window(struct dc *dc, struct dc_stream_state *stream,
+ struct crc_params *crc_window);
+bool dc_stream_stop_dmcu_crc_win_update(struct dc *dc,
+ struct dc_stream_state *stream);
+#endif
+
bool dc_stream_configure_crc(struct dc *dc,
struct dc_stream_state *stream,
struct crc_params *crc_window,
diff --git a/drivers/gpu/drm/amd/display/dc/dc_types.h b/drivers/gpu/drm/amd/display/dc/dc_types.h
index 80757a0ea7c6..432754eaf10b 100644
--- a/drivers/gpu/drm/amd/display/dc/dc_types.h
+++ b/drivers/gpu/drm/amd/display/dc/dc_types.h
@@ -113,6 +113,7 @@ struct dc_context {
struct gpio_service *gpio_service;
uint32_t dc_sink_id_count;
uint32_t dc_stream_id_count;
+ uint32_t dc_edp_id_count;
uint64_t fbc_gpu_addr;
struct dc_dmub_srv *dmub_srv;
@@ -687,7 +688,8 @@ enum dc_psr_state {
PSR_STATE5,
PSR_STATE5a,
PSR_STATE5b,
- PSR_STATE5c
+ PSR_STATE5c,
+ PSR_STATE_INVALID = 0xFF
};
struct psr_config {
@@ -934,4 +936,19 @@ enum dc_psr_version {
DC_PSR_VERSION_UNSUPPORTED = 0xFFFFFFFF,
};
+/* Possible values of display_endpoint_id.endpoint */
+enum display_endpoint_type {
+ DISPLAY_ENDPOINT_PHY = 0, /* Physical connector. */
+ DISPLAY_ENDPOINT_UNKNOWN = -1
+};
+
+/* Extends graphics_object_id with an additional member 'ep_type' for
+ * distinguishing between physical endpoints (with entries in BIOS connector table) and
+ * logical endpoints.
+ */
+struct display_endpoint_id {
+ struct graphics_object_id link_id;
+ enum display_endpoint_type ep_type;
+};
+
#endif /* DC_TYPES_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dce/Makefile b/drivers/gpu/drm/amd/display/dc/dce/Makefile
index 973be8f9fd10..0d7db132a20f 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/Makefile
+++ b/drivers/gpu/drm/amd/display/dc/dce/Makefile
@@ -30,7 +30,7 @@ DCE = dce_audio.o dce_stream_encoder.o dce_link_encoder.o dce_hwseq.o \
dce_mem_input.o dce_clock_source.o dce_scl_filters.o dce_transform.o \
dce_opp.o dce_dmcu.o dce_abm.o dce_ipp.o dce_aux.o \
dce_i2c.o dce_i2c_hw.o dce_i2c_sw.o dmub_psr.o dmub_abm.o dce_panel_cntl.o \
-dmub_hw_lock_mgr.o
+dmub_hw_lock_mgr.o dmub_outbox.o
AMD_DAL_DCE = $(addprefix $(AMDDALPATH)/dc/dce/,$(DCE))
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_abm.c b/drivers/gpu/drm/amd/display/dc/dce/dce_abm.c
index 4e87e70237e3..874b132fe1d7 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_abm.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_abm.c
@@ -283,7 +283,7 @@ struct abm *dce_abm_create(
const struct dce_abm_shift *abm_shift,
const struct dce_abm_mask *abm_mask)
{
- struct dce_abm *abm_dce = kzalloc(sizeof(*abm_dce), GFP_KERNEL);
+ struct dce_abm *abm_dce = kzalloc(sizeof(*abm_dce), GFP_ATOMIC);
if (abm_dce == NULL) {
BREAK_TO_DEBUGGER();
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c b/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c
index d51b5fe91287..87d57e81de12 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_aux.c
@@ -31,6 +31,8 @@
#include "dce_aux.h"
#include "dce/dce_11_0_sh_mask.h"
#include "dm_event_log.h"
+#include "dm_helpers.h"
+#include "dmub/inc/dmub_cmd.h"
#define CTX \
aux110->base.ctx
@@ -324,7 +326,7 @@ static int read_channel_reply(struct dce_aux *engine, uint32_t size,
return 0;
}
-static enum aux_channel_operation_result get_channel_status(
+static enum aux_return_code_type get_channel_status(
struct dce_aux *engine,
uint8_t *returned_bytes)
{
@@ -335,7 +337,7 @@ static enum aux_channel_operation_result get_channel_status(
if (returned_bytes == NULL) {
/*caller pass NULL pointer*/
ASSERT_CRITICAL(false);
- return AUX_CHANNEL_OPERATION_FAILED_REASON_UNKNOWN;
+ return AUX_RET_ERROR_UNKNOWN;
}
*returned_bytes = 0;
@@ -346,7 +348,7 @@ static enum aux_channel_operation_result get_channel_status(
value = REG_READ(AUX_SW_STATUS);
/* in case HPD is LOW, exit AUX transaction */
if ((value & AUX_SW_STATUS__AUX_SW_HPD_DISCON_MASK))
- return AUX_CHANNEL_OPERATION_FAILED_HPD_DISCON;
+ return AUX_RET_ERROR_HPD_DISCON;
/* Note that the following bits are set in 'status.bits'
* during CTS 4.2.1.2 (FW 3.3.1):
@@ -359,14 +361,14 @@ static enum aux_channel_operation_result get_channel_status(
if (value & AUX_SW_STATUS__AUX_SW_DONE_MASK) {
if ((value & AUX_SW_STATUS__AUX_SW_RX_TIMEOUT_STATE_MASK) ||
(value & AUX_SW_STATUS__AUX_SW_RX_TIMEOUT_MASK))
- return AUX_CHANNEL_OPERATION_FAILED_TIMEOUT;
+ return AUX_RET_ERROR_TIMEOUT;
else if ((value & AUX_SW_STATUS__AUX_SW_RX_INVALID_STOP_MASK) ||
(value & AUX_SW_STATUS__AUX_SW_RX_RECV_NO_DET_MASK) ||
(value &
AUX_SW_STATUS__AUX_SW_RX_RECV_INVALID_H_MASK) ||
(value & AUX_SW_STATUS__AUX_SW_RX_RECV_INVALID_L_MASK))
- return AUX_CHANNEL_OPERATION_FAILED_INVALID_REPLY;
+ return AUX_RET_ERROR_INVALID_REPLY;
*returned_bytes = get_reg_field_value(value,
AUX_SW_STATUS,
@@ -374,17 +376,17 @@ static enum aux_channel_operation_result get_channel_status(
if (*returned_bytes == 0)
return
- AUX_CHANNEL_OPERATION_FAILED_INVALID_REPLY;
+ AUX_RET_ERROR_INVALID_REPLY;
else {
*returned_bytes -= 1;
- return AUX_CHANNEL_OPERATION_SUCCEEDED;
+ return AUX_RET_SUCCESS;
}
} else {
/*time_elapsed >= aux_engine->timeout_period
* AUX_SW_STATUS__AUX_SW_HPD_DISCON = at this point
*/
ASSERT_CRITICAL(false);
- return AUX_CHANNEL_OPERATION_FAILED_TIMEOUT;
+ return AUX_RET_ERROR_TIMEOUT;
}
}
@@ -541,7 +543,7 @@ static enum i2caux_transaction_action i2caux_action_from_payload(struct aux_payl
int dce_aux_transfer_raw(struct ddc_service *ddc,
struct aux_payload *payload,
- enum aux_channel_operation_result *operation_result)
+ enum aux_return_code_type *operation_result)
{
struct ddc *ddc_pin = ddc->ddc_pin;
struct dce_aux *aux_engine;
@@ -556,7 +558,7 @@ int dce_aux_transfer_raw(struct ddc_service *ddc,
aux_engine = ddc->ctx->dc->res_pool->engines[ddc_pin->pin_data->en];
if (!acquire(aux_engine, ddc_pin)) {
- *operation_result = AUX_CHANNEL_OPERATION_FAILED_ENGINE_ACQUIRE;
+ *operation_result = AUX_RET_ERROR_ENGINE_ACQUIRE;
return -1;
}
@@ -575,8 +577,9 @@ int dce_aux_transfer_raw(struct ddc_service *ddc,
submit_channel_request(aux_engine, &aux_req);
*operation_result = get_channel_status(aux_engine, &returned_bytes);
- if (*operation_result == AUX_CHANNEL_OPERATION_SUCCEEDED) {
+ if (*operation_result == AUX_RET_SUCCESS) {
int __maybe_unused bytes_replied = 0;
+
bytes_replied = read_channel_reply(aux_engine, payload->length,
payload->data, payload->reply,
&status);
@@ -604,7 +607,7 @@ bool dce_aux_transfer_with_retries(struct ddc_service *ddc,
int i, ret = 0;
uint8_t reply;
bool payload_reply = true;
- enum aux_channel_operation_result operation_result;
+ enum aux_return_code_type operation_result;
bool retry_on_defer = false;
int aux_ack_retries = 0,
@@ -620,8 +623,9 @@ bool dce_aux_transfer_with_retries(struct ddc_service *ddc,
for (i = 0; i < AUX_MAX_RETRIES; i++) {
ret = dce_aux_transfer_raw(ddc, payload, &operation_result);
+
switch (operation_result) {
- case AUX_CHANNEL_OPERATION_SUCCEEDED:
+ case AUX_RET_SUCCESS:
aux_timeout_retries = 0;
aux_invalid_reply_retries = 0;
@@ -667,14 +671,14 @@ bool dce_aux_transfer_with_retries(struct ddc_service *ddc,
}
break;
- case AUX_CHANNEL_OPERATION_FAILED_INVALID_REPLY:
+ case AUX_RET_ERROR_INVALID_REPLY:
if (++aux_invalid_reply_retries >= AUX_MAX_INVALID_REPLY_RETRIES)
goto fail;
else
udelay(400);
break;
- case AUX_CHANNEL_OPERATION_FAILED_TIMEOUT:
+ case AUX_RET_ERROR_TIMEOUT:
// Check whether a DEFER had occurred before the timeout.
// If so, treat timeout as a DEFER.
if (retry_on_defer) {
@@ -696,9 +700,9 @@ bool dce_aux_transfer_with_retries(struct ddc_service *ddc,
}
break;
- case AUX_CHANNEL_OPERATION_FAILED_HPD_DISCON:
- case AUX_CHANNEL_OPERATION_FAILED_ENGINE_ACQUIRE:
- case AUX_CHANNEL_OPERATION_FAILED_REASON_UNKNOWN:
+ case AUX_RET_ERROR_HPD_DISCON:
+ case AUX_RET_ERROR_ENGINE_ACQUIRE:
+ case AUX_RET_ERROR_UNKNOWN:
default:
goto fail;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_aux.h b/drivers/gpu/drm/amd/display/dc/dce/dce_aux.h
index 277484cf853e..566b1bddd8cc 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_aux.h
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_aux.h
@@ -29,6 +29,7 @@
#include "i2caux_interface.h"
#include "inc/hw/aux_engine.h"
+enum aux_return_code_type;
#define AUX_COMMON_REG_LIST0(id)\
SRI(AUX_CONTROL, DP_AUX, id), \
@@ -99,7 +100,6 @@ struct dce110_aux_registers {
AUX_SF(AUX_SW_CONTROL, AUX_SW_GO, mask_sh),\
AUX_SF(AUX_SW_DATA, AUX_SW_AUTOINCREMENT_DISABLE, mask_sh),\
AUX_SF(AUX_SW_DATA, AUX_SW_DATA_RW, mask_sh),\
- AUX_SF(AUX_SW_DATA, AUX_SW_AUTOINCREMENT_DISABLE, mask_sh),\
AUX_SF(AUX_SW_DATA, AUX_SW_INDEX, mask_sh),\
AUX_SF(AUX_SW_DATA, AUX_SW_DATA, mask_sh),\
AUX_SF(AUX_SW_STATUS, AUX_SW_REPLY_BYTE_COUNT, mask_sh),\
@@ -302,7 +302,7 @@ bool dce110_aux_engine_acquire(
int dce_aux_transfer_raw(struct ddc_service *ddc,
struct aux_payload *cmd,
- enum aux_channel_operation_result *operation_result);
+ enum aux_return_code_type *operation_result);
bool dce_aux_transfer_with_retries(struct ddc_service *ddc,
struct aux_payload *cmd);
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c b/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c
index dec58b3c42e4..2c7eb982eabc 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_clock_source.c
@@ -1002,15 +1002,27 @@ static bool get_pixel_clk_frequency_100hz(
{
struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(clock_source);
unsigned int clock_hz = 0;
+ unsigned int modulo_hz = 0;
if (clock_source->id == CLOCK_SOURCE_ID_DP_DTO) {
clock_hz = REG_READ(PHASE[inst]);
- /* NOTE: There is agreement with VBIOS here that MODULO is
- * programmed equal to DPREFCLK, in which case PHASE will be
- * equivalent to pixel clock.
- */
- *pixel_clk_khz = clock_hz / 100;
+ if (clock_source->ctx->dc->hwss.enable_vblanks_synchronization &&
+ clock_source->ctx->dc->config.vblank_alignment_max_frame_time_diff > 0) {
+ /* NOTE: In case VBLANK syncronization is enabled, MODULO may
+ * not be programmed equal to DPREFCLK
+ */
+ modulo_hz = REG_READ(MODULO[inst]);
+ *pixel_clk_khz = div_u64((uint64_t)clock_hz*
+ clock_source->ctx->dc->clk_mgr->dprefclk_khz*10,
+ modulo_hz);
+ } else {
+ /* NOTE: There is agreement with VBIOS here that MODULO is
+ * programmed equal to DPREFCLK, in which case PHASE will be
+ * equivalent to pixel clock.
+ */
+ *pixel_clk_khz = clock_hz / 100;
+ }
return true;
}
@@ -1074,8 +1086,35 @@ static bool dcn20_program_pix_clk(
struct pixel_clk_params *pix_clk_params,
struct pll_settings *pll_settings)
{
+ struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(clock_source);
+ unsigned int inst = pix_clk_params->controller_id - CONTROLLER_ID_D0;
+
dce112_program_pix_clk(clock_source, pix_clk_params, pll_settings);
+ if (clock_source->ctx->dc->hwss.enable_vblanks_synchronization &&
+ clock_source->ctx->dc->config.vblank_alignment_max_frame_time_diff > 0) {
+ /* NOTE: In case VBLANK syncronization is enabled,
+ * we need to set modulo to default DPREFCLK first
+ * dce112_program_pix_clk does not set default DPREFCLK
+ */
+ REG_WRITE(MODULO[inst],
+ clock_source->ctx->dc->clk_mgr->dprefclk_khz*1000);
+ }
+ return true;
+}
+
+static bool dcn20_override_dp_pix_clk(
+ struct clock_source *clock_source,
+ unsigned int inst,
+ unsigned int pixel_clk,
+ unsigned int ref_clk)
+{
+ struct dce110_clk_src *clk_src = TO_DCE110_CLK_SRC(clock_source);
+
+ REG_UPDATE(PIXEL_RATE_CNTL[inst], DP_DTO0_ENABLE, 0);
+ REG_WRITE(PHASE[inst], pixel_clk);
+ REG_WRITE(MODULO[inst], ref_clk);
+ REG_UPDATE(PIXEL_RATE_CNTL[inst], DP_DTO0_ENABLE, 1);
return true;
}
@@ -1083,7 +1122,8 @@ static const struct clock_source_funcs dcn20_clk_src_funcs = {
.cs_power_down = dce110_clock_source_power_down,
.program_pix_clk = dcn20_program_pix_clk,
.get_pix_clk_dividers = dce112_get_pix_clk_dividers,
- .get_pixel_clk_frequency_100hz = get_pixel_clk_frequency_100hz
+ .get_pixel_clk_frequency_100hz = get_pixel_clk_frequency_100hz,
+ .override_dp_pix_clk = dcn20_override_dp_pix_clk
};
#if defined(CONFIG_DRM_AMD_DC_DCN)
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c b/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c
index ddc789daf3b1..8cd841320ded 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_dmcu.c
@@ -57,6 +57,8 @@
#define MCP_SYNC_PHY_LOCK 0x90
#define MCP_SYNC_PHY_UNLOCK 0x91
#define MCP_BL_SET_PWM_FRAC 0x6A /* Enable or disable Fractional PWM */
+#define CRC_WIN_NOTIFY 0x92
+#define CRC_STOP_UPDATE 0x93
#define MCP_SEND_EDID_CEA 0xA0
#define EDID_CEA_CMD_ACK 1
#define EDID_CEA_CMD_NACK 2
@@ -930,6 +932,84 @@ static bool dcn10_recv_edid_cea_ack(struct dmcu *dmcu, int *offset)
#endif //(CONFIG_DRM_AMD_DC_DCN)
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+static void dcn10_forward_crc_window(struct dmcu *dmcu,
+ struct crc_region *crc_win,
+ struct otg_phy_mux *mux_mapping)
+{
+ struct dce_dmcu *dmcu_dce = TO_DCE_DMCU(dmcu);
+ unsigned int dmcu_max_retry_on_wait_reg_ready = 801;
+ unsigned int dmcu_wait_reg_ready_interval = 100;
+ unsigned int crc_start = 0, crc_end = 0, otg_phy_mux = 0;
+
+ /* If microcontroller is not running, do nothing */
+ if (dmcu->dmcu_state != DMCU_RUNNING)
+ return;
+
+ if (!crc_win)
+ return;
+
+ /* waitDMCUReadyForCmd */
+ REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 0,
+ dmcu_wait_reg_ready_interval,
+ dmcu_max_retry_on_wait_reg_ready);
+
+ /* build up nitification data */
+ crc_start = (((unsigned int) crc_win->x_start) << 16) | crc_win->y_start;
+ crc_end = (((unsigned int) crc_win->x_end) << 16) | crc_win->y_end;
+ otg_phy_mux =
+ (((unsigned int) mux_mapping->otg_output_num) << 16) | mux_mapping->phy_output_num;
+
+ dm_write_reg(dmcu->ctx, REG(MASTER_COMM_DATA_REG1),
+ crc_start);
+
+ dm_write_reg(dmcu->ctx, REG(MASTER_COMM_DATA_REG2),
+ crc_end);
+
+ dm_write_reg(dmcu->ctx, REG(MASTER_COMM_DATA_REG3),
+ otg_phy_mux);
+
+ /* setDMCUParam_Cmd */
+ REG_UPDATE(MASTER_COMM_CMD_REG, MASTER_COMM_CMD_REG_BYTE0,
+ CRC_WIN_NOTIFY);
+
+ /* notifyDMCUMsg */
+ REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1);
+}
+
+static void dcn10_stop_crc_win_update(struct dmcu *dmcu,
+ struct otg_phy_mux *mux_mapping)
+{
+ struct dce_dmcu *dmcu_dce = TO_DCE_DMCU(dmcu);
+ unsigned int dmcu_max_retry_on_wait_reg_ready = 801;
+ unsigned int dmcu_wait_reg_ready_interval = 100;
+ unsigned int otg_phy_mux = 0;
+
+ /* If microcontroller is not running, do nothing */
+ if (dmcu->dmcu_state != DMCU_RUNNING)
+ return;
+
+ /* waitDMCUReadyForCmd */
+ REG_WAIT(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 0,
+ dmcu_wait_reg_ready_interval,
+ dmcu_max_retry_on_wait_reg_ready);
+
+ /* build up nitification data */
+ otg_phy_mux =
+ (((unsigned int) mux_mapping->otg_output_num) << 16) | mux_mapping->phy_output_num;
+
+ dm_write_reg(dmcu->ctx, REG(MASTER_COMM_DATA_REG1),
+ otg_phy_mux);
+
+ /* setDMCUParam_Cmd */
+ REG_UPDATE(MASTER_COMM_CMD_REG, MASTER_COMM_CMD_REG_BYTE0,
+ CRC_STOP_UPDATE);
+
+ /* notifyDMCUMsg */
+ REG_UPDATE(MASTER_COMM_CNTL_REG, MASTER_COMM_INTERRUPT, 1);
+}
+#endif
+
static const struct dmcu_funcs dce_funcs = {
.dmcu_init = dce_dmcu_init,
.load_iram = dce_dmcu_load_iram,
@@ -953,6 +1033,10 @@ static const struct dmcu_funcs dcn10_funcs = {
.send_edid_cea = dcn10_send_edid_cea,
.recv_amd_vsdb = dcn10_recv_amd_vsdb,
.recv_edid_cea_ack = dcn10_recv_edid_cea_ack,
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ .forward_crc_window = dcn10_forward_crc_window,
+ .stop_crc_win_update = dcn10_stop_crc_win_update,
+#endif
.is_dmcu_initialized = dcn10_is_dmcu_initialized
};
@@ -1049,7 +1133,7 @@ struct dmcu *dcn10_dmcu_create(
const struct dce_dmcu_shift *dmcu_shift,
const struct dce_dmcu_mask *dmcu_mask)
{
- struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_KERNEL);
+ struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_ATOMIC);
if (dmcu_dce == NULL) {
BREAK_TO_DEBUGGER();
@@ -1070,7 +1154,7 @@ struct dmcu *dcn20_dmcu_create(
const struct dce_dmcu_shift *dmcu_shift,
const struct dce_dmcu_mask *dmcu_mask)
{
- struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_KERNEL);
+ struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_ATOMIC);
if (dmcu_dce == NULL) {
BREAK_TO_DEBUGGER();
@@ -1091,7 +1175,7 @@ struct dmcu *dcn21_dmcu_create(
const struct dce_dmcu_shift *dmcu_shift,
const struct dce_dmcu_mask *dmcu_mask)
{
- struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_KERNEL);
+ struct dce_dmcu *dmcu_dce = kzalloc(sizeof(*dmcu_dce), GFP_ATOMIC);
if (dmcu_dce == NULL) {
BREAK_TO_DEBUGGER();
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dce_opp.c b/drivers/gpu/drm/amd/display/dc/dce/dce_opp.c
index 4600231da6cb..895b015b02e8 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dce_opp.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dce_opp.c
@@ -216,9 +216,7 @@ static void set_spatial_dither(
REG_UPDATE(FMT_BIT_DEPTH_CONTROL,
FMT_TEMPORAL_DITHER_EN, 0);
- /* no 10bpc on DCE11*/
- if (params->flags.SPATIAL_DITHER_ENABLED == 0 ||
- params->flags.SPATIAL_DITHER_DEPTH == 2)
+ if (params->flags.SPATIAL_DITHER_ENABLED == 0)
return;
/* only use FRAME_COUNTER_MAX if frameRandom == 1*/
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_abm.c b/drivers/gpu/drm/amd/display/dc/dce/dmub_abm.c
index 453aaa5757bd..eb1698d54a48 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dmub_abm.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_abm.c
@@ -72,11 +72,11 @@ static void dmub_abm_init(struct abm *abm, uint32_t backlight)
{
struct dce_abm *dce_abm = TO_DMUB_ABM(abm);
- REG_WRITE(DC_ABM1_HG_SAMPLE_RATE, 0x103);
- REG_WRITE(DC_ABM1_HG_SAMPLE_RATE, 0x101);
- REG_WRITE(DC_ABM1_LS_SAMPLE_RATE, 0x103);
- REG_WRITE(DC_ABM1_LS_SAMPLE_RATE, 0x101);
- REG_WRITE(BL1_PWM_BL_UPDATE_SAMPLE_RATE, 0x101);
+ REG_WRITE(DC_ABM1_HG_SAMPLE_RATE, 0x3);
+ REG_WRITE(DC_ABM1_HG_SAMPLE_RATE, 0x1);
+ REG_WRITE(DC_ABM1_LS_SAMPLE_RATE, 0x3);
+ REG_WRITE(DC_ABM1_LS_SAMPLE_RATE, 0x1);
+ REG_WRITE(BL1_PWM_BL_UPDATE_SAMPLE_RATE, 0x1);
REG_SET_3(DC_ABM1_HG_MISC_CTRL, 0,
ABM1_HG_NUM_OF_BINS_SEL, 0,
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.c b/drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.c
new file mode 100644
index 000000000000..295596d1f47f
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.c
@@ -0,0 +1,60 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ */
+
+#include "dmub_outbox.h"
+#include "dc_dmub_srv.h"
+#include "dmub/inc/dmub_cmd.h"
+
+/**
+ *****************************************************************************
+ * Function: dmub_enable_outbox_notification
+ *
+ * @brief
+ * Sends inbox cmd to dmub to enable outbox1 messages with interrupt.
+ * Dmub sends outbox1 message and triggers outbox1 interrupt.
+ *
+ * @param
+ * [in] dc: dc structure
+ *
+ * @return
+ * None
+ *****************************************************************************
+ */
+void dmub_enable_outbox_notification(struct dc *dc)
+{
+ union dmub_rb_cmd cmd;
+ struct dc_context *dc_ctx = dc->ctx;
+
+ memset(&cmd, 0x0, sizeof(cmd));
+ cmd.outbox1_enable.header.type = DMUB_CMD__OUTBOX1_ENABLE;
+ cmd.outbox1_enable.header.sub_type = 0;
+ cmd.outbox1_enable.header.payload_bytes =
+ sizeof(cmd.outbox1_enable) -
+ sizeof(cmd.outbox1_enable.header);
+ cmd.outbox1_enable.enable = true;
+
+ dc_dmub_srv_cmd_queue(dc_ctx->dmub_srv, &cmd);
+ dc_dmub_srv_cmd_execute(dc_ctx->dmub_srv);
+ dc_dmub_srv_wait_idle(dc_ctx->dmub_srv);
+}
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.h b/drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.h
new file mode 100644
index 000000000000..4e0aa0d1a2d5
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_outbox.h
@@ -0,0 +1,33 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef _DMUB_OUTBOX_H_
+#define _DMUB_OUTBOX_H_
+
+#include "dc.h"
+
+void dmub_enable_outbox_notification(struct dc *dc);
+
+#endif /* _DMUB_OUTBOX_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
index 69e34bef274c..5e99553fcdd4 100644
--- a/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
+++ b/drivers/gpu/drm/amd/display/dc/dce/dmub_psr.c
@@ -80,14 +80,26 @@ static enum dc_psr_state convert_psr_state(uint32_t raw_state)
static void dmub_psr_get_state(struct dmub_psr *dmub, enum dc_psr_state *state)
{
struct dmub_srv *srv = dmub->ctx->dmub_srv->dmub;
- uint32_t raw_state;
-
- // Send gpint command and wait for ack
- dmub_srv_send_gpint_command(srv, DMUB_GPINT__GET_PSR_STATE, 0, 30);
-
- dmub_srv_get_gpint_response(srv, &raw_state);
-
- *state = convert_psr_state(raw_state);
+ uint32_t raw_state = 0;
+ uint32_t retry_count = 0;
+ enum dmub_status status;
+
+ do {
+ // Send gpint command and wait for ack
+ status = dmub_srv_send_gpint_command(srv, DMUB_GPINT__GET_PSR_STATE, 0, 30);
+
+ if (status == DMUB_STATUS_OK) {
+ // GPINT was executed, get response
+ dmub_srv_get_gpint_response(srv, &raw_state);
+ *state = convert_psr_state(raw_state);
+ } else
+ // Return invalid state when GPINT times out
+ *state = PSR_STATE_INVALID;
+
+ // Assert if max retry hit
+ if (retry_count >= 1000)
+ ASSERT(0);
+ } while (++retry_count <= 1000 && *state == PSR_STATE_INVALID);
}
/*
@@ -216,6 +228,7 @@ static bool dmub_psr_copy_settings(struct dmub_psr *dmub,
res_ctx->pipe_ctx[i].stream->link == link &&
res_ctx->pipe_ctx[i].stream->link->connector_signal == SIGNAL_TYPE_EDP) {
pipe_ctx = &res_ctx->pipe_ctx[i];
+ //TODO: refactor for multi edp support
break;
}
}
@@ -269,9 +282,10 @@ static bool dmub_psr_copy_settings(struct dmub_psr *dmub,
copy_settings_data->frame_cap_ind = psr_context->psrFrameCaptureIndicationReq;
copy_settings_data->init_sdp_deadline = psr_context->sdpTransmitLineNumDeadline;
copy_settings_data->debug.u32All = 0;
- copy_settings_data->debug.bitfields.visual_confirm = dc->dc->debug.visual_confirm == VISUAL_CONFIRM_PSR ?
- true : false;
+ copy_settings_data->debug.bitfields.visual_confirm = dc->dc->debug.visual_confirm == VISUAL_CONFIRM_PSR;
copy_settings_data->debug.bitfields.use_hw_lock_mgr = 1;
+ copy_settings_data->fec_enable_status = (link->fec_state == dc_link_fec_enabled);
+ copy_settings_data->fec_enable_delay_in100us = link->dc->debug.fec_enable_delay_in100us;
dc_dmub_srv_cmd_queue(dc->dmub_srv, &cmd);
dc_dmub_srv_cmd_execute(dc->dmub_srv);
diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
index caee1c9f54bd..5ddeee96bf23 100644
--- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_hw_sequencer.c
@@ -48,6 +48,7 @@
#include "stream_encoder.h"
#include "link_encoder.h"
#include "link_hwss.h"
+#include "dc_link_dp.h"
#include "clock_source.h"
#include "clk_mgr.h"
#include "abm.h"
@@ -797,6 +798,7 @@ void dce110_edp_power_control(
struct dc_context *ctx = link->ctx;
struct bp_transmitter_control cntl = { 0 };
enum bp_result bp_result;
+ uint8_t panel_instance;
if (dal_graphics_object_id_get_connector_id(link->link_enc->connector)
@@ -807,7 +809,6 @@ void dce110_edp_power_control(
if (!link->panel_cntl)
return;
-
if (power_up !=
link->panel_cntl->funcs->is_panel_powered_on(link->panel_cntl)) {
@@ -880,15 +881,18 @@ void dce110_edp_power_control(
cntl.coherent = false;
cntl.lanes_number = LANE_COUNT_FOUR;
cntl.hpd_sel = link->link_enc->hpd_source;
+ panel_instance = link->panel_cntl->inst;
if (ctx->dc->ctx->dmub_srv &&
ctx->dc->debug.dmub_command_table) {
if (cntl.action == TRANSMITTER_CONTROL_POWER_ON)
bp_result = ctx->dc_bios->funcs->enable_lvtma_control(ctx->dc_bios,
- LVTMA_CONTROL_POWER_ON);
+ LVTMA_CONTROL_POWER_ON,
+ panel_instance);
else
bp_result = ctx->dc_bios->funcs->enable_lvtma_control(ctx->dc_bios,
- LVTMA_CONTROL_POWER_OFF);
+ LVTMA_CONTROL_POWER_OFF,
+ panel_instance);
}
bp_result = link_transmitter_control(ctx->dc_bios, &cntl);
@@ -963,6 +967,7 @@ void dce110_edp_backlight_control(
{
struct dc_context *ctx = link->ctx;
struct bp_transmitter_control cntl = { 0 };
+ uint8_t panel_instance;
if (dal_graphics_object_id_get_connector_id(link->link_enc->connector)
!= CONNECTOR_ID_EDP) {
@@ -1011,6 +1016,7 @@ void dce110_edp_backlight_control(
*/
/* dc_service_sleep_in_milliseconds(50); */
/*edp 1.2*/
+ panel_instance = link->panel_cntl->inst;
if (cntl.action == TRANSMITTER_CONTROL_BACKLIGHT_ON)
edp_receiver_ready_T7(link);
@@ -1018,10 +1024,12 @@ void dce110_edp_backlight_control(
ctx->dc->debug.dmub_command_table) {
if (cntl.action == TRANSMITTER_CONTROL_BACKLIGHT_ON)
ctx->dc_bios->funcs->enable_lvtma_control(ctx->dc_bios,
- LVTMA_CONTROL_LCD_BLON);
+ LVTMA_CONTROL_LCD_BLON,
+ panel_instance);
else
ctx->dc_bios->funcs->enable_lvtma_control(ctx->dc_bios,
- LVTMA_CONTROL_LCD_BLOFF);
+ LVTMA_CONTROL_LCD_BLOFF,
+ panel_instance);
}
link_transmitter_control(ctx->dc_bios, &cntl);
@@ -1629,34 +1637,39 @@ static void disable_vga_and_power_gate_all_controllers(
}
-static struct dc_stream_state *get_edp_stream(struct dc_state *context)
+static void get_edp_streams(struct dc_state *context,
+ struct dc_stream_state **edp_streams,
+ int *edp_stream_num)
{
int i;
+ *edp_stream_num = 0;
for (i = 0; i < context->stream_count; i++) {
- if (context->streams[i]->signal == SIGNAL_TYPE_EDP)
- return context->streams[i];
+ if (context->streams[i]->signal == SIGNAL_TYPE_EDP) {
+ edp_streams[*edp_stream_num] = context->streams[i];
+ if (++(*edp_stream_num) == MAX_NUM_EDP)
+ return;
+ }
}
- return NULL;
}
-static struct dc_link *get_edp_link_with_sink(
+static void get_edp_links_with_sink(
struct dc *dc,
- struct dc_state *context)
+ struct dc_link **edp_links_with_sink,
+ int *edp_with_sink_num)
{
int i;
- struct dc_link *link = NULL;
/* check if there is an eDP panel not in use */
+ *edp_with_sink_num = 0;
for (i = 0; i < dc->link_count; i++) {
if (dc->links[i]->local_sink &&
dc->links[i]->local_sink->sink_signal == SIGNAL_TYPE_EDP) {
- link = dc->links[i];
- break;
+ edp_links_with_sink[*edp_with_sink_num] = dc->links[i];
+ if (++(*edp_with_sink_num) == MAX_NUM_EDP)
+ return;
}
}
-
- return link;
}
/*
@@ -1668,36 +1681,53 @@ static struct dc_link *get_edp_link_with_sink(
*/
void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context)
{
- int i;
- struct dc_link *edp_link_with_sink = get_edp_link_with_sink(dc, context);
- struct dc_link *edp_link = get_edp_link(dc);
+ struct dc_link *edp_links_with_sink[MAX_NUM_EDP];
+ struct dc_link *edp_links[MAX_NUM_EDP];
+ struct dc_stream_state *edp_streams[MAX_NUM_EDP];
+ struct dc_link *edp_link_with_sink = NULL;
+ struct dc_link *edp_link = NULL;
struct dc_stream_state *edp_stream = NULL;
+ struct dce_hwseq *hws = dc->hwseq;
+ int edp_with_sink_num;
+ int edp_num;
+ int edp_stream_num;
+ int i;
bool can_apply_edp_fast_boot = false;
bool can_apply_seamless_boot = false;
bool keep_edp_vdd_on = false;
- struct dce_hwseq *hws = dc->hwseq;
+ DC_LOGGER_INIT();
+
+
+ get_edp_links_with_sink(dc, edp_links_with_sink, &edp_with_sink_num);
+ get_edp_links(dc, edp_links, &edp_num);
if (hws->funcs.init_pipes)
hws->funcs.init_pipes(dc, context);
- edp_stream = get_edp_stream(context);
+ get_edp_streams(context, edp_streams, &edp_stream_num);
// Check fastboot support, disable on DCE8 because of blank screens
- if (edp_link && dc->ctx->dce_version != DCE_VERSION_8_0 &&
+ if (edp_num && dc->ctx->dce_version != DCE_VERSION_8_0 &&
dc->ctx->dce_version != DCE_VERSION_8_1 &&
dc->ctx->dce_version != DCE_VERSION_8_3) {
-
- // enable fastboot if backend is enabled on eDP
- if (edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc)) {
- /* Set optimization flag on eDP stream*/
- if (edp_stream && edp_link->link_status.link_active) {
- edp_stream->apply_edp_fast_boot_optimization = true;
- can_apply_edp_fast_boot = true;
+ for (i = 0; i < edp_num; i++) {
+ edp_link = edp_links[i];
+ // enable fastboot if backend is enabled on eDP
+ if (edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc)) {
+ /* Set optimization flag on eDP stream*/
+ if (edp_stream_num && edp_link->link_status.link_active) {
+ edp_stream = edp_streams[0];
+ can_apply_edp_fast_boot = !is_edp_ilr_optimization_required(edp_stream->link, &edp_stream->timing);
+ edp_stream->apply_edp_fast_boot_optimization = can_apply_edp_fast_boot;
+ if (can_apply_edp_fast_boot)
+ DC_LOG_EVENT_LINK_TRAINING("eDP fast boot disabled to optimize link rate\n");
+
+ break;
+ }
}
}
-
// We are trying to enable eDP, don't power down VDD
- if (edp_stream)
+ if (edp_stream_num)
keep_edp_vdd_on = true;
}
@@ -1712,6 +1742,9 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context)
/* eDP should not have stream in resume from S4 and so even with VBios post
* it should get turned off
*/
+ if (edp_with_sink_num)
+ edp_link_with_sink = edp_links_with_sink[0];
+
if (!can_apply_edp_fast_boot && !can_apply_seamless_boot) {
if (edp_link_with_sink && !keep_edp_vdd_on) {
/*turn off backlight before DP_blank and encoder powered down*/
@@ -1723,7 +1756,7 @@ void dce110_enable_accelerated_mode(struct dc *dc, struct dc_state *context)
if (edp_link_with_sink && !keep_edp_vdd_on)
dc->hwss.edp_power_control(edp_link_with_sink, false);
}
- bios_set_scratch_acc_mode_change(dc->ctx->dc_bios);
+ bios_set_scratch_acc_mode_change(dc->ctx->dc_bios, 1);
}
static uint32_t compute_pstate_blackout_duration(
@@ -1819,8 +1852,7 @@ void dce110_set_safe_displaymarks(
******************************************************************************/
static void set_drr(struct pipe_ctx **pipe_ctx,
- int num_pipes, unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number)
+ int num_pipes, struct dc_crtc_timing_adjust adjust)
{
int i = 0;
struct drr_params params = {0};
@@ -1829,8 +1861,8 @@ static void set_drr(struct pipe_ctx **pipe_ctx,
// Note DRR trigger events are generated regardless of whether num frames met.
unsigned int num_frames = 2;
- params.vertical_total_max = vmax;
- params.vertical_total_min = vmin;
+ params.vertical_total_max = adjust.v_total_max;
+ params.vertical_total_min = adjust.v_total_min;
/* TODO: If multiple pipes are to be supported, you need
* some GSL stuff. Static screen triggers may be programmed differently
@@ -1840,7 +1872,7 @@ static void set_drr(struct pipe_ctx **pipe_ctx,
pipe_ctx[i]->stream_res.tg->funcs->set_drr(
pipe_ctx[i]->stream_res.tg, &params);
- if (vmax != 0 && vmin != 0)
+ if (adjust.v_total_max != 0 && adjust.v_total_min != 0)
pipe_ctx[i]->stream_res.tg->funcs->set_static_screen_control(
pipe_ctx[i]->stream_res.tg,
event_triggers, num_frames);
diff --git a/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c b/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c
index d7fcc5cccdce..ef56eab4e5da 100644
--- a/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dce110/dce110_resource.c
@@ -1272,6 +1272,8 @@ static bool underlay_create(struct dc_context *ctx, struct resource_pool *pool)
/* update the public caps to indicate an underlay is available */
ctx->dc->caps.max_slave_planes = 1;
+ ctx->dc->caps.max_slave_yuv_planes = 1;
+ ctx->dc->caps.max_slave_rgb_planes = 0;
return true;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c b/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c
index 612450f99278..725d92e40cd3 100644
--- a/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dce80/dce80_resource.c
@@ -526,7 +526,7 @@ static struct output_pixel_processor *dce80_opp_create(
return &opp->base;
}
-struct dce_aux *dce80_aux_engine_create(
+static struct dce_aux *dce80_aux_engine_create(
struct dc_context *ctx,
uint32_t inst)
{
@@ -564,7 +564,7 @@ static const struct dce_i2c_mask i2c_masks = {
I2C_COMMON_MASK_SH_LIST_DCE_COMMON_BASE(_MASK)
};
-struct dce_i2c_hw *dce80_i2c_hw_create(
+static struct dce_i2c_hw *dce80_i2c_hw_create(
struct dc_context *ctx,
uint32_t inst)
{
@@ -580,7 +580,7 @@ struct dce_i2c_hw *dce80_i2c_hw_create(
return dce_i2c_hw;
}
-struct dce_i2c_sw *dce80_i2c_sw_create(
+static struct dce_i2c_sw *dce80_i2c_sw_create(
struct dc_context *ctx)
{
struct dce_i2c_sw *dce_i2c_sw =
@@ -714,7 +714,7 @@ static const struct encoder_feature_support link_enc_feature = {
.flags.bits.IS_TPS3_CAPABLE = true
};
-struct link_encoder *dce80_link_encoder_create(
+static struct link_encoder *dce80_link_encoder_create(
const struct encoder_init_data *enc_init_data)
{
struct dce110_link_encoder *enc110 =
@@ -753,7 +753,7 @@ static struct panel_cntl *dce80_panel_cntl_create(const struct panel_cntl_init_d
return &panel_cntl->base;
}
-struct clock_source *dce80_clock_source_create(
+static struct clock_source *dce80_clock_source_create(
struct dc_context *ctx,
struct dc_bios *bios,
enum clock_source_id id,
@@ -777,7 +777,7 @@ struct clock_source *dce80_clock_source_create(
return NULL;
}
-void dce80_clock_source_destroy(struct clock_source **clk_src)
+static void dce80_clock_source_destroy(struct clock_source **clk_src)
{
kfree(TO_DCE110_CLK_SRC(*clk_src));
*clk_src = NULL;
@@ -867,7 +867,7 @@ static void dce80_resource_destruct(struct dce110_resource_pool *pool)
}
}
-bool dce80_validate_bandwidth(
+static bool dce80_validate_bandwidth(
struct dc *dc,
struct dc_state *context,
bool fast_validate)
@@ -912,7 +912,7 @@ static bool dce80_validate_surface_sets(
return true;
}
-enum dc_status dce80_validate_global(
+static enum dc_status dce80_validate_global(
struct dc *dc,
struct dc_state *context)
{
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.c
index 9e796dfeac20..714c71a5fbde 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.c
@@ -1257,6 +1257,16 @@ void hubp1_soft_reset(struct hubp *hubp, bool reset)
REG_UPDATE(DCHUBP_CNTL, HUBP_DISABLE, reset ? 1 : 0);
}
+void hubp1_set_flip_int(struct hubp *hubp)
+{
+ struct dcn10_hubp *hubp1 = TO_DCN10_HUBP(hubp);
+
+ REG_UPDATE(DCSURF_SURFACE_FLIP_INTERRUPT,
+ SURFACE_FLIP_INT_MASK, 1);
+
+ return;
+}
+
void hubp1_init(struct hubp *hubp)
{
//do nothing
@@ -1290,6 +1300,7 @@ static const struct hubp_funcs dcn10_hubp_funcs = {
.dmdata_load = NULL,
.hubp_soft_reset = hubp1_soft_reset,
.hubp_in_blank = hubp1_in_blank,
+ .hubp_set_flip_int = hubp1_set_flip_int,
};
/*****************************************/
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.h b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.h
index a9a6ed7f4f99..e2f2f6995935 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hubp.h
@@ -74,6 +74,7 @@
SRI(DCSURF_SURFACE_EARLIEST_INUSE_C, HUBPREQ, id),\
SRI(DCSURF_SURFACE_EARLIEST_INUSE_HIGH_C, HUBPREQ, id),\
SRI(DCSURF_SURFACE_CONTROL, HUBPREQ, id),\
+ SRI(DCSURF_SURFACE_FLIP_INTERRUPT, HUBPREQ, id),\
SRI(HUBPRET_CONTROL, HUBPRET, id),\
SRI(DCN_EXPANSION_MODE, HUBPREQ, id),\
SRI(DCHUBP_REQ_SIZE_CONFIG, HUBP, id),\
@@ -183,6 +184,7 @@
uint32_t DCSURF_SURFACE_EARLIEST_INUSE_C; \
uint32_t DCSURF_SURFACE_EARLIEST_INUSE_HIGH_C; \
uint32_t DCSURF_SURFACE_CONTROL; \
+ uint32_t DCSURF_SURFACE_FLIP_INTERRUPT; \
uint32_t HUBPRET_CONTROL; \
uint32_t DCN_EXPANSION_MODE; \
uint32_t DCHUBP_REQ_SIZE_CONFIG; \
@@ -332,6 +334,7 @@
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_META_SURFACE_TMZ_C, mask_sh),\
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_EN, mask_sh),\
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_IND_64B_BLK, mask_sh),\
+ HUBP_SF(HUBPREQ0_DCSURF_SURFACE_FLIP_INTERRUPT, SURFACE_FLIP_INT_MASK, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, DET_BUF_PLANE1_BASE_ADDRESS, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CB_B, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CR_R, mask_sh),\
@@ -531,6 +534,7 @@
type PRIMARY_SURFACE_DCC_IND_64B_BLK;\
type SECONDARY_SURFACE_DCC_EN;\
type SECONDARY_SURFACE_DCC_IND_64B_BLK;\
+ type SURFACE_FLIP_INT_MASK;\
type DET_BUF_PLANE1_BASE_ADDRESS;\
type CROSSBAR_SRC_CB_B;\
type CROSSBAR_SRC_CR_R;\
@@ -777,4 +781,6 @@ void hubp1_read_state_common(struct hubp *hubp);
bool hubp1_in_blank(struct hubp *hubp);
void hubp1_soft_reset(struct hubp *hubp, bool reset);
+void hubp1_set_flip_int(struct hubp *hubp);
+
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
index 89912bb5014f..7c939c0a977b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
@@ -53,6 +53,7 @@
#include "dsc.h"
#include "dce/dmub_hw_lock_mgr.h"
#include "dc_trace.h"
+#include "dce/dmub_outbox.h"
#define DC_LOGGER_INIT(logger)
@@ -1355,6 +1356,10 @@ void dcn10_init_hw(struct dc *dc)
hws->funcs.dsc_pg_control(hws, res_pool->dscs[i]->inst, false);
}
+ /* Enable outbox notification feature of dmub */
+ if (dc->debug.enable_dmub_aux_for_legacy_ddc)
+ dmub_enable_outbox_notification(dc);
+
/* we want to turn off all dp displays before doing detection */
if (dc->config.power_down_display_on_boot) {
uint8_t dpcd_power_state = '\0';
@@ -1457,19 +1462,26 @@ void dcn10_init_hw(struct dc *dc)
*/
void dcn10_power_down_on_boot(struct dc *dc)
{
- int i = 0;
+ struct dc_link *edp_links[MAX_NUM_EDP];
struct dc_link *edp_link;
+ int edp_num;
+ int i = 0;
- edp_link = get_edp_link(dc);
- if (edp_link &&
- edp_link->link_enc->funcs->is_dig_enabled &&
- edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc) &&
- dc->hwseq->funcs.edp_backlight_control &&
- dc->hwss.power_down &&
- dc->hwss.edp_power_control) {
- dc->hwseq->funcs.edp_backlight_control(edp_link, false);
- dc->hwss.power_down(dc);
- dc->hwss.edp_power_control(edp_link, false);
+ get_edp_links(dc, edp_links, &edp_num);
+
+ if (edp_num) {
+ for (i = 0; i < edp_num; i++) {
+ edp_link = edp_links[i];
+ if (edp_link->link_enc->funcs->is_dig_enabled &&
+ edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc) &&
+ dc->hwseq->funcs.edp_backlight_control &&
+ dc->hwss.power_down &&
+ dc->hwss.edp_power_control) {
+ dc->hwseq->funcs.edp_backlight_control(edp_link, false);
+ dc->hwss.power_down(dc);
+ dc->hwss.edp_power_control(edp_link, false);
+ }
+ }
} else {
for (i = 0; i < dc->link_count; i++) {
struct dc_link *link = dc->links[i];
@@ -1851,6 +1863,230 @@ static bool wait_for_reset_trigger_to_occur(
return rc;
}
+uint64_t reduceSizeAndFraction(
+ uint64_t *numerator,
+ uint64_t *denominator,
+ bool checkUint32Bounary)
+{
+ int i;
+ bool ret = checkUint32Bounary == false;
+ uint64_t max_int32 = 0xffffffff;
+ uint64_t num, denom;
+ static const uint16_t prime_numbers[] = {
+ 2, 3, 5, 7, 11, 13, 17, 19, 23, 29, 31, 37, 41, 43,
+ 47, 53, 59, 61, 67, 71, 73, 79, 83, 89, 97, 101, 103,
+ 107, 109, 113, 127, 131, 137, 139, 149, 151, 157, 163,
+ 167, 173, 179, 181, 191, 193, 197, 199, 211, 223, 227,
+ 229, 233, 239, 241, 251, 257, 263, 269, 271, 277, 281,
+ 283, 293, 307, 311, 313, 317, 331, 337, 347, 349, 353,
+ 359, 367, 373, 379, 383, 389, 397, 401, 409, 419, 421,
+ 431, 433, 439, 443, 449, 457, 461, 463, 467, 479, 487,
+ 491, 499, 503, 509, 521, 523, 541, 547, 557, 563, 569,
+ 571, 577, 587, 593, 599, 601, 607, 613, 617, 619, 631,
+ 641, 643, 647, 653, 659, 661, 673, 677, 683, 691, 701,
+ 709, 719, 727, 733, 739, 743, 751, 757, 761, 769, 773,
+ 787, 797, 809, 811, 821, 823, 827, 829, 839, 853, 857,
+ 859, 863, 877, 881, 883, 887, 907, 911, 919, 929, 937,
+ 941, 947, 953, 967, 971, 977, 983, 991, 997};
+ int count = ARRAY_SIZE(prime_numbers);
+
+ num = *numerator;
+ denom = *denominator;
+ for (i = 0; i < count; i++) {
+ uint32_t num_remainder, denom_remainder;
+ uint64_t num_result, denom_result;
+ if (checkUint32Bounary &&
+ num <= max_int32 && denom <= max_int32) {
+ ret = true;
+ break;
+ }
+ do {
+ num_result = div_u64_rem(num, prime_numbers[i], &num_remainder);
+ denom_result = div_u64_rem(denom, prime_numbers[i], &denom_remainder);
+ if (num_remainder == 0 && denom_remainder == 0) {
+ num = num_result;
+ denom = denom_result;
+ }
+ } while (num_remainder == 0 && denom_remainder == 0);
+ }
+ *numerator = num;
+ *denominator = denom;
+ return ret;
+}
+
+bool is_low_refresh_rate(struct pipe_ctx *pipe)
+{
+ uint32_t master_pipe_refresh_rate =
+ pipe->stream->timing.pix_clk_100hz * 100 /
+ pipe->stream->timing.h_total /
+ pipe->stream->timing.v_total;
+ return master_pipe_refresh_rate <= 30;
+}
+
+uint8_t get_clock_divider(struct pipe_ctx *pipe, bool account_low_refresh_rate)
+{
+ uint32_t clock_divider = 1;
+ uint32_t numpipes = 1;
+
+ if (account_low_refresh_rate && is_low_refresh_rate(pipe))
+ clock_divider *= 2;
+
+ if (pipe->stream_res.pix_clk_params.pixel_encoding == PIXEL_ENCODING_YCBCR420)
+ clock_divider *= 2;
+
+ while (pipe->next_odm_pipe) {
+ pipe = pipe->next_odm_pipe;
+ numpipes++;
+ }
+ clock_divider *= numpipes;
+
+ return clock_divider;
+}
+
+int dcn10_align_pixel_clocks(
+ struct dc *dc,
+ int group_size,
+ struct pipe_ctx *grouped_pipes[])
+{
+ struct dc_context *dc_ctx = dc->ctx;
+ int i, master = -1, embedded = -1;
+ struct dc_crtc_timing hw_crtc_timing[MAX_PIPES] = {0};
+ uint64_t phase[MAX_PIPES];
+ uint64_t modulo[MAX_PIPES];
+ unsigned int pclk;
+
+ uint32_t embedded_pix_clk_100hz;
+ uint16_t embedded_h_total;
+ uint16_t embedded_v_total;
+ bool clamshell_closed = false;
+ uint32_t dp_ref_clk_100hz =
+ dc->res_pool->dp_clock_source->ctx->dc->clk_mgr->dprefclk_khz*10;
+
+ if (dc->config.vblank_alignment_dto_params &&
+ dc->res_pool->dp_clock_source->funcs->override_dp_pix_clk) {
+ clamshell_closed =
+ (dc->config.vblank_alignment_dto_params >> 63);
+ embedded_h_total =
+ (dc->config.vblank_alignment_dto_params >> 32) & 0x7FFF;
+ embedded_v_total =
+ (dc->config.vblank_alignment_dto_params >> 48) & 0x7FFF;
+ embedded_pix_clk_100hz =
+ dc->config.vblank_alignment_dto_params & 0xFFFFFFFF;
+
+ for (i = 0; i < group_size; i++) {
+ grouped_pipes[i]->stream_res.tg->funcs->get_hw_timing(
+ grouped_pipes[i]->stream_res.tg,
+ &hw_crtc_timing[i]);
+ dc->res_pool->dp_clock_source->funcs->get_pixel_clk_frequency_100hz(
+ dc->res_pool->dp_clock_source,
+ grouped_pipes[i]->stream_res.tg->inst,
+ &pclk);
+ hw_crtc_timing[i].pix_clk_100hz = pclk;
+ if (dc_is_embedded_signal(
+ grouped_pipes[i]->stream->signal)) {
+ embedded = i;
+ master = i;
+ phase[i] = embedded_pix_clk_100hz*100;
+ modulo[i] = dp_ref_clk_100hz*100;
+ } else {
+
+ phase[i] = (uint64_t)embedded_pix_clk_100hz*
+ hw_crtc_timing[i].h_total*
+ hw_crtc_timing[i].v_total;
+ phase[i] = div_u64(phase[i], get_clock_divider(grouped_pipes[i], true));
+ modulo[i] = (uint64_t)dp_ref_clk_100hz*
+ embedded_h_total*
+ embedded_v_total;
+
+ if (reduceSizeAndFraction(&phase[i],
+ &modulo[i], true) == false) {
+ /*
+ * this will help to stop reporting
+ * this timing synchronizable
+ */
+ DC_SYNC_INFO("Failed to reduce DTO parameters\n");
+ grouped_pipes[i]->stream->has_non_synchronizable_pclk = true;
+ }
+ }
+ }
+
+ for (i = 0; i < group_size; i++) {
+ if (i != embedded && !grouped_pipes[i]->stream->has_non_synchronizable_pclk) {
+ dc->res_pool->dp_clock_source->funcs->override_dp_pix_clk(
+ dc->res_pool->dp_clock_source,
+ grouped_pipes[i]->stream_res.tg->inst,
+ phase[i], modulo[i]);
+ dc->res_pool->dp_clock_source->funcs->get_pixel_clk_frequency_100hz(
+ dc->res_pool->dp_clock_source,
+ grouped_pipes[i]->stream_res.tg->inst, &pclk);
+ grouped_pipes[i]->stream->timing.pix_clk_100hz =
+ pclk*get_clock_divider(grouped_pipes[i], false);
+ if (master == -1)
+ master = i;
+ }
+ }
+
+ }
+ return master;
+}
+
+void dcn10_enable_vblanks_synchronization(
+ struct dc *dc,
+ int group_index,
+ int group_size,
+ struct pipe_ctx *grouped_pipes[])
+{
+ struct dc_context *dc_ctx = dc->ctx;
+ struct output_pixel_processor *opp;
+ struct timing_generator *tg;
+ int i, width, height, master;
+
+ for (i = 1; i < group_size; i++) {
+ opp = grouped_pipes[i]->stream_res.opp;
+ tg = grouped_pipes[i]->stream_res.tg;
+ tg->funcs->get_otg_active_size(tg, &width, &height);
+ if (opp->funcs->opp_program_dpg_dimensions)
+ opp->funcs->opp_program_dpg_dimensions(opp, width, 2*(height) + 1);
+ }
+
+ for (i = 0; i < group_size; i++) {
+ if (grouped_pipes[i]->stream == NULL)
+ continue;
+ grouped_pipes[i]->stream->vblank_synchronized = false;
+ grouped_pipes[i]->stream->has_non_synchronizable_pclk = false;
+ }
+
+ DC_SYNC_INFO("Aligning DP DTOs\n");
+
+ master = dcn10_align_pixel_clocks(dc, group_size, grouped_pipes);
+
+ DC_SYNC_INFO("Synchronizing VBlanks\n");
+
+ if (master >= 0) {
+ for (i = 0; i < group_size; i++) {
+ if (i != master && !grouped_pipes[i]->stream->has_non_synchronizable_pclk)
+ grouped_pipes[i]->stream_res.tg->funcs->align_vblanks(
+ grouped_pipes[master]->stream_res.tg,
+ grouped_pipes[i]->stream_res.tg,
+ grouped_pipes[master]->stream->timing.pix_clk_100hz,
+ grouped_pipes[i]->stream->timing.pix_clk_100hz,
+ get_clock_divider(grouped_pipes[master], false),
+ get_clock_divider(grouped_pipes[i], false));
+ grouped_pipes[i]->stream->vblank_synchronized = true;
+ }
+ grouped_pipes[master]->stream->vblank_synchronized = true;
+ DC_SYNC_INFO("Sync complete\n");
+ }
+
+ for (i = 1; i < group_size; i++) {
+ opp = grouped_pipes[i]->stream_res.opp;
+ tg = grouped_pipes[i]->stream_res.tg;
+ tg->funcs->get_otg_active_size(tg, &width, &height);
+ if (opp->funcs->opp_program_dpg_dimensions)
+ opp->funcs->opp_program_dpg_dimensions(opp, width, height);
+ }
+}
+
void dcn10_enable_timing_synchronization(
struct dc *dc,
int group_index,
@@ -1872,6 +2108,12 @@ void dcn10_enable_timing_synchronization(
opp->funcs->opp_program_dpg_dimensions(opp, width, 2*(height) + 1);
}
+ for (i = 0; i < group_size; i++) {
+ if (grouped_pipes[i]->stream == NULL)
+ continue;
+ grouped_pipes[i]->stream->vblank_synchronized = false;
+ }
+
for (i = 1; i < group_size; i++)
grouped_pipes[i]->stream_res.tg->funcs->enable_reset_trigger(
grouped_pipes[i]->stream_res.tg,
@@ -2196,6 +2438,13 @@ static void dcn10_enable_plane(
if (dc->debug.sanity_checks) {
hws->funcs.verify_allow_pstate_change_high(dc);
}
+
+ if (!pipe_ctx->top_pipe
+ && pipe_ctx->plane_state
+ && pipe_ctx->plane_state->flip_int_enabled
+ && pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int)
+ pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int(pipe_ctx->plane_res.hubp);
+
}
void dcn10_program_gamut_remap(struct pipe_ctx *pipe_ctx)
@@ -2635,7 +2884,7 @@ static void dcn10_update_dchubp_dpp(
hws->funcs.update_plane_addr(dc, pipe_ctx);
if (is_pipe_tree_visible(pipe_ctx))
- dc->hwss.set_hubp_blank(dc, pipe_ctx, false);
+ hubp->funcs->set_blank(hubp, false);
}
void dcn10_blank_pixel_data(
@@ -3022,8 +3271,7 @@ void dcn10_optimize_bandwidth(
}
void dcn10_set_drr(struct pipe_ctx **pipe_ctx,
- int num_pipes, unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number)
+ int num_pipes, struct dc_crtc_timing_adjust adjust)
{
int i = 0;
struct drr_params params = {0};
@@ -3032,11 +3280,10 @@ void dcn10_set_drr(struct pipe_ctx **pipe_ctx,
// Note DRR trigger events are generated regardless of whether num frames met.
unsigned int num_frames = 2;
- params.vertical_total_max = vmax;
- params.vertical_total_min = vmin;
- params.vertical_total_mid = vmid;
- params.vertical_total_mid_frame_num = vmid_frame_number;
-
+ params.vertical_total_max = adjust.v_total_max;
+ params.vertical_total_min = adjust.v_total_min;
+ params.vertical_total_mid = adjust.v_total_mid;
+ params.vertical_total_mid_frame_num = adjust.v_total_mid_frame_num;
/* TODO: If multiple pipes are to be supported, you need
* some GSL stuff. Static screen triggers may be programmed differently
* as well.
@@ -3044,7 +3291,7 @@ void dcn10_set_drr(struct pipe_ctx **pipe_ctx,
for (i = 0; i < num_pipes; i++) {
pipe_ctx[i]->stream_res.tg->funcs->set_drr(
pipe_ctx[i]->stream_res.tg, &params);
- if (vmax != 0 && vmin != 0)
+ if (adjust.v_total_max != 0 && adjust.v_total_min != 0)
pipe_ctx[i]->stream_res.tg->funcs->set_static_screen_control(
pipe_ctx[i]->stream_res.tg,
event_triggers, num_frames);
@@ -3146,16 +3393,13 @@ void dcn10_setup_stereo(struct pipe_ctx *pipe_ctx, struct dc *dc)
return;
}
-static struct pipe_ctx *get_pipe_ctx_by_hubp_inst(struct dc_state *context, int mpcc_inst)
+static struct hubp *get_hubp_by_inst(struct resource_pool *res_pool, int mpcc_inst)
{
int i;
- for (i = 0; i < MAX_PIPES; i++) {
- if (context->res_ctx.pipe_ctx[i].plane_res.hubp
- && context->res_ctx.pipe_ctx[i].plane_res.hubp->inst == mpcc_inst) {
- return &context->res_ctx.pipe_ctx[i];
- }
-
+ for (i = 0; i < res_pool->pipe_count; i++) {
+ if (res_pool->hubps[i]->inst == mpcc_inst)
+ return res_pool->hubps[i];
}
ASSERT(false);
return NULL;
@@ -3178,23 +3422,11 @@ void dcn10_wait_for_mpcc_disconnect(
for (mpcc_inst = 0; mpcc_inst < MAX_PIPES; mpcc_inst++) {
if (pipe_ctx->stream_res.opp->mpcc_disconnect_pending[mpcc_inst]) {
- struct pipe_ctx *restore_bottom_pipe;
- struct pipe_ctx *restore_top_pipe;
- struct pipe_ctx *inst_pipe_ctx = get_pipe_ctx_by_hubp_inst(dc->current_state, mpcc_inst);
+ struct hubp *hubp = get_hubp_by_inst(res_pool, mpcc_inst);
- ASSERT(inst_pipe_ctx);
res_pool->mpc->funcs->wait_for_idle(res_pool->mpc, mpcc_inst);
pipe_ctx->stream_res.opp->mpcc_disconnect_pending[mpcc_inst] = false;
- /*
- * Set top and bottom pipes NULL, as we don't want
- * to blank those pipes when disconnecting from MPCC
- */
- restore_bottom_pipe = inst_pipe_ctx->bottom_pipe;
- restore_top_pipe = inst_pipe_ctx->top_pipe;
- inst_pipe_ctx->top_pipe = inst_pipe_ctx->bottom_pipe = NULL;
- dc->hwss.set_hubp_blank(dc, inst_pipe_ctx, true);
- inst_pipe_ctx->top_pipe = restore_top_pipe;
- inst_pipe_ctx->bottom_pipe = restore_bottom_pipe;
+ hubp->funcs->set_blank(hubp, true);
}
}
@@ -3748,9 +3980,18 @@ void dcn10_get_clock(struct dc *dc,
}
-void dcn10_set_hubp_blank(const struct dc *dc,
- struct pipe_ctx *pipe_ctx,
- bool blank_enable)
+void dcn10_get_dcc_en_bits(struct dc *dc, int *dcc_en_bits)
{
- pipe_ctx->plane_res.hubp->funcs->set_blank(pipe_ctx->plane_res.hubp, blank_enable);
+ struct resource_pool *pool = dc->res_pool;
+ int i;
+
+ for (i = 0; i < pool->pipe_count; i++) {
+ struct hubp *hubp = pool->hubps[i];
+ struct dcn_hubp_state *s = &(TO_DCN10_HUBP(hubp)->state);
+
+ hubp->funcs->hubp_read_state(hubp);
+
+ if (!s->blank_en)
+ dcc_en_bits[i] = s->dcc_en ? 1 : 0;
+ }
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.h
index 89e6dfb63da0..37bec421fde8 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.h
@@ -1,5 +1,5 @@
/*
-* Copyright 2016 Advanced Micro Devices, Inc.
+* Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -123,6 +123,11 @@ void dcn10_enable_timing_synchronization(
int group_index,
int group_size,
struct pipe_ctx *grouped_pipes[]);
+void dcn10_enable_vblanks_synchronization(
+ struct dc *dc,
+ int group_index,
+ int group_size,
+ struct pipe_ctx *grouped_pipes[]);
void dcn10_enable_per_frame_crtc_position_reset(
struct dc *dc,
int group_size,
@@ -140,8 +145,7 @@ bool dcn10_dummy_display_power_gating(
struct dc_bios *dcb,
enum pipe_gating_control power_gating);
void dcn10_set_drr(struct pipe_ctx **pipe_ctx,
- int num_pipes, unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number);
+ int num_pipes, struct dc_crtc_timing_adjust adjust);
void dcn10_get_position(struct pipe_ctx **pipe_ctx,
int num_pipes,
struct crtc_position *position);
@@ -204,8 +208,7 @@ void dcn10_wait_for_pending_cleared(struct dc *dc,
struct dc_state *context);
void dcn10_set_hdr_multiplier(struct pipe_ctx *pipe_ctx);
void dcn10_verify_allow_pstate_change_high(struct dc *dc);
-void dcn10_set_hubp_blank(const struct dc *dc,
- struct pipe_ctx *pipe_ctx,
- bool blank_enable);
+
+void dcn10_get_dcc_en_bits(struct dc *dc, int *dcc_en_bits);
#endif /* __DC_HWSS_DCN10_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c
index 2f1b802e66a1..d532c78ee764 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_init.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2016 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -79,7 +79,7 @@ static const struct hw_sequencer_funcs dcn10_funcs = {
.set_backlight_level = dce110_set_backlight_level,
.set_abm_immediate_disable = dce110_set_abm_immediate_disable,
.set_pipe = dce110_set_pipe,
- .set_hubp_blank = dcn10_set_hubp_blank,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn10_private_funcs = {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
index 6138f4887de7..677663cc7bff 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.c
@@ -132,6 +132,22 @@ void optc1_setup_vertical_interrupt2(
}
/**
+ * Vupdate keepout can be set to a window to block the update lock for that pipe from changing.
+ * Start offset begins with vstartup and goes for x number of clocks,
+ * end offset starts from end of vupdate to x number of clocks.
+ */
+void optc1_set_vupdate_keepout(struct timing_generator *optc,
+ struct vupdate_keepout_params *params)
+{
+ struct optc *optc1 = DCN10TG_FROM_TG(optc);
+
+ REG_SET_3(OTG_VUPDATE_KEEPOUT, 0,
+ MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, params->start_offset,
+ MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, params->end_offset,
+ OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, params->enable);
+}
+
+/**
* program_timing_generator used by mode timing set
* Program CRTC Timing Registers - OTG_H_*, OTG_V_*, Pixel repetition.
* Including SYNC. Call BIOS command table to program Timings.
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.h b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.h
index b222c67973d4..cabfe83fd634 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_optc.h
@@ -194,6 +194,9 @@ struct dcn_optc_registers {
SF(OTG0_OTG_DOUBLE_BUFFER_CONTROL, OTG_UPDATE_PENDING, mask_sh),\
SF(OTG0_OTG_DOUBLE_BUFFER_CONTROL, OTG_BLANK_DATA_DOUBLE_BUFFER_EN, mask_sh),\
SF(OTG0_OTG_DOUBLE_BUFFER_CONTROL, OTG_RANGE_TIMING_DBUF_UPDATE_MODE, mask_sh),\
+ SF(OTG0_OTG_VUPDATE_KEEPOUT, OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, mask_sh), \
+ SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, mask_sh), \
+ SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, mask_sh), \
SF(OTG0_OTG_H_TOTAL, OTG_H_TOTAL, mask_sh),\
SF(OTG0_OTG_H_BLANK_START_END, OTG_H_BLANK_START, mask_sh),\
SF(OTG0_OTG_H_BLANK_START_END, OTG_H_BLANK_END, mask_sh),\
@@ -212,6 +215,7 @@ struct dcn_optc_registers {
SF(OTG0_OTG_CONTROL, OTG_START_POINT_CNTL, mask_sh),\
SF(OTG0_OTG_CONTROL, OTG_DISABLE_POINT_CNTL, mask_sh),\
SF(OTG0_OTG_CONTROL, OTG_FIELD_NUMBER_CNTL, mask_sh),\
+ SF(OTG0_OTG_CONTROL, OTG_CURRENT_MASTER_EN_STATE, mask_sh),\
SF(OTG0_OTG_STEREO_CONTROL, OTG_STEREO_EN, mask_sh),\
SF(OTG0_OTG_STEREO_CONTROL, OTG_STEREO_SYNC_OUTPUT_LINE_NUM, mask_sh),\
SF(OTG0_OTG_STEREO_CONTROL, OTG_STEREO_SYNC_OUTPUT_POLARITY, mask_sh),\
@@ -352,6 +356,7 @@ struct dcn_optc_registers {
type OTG_START_POINT_CNTL;\
type OTG_DISABLE_POINT_CNTL;\
type OTG_FIELD_NUMBER_CNTL;\
+ type OTG_CURRENT_MASTER_EN_STATE;\
type OTG_STEREO_EN;\
type OTG_STEREO_SYNC_OUTPUT_LINE_NUM;\
type OTG_STEREO_SYNC_OUTPUT_POLARITY;\
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
index 90e912fef2b3..f962b905e79e 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_resource.c
@@ -1420,6 +1420,8 @@ static bool dcn10_resource_construct(
dc->caps.max_cursor_size = 256;
dc->caps.min_horizontal_blanking_period = 80;
dc->caps.max_slave_planes = 1;
+ dc->caps.max_slave_yuv_planes = 1;
+ dc->caps.max_slave_rgb_planes = 0;
dc->caps.is_apu = true;
dc->caps.post_blend_color_processing = false;
dc->caps.extended_aux_timeout_support = false;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c
index 73ac78b16bd4..f1a08a7736ac 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_stream_encoder.c
@@ -738,7 +738,6 @@ void enc1_stream_encoder_update_dp_info_packets(
REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP2_ENABLE, info_frame->spd.valid);
REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP3_ENABLE, info_frame->hdrsmd.valid);
-
/* This bit is the master enable bit.
* When enabling secondary stream engine,
* this master bit must also be set.
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dccg.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dccg.c
index 62cc2651e00c..8774406120fc 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dccg.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_dccg.c
@@ -112,7 +112,7 @@ struct dccg *dccg2_create(
const struct dccg_shift *dccg_shift,
const struct dccg_mask *dccg_mask)
{
- struct dcn_dccg *dccg_dcn = kzalloc(sizeof(*dccg_dcn), GFP_KERNEL);
+ struct dcn_dccg *dccg_dcn = kzalloc(sizeof(*dccg_dcn), GFP_ATOMIC);
struct dccg *base;
if (dccg_dcn == NULL) {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c
index 0df0da2e6a4d..a1318c31bcfa 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hubp.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2012-17 Advanced Micro Devices, Inc.
+ * Copyright 2012-2021 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -181,11 +181,14 @@ void hubp2_vready_at_or_After_vsync(struct hubp *hubp,
else
Set HUBP_VREADY_AT_OR_AFTER_VSYNC = 0
*/
- if ((pipe_dest->vstartup_start - (pipe_dest->vready_offset+pipe_dest->vupdate_width
- + pipe_dest->vupdate_offset) / pipe_dest->htotal) <= pipe_dest->vblank_end) {
- value = 1;
- } else
- value = 0;
+ if (pipe_dest->htotal != 0) {
+ if ((pipe_dest->vstartup_start - (pipe_dest->vready_offset+pipe_dest->vupdate_width
+ + pipe_dest->vupdate_offset) / pipe_dest->htotal) <= pipe_dest->vblank_end) {
+ value = 1;
+ } else
+ value = 0;
+ }
+
REG_UPDATE(DCHUBP_CNTL, HUBP_VREADY_AT_OR_AFTER_VSYNC, value);
}
@@ -1597,6 +1600,7 @@ static struct hubp_funcs dcn20_hubp_funcs = {
.validate_dml_output = hubp2_validate_dml_output,
.hubp_in_blank = hubp1_in_blank,
.hubp_soft_reset = hubp1_soft_reset,
+ .hubp_set_flip_int = hubp1_set_flip_int,
};
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
index 0726fb435e2a..6a10daec15cc 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_hwseq.c
@@ -1146,6 +1146,12 @@ void dcn20_enable_plane(
pipe_ctx->plane_res.hubp->funcs->hubp_set_vm_system_aperture_settings(pipe_ctx->plane_res.hubp, &apt);
}
+ if (!pipe_ctx->top_pipe
+ && pipe_ctx->plane_state
+ && pipe_ctx->plane_state->flip_int_enabled
+ && pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int)
+ pipe_ctx->plane_res.hubp->funcs->hubp_set_flip_int(pipe_ctx->plane_res.hubp);
+
// if (dc->debug.sanity_checks) {
// dcn10_verify_allow_pstate_change_high(dc);
// }
@@ -1501,38 +1507,8 @@ static void dcn20_update_dchubp_dpp(
if (pipe_ctx->update_flags.bits.enable || pipe_ctx->update_flags.bits.opp_changed
|| pipe_ctx->stream->update_flags.bits.gamut_remap
|| pipe_ctx->stream->update_flags.bits.out_csc) {
- struct mpc *mpc = pipe_ctx->stream_res.opp->ctx->dc->res_pool->mpc;
-
- if (mpc->funcs->set_gamut_remap) {
- int i;
- int mpcc_id = hubp->inst;
- struct mpc_grph_gamut_adjustment adjust;
- bool enable_remap_dpp = false;
-
- memset(&adjust, 0, sizeof(adjust));
- adjust.gamut_adjust_type = GRAPHICS_GAMUT_ADJUST_TYPE_BYPASS;
-
- /* save the enablement of gamut remap for dpp */
- enable_remap_dpp = pipe_ctx->stream->gamut_remap_matrix.enable_remap;
-
- /* force bypass gamut remap for dpp/cm */
- pipe_ctx->stream->gamut_remap_matrix.enable_remap = false;
- dc->hwss.program_gamut_remap(pipe_ctx);
-
- /* restore gamut remap flag and use this remap into mpc */
- pipe_ctx->stream->gamut_remap_matrix.enable_remap = enable_remap_dpp;
-
- /* build remap matrix for top plane if enabled */
- if (enable_remap_dpp && pipe_ctx->top_pipe == NULL) {
- adjust.gamut_adjust_type = GRAPHICS_GAMUT_ADJUST_TYPE_SW;
- for (i = 0; i < CSC_TEMPERATURE_MATRIX_SIZE; i++)
- adjust.temperature_matrix[i] =
- pipe_ctx->stream->gamut_remap_matrix.matrix[i];
- }
- mpc->funcs->set_gamut_remap(mpc, mpcc_id, &adjust);
- } else
- /* dpp/cm gamut remap*/
- dc->hwss.program_gamut_remap(pipe_ctx);
+ /* dpp/cm gamut remap*/
+ dc->hwss.program_gamut_remap(pipe_ctx);
/*call the dcn2 method which uses mpc csc*/
dc->hwss.program_output_csc(dc,
@@ -1575,8 +1551,8 @@ static void dcn20_update_dchubp_dpp(
- if (is_pipe_tree_visible(pipe_ctx))
- dc->hwss.set_hubp_blank(dc, pipe_ctx, false);
+ if (pipe_ctx->update_flags.bits.enable)
+ hubp->funcs->set_blank(hubp, false);
}
@@ -1772,10 +1748,19 @@ void dcn20_post_unlock_program_front_end(
for (i = 0; i < dc->res_pool->pipe_count; i++) {
struct pipe_ctx *pipe = &context->res_ctx.pipe_ctx[i];
+ struct pipe_ctx *mpcc_pipe;
if (pipe->vtp_locked) {
- dc->hwss.set_hubp_blank(dc, pipe, true);
+ dc->hwseq->funcs.wait_for_blank_complete(pipe->stream_res.opp);
+ pipe->plane_res.hubp->funcs->set_blank(pipe->plane_res.hubp, true);
pipe->vtp_locked = false;
+
+ for (mpcc_pipe = pipe->bottom_pipe; mpcc_pipe; mpcc_pipe = mpcc_pipe->bottom_pipe)
+ mpcc_pipe->plane_res.hubp->funcs->set_blank(mpcc_pipe->plane_res.hubp, true);
+
+ for (i = 0; i < dc->res_pool->pipe_count; i++)
+ if (context->res_ctx.pipe_ctx[i].update_flags.bits.disable)
+ dc->hwss.disable_plane(dc, &dc->current_state->res_ctx.pipe_ctx[i]);
}
}
/* WA to apply WM setting*/
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c
index 51a4166e9750..b5bb613eed4d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_init.c
@@ -42,6 +42,7 @@ static const struct hw_sequencer_funcs dcn20_funcs = {
.program_output_csc = dcn20_program_output_csc,
.enable_accelerated_mode = dce110_enable_accelerated_mode,
.enable_timing_synchronization = dcn10_enable_timing_synchronization,
+ .enable_vblanks_synchronization = dcn10_enable_vblanks_synchronization,
.enable_per_frame_crtc_position_reset = dcn10_enable_per_frame_crtc_position_reset,
.update_info_frame = dce110_update_info_frame,
.send_immediate_sdp_message = dcn10_send_immediate_sdp_message,
@@ -94,7 +95,7 @@ static const struct hw_sequencer_funcs dcn20_funcs = {
.optimize_timing_for_fsft = dcn20_optimize_timing_for_fsft,
#endif
.set_disp_pattern_generator = dcn20_set_disp_pattern_generator,
- .set_hubp_blank = dcn10_set_hubp_blank,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn20_private_funcs = {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_link_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_link_encoder.c
index fa013496e26b..2f9bfaeaba8d 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_link_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_link_encoder.c
@@ -341,8 +341,7 @@ void enc2_hw_init(struct link_encoder *enc)
} else {
AUX_REG_WRITE(AUX_DPHY_RX_CONTROL0, 0x103d1110);
- AUX_REG_WRITE(AUX_DPHY_TX_CONTROL, 0x21c4d);
-
+ AUX_REG_WRITE(AUX_DPHY_TX_CONTROL, 0x21c7a);
}
//AUX_DPHY_TX_REF_CONTROL'AUX_TX_REF_DIV HW default is 0x32;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c
index d8b18c515d06..3139d90017ee 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.c
@@ -134,22 +134,6 @@ void optc2_set_gsl_window(struct timing_generator *optc,
OTG_GSL_WINDOW_END_Y, params->gsl_window_end_y);
}
-/**
- * Vupdate keepout can be set to a window to block the update lock for that pipe from changing.
- * Start offset begins with vstartup and goes for x number of clocks,
- * end offset starts from end of vupdate to x number of clocks.
- */
-void optc2_set_vupdate_keepout(struct timing_generator *optc,
- const struct vupdate_keepout_params *params)
-{
- struct optc *optc1 = DCN10TG_FROM_TG(optc);
-
- REG_SET_3(OTG_VUPDATE_KEEPOUT, 0,
- MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, params->start_offset,
- MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, params->end_offset,
- OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, params->enable);
-}
-
void optc2_set_gsl_source_select(
struct timing_generator *optc,
int group_idx,
@@ -309,6 +293,129 @@ void optc2_set_dwb_source(struct timing_generator *optc,
OPTC_DWB1_SOURCE_SELECT, optc->inst);
}
+void optc2_align_vblanks(
+ struct timing_generator *optc_master,
+ struct timing_generator *optc_slave,
+ uint32_t master_pixel_clock_100Hz,
+ uint32_t slave_pixel_clock_100Hz,
+ uint8_t master_clock_divider,
+ uint8_t slave_clock_divider)
+{
+ /* accessing slave OTG registers */
+ struct optc *optc1 = DCN10TG_FROM_TG(optc_slave);
+
+ uint32_t master_v_active = 0;
+ uint32_t master_h_total = 0;
+ uint32_t slave_h_total = 0;
+ uint64_t L, XY;
+ uint32_t X, Y, p = 10000;
+ uint32_t master_update_lock;
+
+ /* disable slave OTG */
+ REG_UPDATE(OTG_CONTROL, OTG_MASTER_EN, 0);
+ /* wait until disabled */
+ REG_WAIT(OTG_CONTROL,
+ OTG_CURRENT_MASTER_EN_STATE,
+ 0, 10, 5000);
+
+ REG_GET(OTG_H_TOTAL, OTG_H_TOTAL, &slave_h_total);
+
+ /* assign slave OTG to be controlled by master update lock */
+ REG_SET(OTG_GLOBAL_CONTROL0, 0,
+ OTG_MASTER_UPDATE_LOCK_SEL, optc_master->inst);
+
+ /* accessing master OTG registers */
+ optc1 = DCN10TG_FROM_TG(optc_master);
+
+ /* saving update lock state, not sure if it's needed */
+ REG_GET(OTG_MASTER_UPDATE_LOCK,
+ OTG_MASTER_UPDATE_LOCK, &master_update_lock);
+ /* unlocking master OTG */
+ REG_SET(OTG_MASTER_UPDATE_LOCK, 0,
+ OTG_MASTER_UPDATE_LOCK, 0);
+
+ REG_GET(OTG_V_BLANK_START_END,
+ OTG_V_BLANK_START, &master_v_active);
+ REG_GET(OTG_H_TOTAL, OTG_H_TOTAL, &master_h_total);
+
+ /* calculate when to enable slave OTG */
+ L = (uint64_t)p * slave_h_total * master_pixel_clock_100Hz;
+ L = div_u64(L, master_h_total);
+ L = div_u64(L, slave_pixel_clock_100Hz);
+ XY = div_u64(L, p);
+ Y = master_v_active - XY - 1;
+ X = div_u64(((XY + 1) * p - L) * master_h_total, p * master_clock_divider);
+
+ /*
+ * set master OTG to unlock when V/H
+ * counters reach calculated values
+ */
+ REG_UPDATE(OTG_GLOBAL_CONTROL1,
+ MASTER_UPDATE_LOCK_DB_EN, 1);
+ REG_UPDATE_2(OTG_GLOBAL_CONTROL1,
+ MASTER_UPDATE_LOCK_DB_X,
+ X,
+ MASTER_UPDATE_LOCK_DB_Y,
+ Y);
+
+ /* lock master OTG */
+ REG_SET(OTG_MASTER_UPDATE_LOCK, 0,
+ OTG_MASTER_UPDATE_LOCK, 1);
+ REG_WAIT(OTG_MASTER_UPDATE_LOCK,
+ UPDATE_LOCK_STATUS, 1, 1, 10);
+
+ /* accessing slave OTG registers */
+ optc1 = DCN10TG_FROM_TG(optc_slave);
+
+ /*
+ * enable slave OTG, the OTG is locked with
+ * master's update lock, so it will not run
+ */
+ REG_UPDATE(OTG_CONTROL,
+ OTG_MASTER_EN, 1);
+
+ /* accessing master OTG registers */
+ optc1 = DCN10TG_FROM_TG(optc_master);
+
+ /*
+ * unlock master OTG. When master H/V counters reach
+ * DB_XY point, slave OTG will start
+ */
+ REG_SET(OTG_MASTER_UPDATE_LOCK, 0,
+ OTG_MASTER_UPDATE_LOCK, 0);
+
+ /* accessing slave OTG registers */
+ optc1 = DCN10TG_FROM_TG(optc_slave);
+
+ /* wait for slave OTG to start running*/
+ REG_WAIT(OTG_CONTROL,
+ OTG_CURRENT_MASTER_EN_STATE,
+ 1, 10, 5000);
+
+ /* accessing master OTG registers */
+ optc1 = DCN10TG_FROM_TG(optc_master);
+
+ /* disable the XY point*/
+ REG_UPDATE(OTG_GLOBAL_CONTROL1,
+ MASTER_UPDATE_LOCK_DB_EN, 0);
+ REG_UPDATE_2(OTG_GLOBAL_CONTROL1,
+ MASTER_UPDATE_LOCK_DB_X,
+ 0,
+ MASTER_UPDATE_LOCK_DB_Y,
+ 0);
+
+ /*restore master update lock*/
+ REG_SET(OTG_MASTER_UPDATE_LOCK, 0,
+ OTG_MASTER_UPDATE_LOCK, master_update_lock);
+
+ /* accessing slave OTG registers */
+ optc1 = DCN10TG_FROM_TG(optc_slave);
+ /* restore slave to be controlled by it's own */
+ REG_SET(OTG_GLOBAL_CONTROL0, 0,
+ OTG_MASTER_UPDATE_LOCK_SEL, optc_slave->inst);
+
+}
+
void optc2_triplebuffer_lock(struct timing_generator *optc)
{
struct optc *optc1 = DCN10TG_FROM_TG(optc);
@@ -468,6 +575,7 @@ static struct timing_generator_funcs dcn20_tg_funcs = {
.program_manual_trigger = optc2_program_manual_trigger,
.setup_manual_trigger = optc2_setup_manual_trigger,
.get_hw_timing = optc1_get_hw_timing,
+ .align_vblanks = optc2_align_vblanks,
};
void dcn20_timing_generator_init(struct optc *optc1)
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h
index e0a0a8a8e2c6..3dee2ec2a1bb 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_optc.h
@@ -56,9 +56,6 @@
SF(OTG0_OTG_GSL_WINDOW_X, OTG_GSL_WINDOW_END_X, mask_sh), \
SF(OTG0_OTG_GSL_WINDOW_Y, OTG_GSL_WINDOW_START_Y, mask_sh),\
SF(OTG0_OTG_GSL_WINDOW_Y, OTG_GSL_WINDOW_END_Y, mask_sh),\
- SF(OTG0_OTG_VUPDATE_KEEPOUT, OTG_MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_EN, mask_sh), \
- SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_START_OFFSET, mask_sh), \
- SF(OTG0_OTG_VUPDATE_KEEPOUT, MASTER_UPDATE_LOCK_VUPDATE_KEEPOUT_END_OFFSET, mask_sh), \
SF(OTG0_OTG_GSL_CONTROL, OTG_GSL_MASTER_MODE, mask_sh), \
SF(OTG0_OTG_GSL_CONTROL, OTG_MASTER_UPDATE_LOCK_GSL_EN, mask_sh), \
SF(OTG0_OTG_DSC_START_POSITION, OTG_DSC_START_POSITION_X, mask_sh), \
diff --git a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
index 2c2dbfcd8957..527e56c353cb 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn20/dcn20_resource.c
@@ -1104,7 +1104,7 @@ struct dpp *dcn20_dpp_create(
uint32_t inst)
{
struct dcn20_dpp *dpp =
- kzalloc(sizeof(struct dcn20_dpp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_dpp), GFP_ATOMIC);
if (!dpp)
return NULL;
@@ -1122,7 +1122,7 @@ struct input_pixel_processor *dcn20_ipp_create(
struct dc_context *ctx, uint32_t inst)
{
struct dcn10_ipp *ipp =
- kzalloc(sizeof(struct dcn10_ipp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn10_ipp), GFP_ATOMIC);
if (!ipp) {
BREAK_TO_DEBUGGER();
@@ -1139,7 +1139,7 @@ struct output_pixel_processor *dcn20_opp_create(
struct dc_context *ctx, uint32_t inst)
{
struct dcn20_opp *opp =
- kzalloc(sizeof(struct dcn20_opp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_opp), GFP_ATOMIC);
if (!opp) {
BREAK_TO_DEBUGGER();
@@ -1156,7 +1156,7 @@ struct dce_aux *dcn20_aux_engine_create(
uint32_t inst)
{
struct aux_engine_dce110 *aux_engine =
- kzalloc(sizeof(struct aux_engine_dce110), GFP_KERNEL);
+ kzalloc(sizeof(struct aux_engine_dce110), GFP_ATOMIC);
if (!aux_engine)
return NULL;
@@ -1194,7 +1194,7 @@ struct dce_i2c_hw *dcn20_i2c_hw_create(
uint32_t inst)
{
struct dce_i2c_hw *dce_i2c_hw =
- kzalloc(sizeof(struct dce_i2c_hw), GFP_KERNEL);
+ kzalloc(sizeof(struct dce_i2c_hw), GFP_ATOMIC);
if (!dce_i2c_hw)
return NULL;
@@ -1207,7 +1207,7 @@ struct dce_i2c_hw *dcn20_i2c_hw_create(
struct mpc *dcn20_mpc_create(struct dc_context *ctx)
{
struct dcn20_mpc *mpc20 = kzalloc(sizeof(struct dcn20_mpc),
- GFP_KERNEL);
+ GFP_ATOMIC);
if (!mpc20)
return NULL;
@@ -1225,7 +1225,7 @@ struct hubbub *dcn20_hubbub_create(struct dc_context *ctx)
{
int i;
struct dcn20_hubbub *hubbub = kzalloc(sizeof(struct dcn20_hubbub),
- GFP_KERNEL);
+ GFP_ATOMIC);
if (!hubbub)
return NULL;
@@ -1253,7 +1253,7 @@ struct timing_generator *dcn20_timing_generator_create(
uint32_t instance)
{
struct optc *tgn10 =
- kzalloc(sizeof(struct optc), GFP_KERNEL);
+ kzalloc(sizeof(struct optc), GFP_ATOMIC);
if (!tgn10)
return NULL;
@@ -1332,7 +1332,7 @@ static struct clock_source *dcn20_clock_source_create(
bool dp_clk_src)
{
struct dce110_clk_src *clk_src =
- kzalloc(sizeof(struct dce110_clk_src), GFP_KERNEL);
+ kzalloc(sizeof(struct dce110_clk_src), GFP_ATOMIC);
if (!clk_src)
return NULL;
@@ -1438,7 +1438,7 @@ struct display_stream_compressor *dcn20_dsc_create(
struct dc_context *ctx, uint32_t inst)
{
struct dcn20_dsc *dsc =
- kzalloc(sizeof(struct dcn20_dsc), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_dsc), GFP_ATOMIC);
if (!dsc) {
BREAK_TO_DEBUGGER();
@@ -1572,7 +1572,7 @@ struct hubp *dcn20_hubp_create(
uint32_t inst)
{
struct dcn20_hubp *hubp2 =
- kzalloc(sizeof(struct dcn20_hubp), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_hubp), GFP_ATOMIC);
if (!hubp2)
return NULL;
@@ -2033,9 +2033,13 @@ int dcn20_populate_dml_pipes_from_context(
if (res_ctx->pipe_ctx[pipe_cnt].stream == res_ctx->pipe_ctx[i].stream)
continue;
- if (dc->debug.disable_timing_sync || !resource_are_streams_timing_synchronizable(
+ if (dc->debug.disable_timing_sync ||
+ (!resource_are_streams_timing_synchronizable(
res_ctx->pipe_ctx[pipe_cnt].stream,
- res_ctx->pipe_ctx[i].stream)) {
+ res_ctx->pipe_ctx[i].stream) &&
+ !resource_are_vblanks_synchronizable(
+ res_ctx->pipe_ctx[pipe_cnt].stream,
+ res_ctx->pipe_ctx[i].stream))) {
synchronized_vblank = false;
break;
}
@@ -2197,10 +2201,11 @@ int dcn20_populate_dml_pipes_from_context(
pipes[pipe_cnt].dout.output_bpp = (output_bpc * 3.0) / 2;
break;
case PIXEL_ENCODING_YCBCR422:
- if (true) /* todo */
- pipes[pipe_cnt].dout.output_format = dm_s422;
- else
+ if (res_ctx->pipe_ctx[i].stream->timing.flags.DSC &&
+ !res_ctx->pipe_ctx[i].stream->timing.dsc_cfg.ycbcr422_simple)
pipes[pipe_cnt].dout.output_format = dm_n422;
+ else
+ pipes[pipe_cnt].dout.output_format = dm_s422;
pipes[pipe_cnt].dout.output_bpp = output_bpc * 2;
break;
default:
@@ -2212,7 +2217,7 @@ int dcn20_populate_dml_pipes_from_context(
pipes[pipe_cnt].dout.output_bpp = res_ctx->pipe_ctx[i].stream->timing.dsc_cfg.bits_per_pixel / 16.0;
/* todo: default max for now, until there is logic reflecting this in dc*/
- pipes[pipe_cnt].dout.output_bpc = 12;
+ pipes[pipe_cnt].dout.dsc_input_bpc = 12;
/*fill up the audio sample rate (unit in kHz)*/
get_audio_check(&res_ctx->pipe_ctx[i].stream->audio_info, &aud_check);
pipes[pipe_cnt].dout.max_audio_sample_rate = aud_check.max_audiosample_rate / 1000;
@@ -3390,7 +3395,7 @@ bool dcn20_mmhubbub_create(struct dc_context *ctx, struct resource_pool *pool)
static struct pp_smu_funcs *dcn20_pp_smu_create(struct dc_context *ctx)
{
- struct pp_smu_funcs *pp_smu = kzalloc(sizeof(*pp_smu), GFP_KERNEL);
+ struct pp_smu_funcs *pp_smu = kzalloc(sizeof(*pp_smu), GFP_ATOMIC);
if (!pp_smu)
return pp_smu;
@@ -3697,6 +3702,8 @@ static bool dcn20_resource_construct(
dc->caps.dmdata_alloc_size = 2048;
dc->caps.max_slave_planes = 1;
+ dc->caps.max_slave_yuv_planes = 1;
+ dc->caps.max_slave_rgb_planes = 1;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
dc->caps.extended_aux_timeout_support = true;
@@ -4034,7 +4041,7 @@ struct resource_pool *dcn20_create_resource_pool(
struct dc *dc)
{
struct dcn20_resource_pool *pool =
- kzalloc(sizeof(struct dcn20_resource_pool), GFP_KERNEL);
+ kzalloc(sizeof(struct dcn20_resource_pool), GFP_ATOMIC);
if (!pool)
return NULL;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c
index f9045852728f..b0c9180b808f 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hubp.c
@@ -838,6 +838,7 @@ static struct hubp_funcs dcn21_hubp_funcs = {
.hubp_set_flip_control_surface_gsl = hubp2_set_flip_control_surface_gsl,
.hubp_init = hubp21_init,
.validate_dml_output = hubp21_validate_dml_output,
+ .hubp_set_flip_int = hubp1_set_flip_int,
};
bool hubp21_construct(
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hwseq.c
index d3b643089603..8fccee5a3036 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_hwseq.c
@@ -218,6 +218,8 @@ bool dcn21_set_backlight_level(struct pipe_ctx *pipe_ctx,
cmd.abm_set_backlight.header.sub_type = DMUB_CMD__ABM_SET_BACKLIGHT;
cmd.abm_set_backlight.abm_set_backlight_data.frame_ramp = frame_ramp;
cmd.abm_set_backlight.abm_set_backlight_data.backlight_user_level = backlight_pwm_u16_16;
+ cmd.abm_set_backlight.abm_set_backlight_data.version = DMUB_CMD_ABM_SET_BACKLIGHT_VERSION_1;
+ cmd.abm_set_backlight.abm_set_backlight_data.panel_mask = (0x01 << panel_cntl->inst);
cmd.abm_set_backlight.header.payload_bytes = sizeof(struct dmub_cmd_abm_set_backlight_data);
dc_dmub_srv_cmd_queue(dc->dmub_srv, &cmd);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c
index 0597391b2171..4f20a85ff396 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_init.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2016 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -99,7 +99,7 @@ static const struct hw_sequencer_funcs dcn21_funcs = {
#endif
.is_abm_supported = dcn21_is_abm_supported,
.set_disp_pattern_generator = dcn20_set_disp_pattern_generator,
- .set_hubp_blank = dcn10_set_hubp_blank,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn21_private_funcs = {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
index 072f8c880924..8e3f1d0b4cc3 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn21/dcn21_resource.c
@@ -55,13 +55,11 @@
#include "dce/dce_audio.h"
#include "dce/dce_hwseq.h"
#include "virtual/virtual_stream_encoder.h"
-#include "dce110/dce110_resource.h"
#include "dml/display_mode_vba.h"
#include "dcn20/dcn20_dccg.h"
#include "dcn21/dcn21_dccg.h"
#include "dcn21_hubbub.h"
#include "dcn10/dcn10_resource.h"
-#include "dce110/dce110_resource.h"
#include "dce/dce_panel_cntl.h"
#include "dcn20/dcn20_dwb.h"
@@ -296,7 +294,7 @@ struct _vcs_dpi_soc_bounding_box_st dcn2_1_soc = {
.num_banks = 8,
.num_chans = 4,
.vmm_page_size_bytes = 4096,
- .dram_clock_change_latency_us = 11.72,
+ .dram_clock_change_latency_us = 23.84,
.return_bus_width_bytes = 64,
.dispclk_dppclk_vco_speed_mhz = 3600,
.xfc_bus_transport_time_us = 4,
@@ -883,7 +881,9 @@ static const struct dc_debug_options debug_defaults_drv = {
.scl_reset_length10 = true,
.sanity_checks = true,
.disable_48mhz_pwrdwn = false,
- .usbc_combo_phy_reset_wa = true
+ .usbc_combo_phy_reset_wa = true,
+ .dmub_command_table = true,
+ .use_max_lb = true
};
static const struct dc_debug_options debug_defaults_diags = {
@@ -899,7 +899,8 @@ static const struct dc_debug_options debug_defaults_diags = {
.disable_stutter = true,
.disable_48mhz_pwrdwn = true,
.disable_psr = true,
- .enable_tri_buf = true
+ .enable_tri_buf = true,
+ .use_max_lb = true
};
enum dcn20_clk_src_array_id {
@@ -1062,8 +1063,6 @@ static void patch_bounding_box(struct dc *dc, struct _vcs_dpi_soc_bounding_box_s
{
int i;
- DC_FP_START();
-
if (dc->bb_overrides.sr_exit_time_ns) {
for (i = 0; i < WM_SET_COUNT; i++) {
dc->clk_mgr->bw_params->wm_table.entries[i].sr_exit_time_us =
@@ -1088,8 +1087,6 @@ static void patch_bounding_box(struct dc *dc, struct _vcs_dpi_soc_bounding_box_s
dc->bb_overrides.dram_clock_change_latency_ns / 1000.0;
}
}
-
- DC_FP_END();
}
void dcn21_calculate_wm(
@@ -1339,7 +1336,7 @@ static noinline bool dcn21_validate_bandwidth_fp(struct dc *dc,
int vlevel = 0;
int pipe_split_from[MAX_PIPES];
int pipe_cnt = 0;
- display_e2e_pipe_params_st *pipes = kzalloc(dc->res_pool->pipe_count * sizeof(display_e2e_pipe_params_st), GFP_KERNEL);
+ display_e2e_pipe_params_st *pipes = kzalloc(dc->res_pool->pipe_count * sizeof(display_e2e_pipe_params_st), GFP_ATOMIC);
DC_LOGGER_INIT(dc->ctx->logger);
BW_VAL_TRACE_COUNT();
@@ -1599,6 +1596,11 @@ static void update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_param
dcn2_1_soc.num_chans = bw_params->num_channels;
ASSERT(clk_table->num_entries);
+ /* Copy dcn2_1_soc.clock_limits to clock_limits to avoid copying over null states later */
+ for (i = 0; i < dcn2_1_soc.num_states + 1; i++) {
+ clock_limits[i] = dcn2_1_soc.clock_limits[i];
+ }
+
for (i = 0; i < clk_table->num_entries; i++) {
/* loop backwards*/
for (closest_clk_lvl = 0, j = dcn2_1_soc.num_states - 1; j >= 0; j--) {
@@ -1632,11 +1634,11 @@ static void update_bw_bounding_box(struct dc *dc, struct clk_bw_params *bw_param
dcn2_1_soc.clock_limits[i] = clock_limits[i];
if (clk_table->num_entries) {
dcn2_1_soc.num_states = clk_table->num_entries + 1;
+ /* fill in min DF PState */
+ dcn2_1_soc.clock_limits[1] = construct_low_pstate_lvl(clk_table, closest_clk_lvl);
/* duplicate last level */
dcn2_1_soc.clock_limits[dcn2_1_soc.num_states] = dcn2_1_soc.clock_limits[dcn2_1_soc.num_states - 1];
dcn2_1_soc.clock_limits[dcn2_1_soc.num_states].state = dcn2_1_soc.num_states;
- /* fill in min DF PState */
- dcn2_1_soc.clock_limits[1] = construct_low_pstate_lvl(clk_table, closest_clk_lvl);
}
dml_init_instance(&dc->dml, &dcn2_1_soc, &dcn2_1_ip, DML_PROJECT_DCN21);
@@ -1979,6 +1981,8 @@ static bool dcn21_resource_construct(
dc->caps.dmdata_alloc_size = 2048;
dc->caps.max_slave_planes = 1;
+ dc->caps.max_slave_yuv_planes = 1;
+ dc->caps.max_slave_rgb_planes = 1;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
dc->caps.extended_aux_timeout_support = true;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_cm_common.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_cm_common.c
index 41a1d0e9b7e2..e0df9b0065f9 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_cm_common.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_cm_common.c
@@ -113,6 +113,7 @@ bool cm3_helper_translate_curve_to_hw_format(
struct pwl_result_data *rgb_resulted;
struct pwl_result_data *rgb;
struct pwl_result_data *rgb_plus_1;
+ struct pwl_result_data *rgb_minus_1;
struct fixed31_32 end_value;
int32_t region_start, region_end;
@@ -140,7 +141,7 @@ bool cm3_helper_translate_curve_to_hw_format(
region_start = -MAX_LOW_POINT;
region_end = NUMBER_REGIONS - MAX_LOW_POINT;
} else {
- /* 10 segments
+ /* 11 segments
* segment is from 2^-10 to 2^0
* There are less than 256 points, for optimization
*/
@@ -154,9 +155,10 @@ bool cm3_helper_translate_curve_to_hw_format(
seg_distr[7] = 4;
seg_distr[8] = 4;
seg_distr[9] = 4;
+ seg_distr[10] = 1;
region_start = -10;
- region_end = 0;
+ region_end = 1;
}
for (i = region_end - region_start; i < MAX_REGIONS_NUMBER ; i++)
@@ -189,6 +191,10 @@ bool cm3_helper_translate_curve_to_hw_format(
rgb_resulted[hw_points - 1].green = output_tf->tf_pts.green[start_index];
rgb_resulted[hw_points - 1].blue = output_tf->tf_pts.blue[start_index];
+ rgb_resulted[hw_points].red = rgb_resulted[hw_points - 1].red;
+ rgb_resulted[hw_points].green = rgb_resulted[hw_points - 1].green;
+ rgb_resulted[hw_points].blue = rgb_resulted[hw_points - 1].blue;
+
// All 3 color channels have same x
corner_points[0].red.x = dc_fixpt_pow(dc_fixpt_from_int(2),
dc_fixpt_from_int(region_start));
@@ -259,15 +265,18 @@ bool cm3_helper_translate_curve_to_hw_format(
rgb = rgb_resulted;
rgb_plus_1 = rgb_resulted + 1;
+ rgb_minus_1 = rgb;
i = 1;
while (i != hw_points + 1) {
- if (dc_fixpt_lt(rgb_plus_1->red, rgb->red))
- rgb_plus_1->red = rgb->red;
- if (dc_fixpt_lt(rgb_plus_1->green, rgb->green))
- rgb_plus_1->green = rgb->green;
- if (dc_fixpt_lt(rgb_plus_1->blue, rgb->blue))
- rgb_plus_1->blue = rgb->blue;
+ if (i >= hw_points - 1) {
+ if (dc_fixpt_lt(rgb_plus_1->red, rgb->red))
+ rgb_plus_1->red = dc_fixpt_add(rgb->red, rgb_minus_1->delta_red);
+ if (dc_fixpt_lt(rgb_plus_1->green, rgb->green))
+ rgb_plus_1->green = dc_fixpt_add(rgb->green, rgb_minus_1->delta_green);
+ if (dc_fixpt_lt(rgb_plus_1->blue, rgb->blue))
+ rgb_plus_1->blue = dc_fixpt_add(rgb->blue, rgb_minus_1->delta_blue);
+ }
rgb->delta_red = dc_fixpt_sub(rgb_plus_1->red, rgb->red);
rgb->delta_green = dc_fixpt_sub(rgb_plus_1->green, rgb->green);
@@ -283,6 +292,7 @@ bool cm3_helper_translate_curve_to_hw_format(
}
++rgb_plus_1;
+ rgb_minus_1 = rgb;
++rgb;
++i;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c
index 6c0f7ef0a3df..72bee637c1e4 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dio_stream_encoder.c
@@ -454,7 +454,6 @@ static void enc3_stream_encoder_update_dp_info_packets(
REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP2_ENABLE, info_frame->spd.valid);
REG_UPDATE(DP_SEC_CNTL, DP_SEC_GSP3_ENABLE, info_frame->hdrsmd.valid);
-
/* This bit is the master enable bit.
* When enabling secondary stream engine,
* this master bit must also be set.
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c
index 6e864b1a95c4..434d3c46cad4 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp.c
@@ -718,7 +718,7 @@ bool dpp3_program_blnd_lut(
next_mode = LUT_RAM_B;
dpp3_power_on_blnd_lut(dpp_base, true);
- dpp3_configure_blnd_lut(dpp_base, next_mode == LUT_RAM_A ? true:false);
+ dpp3_configure_blnd_lut(dpp_base, next_mode == LUT_RAM_A);
if (next_mode == LUT_RAM_A)
dpp3_program_blnd_luta_settings(dpp_base, params);
@@ -1136,7 +1136,7 @@ bool dpp3_program_shaper(
else
next_mode = LUT_RAM_A;
- dpp3_configure_shaper_lut(dpp_base, next_mode == LUT_RAM_A ? true:false);
+ dpp3_configure_shaper_lut(dpp_base, next_mode == LUT_RAM_A);
if (next_mode == LUT_RAM_A)
dpp3_program_shaper_luta_settings(dpp_base, params);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp_cm.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp_cm.c
index 33985401f25c..72c5687adc68 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp_cm.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dpp_cm.c
@@ -240,7 +240,7 @@ bool dpp3_program_gamcor_lut(
next_mode = LUT_RAM_A;
dpp3_power_on_gamcor_lut(dpp_base, true);
- dpp3_configure_gamcor_lut(dpp_base, next_mode == LUT_RAM_A ? true:false);
+ dpp3_configure_gamcor_lut(dpp_base, next_mode == LUT_RAM_A);
if (next_mode == LUT_RAM_B) {
gam_regs.start_cntl_b = REG(CM_GAMCOR_RAMB_START_CNTL_B);
@@ -295,7 +295,7 @@ bool dpp3_program_gamcor_lut(
cm_helper_program_gamcor_xfer_func(dpp_base->ctx, params, &gam_regs);
dpp3_program_gammcor_lut(dpp_base, params->rgb_resulted, params->hw_points_num,
- next_mode == LUT_RAM_A ? true:false);
+ next_mode == LUT_RAM_A);
//select Gamma LUT to use for next frame
REG_UPDATE(CM_GAMCOR_CONTROL, CM_GAMCOR_SELECT, next_mode == LUT_RAM_A ? 0:1);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb_cm.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb_cm.c
index 8593145379d9..3fe9e41e4dbd 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb_cm.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_dwb_cm.c
@@ -217,7 +217,7 @@ static bool dwb3_program_ogam_lut(
else
next_mode = LUT_RAM_A;
- dwb3_configure_ogam_lut(dwbc30, next_mode == LUT_RAM_A ? true : false);
+ dwb3_configure_ogam_lut(dwbc30, next_mode == LUT_RAM_A);
if (next_mode == LUT_RAM_A)
dwb3_program_ogam_luta_settings(dwbc30, params);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c
index 88ffa9ff1ed1..f24612523248 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.c
@@ -511,6 +511,7 @@ static struct hubp_funcs dcn30_hubp_funcs = {
.hubp_init = hubp3_init,
.hubp_in_blank = hubp1_in_blank,
.hubp_soft_reset = hubp1_soft_reset,
+ .hubp_set_flip_int = hubp1_set_flip_int,
};
bool hubp3_construct(
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.h b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.h
index 705fbfc37502..8a32772d4e91 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hubp.h
@@ -134,6 +134,7 @@
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_EN, mask_sh),\
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_IND_BLK, mask_sh),\
HUBP_SF(HUBPREQ0_DCSURF_SURFACE_CONTROL, SECONDARY_SURFACE_DCC_IND_BLK_C, mask_sh),\
+ HUBP_SF(HUBPREQ0_DCSURF_SURFACE_FLIP_INTERRUPT, SURFACE_FLIP_INT_MASK, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, DET_BUF_PLANE1_BASE_ADDRESS, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CB_B, mask_sh),\
HUBP_SF(HUBPRET0_HUBPRET_CONTROL, CROSSBAR_SRC_CR_R, mask_sh),\
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c
index 06dc1e2e8383..d53f8b39699b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.c
@@ -421,11 +421,12 @@ void dcn30_program_all_writeback_pipes_in_tree(
void dcn30_init_hw(struct dc *dc)
{
- int i, j;
struct abm **abms = dc->res_pool->multiple_abms;
struct dce_hwseq *hws = dc->hwseq;
struct dc_bios *dcb = dc->ctx->dc_bios;
struct resource_pool *res_pool = dc->res_pool;
+ int i, j;
+ int edp_num;
uint32_t backlight = MAX_BACKLIGHT_LEVEL;
if (dc->clk_mgr && dc->clk_mgr->funcs->init_clocks)
@@ -574,17 +575,23 @@ void dcn30_init_hw(struct dc *dc)
* if DIG is turned on and seamless boot not enabled
*/
if (dc->config.power_down_display_on_boot) {
- struct dc_link *edp_link = get_edp_link(dc);
-
- if (edp_link &&
- edp_link->link_enc->funcs->is_dig_enabled &&
- edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc) &&
- dc->hwss.edp_backlight_control &&
- dc->hwss.power_down &&
- dc->hwss.edp_power_control) {
- dc->hwss.edp_backlight_control(edp_link, false);
- dc->hwss.power_down(dc);
- dc->hwss.edp_power_control(edp_link, false);
+ struct dc_link *edp_links[MAX_NUM_EDP];
+ struct dc_link *edp_link;
+
+ get_edp_links(dc, edp_links, &edp_num);
+ if (edp_num) {
+ for (i = 0; i < edp_num; i++) {
+ edp_link = edp_links[i];
+ if (edp_link->link_enc->funcs->is_dig_enabled &&
+ edp_link->link_enc->funcs->is_dig_enabled(edp_link->link_enc) &&
+ dc->hwss.edp_backlight_control &&
+ dc->hwss.power_down &&
+ dc->hwss.edp_power_control) {
+ dc->hwss.edp_backlight_control(edp_link, false);
+ dc->hwss.power_down(dc);
+ dc->hwss.edp_power_control(edp_link, false);
+ }
+ }
} else {
for (i = 0; i < dc->link_count; i++) {
struct dc_link *link = dc->links[i];
@@ -651,7 +658,7 @@ void dcn30_set_avmute(struct pipe_ctx *pipe_ctx, bool enable)
if (pipe_ctx == NULL)
return;
- if (dc_is_hdmi_tmds_signal(pipe_ctx->stream->signal) && pipe_ctx->stream_res.stream_enc != NULL)
+ if (dc_is_hdmi_signal(pipe_ctx->stream->signal) && pipe_ctx->stream_res.stream_enc != NULL)
pipe_ctx->stream_res.stream_enc->funcs->set_avmute(
pipe_ctx->stream_res.stream_enc,
enable);
@@ -848,7 +855,7 @@ bool dcn30_apply_idle_power_optimizations(struct dc *dc, bool enable)
cmd.mall.cursor_copy_src.quad_part = cursor_attr.address.quad_part;
cmd.mall.cursor_copy_dst.quad_part =
- plane->address.grph.cursor_cache_addr.quad_part;
+ (plane->address.grph.cursor_cache_addr.quad_part + 2047) & ~2047;
cmd.mall.cursor_width = cursor_attr.width;
cmd.mall.cursor_height = cursor_attr.height;
cmd.mall.cursor_pitch = cursor_attr.pitch;
@@ -858,8 +865,7 @@ bool dcn30_apply_idle_power_optimizations(struct dc *dc, bool enable)
dc_dmub_srv_wait_idle(dc->ctx->dmub_srv);
/* Use copied cursor, and it's okay to not switch back */
- cursor_attr.address.quad_part =
- plane->address.grph.cursor_cache_addr.quad_part;
+ cursor_attr.address.quad_part = cmd.mall.cursor_copy_dst.quad_part;
dc_stream_set_cursor_attributes(stream, &cursor_attr);
}
@@ -940,53 +946,6 @@ void dcn30_hardware_release(struct dc *dc)
dc->res_pool->hubbub, true, true);
}
-void dcn30_set_hubp_blank(const struct dc *dc,
- struct pipe_ctx *pipe_ctx,
- bool blank_enable)
-{
- struct pipe_ctx *mpcc_pipe;
- struct pipe_ctx *odm_pipe;
-
- if (blank_enable) {
- struct plane_resource *plane_res = &pipe_ctx->plane_res;
- struct stream_resource *stream_res = &pipe_ctx->stream_res;
-
- /* Wait for enter vblank */
- stream_res->tg->funcs->wait_for_state(stream_res->tg, CRTC_STATE_VBLANK);
-
- /* Blank HUBP to allow p-state during blank on all timings */
- pipe_ctx->plane_res.hubp->funcs->set_blank(pipe_ctx->plane_res.hubp, true);
- /* Confirm hubp in blank */
- ASSERT(plane_res->hubp->funcs->hubp_in_blank(plane_res->hubp));
- /* Toggle HUBP_DISABLE */
- plane_res->hubp->funcs->hubp_soft_reset(plane_res->hubp, true);
- plane_res->hubp->funcs->hubp_soft_reset(plane_res->hubp, false);
- for (mpcc_pipe = pipe_ctx->bottom_pipe; mpcc_pipe; mpcc_pipe = mpcc_pipe->bottom_pipe) {
- mpcc_pipe->plane_res.hubp->funcs->set_blank(mpcc_pipe->plane_res.hubp, true);
- /* Confirm hubp in blank */
- ASSERT(mpcc_pipe->plane_res.hubp->funcs->hubp_in_blank(mpcc_pipe->plane_res.hubp));
- /* Toggle HUBP_DISABLE */
- mpcc_pipe->plane_res.hubp->funcs->hubp_soft_reset(mpcc_pipe->plane_res.hubp, true);
- mpcc_pipe->plane_res.hubp->funcs->hubp_soft_reset(mpcc_pipe->plane_res.hubp, false);
-
- }
- for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe) {
- odm_pipe->plane_res.hubp->funcs->set_blank(odm_pipe->plane_res.hubp, true);
- /* Confirm hubp in blank */
- ASSERT(odm_pipe->plane_res.hubp->funcs->hubp_in_blank(odm_pipe->plane_res.hubp));
- /* Toggle HUBP_DISABLE */
- odm_pipe->plane_res.hubp->funcs->hubp_soft_reset(odm_pipe->plane_res.hubp, true);
- odm_pipe->plane_res.hubp->funcs->hubp_soft_reset(odm_pipe->plane_res.hubp, false);
- }
- } else {
- pipe_ctx->plane_res.hubp->funcs->set_blank(pipe_ctx->plane_res.hubp, false);
- for (mpcc_pipe = pipe_ctx->bottom_pipe; mpcc_pipe; mpcc_pipe = mpcc_pipe->bottom_pipe)
- mpcc_pipe->plane_res.hubp->funcs->set_blank(mpcc_pipe->plane_res.hubp, false);
- for (odm_pipe = pipe_ctx->next_odm_pipe; odm_pipe; odm_pipe = odm_pipe->next_odm_pipe)
- odm_pipe->plane_res.hubp->funcs->set_blank(odm_pipe->plane_res.hubp, false);
- }
-}
-
void dcn30_set_disp_pattern_generator(const struct dc *dc,
struct pipe_ctx *pipe_ctx,
enum controller_dp_test_pattern test_pattern,
@@ -996,6 +955,7 @@ void dcn30_set_disp_pattern_generator(const struct dc *dc,
int width, int height, int offset)
{
struct stream_resource *stream_res = &pipe_ctx->stream_res;
+ struct pipe_ctx *mpcc_pipe;
if (test_pattern != CONTROLLER_DP_TEST_PATTERN_VIDEOMODE) {
pipe_ctx->vtp_locked = false;
@@ -1007,12 +967,20 @@ void dcn30_set_disp_pattern_generator(const struct dc *dc,
if (stream_res->tg->funcs->is_tg_enabled(stream_res->tg)) {
if (stream_res->tg->funcs->is_locked(stream_res->tg))
pipe_ctx->vtp_locked = true;
- else
- dc->hwss.set_hubp_blank(dc, pipe_ctx, true);
+ else {
+ /* Blank HUBP to allow p-state during blank on all timings */
+ pipe_ctx->plane_res.hubp->funcs->set_blank(pipe_ctx->plane_res.hubp, true);
+
+ for (mpcc_pipe = pipe_ctx->bottom_pipe; mpcc_pipe; mpcc_pipe = mpcc_pipe->bottom_pipe)
+ mpcc_pipe->plane_res.hubp->funcs->set_blank(mpcc_pipe->plane_res.hubp, true);
+ }
}
} else {
- dc->hwss.set_hubp_blank(dc, pipe_ctx, false);
/* turning off DPG */
+ pipe_ctx->plane_res.hubp->funcs->set_blank(pipe_ctx->plane_res.hubp, false);
+ for (mpcc_pipe = pipe_ctx->bottom_pipe; mpcc_pipe; mpcc_pipe = mpcc_pipe->bottom_pipe)
+ mpcc_pipe->plane_res.hubp->funcs->set_blank(mpcc_pipe->plane_res.hubp, false);
+
stream_res->opp->funcs->opp_set_disp_pattern_generator(stream_res->opp, test_pattern, color_space,
color_depth, solid_color, width, height, offset);
}
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h
index 3b7d4812e311..e9a0005288d3 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_hwseq.h
@@ -80,8 +80,4 @@ void dcn30_set_disp_pattern_generator(const struct dc *dc,
const struct tg_color *solid_color,
int width, int height, int offset);
-void dcn30_set_hubp_blank(const struct dc *dc,
- struct pipe_ctx *pipe_ctx,
- bool blank_enable);
-
#endif /* __DC_HWSS_DCN30_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c
index 204444fead97..bf7fa98b39eb 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_init.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2020 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -98,7 +98,7 @@ static const struct hw_sequencer_funcs dcn30_funcs = {
.hardware_release = dcn30_hardware_release,
.set_pipe = dcn21_set_pipe,
.set_disp_pattern_generator = dcn30_set_disp_pattern_generator,
- .set_hubp_blank = dcn30_set_hubp_blank,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
};
static const struct hwseq_private_funcs dcn30_private_funcs = {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c
index 3e6f76096119..910c17fd4278 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_mpc.c
@@ -143,16 +143,18 @@ static void mpc3_power_on_ogam_lut(
{
struct dcn30_mpc *mpc30 = TO_DCN30_MPC(mpc);
- if (mpc->ctx->dc->debug.enable_mem_low_power.bits.mpc) {
- // Force power on
- REG_UPDATE(MPCC_MEM_PWR_CTRL[mpcc_id], MPCC_OGAM_MEM_PWR_DIS, power_on == true ? 1:0);
- // Wait for confirmation when powering on
- if (power_on)
- REG_WAIT(MPCC_MEM_PWR_CTRL[mpcc_id], MPCC_OGAM_MEM_PWR_STATE, 0, 10, 10);
- } else {
- REG_SET(MPCC_MEM_PWR_CTRL[mpcc_id], 0,
- MPCC_OGAM_MEM_PWR_FORCE, power_on == true ? 0 : 1);
- }
+ /*
+ * Powering on: force memory active so the LUT can be updated.
+ * Powering off: allow entering memory low power mode
+ *
+ * Memory low power mode is controlled during MPC OGAM LUT init.
+ */
+ REG_UPDATE(MPCC_MEM_PWR_CTRL[mpcc_id],
+ MPCC_OGAM_MEM_PWR_DIS, power_on != 0);
+
+ /* Wait for memory to be powered on - we won't be able to write to it otherwise. */
+ if (power_on)
+ REG_WAIT(MPCC_MEM_PWR_CTRL[mpcc_id], MPCC_OGAM_MEM_PWR_STATE, 0, 10, 10);
}
static void mpc3_configure_ogam_lut(
@@ -355,7 +357,7 @@ void mpc3_set_output_gamma(
next_mode = LUT_RAM_A;
mpc3_power_on_ogam_lut(mpc, mpcc_id, true);
- mpc3_configure_ogam_lut(mpc, mpcc_id, next_mode == LUT_RAM_A ? true:false);
+ mpc3_configure_ogam_lut(mpc, mpcc_id, next_mode == LUT_RAM_A);
if (next_mode == LUT_RAM_A)
mpc3_program_luta(mpc, mpcc_id, params);
@@ -1427,7 +1429,7 @@ const struct mpc_funcs dcn30_mpc_funcs = {
.acquire_rmu = mpcc3_acquire_rmu,
.program_3dlut = mpc3_program_3dlut,
.release_rmu = mpcc3_release_rmu,
- .power_on_mpc_mem_pwr = mpc20_power_on_ogam_lut,
+ .power_on_mpc_mem_pwr = mpc3_power_on_ogam_lut,
.get_mpc_out_mux = mpc1_get_mpc_out_mux,
};
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
index 8d0f663489ac..4a5fa23d8e7b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.c
@@ -181,7 +181,7 @@ struct _vcs_dpi_soc_bounding_box_st dcn3_0_soc = {
},
.min_dcfclk = 500.0, /* TODO: set this to actual min DCFCLK */
.num_states = 1,
- .sr_exit_time_us = 12,
+ .sr_exit_time_us = 15.5,
.sr_enter_plus_exit_time_us = 20,
.urgent_latency_us = 4.0,
.urgent_latency_pixel_data_only_us = 4.0,
@@ -852,6 +852,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.dwb_fi_phase = -1, // -1 = disable,
.dmub_command_table = true,
.disable_psr = false,
+ .use_max_lb = true
};
static const struct dc_debug_options debug_defaults_diags = {
@@ -870,6 +871,7 @@ static const struct dc_debug_options debug_defaults_diags = {
.dmub_command_table = true,
.disable_psr = true,
.enable_tri_buf = true,
+ .use_max_lb = true
};
void dcn30_dpp_destroy(struct dpp **dpp)
@@ -1874,6 +1876,7 @@ static noinline bool dcn30_internal_validate_bw(
if (!pipes)
return false;
+ dc->res_pool->funcs->update_soc_for_wm_a(dc, context);
pipe_cnt = dc->res_pool->funcs->populate_dml_pipes(dc, context, pipes, fast_validate);
DC_FP_START();
@@ -2223,11 +2226,7 @@ static noinline void dcn30_calculate_wm_and_dlg_fp(
*
* Set A calculated last so that following calculations are based on Set A
*/
- if (dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].valid) {
- context->bw_ctx.dml.soc.dram_clock_change_latency_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.pstate_latency_us;
- context->bw_ctx.dml.soc.sr_enter_plus_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.sr_enter_plus_exit_time_us;
- context->bw_ctx.dml.soc.sr_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.sr_exit_time_us;
- }
+ dc->res_pool->funcs->update_soc_for_wm_a(dc, context);
context->bw_ctx.bw.dcn.watermarks.a.urgent_ns = get_wm_urgent(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
context->bw_ctx.bw.dcn.watermarks.a.cstate_pstate.cstate_enter_plus_exit_ns = get_wm_stutter_enter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
context->bw_ctx.bw.dcn.watermarks.a.cstate_pstate.cstate_exit_ns = get_wm_stutter_exit(&context->bw_ctx.dml, pipes, pipe_cnt) * 1000;
@@ -2270,6 +2269,15 @@ static noinline void dcn30_calculate_wm_and_dlg_fp(
dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.pstate_latency_us;
}
+void dcn30_update_soc_for_wm_a(struct dc *dc, struct dc_state *context)
+{
+ if (dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].valid) {
+ context->bw_ctx.dml.soc.dram_clock_change_latency_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.pstate_latency_us;
+ context->bw_ctx.dml.soc.sr_enter_plus_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.sr_enter_plus_exit_time_us;
+ context->bw_ctx.dml.soc.sr_exit_time_us = dc->clk_mgr->bw_params->wm_table.nv_entries[WM_A].dml_input.sr_exit_time_us;
+ }
+}
+
void dcn30_calculate_wm_and_dlg(
struct dc *dc, struct dc_state *context,
display_e2e_pipe_params_st *pipes,
@@ -2494,6 +2502,7 @@ static const struct resource_funcs dcn30_res_pool_funcs = {
.panel_cntl_create = dcn30_panel_cntl_create,
.validate_bandwidth = dcn30_validate_bandwidth,
.calculate_wm_and_dlg = dcn30_calculate_wm_and_dlg,
+ .update_soc_for_wm_a = dcn30_update_soc_for_wm_a,
.populate_dml_pipes = dcn30_populate_dml_pipes_from_context,
.acquire_idle_pipe_for_layer = dcn20_acquire_idle_pipe_for_layer,
.add_stream_to_ctx = dcn30_add_stream_to_ctx,
@@ -2508,6 +2517,19 @@ static const struct resource_funcs dcn30_res_pool_funcs = {
.patch_unknown_plane_state = dcn20_patch_unknown_plane_state,
};
+#define CTX ctx
+
+#define REG(reg_name) \
+ (DCN_BASE.instance[0].segment[mm ## reg_name ## _BASE_IDX] + mm ## reg_name)
+
+static uint32_t read_pipe_fuses(struct dc_context *ctx)
+{
+ uint32_t value = REG_READ(CC_DC_PIPE_DIS);
+ /* Support for max 6 pipes */
+ value = value & 0x3f;
+ return value;
+}
+
static bool dcn30_resource_construct(
uint8_t num_virtual_links,
struct dc *dc,
@@ -2517,6 +2539,15 @@ static bool dcn30_resource_construct(
struct dc_context *ctx = dc->ctx;
struct irq_service_init_data init_data;
struct ddc_service_init_data ddc_init_data;
+ uint32_t pipe_fuses = read_pipe_fuses(ctx);
+ uint32_t num_pipes = 0;
+
+ if (!(pipe_fuses == 0 || pipe_fuses == 0x3e)) {
+ BREAK_TO_DEBUGGER();
+ dm_error("DC: Unexpected fuse recipe for navi2x !\n");
+ /* fault to single pipe */
+ pipe_fuses = 0x3e;
+ }
DC_FP_START();
@@ -2544,6 +2575,8 @@ static bool dcn30_resource_construct(
dc->caps.cursor_cache_size = dc->caps.max_cursor_size * dc->caps.max_cursor_size * 8;
dc->caps.max_slave_planes = 1;
+ dc->caps.max_slave_yuv_planes = 1;
+ dc->caps.max_slave_rgb_planes = 1;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
dc->caps.extended_aux_timeout_support = true;
@@ -2650,6 +2683,15 @@ static bool dcn30_resource_construct(
/* PP Lib and SMU interfaces */
init_soc_bounding_box(dc, pool);
+ num_pipes = dcn3_0_ip.max_num_dpp;
+
+ for (i = 0; i < dcn3_0_ip.max_num_dpp; i++)
+ if (pipe_fuses & 1 << i)
+ num_pipes--;
+
+ dcn3_0_ip.max_num_dpp = num_pipes;
+ dcn3_0_ip.max_num_otg = num_pipes;
+
dml_init_instance(&dc->dml, &dcn3_0_soc, &dcn3_0_ip, DML_PROJECT_DCN30);
/* IRQ */
diff --git a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.h b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.h
index 8ce7f6d39a20..b754b89beadf 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.h
+++ b/drivers/gpu/drm/amd/display/dc/dcn30/dcn30_resource.h
@@ -60,6 +60,7 @@ void dcn30_calculate_wm_and_dlg(
display_e2e_pipe_params_st *pipes,
int pipe_cnt,
int vlevel);
+void dcn30_update_soc_for_wm_a(struct dc *dc, struct dc_state *context);
void dcn30_populate_dml_writeback_from_context(
struct dc *dc, struct resource_context *res_ctx, display_e2e_pipe_params_st *pipes);
diff --git a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c
index b8bf6d61005b..70b053d9ba40 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_init.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2020 Advanced Micro Devices, Inc.
+ * Copyright 2016-2020 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -98,7 +98,9 @@ static const struct hw_sequencer_funcs dcn301_funcs = {
.set_abm_immediate_disable = dcn21_set_abm_immediate_disable,
.set_pipe = dcn21_set_pipe,
.set_disp_pattern_generator = dcn30_set_disp_pattern_generator,
- .set_hubp_blank = dcn30_set_hubp_blank,
+ .get_dcc_en_bits = dcn10_get_dcc_en_bits,
+ .optimize_pwr_state = dcn21_optimize_pwr_state,
+ .exit_optimized_pwr_state = dcn21_exit_optimized_pwr_state,
};
static const struct hwseq_private_funcs dcn301_private_funcs = {
diff --git a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
index 5d4b2c60192e..5b54b7fc5105 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn301/dcn301_resource.c
@@ -1,5 +1,5 @@
/*
- * Copyright 2020 Advanced Micro Devices, Inc.
+ * Copyright 2019-2021 Advanced Micro Devices, Inc.
*
* Permission is hereby granted, free of charge, to any person obtaining a
* copy of this software and associated documentation files (the "Software"),
@@ -873,6 +873,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.underflow_assert_delay_us = 0xFFFFFFFF,
.dwb_fi_phase = -1, // -1 = disable
.dmub_command_table = true,
+ .use_max_lb = false,
};
static const struct dc_debug_options debug_defaults_diags = {
@@ -889,6 +890,7 @@ static const struct dc_debug_options debug_defaults_diags = {
.scl_reset_length10 = true,
.dwb_fi_phase = -1, // -1 = disable
.dmub_command_table = true,
+ .use_max_lb = false,
};
void dcn301_dpp_destroy(struct dpp **dpp)
@@ -1619,12 +1621,107 @@ static void dcn301_update_bw_bounding_box(struct dc *dc, struct clk_bw_params *b
dml_init_instance(&dc->dml, &dcn3_01_soc, &dcn3_01_ip, DML_PROJECT_DCN30);
}
+static void calculate_wm_set_for_vlevel(
+ int vlevel,
+ struct wm_range_table_entry *table_entry,
+ struct dcn_watermarks *wm_set,
+ struct display_mode_lib *dml,
+ display_e2e_pipe_params_st *pipes,
+ int pipe_cnt)
+{
+ double dram_clock_change_latency_cached = dml->soc.dram_clock_change_latency_us;
+
+ ASSERT(vlevel < dml->soc.num_states);
+ /* only pipe 0 is read for voltage and dcf/soc clocks */
+ pipes[0].clks_cfg.voltage = vlevel;
+ pipes[0].clks_cfg.dcfclk_mhz = dml->soc.clock_limits[vlevel].dcfclk_mhz;
+ pipes[0].clks_cfg.socclk_mhz = dml->soc.clock_limits[vlevel].socclk_mhz;
+
+ dml->soc.dram_clock_change_latency_us = table_entry->pstate_latency_us;
+ dml->soc.sr_exit_time_us = table_entry->sr_exit_time_us;
+ dml->soc.sr_enter_plus_exit_time_us = table_entry->sr_enter_plus_exit_time_us;
+
+ wm_set->urgent_ns = get_wm_urgent(dml, pipes, pipe_cnt) * 1000;
+ wm_set->cstate_pstate.cstate_enter_plus_exit_ns = get_wm_stutter_enter_exit(dml, pipes, pipe_cnt) * 1000;
+ wm_set->cstate_pstate.cstate_exit_ns = get_wm_stutter_exit(dml, pipes, pipe_cnt) * 1000;
+ wm_set->cstate_pstate.pstate_change_ns = get_wm_dram_clock_change(dml, pipes, pipe_cnt) * 1000;
+ wm_set->pte_meta_urgent_ns = get_wm_memory_trip(dml, pipes, pipe_cnt) * 1000;
+ wm_set->frac_urg_bw_nom = get_fraction_of_urgent_bandwidth(dml, pipes, pipe_cnt) * 1000;
+ wm_set->frac_urg_bw_flip = get_fraction_of_urgent_bandwidth_imm_flip(dml, pipes, pipe_cnt) * 1000;
+ wm_set->urgent_latency_ns = get_urgent_latency(dml, pipes, pipe_cnt) * 1000;
+ dml->soc.dram_clock_change_latency_us = dram_clock_change_latency_cached;
+
+}
+
+static void dcn301_calculate_wm_and_dlg(
+ struct dc *dc, struct dc_state *context,
+ display_e2e_pipe_params_st *pipes,
+ int pipe_cnt,
+ int vlevel_req)
+{
+ int i, pipe_idx;
+ int vlevel, vlevel_max;
+ struct wm_range_table_entry *table_entry;
+ struct clk_bw_params *bw_params = dc->clk_mgr->bw_params;
+
+ ASSERT(bw_params);
+
+ vlevel_max = bw_params->clk_table.num_entries - 1;
+
+ /* WM Set D */
+ table_entry = &bw_params->wm_table.entries[WM_D];
+ if (table_entry->wm_type == WM_TYPE_RETRAINING)
+ vlevel = 0;
+ else
+ vlevel = vlevel_max;
+ calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.d,
+ &context->bw_ctx.dml, pipes, pipe_cnt);
+ /* WM Set C */
+ table_entry = &bw_params->wm_table.entries[WM_C];
+ vlevel = min(max(vlevel_req, 2), vlevel_max);
+ calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.c,
+ &context->bw_ctx.dml, pipes, pipe_cnt);
+ /* WM Set B */
+ table_entry = &bw_params->wm_table.entries[WM_B];
+ vlevel = min(max(vlevel_req, 1), vlevel_max);
+ calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.b,
+ &context->bw_ctx.dml, pipes, pipe_cnt);
+
+ /* WM Set A */
+ table_entry = &bw_params->wm_table.entries[WM_A];
+ vlevel = min(vlevel_req, vlevel_max);
+ calculate_wm_set_for_vlevel(vlevel, table_entry, &context->bw_ctx.bw.dcn.watermarks.a,
+ &context->bw_ctx.dml, pipes, pipe_cnt);
+
+ for (i = 0, pipe_idx = 0; i < dc->res_pool->pipe_count; i++) {
+ if (!context->res_ctx.pipe_ctx[i].stream)
+ continue;
+
+ pipes[pipe_idx].clks_cfg.dispclk_mhz = get_dispclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt);
+ pipes[pipe_idx].clks_cfg.dppclk_mhz = get_dppclk_calculated(&context->bw_ctx.dml, pipes, pipe_cnt, pipe_idx);
+
+ if (dc->config.forced_clocks) {
+ pipes[pipe_idx].clks_cfg.dispclk_mhz = context->bw_ctx.dml.soc.clock_limits[0].dispclk_mhz;
+ pipes[pipe_idx].clks_cfg.dppclk_mhz = context->bw_ctx.dml.soc.clock_limits[0].dppclk_mhz;
+ }
+ if (dc->debug.min_disp_clk_khz > pipes[pipe_idx].clks_cfg.dispclk_mhz * 1000)
+ pipes[pipe_idx].clks_cfg.dispclk_mhz = dc->debug.min_disp_clk_khz / 1000.0;
+ if (dc->debug.min_dpp_clk_khz > pipes[pipe_idx].clks_cfg.dppclk_mhz * 1000)
+ pipes[pipe_idx].clks_cfg.dppclk_mhz = dc->debug.min_dpp_clk_khz / 1000.0;
+
+ pipe_idx++;
+ }
+
+ dcn20_calculate_dlg_params(dc, context, pipes, pipe_cnt, vlevel);
+}
+
static struct resource_funcs dcn301_res_pool_funcs = {
.destroy = dcn301_destroy_resource_pool,
.link_enc_create = dcn301_link_encoder_create,
.panel_cntl_create = dcn301_panel_cntl_create,
.validate_bandwidth = dcn30_validate_bandwidth,
- .calculate_wm_and_dlg = dcn30_calculate_wm_and_dlg,
+ .calculate_wm_and_dlg = dcn301_calculate_wm_and_dlg,
+ .update_soc_for_wm_a = dcn30_update_soc_for_wm_a,
.populate_dml_pipes = dcn30_populate_dml_pipes_from_context,
.acquire_idle_pipe_for_layer = dcn20_acquire_idle_pipe_for_layer,
.add_stream_to_ctx = dcn30_add_stream_to_ctx,
@@ -1670,6 +1767,8 @@ static bool dcn301_resource_construct(
dc->caps.min_horizontal_blanking_period = 80;
dc->caps.dmdata_alloc_size = 2048;
dc->caps.max_slave_planes = 1;
+ dc->caps.max_slave_yuv_planes = 1;
+ dc->caps.max_slave_rgb_planes = 1;
dc->caps.is_apu = true;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
diff --git a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
index 4b659b63f75b..fc2dea243d1b 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn302/dcn302_resource.c
@@ -164,7 +164,7 @@ struct _vcs_dpi_soc_bounding_box_st dcn3_02_soc = {
.min_dcfclk = 500.0, /* TODO: set this to actual min DCFCLK */
.num_states = 1,
- .sr_exit_time_us = 12,
+ .sr_exit_time_us = 15.5,
.sr_enter_plus_exit_time_us = 20,
.urgent_latency_us = 4.0,
.urgent_latency_pixel_data_only_us = 4.0,
@@ -223,6 +223,7 @@ static const struct dc_debug_options debug_defaults_drv = {
.underflow_assert_delay_us = 0xFFFFFFFF,
.dwb_fi_phase = -1, // -1 = disable,
.dmub_command_table = true,
+ .use_max_lb = true
};
static const struct dc_debug_options debug_defaults_diags = {
@@ -241,6 +242,7 @@ static const struct dc_debug_options debug_defaults_diags = {
.dmub_command_table = true,
.enable_tri_buf = true,
.disable_psr = true,
+ .use_max_lb = true
};
enum dcn302_clk_src_array_id {
@@ -1395,6 +1397,7 @@ static struct resource_funcs dcn302_res_pool_funcs = {
.panel_cntl_create = dcn302_panel_cntl_create,
.validate_bandwidth = dcn30_validate_bandwidth,
.calculate_wm_and_dlg = dcn30_calculate_wm_and_dlg,
+ .update_soc_for_wm_a = dcn30_update_soc_for_wm_a,
.populate_dml_pipes = dcn30_populate_dml_pipes_from_context,
.acquire_idle_pipe_for_layer = dcn20_acquire_idle_pipe_for_layer,
.add_stream_to_ctx = dcn30_add_stream_to_ctx,
@@ -1481,6 +1484,8 @@ static bool dcn302_resource_construct(
dc->caps.mall_size_total = dc->caps.mall_size_per_mem_channel * dc->ctx->dc_bios->vram_info.num_chans * 1048576;
dc->caps.cursor_cache_size = dc->caps.max_cursor_size * dc->caps.max_cursor_size * 8;
dc->caps.max_slave_planes = 1;
+ dc->caps.max_slave_yuv_planes = 1;
+ dc->caps.max_slave_rgb_planes = 1;
dc->caps.post_blend_color_processing = true;
dc->caps.force_dp_tps4_for_cp2520 = true;
dc->caps.extended_aux_timeout_support = true;
diff --git a/drivers/gpu/drm/amd/display/dc/dm_helpers.h b/drivers/gpu/drm/amd/display/dc/dm_helpers.h
index 07e349b1067b..7617fab9e1f9 100644
--- a/drivers/gpu/drm/amd/display/dc/dm_helpers.h
+++ b/drivers/gpu/drm/amd/display/dc/dm_helpers.h
@@ -147,6 +147,8 @@ bool dm_helpers_dp_write_dsc_enable(
bool dm_helpers_is_dp_sink_present(
struct dc_link *link);
+void dm_helpers_mst_enable_stream_features(const struct dc_stream_state *stream);
+
enum dc_edid_status dm_helpers_read_local_edid(
struct dc_context *ctx,
struct dc_link *link,
@@ -156,4 +158,6 @@ void dm_set_dcn_clocks(
struct dc_context *ctx,
struct dc_clocks *clks);
+bool dm_helpers_dmub_outbox0_interrupt_control(struct dc_context *ctx, bool enable);
+
#endif /* __DM_HELPERS__ */
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20.c b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20.c
index 0f3f510fd83b..9729cf292e84 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20.c
@@ -3437,6 +3437,7 @@ void dml20_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
mode_lib->vba.DCCEnabledInAnyPlane = true;
}
}
+ mode_lib->vba.UrgentLatency = mode_lib->vba.UrgentLatencyPixelDataOnly;
for (i = 0; i <= mode_lib->vba.soc.num_states; i++) {
locals->FabricAndDRAMBandwidthPerState[i] = dml_min(
mode_lib->vba.DRAMSpeedPerState[i] * mode_lib->vba.NumberOfChannels
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20v2.c b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20v2.c
index 210c96cd5b03..51098c2c9854 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20v2.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_mode_vba_20v2.c
@@ -3544,6 +3544,7 @@ void dml20v2_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode
mode_lib->vba.DCCEnabledInAnyPlane = true;
}
}
+ mode_lib->vba.UrgentLatency = mode_lib->vba.UrgentLatencyPixelDataOnly;
for (i = 0; i <= mode_lib->vba.soc.num_states; i++) {
locals->FabricAndDRAMBandwidthPerState[i] = dml_min(
mode_lib->vba.DRAMSpeedPerState[i] * mode_lib->vba.NumberOfChannels
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20.c b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20.c
index 72423dc425dc..799bae229e67 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20.c
@@ -293,13 +293,31 @@ static void handle_det_buf_split(struct display_mode_lib *mode_lib,
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20v2.c b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20v2.c
index 9c78446c3a9d..6a6d5970d1d5 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20v2.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn20/display_rq_dlg_calc_20v2.c
@@ -293,13 +293,31 @@ static void handle_det_buf_split(struct display_mode_lib *mode_lib,
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c b/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c
index edd41d358291..dc1c81a6e377 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn21/display_rq_dlg_calc_21.c
@@ -277,13 +277,31 @@ static void handle_det_buf_split(
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c b/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c
index bc07082c1357..cb3f70a71b51 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_mode_vba_30.c
@@ -4050,7 +4050,7 @@ void dml30_ModeSupportAndSystemConfigurationFull(struct display_mode_lib *mode_l
v->RequiredDPPCLK[i][j][NumberOfNonSplitPlaneOfMaximumBandwidth] = v->MinDPPCLKUsingSingleDPP[NumberOfNonSplitPlaneOfMaximumBandwidth]
* (1 + v->DISPCLKDPPCLKDSCCLKDownSpreading / 100) / 2;
v->TotalNumberOfActiveDPP[i][j] = v->TotalNumberOfActiveDPP[i][j] + 1;
- v->TotalNumberOfSingleDPPPlanes[i][j] = v->TotalNumberOfSingleDPPPlanes[i][j] + 1;
+ v->TotalNumberOfSingleDPPPlanes[i][j] = v->TotalNumberOfSingleDPPPlanes[i][j] - 1;
}
}
if (v->TotalNumberOfActiveDPP[i][j] > v->MaxNumDPP) {
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_rq_dlg_calc_30.c b/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_rq_dlg_calc_30.c
index 0f14f205ebe5..04601a767a8f 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_rq_dlg_calc_30.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dcn30/display_rq_dlg_calc_30.c
@@ -237,13 +237,31 @@ static void handle_det_buf_split(struct display_mode_lib *mode_lib,
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
diff --git a/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.c b/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.c
index 098d6433f7f3..1f7b6ddf3020 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/display_mode_lib.c
@@ -226,7 +226,7 @@ void dml_log_pipe_params(
dml_print("DML PARAMS: PIPE [%d] DISPLAY OUTPUT PARAMS:\n", i);
dml_print("DML PARAMS: output_type = %d\n", dout->output_type);
dml_print("DML PARAMS: output_format = %d\n", dout->output_format);
- dml_print("DML PARAMS: output_bpc = %d\n", dout->output_bpc);
+ dml_print("DML PARAMS: dsc_input_bpc = %d\n", dout->dsc_input_bpc);
dml_print("DML PARAMS: output_bpp = %3.4f\n", dout->output_bpp);
dml_print("DML PARAMS: dp_lanes = %d\n", dout->dp_lanes);
dml_print("DML PARAMS: dsc_enable = %d\n", dout->dsc_enable);
diff --git a/drivers/gpu/drm/amd/display/dc/dml/display_mode_structs.h b/drivers/gpu/drm/amd/display/dc/dml/display_mode_structs.h
index 0c5128187e08..2ece3690bfa3 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/display_mode_structs.h
+++ b/drivers/gpu/drm/amd/display/dc/dml/display_mode_structs.h
@@ -164,7 +164,7 @@ struct _vcs_dpi_ip_params_st {
double writeback_max_vscl_ratio;
double writeback_min_hscl_ratio;
double writeback_min_vscl_ratio;
- double maximum_dsc_bits_per_component;
+ unsigned int maximum_dsc_bits_per_component;
unsigned int writeback_max_hscl_taps;
unsigned int writeback_max_vscl_taps;
unsigned int writeback_line_buffer_luma_buffer_size;
@@ -292,10 +292,10 @@ struct writeback_st {
struct _vcs_dpi_display_output_params_st {
int dp_lanes;
double output_bpp;
+ unsigned int dsc_input_bpc;
int dsc_enable;
int wb_enable;
int num_active_wb;
- int output_bpc;
int output_type;
int is_virtual;
int output_format;
diff --git a/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.c b/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.c
index bc0485a59018..2a967458065b 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/display_mode_vba.c
@@ -471,7 +471,13 @@ static void fetch_pipe_params(struct display_mode_lib *mode_lib)
mode_lib->vba.DSCEnable[mode_lib->vba.NumberOfActivePlanes] = dout->dsc_enable;
mode_lib->vba.NumberOfDSCSlices[mode_lib->vba.NumberOfActivePlanes] =
dout->dsc_slices;
- mode_lib->vba.DSCInputBitPerComponent[mode_lib->vba.NumberOfActivePlanes] = dout->output_bpc;
+ if (!dout->dsc_input_bpc) {
+ mode_lib->vba.DSCInputBitPerComponent[mode_lib->vba.NumberOfActivePlanes] =
+ ip->maximum_dsc_bits_per_component;
+ } else {
+ mode_lib->vba.DSCInputBitPerComponent[mode_lib->vba.NumberOfActivePlanes] =
+ dout->dsc_input_bpc;
+ }
mode_lib->vba.WritebackEnable[mode_lib->vba.NumberOfActivePlanes] = dout->wb_enable;
mode_lib->vba.ActiveWritebacksPerPlane[mode_lib->vba.NumberOfActivePlanes] =
dout->num_active_wb;
@@ -599,7 +605,6 @@ static void fetch_pipe_params(struct display_mode_lib *mode_lib)
for (k = j + 1; k < mode_lib->vba.cache_num_pipes; ++k) {
display_pipe_source_params_st *src_k = &pipes[k].pipe.src;
display_pipe_dest_params_st *dst_k = &pipes[k].pipe.dest;
- display_output_params_st *dout_k = &pipes[j].dout;
if (src_k->is_hsplit && !visited[k]
&& src->hsplit_grp == src_k->hsplit_grp) {
@@ -620,8 +625,6 @@ static void fetch_pipe_params(struct display_mode_lib *mode_lib)
mode_lib->vba.ViewportHeightChroma[mode_lib->vba.NumberOfActivePlanes] +=
src_k->viewport_height_c;
}
- mode_lib->vba.NumberOfDSCSlices[mode_lib->vba.NumberOfActivePlanes] +=
- dout_k->dsc_slices;
visited[k] = true;
}
diff --git a/drivers/gpu/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c b/drivers/gpu/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c
index 4c3e9cc30167..414da64f5734 100644
--- a/drivers/gpu/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c
+++ b/drivers/gpu/drm/amd/display/dc/dml/dml1_display_rq_dlg_calc.c
@@ -344,13 +344,31 @@ static void handle_det_buf_split(
if (surf_linear) {
log2_swath_height_l = 0;
log2_swath_height_c = 0;
- } else if (!surf_vert) {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_height) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_height) - req128_c;
} else {
- log2_swath_height_l = dml_log2(rq_param->misc.rq_l.blk256_width) - req128_l;
- log2_swath_height_c = dml_log2(rq_param->misc.rq_c.blk256_width) - req128_c;
+ unsigned int swath_height_l;
+ unsigned int swath_height_c;
+
+ if (!surf_vert) {
+ swath_height_l = rq_param->misc.rq_l.blk256_height;
+ swath_height_c = rq_param->misc.rq_c.blk256_height;
+ } else {
+ swath_height_l = rq_param->misc.rq_l.blk256_width;
+ swath_height_c = rq_param->misc.rq_c.blk256_width;
+ }
+
+ if (swath_height_l > 0)
+ log2_swath_height_l = dml_log2(swath_height_l);
+
+ if (req128_l && log2_swath_height_l > 0)
+ log2_swath_height_l -= 1;
+
+ if (swath_height_c > 0)
+ log2_swath_height_c = dml_log2(swath_height_c);
+
+ if (req128_c && log2_swath_height_c > 0)
+ log2_swath_height_c -= 1;
}
+
rq_param->dlg.rq_l.swath_height = 1 << log2_swath_height_l;
rq_param->dlg.rq_c.swath_height = 1 << log2_swath_height_c;
diff --git a/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c b/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c
index c62d0eddc9c6..be57088d185d 100644
--- a/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c
+++ b/drivers/gpu/drm/amd/display/dc/dsc/dc_dsc.c
@@ -37,59 +37,6 @@ static uint32_t dsc_policy_max_target_bpp_limit = 16;
/* default DSC policy enables DSC only when needed */
static bool dsc_policy_enable_dsc_when_not_needed;
-static uint32_t dc_dsc_bandwidth_in_kbps_from_timing(
- const struct dc_crtc_timing *timing)
-{
- uint32_t bits_per_channel = 0;
- uint32_t kbps;
-
- if (timing->flags.DSC) {
- kbps = (timing->pix_clk_100hz * timing->dsc_cfg.bits_per_pixel);
- kbps = kbps / 160 + ((kbps % 160) ? 1 : 0);
- return kbps;
- }
-
- switch (timing->display_color_depth) {
- case COLOR_DEPTH_666:
- bits_per_channel = 6;
- break;
- case COLOR_DEPTH_888:
- bits_per_channel = 8;
- break;
- case COLOR_DEPTH_101010:
- bits_per_channel = 10;
- break;
- case COLOR_DEPTH_121212:
- bits_per_channel = 12;
- break;
- case COLOR_DEPTH_141414:
- bits_per_channel = 14;
- break;
- case COLOR_DEPTH_161616:
- bits_per_channel = 16;
- break;
- default:
- break;
- }
-
- ASSERT(bits_per_channel != 0);
-
- kbps = timing->pix_clk_100hz / 10;
- kbps *= bits_per_channel;
-
- if (timing->flags.Y_ONLY != 1) {
- /*Only YOnly make reduce bandwidth by 1/3 compares to RGB*/
- kbps *= 3;
- if (timing->pixel_encoding == PIXEL_ENCODING_YCBCR420)
- kbps /= 2;
- else if (timing->pixel_encoding == PIXEL_ENCODING_YCBCR422)
- kbps = kbps * 2 / 3;
- }
-
- return kbps;
-
-}
-
static bool dsc_buff_block_size_from_dpcd(int dpcd_buff_block_size, int *buff_block_size)
{
@@ -315,18 +262,18 @@ static inline uint32_t dsc_div_by_10_round_up(uint32_t value)
* and uncompressed bandwidth.
*/
static void get_dsc_bandwidth_range(
- const uint32_t min_bpp,
- const uint32_t max_bpp,
+ const uint32_t min_bpp_x16,
+ const uint32_t max_bpp_x16,
const struct dsc_enc_caps *dsc_caps,
const struct dc_crtc_timing *timing,
struct dc_dsc_bw_range *range)
{
/* native stream bandwidth */
- range->stream_kbps = dc_dsc_bandwidth_in_kbps_from_timing(timing);
+ range->stream_kbps = dc_bandwidth_in_kbps_from_timing(timing);
/* max dsc target bpp */
- range->max_kbps = dsc_div_by_10_round_up(max_bpp * timing->pix_clk_100hz);
- range->max_target_bpp_x16 = max_bpp * 16;
+ range->max_kbps = dc_dsc_stream_bandwidth_in_kbps(timing->pix_clk_100hz, max_bpp_x16);
+ range->max_target_bpp_x16 = max_bpp_x16;
if (range->max_kbps > range->stream_kbps) {
/* max dsc target bpp is capped to native bandwidth */
range->max_kbps = range->stream_kbps;
@@ -334,8 +281,8 @@ static void get_dsc_bandwidth_range(
}
/* min dsc target bpp */
- range->min_kbps = dsc_div_by_10_round_up(min_bpp * timing->pix_clk_100hz);
- range->min_target_bpp_x16 = min_bpp * 16;
+ range->min_kbps = dc_dsc_stream_bandwidth_in_kbps(timing->pix_clk_100hz, min_bpp_x16);
+ range->min_target_bpp_x16 = min_bpp_x16;
if (range->min_kbps > range->max_kbps) {
/* min dsc target bpp is capped to max dsc bandwidth*/
range->min_kbps = range->max_kbps;
@@ -363,12 +310,17 @@ static bool decide_dsc_target_bpp_x16(
memset(&range, 0, sizeof(range));
- get_dsc_bandwidth_range(policy->min_target_bpp, policy->max_target_bpp,
+ get_dsc_bandwidth_range(policy->min_target_bpp * 16, policy->max_target_bpp * 16,
dsc_common_caps, timing, &range);
if (!policy->enable_dsc_when_not_needed && target_bandwidth_kbps >= range.stream_kbps) {
/* enough bandwidth without dsc */
*target_bpp_x16 = 0;
should_use_dsc = false;
+ } else if (policy->preferred_bpp_x16 > 0 &&
+ policy->preferred_bpp_x16 <= range.max_target_bpp_x16 &&
+ policy->preferred_bpp_x16 >= range.min_target_bpp_x16) {
+ *target_bpp_x16 = policy->preferred_bpp_x16;
+ should_use_dsc = true;
} else if (target_bandwidth_kbps >= range.max_kbps) {
/* use max target bpp allowed */
*target_bpp_x16 = range.max_target_bpp_x16;
@@ -545,7 +497,7 @@ static bool setup_dsc_config(
int target_bandwidth_kbps,
const struct dc_crtc_timing *timing,
int min_slice_height_override,
- int max_dsc_target_bpp_limit_override,
+ int max_dsc_target_bpp_limit_override_x16,
struct dc_dsc_config *dsc_cfg)
{
struct dsc_enc_caps dsc_common_caps;
@@ -564,7 +516,7 @@ static bool setup_dsc_config(
memset(dsc_cfg, 0, sizeof(struct dc_dsc_config));
- dc_dsc_get_policy_for_timing(timing, max_dsc_target_bpp_limit_override, &policy);
+ dc_dsc_get_policy_for_timing(timing, max_dsc_target_bpp_limit_override_x16, &policy);
pic_width = timing->h_addressable + timing->h_border_left + timing->h_border_right;
pic_height = timing->v_addressable + timing->v_border_top + timing->v_border_bottom;
@@ -865,8 +817,8 @@ bool dc_dsc_parse_dsc_dpcd(const struct dc *dc, const uint8_t *dpcd_dsc_basic_da
bool dc_dsc_compute_bandwidth_range(
const struct display_stream_compressor *dsc,
uint32_t dsc_min_slice_height_override,
- uint32_t min_bpp,
- uint32_t max_bpp,
+ uint32_t min_bpp_x16,
+ uint32_t max_bpp_x16,
const struct dsc_dec_dpcd_caps *dsc_sink_caps,
const struct dc_crtc_timing *timing,
struct dc_dsc_bw_range *range)
@@ -883,10 +835,10 @@ bool dc_dsc_compute_bandwidth_range(
if (is_dsc_possible)
is_dsc_possible = setup_dsc_config(dsc_sink_caps, &dsc_enc_caps, 0, timing,
- dsc_min_slice_height_override, max_bpp, &config);
+ dsc_min_slice_height_override, max_bpp_x16, &config);
if (is_dsc_possible)
- get_dsc_bandwidth_range(min_bpp, max_bpp, &dsc_common_caps, timing, range);
+ get_dsc_bandwidth_range(min_bpp_x16, max_bpp_x16, &dsc_common_caps, timing, range);
return is_dsc_possible;
}
@@ -908,11 +860,20 @@ bool dc_dsc_compute_config(
&dsc_enc_caps,
target_bandwidth_kbps,
timing, dsc_min_slice_height_override,
- max_target_bpp_limit_override, dsc_cfg);
+ max_target_bpp_limit_override * 16, dsc_cfg);
return is_dsc_possible;
}
-void dc_dsc_get_policy_for_timing(const struct dc_crtc_timing *timing, uint32_t max_target_bpp_limit_override, struct dc_dsc_policy *policy)
+uint32_t dc_dsc_stream_bandwidth_in_kbps(uint32_t pix_clk_100hz, uint32_t bpp_x16)
+{
+ struct fixed31_32 link_bw_kbps;
+ link_bw_kbps = dc_fixpt_from_int(pix_clk_100hz);
+ link_bw_kbps = dc_fixpt_div_int(link_bw_kbps, 160);
+ link_bw_kbps = dc_fixpt_mul_int(link_bw_kbps, bpp_x16);
+ return dc_fixpt_ceil(link_bw_kbps);
+}
+
+void dc_dsc_get_policy_for_timing(const struct dc_crtc_timing *timing, uint32_t max_target_bpp_limit_override_x16, struct dc_dsc_policy *policy)
{
uint32_t bpc = 0;
@@ -967,13 +928,15 @@ void dc_dsc_get_policy_for_timing(const struct dc_crtc_timing *timing, uint32_t
return;
}
+ policy->preferred_bpp_x16 = timing->dsc_fixed_bits_per_pixel_x16;
+
/* internal upper limit, default 16 bpp */
if (policy->max_target_bpp > dsc_policy_max_target_bpp_limit)
policy->max_target_bpp = dsc_policy_max_target_bpp_limit;
/* apply override */
- if (max_target_bpp_limit_override && policy->max_target_bpp > max_target_bpp_limit_override)
- policy->max_target_bpp = max_target_bpp_limit_override;
+ if (max_target_bpp_limit_override_x16 && policy->max_target_bpp > max_target_bpp_limit_override_x16 / 16)
+ policy->max_target_bpp = max_target_bpp_limit_override_x16 / 16;
/* enable DSC when not needed, default false */
if (dsc_policy_enable_dsc_when_not_needed)
diff --git a/drivers/gpu/drm/amd/display/dc/gpio/dce110/hw_factory_dce110.c b/drivers/gpu/drm/amd/display/dc/gpio/dce110/hw_factory_dce110.c
index 66e4841f41e4..ca335ea60412 100644
--- a/drivers/gpu/drm/amd/display/dc/gpio/dce110/hw_factory_dce110.c
+++ b/drivers/gpu/drm/amd/display/dc/gpio/dce110/hw_factory_dce110.c
@@ -48,10 +48,6 @@
#define REGI(reg_name, block, id)\
mm ## block ## id ## _ ## reg_name
-#include "../hw_gpio.h"
-#include "../hw_ddc.h"
-#include "../hw_hpd.h"
-
#include "reg_helper.h"
#include "../hpd_regs.h"
diff --git a/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c b/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c
index 5e384a8a83dc..51855a2624cf 100644
--- a/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c
+++ b/drivers/gpu/drm/amd/display/dc/hdcp/hdcp_msg.c
@@ -39,7 +39,7 @@
#define HDCP14_KSV_SIZE 5
#define HDCP14_MAX_KSV_FIFO_SIZE 127*HDCP14_KSV_SIZE
-static const bool hdcp_cmd_is_read[] = {
+static const bool hdcp_cmd_is_read[HDCP_MESSAGE_ID_MAX] = {
[HDCP_MESSAGE_ID_READ_BKSV] = true,
[HDCP_MESSAGE_ID_READ_RI_R0] = true,
[HDCP_MESSAGE_ID_READ_PJ] = true,
@@ -75,7 +75,7 @@ static const bool hdcp_cmd_is_read[] = {
[HDCP_MESSAGE_ID_WRITE_CONTENT_STREAM_TYPE] = false
};
-static const uint8_t hdcp_i2c_offsets[] = {
+static const uint8_t hdcp_i2c_offsets[HDCP_MESSAGE_ID_MAX] = {
[HDCP_MESSAGE_ID_READ_BKSV] = 0x0,
[HDCP_MESSAGE_ID_READ_RI_R0] = 0x8,
[HDCP_MESSAGE_ID_READ_PJ] = 0xA,
@@ -106,7 +106,8 @@ static const uint8_t hdcp_i2c_offsets[] = {
[HDCP_MESSAGE_ID_WRITE_REPEATER_AUTH_SEND_ACK] = 0x60,
[HDCP_MESSAGE_ID_WRITE_REPEATER_AUTH_STREAM_MANAGE] = 0x60,
[HDCP_MESSAGE_ID_READ_REPEATER_AUTH_STREAM_READY] = 0x80,
- [HDCP_MESSAGE_ID_READ_RXSTATUS] = 0x70
+ [HDCP_MESSAGE_ID_READ_RXSTATUS] = 0x70,
+ [HDCP_MESSAGE_ID_WRITE_CONTENT_STREAM_TYPE] = 0x0,
};
struct protection_properties {
@@ -184,7 +185,7 @@ static const struct protection_properties hdmi_14_protection = {
.process_transaction = hdmi_14_process_transaction
};
-static const uint32_t hdcp_dpcd_addrs[] = {
+static const uint32_t hdcp_dpcd_addrs[HDCP_MESSAGE_ID_MAX] = {
[HDCP_MESSAGE_ID_READ_BKSV] = 0x68000,
[HDCP_MESSAGE_ID_READ_RI_R0] = 0x68005,
[HDCP_MESSAGE_ID_READ_PJ] = 0xFFFFFFFF,
diff --git a/drivers/gpu/drm/amd/display/dc/inc/clock_source.h b/drivers/gpu/drm/amd/display/dc/inc/clock_source.h
index 1b01a9a58d14..e2b3a2c7a927 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/clock_source.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/clock_source.h
@@ -170,6 +170,11 @@ struct clock_source_funcs {
const struct clock_source *clock_source,
unsigned int inst,
unsigned int *pixel_clk_khz);
+ bool (*override_dp_pix_clk)(
+ struct clock_source *clock_source,
+ unsigned int inst,
+ unsigned int pixel_clk,
+ unsigned int ref_clk);
};
struct clock_source {
diff --git a/drivers/gpu/drm/amd/display/dc/inc/core_types.h b/drivers/gpu/drm/amd/display/dc/inc/core_types.h
index 8efa1b80546d..81b92f20d5b6 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/core_types.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/core_types.h
@@ -97,6 +97,10 @@ struct resource_funcs {
const struct panel_cntl_init_data *panel_cntl_init_data);
struct link_encoder *(*link_enc_create)(
const struct encoder_init_data *init);
+ /* Create a minimal link encoder object with no dc_link object
+ * associated with it. */
+ struct link_encoder *(*link_enc_create_minimal)(struct dc_context *ctx, enum engine_id eng_id);
+
bool (*validate_bandwidth)(
struct dc *dc,
struct dc_state *context,
@@ -106,12 +110,35 @@ struct resource_funcs {
display_e2e_pipe_params_st *pipes,
int pipe_cnt,
int vlevel);
+ void (*update_soc_for_wm_a)(
+ struct dc *dc, struct dc_state *context);
int (*populate_dml_pipes)(
struct dc *dc,
struct dc_state *context,
display_e2e_pipe_params_st *pipes,
bool fast_validate);
+ /*
+ * Algorithm for assigning available link encoders to links.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ */
+ void (*link_encs_assign)(
+ struct dc *dc,
+ struct dc_state *state,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count);
+ /*
+ * Unassign a link encoder from a stream.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ */
+ void (*link_enc_unassign)(
+ struct dc_state *state,
+ struct dc_stream_state *stream);
+
enum dc_status (*validate_global)(
struct dc *dc,
struct dc_state *context);
@@ -210,6 +237,15 @@ struct resource_pool {
unsigned int underlay_pipe_index;
unsigned int stream_enc_count;
+ /* An array for accessing the link encoder objects that have been created.
+ * Index in array corresponds to engine ID - viz. 0: ENGINE_ID_DIGA
+ */
+ struct link_encoder *link_encoders[MAX_DIG_LINK_ENCODERS];
+ /* Number of DIG link encoder objects created - i.e. number of valid
+ * entries in link_encoders array.
+ */
+ unsigned int dig_link_enc_count;
+
#if defined(CONFIG_DRM_AMD_DC_DCN)
struct dc_3dlut *mpc_lut[MAX_PIPES];
struct dc_transfer_func *mpc_shaper[MAX_PIPES];
@@ -343,6 +379,12 @@ struct resource_context {
uint8_t clock_source_ref_count[MAX_CLOCK_SOURCES];
uint8_t dp_clock_source_ref_count;
bool is_dsc_acquired[MAX_PIPES];
+ /* A table/array of encoder-to-link assignments. One entry per stream.
+ * Indexed by stream index in dc_state.
+ */
+ struct link_enc_assignment link_enc_assignments[MAX_PIPES];
+ /* List of available link encoders. Uses engine ID as encoder identifier. */
+ enum engine_id link_enc_avail[MAX_DIG_LINK_ENCODERS];
#if defined(CONFIG_DRM_AMD_DC_DCN)
bool is_mpc_3dlut_acquired[MAX_PIPES];
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/inc/dc_link_ddc.h b/drivers/gpu/drm/amd/display/dc/inc/dc_link_ddc.h
index b324e13f3f78..4d7b271b6409 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/dc_link_ddc.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/dc_link_ddc.h
@@ -56,6 +56,7 @@ struct dp_receiver_id_info;
struct i2c_payloads;
struct aux_payloads;
+enum aux_return_code_type;
void dal_ddc_i2c_payloads_add(
struct i2c_payloads *payloads,
@@ -100,7 +101,7 @@ bool dal_ddc_submit_aux_command(struct ddc_service *ddc,
int dc_link_aux_transfer_raw(struct ddc_service *ddc,
struct aux_payload *payload,
- enum aux_channel_operation_result *operation_result);
+ enum aux_return_code_type *operation_result);
bool dc_link_aux_transfer_with_retries(struct ddc_service *ddc,
struct aux_payload *payload);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
index b970a32177af..3ae05c96d557 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/dc_link_dp.h
@@ -52,6 +52,10 @@ bool dp_validate_mode_timing(
struct dc_link *link,
const struct dc_crtc_timing *timing);
+bool decide_edp_link_settings(struct dc_link *link,
+ struct dc_link_settings *link_setting,
+ uint32_t req_bw);
+
void decide_link_settings(
struct dc_stream_state *stream,
struct dc_link_settings *link_setting);
@@ -71,6 +75,8 @@ void detect_edp_sink_caps(struct dc_link *link);
bool is_dp_active_dongle(const struct dc_link *link);
+bool is_edp_ilr_optimization_required(struct dc_link *link, struct dc_crtc_timing *crtc_timing);
+
void dp_enable_mst_on_sink(struct dc_link *link, bool enable);
enum dp_panel_mode dp_get_panel_mode(struct dc_link *link);
@@ -86,5 +92,7 @@ bool dp_set_dsc_enable(struct pipe_ctx *pipe_ctx, bool enable);
bool dp_set_dsc_pps_sdp(struct pipe_ctx *pipe_ctx, bool enable);
void dp_set_dsc_on_stream(struct pipe_ctx *pipe_ctx, bool enable);
bool dp_update_dsc_config(struct pipe_ctx *pipe_ctx);
+bool dp_set_dsc_on_rx(struct pipe_ctx *pipe_ctx, bool enable);
+
#endif /* __DC_LINK_DP_H__ */
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/aux_engine.h b/drivers/gpu/drm/amd/display/dc/inc/hw/aux_engine.h
index e77b3a76766d..2ae630bf2aee 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/aux_engine.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/aux_engine.h
@@ -29,6 +29,8 @@
#include "dc_ddc_types.h"
#include "include/i2caux_interface.h"
+enum aux_return_code_type;
+
enum i2caux_transaction_operation {
I2CAUX_TRANSACTION_READ,
I2CAUX_TRANSACTION_WRITE
@@ -162,7 +164,7 @@ struct aux_engine_funcs {
uint8_t *buffer,
uint8_t *reply_result,
uint32_t *sw_status);
- enum aux_channel_operation_result (*get_channel_status)(
+ enum aux_return_code_type (*get_channel_status)(
struct aux_engine *engine,
uint8_t *returned_bytes);
bool (*is_engine_available)(struct aux_engine *engine);
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h b/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h
index 3a29f379d0c8..5dc8d02b40c3 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/clk_mgr.h
@@ -262,14 +262,9 @@ struct clk_mgr_funcs {
/* Get current memclk states from PMFW, update relevant structures */
void (*get_memclk_states_from_smu)(struct clk_mgr *clk_mgr);
-};
-
-struct dpm_clocks;
-struct wartermarks;
-struct smu_watermark_set {
- struct watermarks *wm_set;
- union large_integer mc_address;
+ /* Get SMU present */
+ bool (*is_smu_present)(struct clk_mgr *clk_mgr);
};
struct clk_mgr {
@@ -283,7 +278,6 @@ struct clk_mgr {
struct clk_state_registers_and_bypass boot_snapshot;
struct clk_bw_params *bw_params;
struct pp_smu_wm_range_sets ranges;
- struct smu_watermark_set smu_wm_set;
};
/* forward declarations */
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h b/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h
index cd1c0dc32bf8..8df2765cce78 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/dmcu.h
@@ -56,6 +56,20 @@ struct dmcu {
bool auto_load_dmcu;
};
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+struct crc_region {
+ uint16_t x_start;
+ uint16_t y_start;
+ uint16_t x_end;
+ uint16_t y_end;
+};
+
+struct otg_phy_mux {
+ uint8_t phy_output_num;
+ uint8_t otg_output_num;
+};
+#endif
+
struct dmcu_funcs {
bool (*dmcu_init)(struct dmcu *dmcu);
bool (*load_iram)(struct dmcu *dmcu,
@@ -84,6 +98,13 @@ struct dmcu_funcs {
int *min_frame_rate,
int *max_frame_rate);
bool (*recv_edid_cea_ack)(struct dmcu *dmcu, int *offset);
+#if defined(CONFIG_DRM_AMD_SECURE_DISPLAY)
+ void (*forward_crc_window)(struct dmcu *dmcu,
+ struct crc_region *crc_win,
+ struct otg_phy_mux *mux_mapping);
+ void (*stop_crc_win_update)(struct dmcu *dmcu,
+ struct otg_phy_mux *mux_mapping);
+#endif
};
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h b/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h
index 22f3f643ed1b..80e1a32bc63d 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/hubp.h
@@ -29,6 +29,7 @@
#include "mem_input.h"
#define OPP_ID_INVALID 0xf
+#define MAX_TTU 0xffffff
enum cursor_pitch {
@@ -191,6 +192,8 @@ struct hubp_funcs {
bool (*hubp_in_blank)(struct hubp *hubp);
void (*hubp_soft_reset)(struct hubp *hubp, bool reset);
+ void (*hubp_set_flip_int)(struct hubp *hubp);
+
};
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h b/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h
index 43e33f47734d..31a1713bb49f 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/hw_shared.h
@@ -36,6 +36,7 @@
#define MAX_AUDIOS 7
#define MAX_PIPES 6
+#define MAX_DIG_LINK_ENCODERS 7
#define MAX_DWB_PIPES 1
struct gamma_curve {
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h b/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h
index 7f5acd8fb918..80bc99500645 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/link_encoder.h
@@ -187,4 +187,17 @@ struct link_encoder_funcs {
struct link_encoder *enc);
};
+/*
+ * Used to track assignments of links (display endpoints) to link encoders.
+ *
+ * Entry in link_enc_assignments table in struct resource_context.
+ * Entries only marked valid once encoder assigned to a link and invalidated once unassigned.
+ * Uses engine ID as identifier since PHY ID not relevant for USB4 DPIA endpoint.
+ */
+struct link_enc_assignment {
+ bool valid;
+ struct display_endpoint_id ep_id;
+ enum engine_id eng_id;
+};
+
#endif /* LINK_ENCODER_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h b/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
index 754832d216fd..9ff68b67780c 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw/timing_generator.h
@@ -109,6 +109,12 @@ enum h_timing_div_mode {
H_TIMING_DIV_BY4,
};
+enum timing_synchronization_type {
+ NOT_SYNCHRONIZABLE,
+ TIMING_SYNCHRONIZABLE,
+ VBLANK_SYNCHRONIZABLE
+};
+
struct crc_params {
/* Regions used to calculate CRC*/
uint16_t windowa_x_start;
@@ -292,6 +298,12 @@ struct timing_generator_funcs {
uint32_t window_start, uint32_t window_end);
void (*set_vtotal_change_limit)(struct timing_generator *optc,
uint32_t limit);
+ void (*align_vblanks)(struct timing_generator *master_optc,
+ struct timing_generator *slave_optc,
+ uint32_t master_pixel_clock_100Hz,
+ uint32_t slave_pixel_clock_100Hz,
+ uint8_t master_clock_divider,
+ uint8_t slave_clock_divider);
};
#endif
diff --git a/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h b/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
index 0586ab2ffd6a..1d5853c95448 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/hw_sequencer.h
@@ -111,12 +111,14 @@ struct hw_sequencer_funcs {
void (*enable_timing_synchronization)(struct dc *dc,
int group_index, int group_size,
struct pipe_ctx *grouped_pipes[]);
+ void (*enable_vblanks_synchronization)(struct dc *dc,
+ int group_index, int group_size,
+ struct pipe_ctx *grouped_pipes[]);
void (*setup_periodic_interrupt)(struct dc *dc,
struct pipe_ctx *pipe_ctx,
enum vline_select vline);
void (*set_drr)(struct pipe_ctx **pipe_ctx, int num_pipes,
- unsigned int vmin, unsigned int vmax,
- unsigned int vmid, unsigned int vmid_frame_number);
+ struct dc_crtc_timing_adjust adjust);
void (*set_static_screen_control)(struct pipe_ctx **pipe_ctx,
int num_pipes,
const struct dc_static_screen_params *events);
@@ -215,6 +217,8 @@ struct hw_sequencer_funcs {
void (*set_pipe)(struct pipe_ctx *pipe_ctx);
+ void (*get_dcc_en_bits)(struct dc *dc, int *dcc_en_bits);
+
/* Idle Optimization Related */
bool (*apply_idle_power_optimizations)(struct dc *dc, bool enable);
@@ -231,10 +235,6 @@ struct hw_sequencer_funcs {
enum dc_color_depth color_depth,
const struct tg_color *solid_color,
int width, int height, int offset);
-
- void (*set_hubp_blank)(const struct dc *dc,
- struct pipe_ctx *pipe_ctx,
- bool blank_enable);
};
void color_space_to_black_color(
diff --git a/drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h b/drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h
new file mode 100644
index 000000000000..7d36e55f3097
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dc/inc/link_enc_cfg.h
@@ -0,0 +1,86 @@
+/*
+ * Copyright 2021 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef DC_INC_LINK_ENC_CFG_H_
+#define DC_INC_LINK_ENC_CFG_H_
+
+/* This module implements functionality for dynamically assigning DIG link
+ * encoder resources to display endpoints (links).
+ */
+
+#include "core_types.h"
+
+/*
+ * Initialise link encoder resource tracking.
+ */
+void link_enc_cfg_init(
+ struct dc *dc,
+ struct dc_state *state);
+
+/*
+ * Algorithm for assigning available DIG link encoders to streams.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ *
+ * Loop over all streams twice:
+ * a) First assign encoders to unmappable endpoints.
+ * b) Then assign encoders to mappable endpoints.
+ */
+void link_enc_cfg_link_encs_assign(
+ struct dc *dc,
+ struct dc_state *state,
+ struct dc_stream_state *streams[],
+ uint8_t stream_count);
+
+/*
+ * Unassign a link encoder from a stream.
+ *
+ * Update link_enc_assignments table and link_enc_avail list accordingly in
+ * struct resource_context.
+ */
+void link_enc_cfg_link_enc_unassign(
+ struct dc_state *state,
+ struct dc_stream_state *stream);
+
+/*
+ * Check whether the transmitter driven by a link encoder is a mappable
+ * endpoint.
+ */
+bool link_enc_cfg_is_transmitter_mappable(
+ struct dc_state *state,
+ struct link_encoder *link_enc);
+
+/* Return link using DIG link encoder resource. NULL if unused. */
+struct dc_link *link_enc_cfg_get_link_using_link_enc(
+ struct dc_state *state,
+ enum engine_id eng_id);
+
+/* Return DIG link encoder used by link. NULL if unused. */
+struct link_encoder *link_enc_cfg_get_link_enc_used_by_link(
+ struct dc_state *state,
+ struct dc_link *link);
+
+#endif /* DC_INC_LINK_ENC_CFG_H_ */
diff --git a/drivers/gpu/drm/amd/display/dc/inc/resource.h b/drivers/gpu/drm/amd/display/dc/inc/resource.h
index d89815a46190..fe1e5833c96a 100644
--- a/drivers/gpu/drm/amd/display/dc/inc/resource.h
+++ b/drivers/gpu/drm/amd/display/dc/inc/resource.h
@@ -48,6 +48,7 @@ struct resource_caps {
int num_ddc;
int num_vmid;
int num_dsc;
+ unsigned int num_dig_link_enc; // Total number of DIGs (digital encoders) in DIO (Display Input/Output).
int num_mpc_3dlut;
};
@@ -115,6 +116,10 @@ bool resource_are_streams_timing_synchronizable(
struct dc_stream_state *stream1,
struct dc_stream_state *stream2);
+bool resource_are_vblanks_synchronizable(
+ struct dc_stream_state *stream1,
+ struct dc_stream_state *stream2);
+
struct clock_source *resource_find_used_clk_src_for_sharing(
struct resource_context *res_ctx,
struct pipe_ctx *pipe_ctx);
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn20/irq_service_dcn20.c b/drivers/gpu/drm/amd/display/dc/irq/dcn20/irq_service_dcn20.c
index 3f1e7a196a23..c4b067d01895 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn20/irq_service_dcn20.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn20/irq_service_dcn20.c
@@ -58,6 +58,18 @@ enum dc_irq_source to_dal_irq_source_dcn20(
return DC_IRQ_SOURCE_VBLANK5;
case DCN_1_0__SRCID__DC_D6_OTG_VSTARTUP:
return DC_IRQ_SOURCE_VBLANK6;
+ case DCN_1_0__SRCID__OTG1_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC1_VLINE0;
+ case DCN_1_0__SRCID__OTG2_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC2_VLINE0;
+ case DCN_1_0__SRCID__OTG3_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC3_VLINE0;
+ case DCN_1_0__SRCID__OTG4_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC4_VLINE0;
+ case DCN_1_0__SRCID__OTG5_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC5_VLINE0;
+ case DCN_1_0__SRCID__OTG6_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC6_VLINE0;
case DCN_1_0__SRCID__HUBP0_FLIP_INTERRUPT:
return DC_IRQ_SOURCE_PFLIP1;
case DCN_1_0__SRCID__HUBP1_FLIP_INTERRUPT:
@@ -172,6 +184,11 @@ static const struct irq_source_info_funcs vupdate_no_lock_irq_info_funcs = {
.ack = NULL
};
+static const struct irq_source_info_funcs vline0_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
@@ -245,6 +262,14 @@ static const struct irq_source_info_funcs vupdate_no_lock_irq_info_funcs = {
.funcs = &vblank_irq_info_funcs\
}
+#define vline0_int_entry(reg_num)\
+ [DC_IRQ_SOURCE_DC1_VLINE0 + reg_num] = {\
+ IRQ_REG_ENTRY(OTG, reg_num,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_INT_ENABLE,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_CLEAR),\
+ .funcs = &vline0_irq_info_funcs\
+ }
+
#define dummy_irq_entry() \
{\
.funcs = &dummy_irq_info_funcs\
@@ -353,6 +378,12 @@ irq_source_info_dcn20[DAL_IRQ_SOURCES_NUMBER] = {
vblank_int_entry(3),
vblank_int_entry(4),
vblank_int_entry(5),
+ vline0_int_entry(0),
+ vline0_int_entry(1),
+ vline0_int_entry(2),
+ vline0_int_entry(3),
+ vline0_int_entry(4),
+ vline0_int_entry(5),
};
static const struct irq_service_funcs irq_service_funcs_dcn20 = {
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c b/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c
index 0e0f494fbb5e..1a5be2792055 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn21/irq_service_dcn21.c
@@ -58,6 +58,20 @@ enum dc_irq_source to_dal_irq_source_dcn21(
return DC_IRQ_SOURCE_VBLANK5;
case DCN_1_0__SRCID__DC_D6_OTG_VSTARTUP:
return DC_IRQ_SOURCE_VBLANK6;
+ case DCN_1_0__SRCID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT:
+ return DC_IRQ_SOURCE_DMCUB_OUTBOX0;
+ case DCN_1_0__SRCID__OTG1_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC1_VLINE0;
+ case DCN_1_0__SRCID__OTG2_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC2_VLINE0;
+ case DCN_1_0__SRCID__OTG3_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC3_VLINE0;
+ case DCN_1_0__SRCID__OTG4_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC4_VLINE0;
+ case DCN_1_0__SRCID__OTG5_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC5_VLINE0;
+ case DCN_1_0__SRCID__OTG6_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC6_VLINE0;
case DCN_1_0__SRCID__HUBP0_FLIP_INTERRUPT:
return DC_IRQ_SOURCE_PFLIP1;
case DCN_1_0__SRCID__HUBP1_FLIP_INTERRUPT:
@@ -173,6 +187,16 @@ static const struct irq_source_info_funcs vupdate_no_lock_irq_info_funcs = {
.ack = NULL
};
+static const struct irq_source_info_funcs dmub_trace_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
+static const struct irq_source_info_funcs vline0_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
#undef BASE_INNER
#define BASE_INNER(seg) DMU_BASE__INST0_SEG ## seg
@@ -185,6 +209,9 @@ static const struct irq_source_info_funcs vupdate_no_lock_irq_info_funcs = {
BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
+#define SRI_DMUB(reg_name)\
+ BASE(mm ## reg_name ## _BASE_IDX) + \
+ mm ## reg_name
#define IRQ_REG_ENTRY(block, reg_num, reg1, mask1, reg2, mask2)\
.enable_reg = SRI(reg1, block, reg_num),\
@@ -200,7 +227,19 @@ static const struct irq_source_info_funcs vupdate_no_lock_irq_info_funcs = {
.ack_value = \
block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK \
-
+#define IRQ_REG_ENTRY_DMUB(reg1, mask1, reg2, mask2)\
+ .enable_reg = SRI_DMUB(reg1),\
+ .enable_mask = \
+ reg1 ## __ ## mask1 ## _MASK,\
+ .enable_value = {\
+ reg1 ## __ ## mask1 ## _MASK,\
+ ~reg1 ## __ ## mask1 ## _MASK \
+ },\
+ .ack_reg = SRI_DMUB(reg2),\
+ .ack_mask = \
+ reg2 ## __ ## mask2 ## _MASK,\
+ .ack_value = \
+ reg2 ## __ ## mask2 ## _MASK \
#define hpd_int_entry(reg_num)\
[DC_IRQ_SOURCE_HPD1 + reg_num] = {\
@@ -254,6 +293,21 @@ static const struct irq_source_info_funcs vupdate_no_lock_irq_info_funcs = {
.funcs = &vblank_irq_info_funcs\
}
+#define vline0_int_entry(reg_num)\
+ [DC_IRQ_SOURCE_DC1_VLINE0 + reg_num] = {\
+ IRQ_REG_ENTRY(OTG, reg_num,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_INT_ENABLE,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_CLEAR),\
+ .funcs = &vline0_irq_info_funcs\
+ }
+
+#define dmub_trace_int_entry()\
+ [DC_IRQ_SOURCE_DMCUB_OUTBOX0] = {\
+ IRQ_REG_ENTRY_DMUB(DMCUB_INTERRUPT_ENABLE, DMCUB_OUTBOX0_READY_INT_EN,\
+ DMCUB_INTERRUPT_ACK, DMCUB_OUTBOX0_READY_INT_ACK),\
+ .funcs = &dmub_trace_irq_info_funcs\
+ }
+
#define dummy_irq_entry() \
{\
.funcs = &dummy_irq_info_funcs\
@@ -366,6 +420,13 @@ irq_source_info_dcn21[DAL_IRQ_SOURCES_NUMBER] = {
vblank_int_entry(3),
vblank_int_entry(4),
vblank_int_entry(5),
+ vline0_int_entry(0),
+ vline0_int_entry(1),
+ vline0_int_entry(2),
+ vline0_int_entry(3),
+ vline0_int_entry(4),
+ vline0_int_entry(5),
+ dmub_trace_int_entry(),
};
static const struct irq_service_funcs irq_service_funcs_dcn21 = {
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn30/irq_service_dcn30.c b/drivers/gpu/drm/amd/display/dc/irq/dcn30/irq_service_dcn30.c
index a35b76772b9d..914ce2ce1c2f 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn30/irq_service_dcn30.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn30/irq_service_dcn30.c
@@ -65,6 +65,20 @@ enum dc_irq_source to_dal_irq_source_dcn30(
return DC_IRQ_SOURCE_VBLANK5;
case DCN_1_0__SRCID__DC_D6_OTG_VSTARTUP:
return DC_IRQ_SOURCE_VBLANK6;
+ case DCN_1_0__SRCID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT:
+ return DC_IRQ_SOURCE_DMCUB_OUTBOX0;
+ case DCN_1_0__SRCID__OTG1_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC1_VLINE0;
+ case DCN_1_0__SRCID__OTG2_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC2_VLINE0;
+ case DCN_1_0__SRCID__OTG3_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC3_VLINE0;
+ case DCN_1_0__SRCID__OTG4_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC4_VLINE0;
+ case DCN_1_0__SRCID__OTG5_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC5_VLINE0;
+ case DCN_1_0__SRCID__OTG6_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC6_VLINE0;
case DCN_1_0__SRCID__HUBP0_FLIP_INTERRUPT:
return DC_IRQ_SOURCE_PFLIP1;
case DCN_1_0__SRCID__HUBP1_FLIP_INTERRUPT:
@@ -179,6 +193,16 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
.ack = NULL
};
+static const struct irq_source_info_funcs dmub_trace_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
+static const struct irq_source_info_funcs vline0_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
@@ -191,6 +215,9 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
+#define SRI_DMUB(reg_name)\
+ BASE(mm ## reg_name ## _BASE_IDX) + \
+ mm ## reg_name
#define IRQ_REG_ENTRY(block, reg_num, reg1, mask1, reg2, mask2)\
.enable_reg = SRI(reg1, block, reg_num),\
@@ -206,7 +233,19 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
.ack_value = \
block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK \
-
+#define IRQ_REG_ENTRY_DMUB(reg1, mask1, reg2, mask2)\
+ .enable_reg = SRI_DMUB(reg1),\
+ .enable_mask = \
+ reg1 ## __ ## mask1 ## _MASK,\
+ .enable_value = {\
+ reg1 ## __ ## mask1 ## _MASK,\
+ ~reg1 ## __ ## mask1 ## _MASK \
+ },\
+ .ack_reg = SRI_DMUB(reg2),\
+ .ack_mask = \
+ reg2 ## __ ## mask2 ## _MASK,\
+ .ack_value = \
+ reg2 ## __ ## mask2 ## _MASK \
#define hpd_int_entry(reg_num)\
[DC_IRQ_SOURCE_HPD1 + reg_num] = {\
@@ -252,6 +291,21 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
.funcs = &vblank_irq_info_funcs\
}
+#define vline0_int_entry(reg_num)\
+ [DC_IRQ_SOURCE_DC1_VLINE0 + reg_num] = {\
+ IRQ_REG_ENTRY(OTG, reg_num,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_INT_ENABLE,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_CLEAR),\
+ .funcs = &vline0_irq_info_funcs\
+ }
+
+#define dmub_trace_int_entry()\
+ [DC_IRQ_SOURCE_DMCUB_OUTBOX0] = {\
+ IRQ_REG_ENTRY_DMUB(DMCUB_INTERRUPT_ENABLE, DMCUB_OUTBOX0_READY_INT_EN,\
+ DMCUB_INTERRUPT_ACK, DMCUB_OUTBOX0_READY_INT_ACK),\
+ .funcs = &dmub_trace_irq_info_funcs\
+ }
+
#define dummy_irq_entry() \
{\
.funcs = &dummy_irq_info_funcs\
@@ -360,6 +414,13 @@ irq_source_info_dcn30[DAL_IRQ_SOURCES_NUMBER] = {
vblank_int_entry(3),
vblank_int_entry(4),
vblank_int_entry(5),
+ vline0_int_entry(0),
+ vline0_int_entry(1),
+ vline0_int_entry(2),
+ vline0_int_entry(3),
+ vline0_int_entry(4),
+ vline0_int_entry(5),
+ dmub_trace_int_entry(),
};
static const struct irq_service_funcs irq_service_funcs_dcn30 = {
diff --git a/drivers/gpu/drm/amd/display/dc/irq/dcn302/irq_service_dcn302.c b/drivers/gpu/drm/amd/display/dc/irq/dcn302/irq_service_dcn302.c
index 927fdc43fb9f..40fd34fb1d5e 100644
--- a/drivers/gpu/drm/amd/display/dc/irq/dcn302/irq_service_dcn302.c
+++ b/drivers/gpu/drm/amd/display/dc/irq/dcn302/irq_service_dcn302.c
@@ -50,6 +50,20 @@ static enum dc_irq_source to_dal_irq_source_dcn302(struct irq_service *irq_servi
return DC_IRQ_SOURCE_VBLANK5;
case DCN_1_0__SRCID__DC_D6_OTG_VSTARTUP:
return DC_IRQ_SOURCE_VBLANK6;
+ case DCN_1_0__SRCID__DMCUB_OUTBOX_HIGH_PRIORITY_READY_INT:
+ return DC_IRQ_SOURCE_DMCUB_OUTBOX0;
+ case DCN_1_0__SRCID__OTG1_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC1_VLINE0;
+ case DCN_1_0__SRCID__OTG2_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC2_VLINE0;
+ case DCN_1_0__SRCID__OTG3_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC3_VLINE0;
+ case DCN_1_0__SRCID__OTG4_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC4_VLINE0;
+ case DCN_1_0__SRCID__OTG5_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC5_VLINE0;
+ case DCN_1_0__SRCID__OTG6_VERTICAL_INTERRUPT0_CONTROL:
+ return DC_IRQ_SOURCE_DC6_VLINE0;
case DCN_1_0__SRCID__HUBP0_FLIP_INTERRUPT:
return DC_IRQ_SOURCE_PFLIP1;
case DCN_1_0__SRCID__HUBP1_FLIP_INTERRUPT:
@@ -154,6 +168,16 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
.ack = NULL
};
+static const struct irq_source_info_funcs dmub_trace_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
+static const struct irq_source_info_funcs vline0_irq_info_funcs = {
+ .set = NULL,
+ .ack = NULL
+};
+
#undef BASE_INNER
#define BASE_INNER(seg) DCN_BASE__INST0_SEG ## seg
@@ -164,6 +188,9 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
BASE(mm ## block ## id ## _ ## reg_name ## _BASE_IDX) + \
mm ## block ## id ## _ ## reg_name
+#define SRI_DMUB(reg_name)\
+ BASE(mm ## reg_name ## _BASE_IDX) + \
+ mm ## reg_name
#define IRQ_REG_ENTRY(block, reg_num, reg1, mask1, reg2, mask2)\
.enable_reg = SRI(reg1, block, reg_num),\
@@ -176,7 +203,26 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
.ack_mask = block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK,\
.ack_value = block ## reg_num ## _ ## reg2 ## __ ## mask2 ## _MASK \
+#define dmub_trace_int_entry()\
+ [DC_IRQ_SOURCE_DMCUB_OUTBOX0] = {\
+ IRQ_REG_ENTRY_DMUB(DMCUB_INTERRUPT_ENABLE, DMCUB_OUTBOX0_READY_INT_EN,\
+ DMCUB_INTERRUPT_ACK, DMCUB_OUTBOX0_READY_INT_ACK),\
+ .funcs = &dmub_trace_irq_info_funcs\
+ }
+#define IRQ_REG_ENTRY_DMUB(reg1, mask1, reg2, mask2)\
+ .enable_reg = SRI_DMUB(reg1),\
+ .enable_mask = \
+ reg1 ## __ ## mask1 ## _MASK,\
+ .enable_value = {\
+ reg1 ## __ ## mask1 ## _MASK,\
+ ~reg1 ## __ ## mask1 ## _MASK \
+ },\
+ .ack_reg = SRI_DMUB(reg2),\
+ .ack_mask = \
+ reg2 ## __ ## mask2 ## _MASK,\
+ .ack_value = \
+ reg2 ## __ ## mask2 ## _MASK \
#define hpd_int_entry(reg_num)\
[DC_IRQ_SOURCE_HPD1 + reg_num] = {\
@@ -222,6 +268,14 @@ static const struct irq_source_info_funcs vblank_irq_info_funcs = {
.funcs = &vblank_irq_info_funcs\
}
+#define vline0_int_entry(reg_num)\
+ [DC_IRQ_SOURCE_DC1_VLINE0 + reg_num] = {\
+ IRQ_REG_ENTRY(OTG, reg_num,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_INT_ENABLE,\
+ OTG_VERTICAL_INTERRUPT0_CONTROL, OTG_VERTICAL_INTERRUPT0_CLEAR),\
+ .funcs = &vline0_irq_info_funcs\
+ }
+
#define dummy_irq_entry() { .funcs = &dummy_irq_info_funcs }
#define i2c_int_entry(reg_num) \
@@ -318,6 +372,12 @@ static const struct irq_source_info irq_source_info_dcn302[DAL_IRQ_SOURCES_NUMBE
vblank_int_entry(2),
vblank_int_entry(3),
vblank_int_entry(4),
+ vline0_int_entry(0),
+ vline0_int_entry(1),
+ vline0_int_entry(2),
+ vline0_int_entry(3),
+ vline0_int_entry(4),
+ dmub_trace_int_entry(),
};
static const struct irq_service_funcs irq_service_funcs_dcn302 = {
diff --git a/drivers/gpu/drm/amd/display/dc/irq_types.h b/drivers/gpu/drm/amd/display/dc/irq_types.h
index 87812d81fed3..ae8f47ec0f8c 100644
--- a/drivers/gpu/drm/amd/display/dc/irq_types.h
+++ b/drivers/gpu/drm/amd/display/dc/irq_types.h
@@ -150,7 +150,8 @@ enum dc_irq_source {
DC_IRQ_SOURCE_DC4_VLINE1,
DC_IRQ_SOURCE_DC5_VLINE1,
DC_IRQ_SOURCE_DC6_VLINE1,
-
+ DC_IRQ_DMCUB_OUTBOX1,
+ DC_IRQ_SOURCE_DMCUB_OUTBOX0,
DAL_IRQ_SOURCES_NUMBER
};
diff --git a/drivers/gpu/drm/amd/display/dmub/dmub_srv.h b/drivers/gpu/drm/amd/display/dmub/dmub_srv.h
index 863cd9cc93ff..b4e14960b164 100644
--- a/drivers/gpu/drm/amd/display/dmub/dmub_srv.h
+++ b/drivers/gpu/drm/amd/display/dmub/dmub_srv.h
@@ -74,6 +74,8 @@ extern "C" {
struct dmub_srv;
struct dmub_srv_common_regs;
+struct dmcub_trace_buf_entry;
+
/* enum dmub_status - return code for dmcub functions */
enum dmub_status {
DMUB_STATUS_OK = 0,
@@ -107,6 +109,15 @@ enum dmub_window_id {
DMUB_WINDOW_TOTAL,
};
+/* enum dmub_notification_type - dmub outbox notification identifier */
+enum dmub_notification_type {
+ DMUB_NOTIFICATION_NO_DATA = 0,
+ DMUB_NOTIFICATION_AUX_REPLY,
+ DMUB_NOTIFICATION_HPD,
+ DMUB_NOTIFICATION_HPD_IRQ,
+ DMUB_NOTIFICATION_MAX
+};
+
/**
* struct dmub_region - dmub hw memory region
* @base: base address for region, must be 256 byte aligned
@@ -256,6 +267,20 @@ struct dmub_srv_hw_funcs {
void (*set_inbox1_wptr)(struct dmub_srv *dmub, uint32_t wptr_offset);
+ void (*setup_out_mailbox)(struct dmub_srv *dmub,
+ const struct dmub_region *outbox1);
+
+ uint32_t (*get_outbox1_wptr)(struct dmub_srv *dmub);
+
+ void (*set_outbox1_rptr)(struct dmub_srv *dmub, uint32_t rptr_offset);
+
+ void (*setup_outbox0)(struct dmub_srv *dmub,
+ const struct dmub_region *outbox0);
+
+ uint32_t (*get_outbox0_wptr)(struct dmub_srv *dmub);
+
+ void (*set_outbox0_rptr)(struct dmub_srv *dmub, uint32_t rptr_offset);
+
uint32_t (*emul_get_inbox1_rptr)(struct dmub_srv *dmub);
void (*emul_set_inbox1_wptr)(struct dmub_srv *dmub, uint32_t wptr_offset);
@@ -279,6 +304,7 @@ struct dmub_srv_hw_funcs {
union dmub_gpint_data_register reg);
uint32_t (*get_gpint_response)(struct dmub_srv *dmub);
+
};
/**
@@ -338,6 +364,13 @@ struct dmub_srv {
struct dmub_srv_base_funcs funcs;
struct dmub_srv_hw_funcs hw_funcs;
struct dmub_rb inbox1_rb;
+ /**
+ * outbox1_rb is accessed without locks (dal & dc)
+ * and to be used only in dmub_srv_stat_get_notification()
+ */
+ struct dmub_rb outbox1_rb;
+
+ struct dmub_rb outbox0_rb;
bool sw_init;
bool hw_init;
@@ -351,6 +384,26 @@ struct dmub_srv {
};
/**
+ * struct dmub_notification - dmub notification data
+ * @type: dmub notification type
+ * @link_index: link index to identify aux connection
+ * @result: USB4 status returned from dmub
+ * @pending_notification: Indicates there are other pending notifications
+ * @aux_reply: aux reply
+ * @hpd_status: hpd status
+ */
+struct dmub_notification {
+ enum dmub_notification_type type;
+ uint8_t link_index;
+ uint8_t result;
+ bool pending_notification;
+ union {
+ struct aux_reply_data aux_reply;
+ enum dp_hpd_status hpd_status;
+ };
+};
+
+/**
* DMUB firmware version helper macro - useful for checking if the version
* of a firmware to know if feature or functionality is supported or present.
*/
@@ -614,6 +667,8 @@ enum dmub_status dmub_srv_get_fw_boot_status(struct dmub_srv *dmub,
enum dmub_status dmub_srv_cmd_with_reply_data(struct dmub_srv *dmub,
union dmub_rb_cmd *cmd);
+bool dmub_srv_get_outbox0_msg(struct dmub_srv *dmub, struct dmcub_trace_buf_entry *entry);
+
#if defined(__cplusplus)
}
#endif
diff --git a/drivers/gpu/drm/amd/display/dmub/dmub_srv_stat.h b/drivers/gpu/drm/amd/display/dmub/dmub_srv_stat.h
new file mode 100644
index 000000000000..6c78aa406e90
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dmub/dmub_srv_stat.h
@@ -0,0 +1,41 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#ifndef _DMUB_SRV_STAT_H_
+#define _DMUB_SRV_STAT_H_
+
+/**
+ * DOC: DMUB_SRV STAT Interface
+ *
+ * These interfaces are called without acquiring DAL and DC locks.
+ * Hence, there is limitations on whese interfaces can access. Only
+ * variables exclusively defined for these interfaces can be modified.
+ */
+#include "dmub_srv.h"
+
+enum dmub_status dmub_srv_stat_get_notification(struct dmub_srv *dmub,
+ struct dmub_notification *notify);
+
+#endif /* _DMUB_SRV_STAT_H_ */
diff --git a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
index 072b4e7e624b..4195ff10c514 100644
--- a/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
+++ b/drivers/gpu/drm/amd/display/dmub/inc/dmub_cmd.h
@@ -47,10 +47,10 @@
/* Firmware versioning. */
#ifdef DMUB_EXPOSE_VERSION
-#define DMUB_FW_VERSION_GIT_HASH 0x6444c02e7
+#define DMUB_FW_VERSION_GIT_HASH 0x23db9b126
#define DMUB_FW_VERSION_MAJOR 0
#define DMUB_FW_VERSION_MINOR 0
-#define DMUB_FW_VERSION_REVISION 51
+#define DMUB_FW_VERSION_REVISION 62
#define DMUB_FW_VERSION_TEST 0
#define DMUB_FW_VERSION_VBIOS 0
#define DMUB_FW_VERSION_HOTFIX 0
@@ -68,25 +68,85 @@
#define __forceinline inline
+/**
+ * Flag from driver to indicate that ABM should be disabled gradually
+ * by slowly reversing all backlight programming and pixel compensation.
+ */
#define SET_ABM_PIPE_GRADUALLY_DISABLE 0
+
+/**
+ * Flag from driver to indicate that ABM should be disabled immediately
+ * and undo all backlight programming and pixel compensation.
+ */
#define SET_ABM_PIPE_IMMEDIATELY_DISABLE 255
+
+/**
+ * Flag from driver to indicate that ABM should be disabled immediately
+ * and keep the current backlight programming and pixel compensation.
+ */
#define SET_ABM_PIPE_IMMEDIATE_KEEP_GAIN_DISABLE 254
+
+/**
+ * Flag from driver to set the current ABM pipe index or ABM operating level.
+ */
#define SET_ABM_PIPE_NORMAL 1
+/**
+ * Number of ambient light levels in ABM algorithm.
+ */
+#define NUM_AMBI_LEVEL 5
+
+/**
+ * Number of operating/aggression levels in ABM algorithm.
+ */
+#define NUM_AGGR_LEVEL 4
+
+/**
+ * Number of segments in the gamma curve.
+ */
+#define NUM_POWER_FN_SEGS 8
+
+/**
+ * Number of segments in the backlight curve.
+ */
+#define NUM_BL_CURVE_SEGS 16
+
/* Maximum number of streams on any ASIC. */
#define DMUB_MAX_STREAMS 6
/* Maximum number of planes on any ASIC. */
#define DMUB_MAX_PLANES 6
+/* Trace buffer offset for entry */
+#define TRACE_BUFFER_ENTRY_OFFSET 16
+
+/**
+ * ABM backlight control version legacy
+ */
+#define DMUB_CMD_ABM_SET_BACKLIGHT_VERSION_UNKNOWN 0x0
+
+/**
+ * ABM backlight control version with multi edp support
+ */
+#define DMUB_CMD_ABM_SET_BACKLIGHT_VERSION_1 0x1
+
+/**
+ * Physical framebuffer address location, 64-bit.
+ */
#ifndef PHYSICAL_ADDRESS_LOC
#define PHYSICAL_ADDRESS_LOC union large_integer
#endif
+/**
+ * OS/FW agnostic memcpy
+ */
#ifndef dmub_memcpy
#define dmub_memcpy(dest, source, bytes) memcpy((dest), (source), (bytes))
#endif
+/**
+ * OS/FW agnostic memset
+ */
#ifndef dmub_memset
#define dmub_memset(dest, val, bytes) memset((dest), (val), (bytes))
#endif
@@ -95,29 +155,62 @@
extern "C" {
#endif
+/**
+ * OS/FW agnostic udelay
+ */
#ifndef dmub_udelay
#define dmub_udelay(microseconds) udelay(microseconds)
#endif
+/**
+ * union dmub_addr - DMUB physical/virtual 64-bit address.
+ */
union dmub_addr {
struct {
- uint32_t low_part;
- uint32_t high_part;
- } u;
- uint64_t quad_part;
+ uint32_t low_part; /**< Lower 32 bits */
+ uint32_t high_part; /**< Upper 32 bits */
+ } u; /*<< Low/high bit access */
+ uint64_t quad_part; /*<< 64 bit address */
};
+/**
+ * Flags that can be set by driver to change some PSR behaviour.
+ */
union dmub_psr_debug_flags {
+ /**
+ * Debug flags.
+ */
struct {
+ /**
+ * Enable visual confirm in FW.
+ */
uint32_t visual_confirm : 1;
+ /**
+ * Use HW Lock Mgr object to do HW locking in FW.
+ */
uint32_t use_hw_lock_mgr : 1;
+
+ /**
+ * Unused.
+ * TODO: Remove.
+ */
uint32_t log_line_nums : 1;
} bitfields;
+ /**
+ * Union for debug flags.
+ */
uint32_t u32All;
};
+/**
+ * DMUB feature capabilities.
+ * After DMUB init, driver will query FW capabilities prior to enabling certain features.
+ */
struct dmub_feature_caps {
+ /**
+ * Max PSR version supported by FW.
+ */
uint8_t psr;
uint8_t reserved[7];
};
@@ -153,23 +246,43 @@ struct dmub_feature_caps {
* @dal_fw: 1 if the firmware is DAL
*/
struct dmub_fw_meta_info {
- uint32_t magic_value;
- uint32_t fw_region_size;
- uint32_t trace_buffer_size;
- uint32_t fw_version;
- uint8_t dal_fw;
- uint8_t reserved[3];
+ uint32_t magic_value; /**< magic value identifying DMUB firmware meta info */
+ uint32_t fw_region_size; /**< size of the firmware state region */
+ uint32_t trace_buffer_size; /**< size of the tracebuffer region */
+ uint32_t fw_version; /**< the firmware version information */
+ uint8_t dal_fw; /**< 1 if the firmware is DAL */
+ uint8_t reserved[3]; /**< padding bits */
};
-/* Ensure that the structure remains 64 bytes. */
+/**
+ * union dmub_fw_meta - ensures that dmub_fw_meta_info remains 64 bytes
+ */
union dmub_fw_meta {
- struct dmub_fw_meta_info info;
- uint8_t reserved[64];
+ struct dmub_fw_meta_info info; /**< metadata info */
+ uint8_t reserved[64]; /**< padding bits */
};
#pragma pack(pop)
//==============================================================================
+//< DMUB Trace Buffer>================================================================
+//==============================================================================
+/**
+ * dmub_trace_code_t - firmware trace code, 32-bits
+ */
+typedef uint32_t dmub_trace_code_t;
+
+/**
+ * struct dmcub_trace_buf_entry - Firmware trace entry
+ */
+struct dmcub_trace_buf_entry {
+ dmub_trace_code_t trace_code; /**< trace code for the event */
+ uint32_t tick_count; /**< the tick count at time of trace */
+ uint32_t param0; /**< trace defined parameter 0 */
+ uint32_t param1; /**< trace defined parameter 1 */
+};
+
+//==============================================================================
//< DMUB_STATUS>================================================================
//==============================================================================
@@ -181,42 +294,49 @@ union dmub_fw_meta {
* SCRATCH15: FW Boot Options register
*/
-/* Register bit definition for SCRATCH0 */
+/**
+ * union dmub_fw_boot_status - Status bit definitions for SCRATCH0.
+ */
union dmub_fw_boot_status {
struct {
- uint32_t dal_fw : 1;
- uint32_t mailbox_rdy : 1;
- uint32_t optimized_init_done : 1;
- uint32_t restore_required : 1;
- } bits;
- uint32_t all;
+ uint32_t dal_fw : 1; /**< 1 if DAL FW */
+ uint32_t mailbox_rdy : 1; /**< 1 if mailbox ready */
+ uint32_t optimized_init_done : 1; /**< 1 if optimized init done */
+ uint32_t restore_required : 1; /**< 1 if driver should call restore */
+ } bits; /**< status bits */
+ uint32_t all; /**< 32-bit access to status bits */
};
+/**
+ * enum dmub_fw_boot_status_bit - Enum bit definitions for SCRATCH0.
+ */
enum dmub_fw_boot_status_bit {
- DMUB_FW_BOOT_STATUS_BIT_DAL_FIRMWARE = (1 << 0),
- DMUB_FW_BOOT_STATUS_BIT_MAILBOX_READY = (1 << 1),
- DMUB_FW_BOOT_STATUS_BIT_OPTIMIZED_INIT_DONE = (1 << 2),
- DMUB_FW_BOOT_STATUS_BIT_RESTORE_REQUIRED = (1 << 3),
+ DMUB_FW_BOOT_STATUS_BIT_DAL_FIRMWARE = (1 << 0), /**< 1 if DAL FW */
+ DMUB_FW_BOOT_STATUS_BIT_MAILBOX_READY = (1 << 1), /**< 1 if mailbox ready */
+ DMUB_FW_BOOT_STATUS_BIT_OPTIMIZED_INIT_DONE = (1 << 2), /**< 1 if init done */
+ DMUB_FW_BOOT_STATUS_BIT_RESTORE_REQUIRED = (1 << 3), /**< 1 if driver should call restore */
};
-/* Register bit definition for SCRATCH15 */
+/**
+ * union dmub_fw_boot_options - Boot option definitions for SCRATCH15
+ */
union dmub_fw_boot_options {
struct {
- uint32_t pemu_env : 1;
- uint32_t fpga_env : 1;
- uint32_t optimized_init : 1;
- uint32_t skip_phy_access : 1;
- uint32_t disable_clk_gate: 1;
- uint32_t skip_phy_init_panel_sequence: 1;
- uint32_t reserved : 26;
- } bits;
- uint32_t all;
+ uint32_t pemu_env : 1; /**< 1 if PEMU */
+ uint32_t fpga_env : 1; /**< 1 if FPGA */
+ uint32_t optimized_init : 1; /**< 1 if optimized init */
+ uint32_t skip_phy_access : 1; /**< 1 if PHY access should be skipped */
+ uint32_t disable_clk_gate: 1; /**< 1 if clock gating should be disabled */
+ uint32_t skip_phy_init_panel_sequence: 1; /**< 1 to skip panel init seq */
+ uint32_t reserved : 26; /**< reserved */
+ } bits; /**< boot bits */
+ uint32_t all; /**< 32-bit access to bits */
};
enum dmub_fw_boot_options_bit {
- DMUB_FW_BOOT_OPTION_BIT_PEMU_ENV = (1 << 0),
- DMUB_FW_BOOT_OPTION_BIT_FPGA_ENV = (1 << 1),
- DMUB_FW_BOOT_OPTION_BIT_OPTIMIZED_INIT_DONE = (1 << 2),
+ DMUB_FW_BOOT_OPTION_BIT_PEMU_ENV = (1 << 0), /**< 1 if PEMU */
+ DMUB_FW_BOOT_OPTION_BIT_FPGA_ENV = (1 << 1), /**< 1 if FPGA */
+ DMUB_FW_BOOT_OPTION_BIT_OPTIMIZED_INIT_DONE = (1 << 2), /**< 1 if optimized init done */
};
//==============================================================================
@@ -226,14 +346,27 @@ enum dmub_fw_boot_options_bit {
//==============================================================================
/*
+ * enum dmub_cmd_vbios_type - VBIOS commands.
+ *
* Command IDs should be treated as stable ABI.
* Do not reuse or modify IDs.
*/
-
enum dmub_cmd_vbios_type {
+ /**
+ * Configures the DIG encoder.
+ */
DMUB_CMD__VBIOS_DIGX_ENCODER_CONTROL = 0,
+ /**
+ * Controls the PHY.
+ */
DMUB_CMD__VBIOS_DIG1_TRANSMITTER_CONTROL = 1,
+ /**
+ * Sets the pixel clock/symbol clock.
+ */
DMUB_CMD__VBIOS_SET_PIXEL_CLOCK = 2,
+ /**
+ * Enables or disables power gating.
+ */
DMUB_CMD__VBIOS_ENABLE_DISP_POWER_GATING = 3,
DMUB_CMD__VBIOS_LVTMA_CONTROL = 15,
};
@@ -262,35 +395,60 @@ enum dmub_cmd_vbios_type {
* Command responses.
*/
+/**
+ * Return response for DMUB_GPINT__STOP_FW command.
+ */
#define DMUB_GPINT__STOP_FW_RESPONSE 0xDEADDEAD
/**
- * The register format for sending a command via the GPINT.
+ * union dmub_gpint_data_register - Format for sending a command via the GPINT.
*/
union dmub_gpint_data_register {
struct {
- uint32_t param : 16;
- uint32_t command_code : 12;
- uint32_t status : 4;
- } bits;
- uint32_t all;
+ uint32_t param : 16; /**< 16-bit parameter */
+ uint32_t command_code : 12; /**< GPINT command */
+ uint32_t status : 4; /**< Command status bit */
+ } bits; /**< GPINT bit access */
+ uint32_t all; /**< GPINT 32-bit access */
};
/*
+ * enum dmub_gpint_command - GPINT command to DMCUB FW
+ *
* Command IDs should be treated as stable ABI.
* Do not reuse or modify IDs.
*/
-
enum dmub_gpint_command {
+ /**
+ * Invalid command, ignored.
+ */
DMUB_GPINT__INVALID_COMMAND = 0,
+ /**
+ * DESC: Queries the firmware version.
+ * RETURN: Firmware version.
+ */
DMUB_GPINT__GET_FW_VERSION = 1,
+ /**
+ * DESC: Halts the firmware.
+ * RETURN: DMUB_GPINT__STOP_FW_RESPONSE (0xDEADDEAD) when halted
+ */
DMUB_GPINT__STOP_FW = 2,
+ /**
+ * DESC: Get PSR state from FW.
+ * RETURN: PSR state enum. This enum may need to be converted to the legacy PSR state value.
+ */
DMUB_GPINT__GET_PSR_STATE = 7,
/**
* DESC: Notifies DMCUB of the currently active streams.
* ARGS: Stream mask, 1 bit per active stream index.
*/
DMUB_GPINT__IDLE_OPT_NOTIFY_STREAM_MASK = 8,
+ /**
+ * DESC: Start PSR residency counter. Stop PSR resdiency counter and get value.
+ * ARGS: We can measure residency from various points. The argument will specify the residency mode.
+ * By default, it is measured from after we powerdown the PHY, to just before we powerup the PHY.
+ * RETURN: PSR residency in milli-percent.
+ */
DMUB_GPINT__PSR_RESIDENCY = 9,
};
@@ -300,52 +458,125 @@ enum dmub_gpint_command {
//< DMUB_CMD>===================================================================
//==============================================================================
+/**
+ * Size in bytes of each DMUB command.
+ */
#define DMUB_RB_CMD_SIZE 64
+
+/**
+ * Maximum number of items in the DMUB ringbuffer.
+ */
#define DMUB_RB_MAX_ENTRY 128
+
+/**
+ * Ringbuffer size in bytes.
+ */
#define DMUB_RB_SIZE (DMUB_RB_CMD_SIZE * DMUB_RB_MAX_ENTRY)
+
+/**
+ * REG_SET mask for reg offload.
+ */
#define REG_SET_MASK 0xFFFF
/*
+ * enum dmub_cmd_type - DMUB inbox command.
+ *
* Command IDs should be treated as stable ABI.
* Do not reuse or modify IDs.
*/
-
enum dmub_cmd_type {
+ /**
+ * Invalid command.
+ */
DMUB_CMD__NULL = 0,
+ /**
+ * Read modify write register sequence offload.
+ */
DMUB_CMD__REG_SEQ_READ_MODIFY_WRITE = 1,
+ /**
+ * Field update register sequence offload.
+ */
DMUB_CMD__REG_SEQ_FIELD_UPDATE_SEQ = 2,
+ /**
+ * Burst write sequence offload.
+ */
DMUB_CMD__REG_SEQ_BURST_WRITE = 3,
+ /**
+ * Reg wait sequence offload.
+ */
DMUB_CMD__REG_REG_WAIT = 4,
+ /**
+ * Workaround to avoid HUBP underflow during NV12 playback.
+ */
DMUB_CMD__PLAT_54186_WA = 5,
+ /**
+ * Command type used to query FW feature caps.
+ */
DMUB_CMD__QUERY_FEATURE_CAPS = 6,
+ /**
+ * Command type used for all PSR commands.
+ */
DMUB_CMD__PSR = 64,
+ /**
+ * Command type used for all MALL commands.
+ */
DMUB_CMD__MALL = 65,
+ /**
+ * Command type used for all ABM commands.
+ */
DMUB_CMD__ABM = 66,
+ /**
+ * Command type used for HW locking in FW.
+ */
DMUB_CMD__HW_LOCK = 69,
+ /**
+ * Command type used to access DP AUX.
+ */
DMUB_CMD__DP_AUX_ACCESS = 70,
+ /**
+ * Command type used for OUTBOX1 notification enable
+ */
DMUB_CMD__OUTBOX1_ENABLE = 71,
+ /**
+ * Command type used for all VBIOS interface commands.
+ */
DMUB_CMD__VBIOS = 128,
};
+/**
+ * enum dmub_out_cmd_type - DMUB outbox commands.
+ */
enum dmub_out_cmd_type {
+ /**
+ * Invalid outbox command, ignored.
+ */
DMUB_OUT_CMD__NULL = 0,
+ /**
+ * Command type used for DP AUX Reply data notification
+ */
DMUB_OUT_CMD__DP_AUX_REPLY = 1,
+ /**
+ * Command type used for DP HPD event notification
+ */
DMUB_OUT_CMD__DP_HPD_NOTIFY = 2,
};
#pragma pack(push, 1)
+/**
+ * struct dmub_cmd_header - Common command header fields.
+ */
struct dmub_cmd_header {
- unsigned int type : 8;
- unsigned int sub_type : 8;
- unsigned int ret_status : 1;
- unsigned int reserved0 : 7;
- unsigned int payload_bytes : 6; /* up to 60 bytes */
- unsigned int reserved1 : 2;
+ unsigned int type : 8; /**< command type */
+ unsigned int sub_type : 8; /**< command sub type */
+ unsigned int ret_status : 1; /**< 1 if returned data, 0 otherwise */
+ unsigned int reserved0 : 7; /**< reserved bits */
+ unsigned int payload_bytes : 6; /* payload excluding header - up to 60 bytes */
+ unsigned int reserved1 : 2; /**< reserved bits */
};
/*
- * Read modify write
+ * struct dmub_cmd_read_modify_write_sequence - Read modify write
*
* 60 payload bytes can hold up to 5 sets of read modify writes,
* each take 3 dwords.
@@ -356,14 +587,24 @@ struct dmub_cmd_header {
* command parser will skip the read and we can use modify_mask = 0xffff'ffff as reg write
*/
struct dmub_cmd_read_modify_write_sequence {
- uint32_t addr;
- uint32_t modify_mask;
- uint32_t modify_value;
+ uint32_t addr; /**< register address */
+ uint32_t modify_mask; /**< modify mask */
+ uint32_t modify_value; /**< modify value */
};
-#define DMUB_READ_MODIFY_WRITE_SEQ__MAX 5
+/**
+ * Maximum number of ops in read modify write sequence.
+ */
+#define DMUB_READ_MODIFY_WRITE_SEQ__MAX 5
+
+/**
+ * struct dmub_cmd_read_modify_write_sequence - Read modify write command.
+ */
struct dmub_rb_cmd_read_modify_write {
- struct dmub_cmd_header header; // type = DMUB_CMD__REG_SEQ_READ_MODIFY_WRITE
+ struct dmub_cmd_header header; /**< command header */
+ /**
+ * Read modify write sequence.
+ */
struct dmub_cmd_read_modify_write_sequence seq[DMUB_READ_MODIFY_WRITE_SEQ__MAX];
};
@@ -381,19 +622,35 @@ struct dmub_rb_cmd_read_modify_write {
*/
struct dmub_cmd_reg_field_update_sequence {
- uint32_t modify_mask; // 0xffff'ffff to skip initial read
- uint32_t modify_value;
+ uint32_t modify_mask; /**< 0xffff'ffff to skip initial read */
+ uint32_t modify_value; /**< value to update with */
};
-#define DMUB_REG_FIELD_UPDATE_SEQ__MAX 7
+/**
+ * Maximum number of ops in field update sequence.
+ */
+#define DMUB_REG_FIELD_UPDATE_SEQ__MAX 7
+
+/**
+ * struct dmub_rb_cmd_reg_field_update_sequence - Field update command.
+ */
struct dmub_rb_cmd_reg_field_update_sequence {
- struct dmub_cmd_header header;
- uint32_t addr;
+ struct dmub_cmd_header header; /**< command header */
+ uint32_t addr; /**< register address */
+ /**
+ * Field update sequence.
+ */
struct dmub_cmd_reg_field_update_sequence seq[DMUB_REG_FIELD_UPDATE_SEQ__MAX];
};
+
+/**
+ * Maximum number of burst write values.
+ */
+#define DMUB_BURST_WRITE_VALUES__MAX 14
+
/*
- * Burst write
+ * struct dmub_rb_cmd_burst_write - Burst write
*
* support use case such as writing out LUTs.
*
@@ -401,96 +658,141 @@ struct dmub_rb_cmd_reg_field_update_sequence {
*
* number of payload = header.payload_bytes / sizeof(struct read_modify_write_sequence)
*/
-#define DMUB_BURST_WRITE_VALUES__MAX 14
struct dmub_rb_cmd_burst_write {
- struct dmub_cmd_header header; // type = DMUB_CMD__REG_SEQ_BURST_WRITE
- uint32_t addr;
+ struct dmub_cmd_header header; /**< command header */
+ uint32_t addr; /**< register start address */
+ /**
+ * Burst write register values.
+ */
uint32_t write_values[DMUB_BURST_WRITE_VALUES__MAX];
};
-
+/**
+ * struct dmub_rb_cmd_common - Common command header
+ */
struct dmub_rb_cmd_common {
- struct dmub_cmd_header header;
+ struct dmub_cmd_header header; /**< command header */
+ /**
+ * Padding to RB_CMD_SIZE
+ */
uint8_t cmd_buffer[DMUB_RB_CMD_SIZE - sizeof(struct dmub_cmd_header)];
};
+/**
+ * struct dmub_cmd_reg_wait_data - Register wait data
+ */
struct dmub_cmd_reg_wait_data {
- uint32_t addr;
- uint32_t mask;
- uint32_t condition_field_value;
- uint32_t time_out_us;
+ uint32_t addr; /**< Register address */
+ uint32_t mask; /**< Mask for register bits */
+ uint32_t condition_field_value; /**< Value to wait for */
+ uint32_t time_out_us; /**< Time out for reg wait in microseconds */
};
+/**
+ * struct dmub_rb_cmd_reg_wait - Register wait command
+ */
struct dmub_rb_cmd_reg_wait {
- struct dmub_cmd_header header;
- struct dmub_cmd_reg_wait_data reg_wait;
+ struct dmub_cmd_header header; /**< Command header */
+ struct dmub_cmd_reg_wait_data reg_wait; /**< Register wait data */
};
+/**
+ * struct dmub_cmd_PLAT_54186_wa - Underflow workaround
+ *
+ * Reprograms surface parameters to avoid underflow.
+ */
struct dmub_cmd_PLAT_54186_wa {
- uint32_t DCSURF_SURFACE_CONTROL;
- uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS_HIGH;
- uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS;
- uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS_HIGH_C;
- uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS_C;
+ uint32_t DCSURF_SURFACE_CONTROL; /**< reg value */
+ uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS_HIGH; /**< reg value */
+ uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS; /**< reg value */
+ uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS_HIGH_C; /**< reg value */
+ uint32_t DCSURF_PRIMARY_SURFACE_ADDRESS_C; /**< reg value */
struct {
- uint8_t hubp_inst : 4;
- uint8_t tmz_surface : 1;
- uint8_t immediate :1;
- uint8_t vmid : 4;
- uint8_t grph_stereo : 1;
- uint32_t reserved : 21;
- } flip_params;
- uint32_t reserved[9];
+ uint8_t hubp_inst : 4; /**< HUBP instance */
+ uint8_t tmz_surface : 1; /**< TMZ enable or disable */
+ uint8_t immediate :1; /**< Immediate flip */
+ uint8_t vmid : 4; /**< VMID */
+ uint8_t grph_stereo : 1; /**< 1 if stereo */
+ uint32_t reserved : 21; /**< Reserved */
+ } flip_params; /**< Pageflip parameters */
+ uint32_t reserved[9]; /**< Reserved bits */
};
+/**
+ * struct dmub_rb_cmd_PLAT_54186_wa - Underflow workaround command
+ */
struct dmub_rb_cmd_PLAT_54186_wa {
- struct dmub_cmd_header header;
- struct dmub_cmd_PLAT_54186_wa flip;
+ struct dmub_cmd_header header; /**< Command header */
+ struct dmub_cmd_PLAT_54186_wa flip; /**< Flip data */
};
+/**
+ * struct dmub_rb_cmd_mall - MALL command data.
+ */
struct dmub_rb_cmd_mall {
- struct dmub_cmd_header header;
- union dmub_addr cursor_copy_src;
- union dmub_addr cursor_copy_dst;
- uint32_t tmr_delay;
- uint32_t tmr_scale;
- uint16_t cursor_width;
- uint16_t cursor_pitch;
- uint16_t cursor_height;
- uint8_t cursor_bpp;
- uint8_t debug_bits;
+ struct dmub_cmd_header header; /**< Common command header */
+ union dmub_addr cursor_copy_src; /**< Cursor copy address */
+ union dmub_addr cursor_copy_dst; /**< Cursor copy destination */
+ uint32_t tmr_delay; /**< Timer delay */
+ uint32_t tmr_scale; /**< Timer scale */
+ uint16_t cursor_width; /**< Cursor width in pixels */
+ uint16_t cursor_pitch; /**< Cursor pitch in pixels */
+ uint16_t cursor_height; /**< Cursor height in pixels */
+ uint8_t cursor_bpp; /**< Cursor bits per pixel */
+ uint8_t debug_bits; /**< Debug bits */
- uint8_t reserved1;
- uint8_t reserved2;
+ uint8_t reserved1; /**< Reserved bits */
+ uint8_t reserved2; /**< Reserved bits */
};
+/**
+ * struct dmub_cmd_digx_encoder_control_data - Encoder control data.
+ */
struct dmub_cmd_digx_encoder_control_data {
- union dig_encoder_control_parameters_v1_5 dig;
+ union dig_encoder_control_parameters_v1_5 dig; /**< payload */
};
+/**
+ * struct dmub_rb_cmd_digx_encoder_control - Encoder control command.
+ */
struct dmub_rb_cmd_digx_encoder_control {
- struct dmub_cmd_header header;
- struct dmub_cmd_digx_encoder_control_data encoder_control;
+ struct dmub_cmd_header header; /**< header */
+ struct dmub_cmd_digx_encoder_control_data encoder_control; /**< payload */
};
+/**
+ * struct dmub_cmd_set_pixel_clock_data - Set pixel clock data.
+ */
struct dmub_cmd_set_pixel_clock_data {
- struct set_pixel_clock_parameter_v1_7 clk;
+ struct set_pixel_clock_parameter_v1_7 clk; /**< payload */
};
+/**
+ * struct dmub_cmd_set_pixel_clock_data - Set pixel clock command.
+ */
struct dmub_rb_cmd_set_pixel_clock {
- struct dmub_cmd_header header;
- struct dmub_cmd_set_pixel_clock_data pixel_clock;
+ struct dmub_cmd_header header; /**< header */
+ struct dmub_cmd_set_pixel_clock_data pixel_clock; /**< payload */
};
+/**
+ * struct dmub_cmd_enable_disp_power_gating_data - Display power gating.
+ */
struct dmub_cmd_enable_disp_power_gating_data {
- struct enable_disp_power_gating_parameters_v2_1 pwr;
+ struct enable_disp_power_gating_parameters_v2_1 pwr; /**< payload */
};
+/**
+ * struct dmub_rb_cmd_enable_disp_power_gating - Display power command.
+ */
struct dmub_rb_cmd_enable_disp_power_gating {
- struct dmub_cmd_header header;
- struct dmub_cmd_enable_disp_power_gating_data power_gating;
+ struct dmub_cmd_header header; /**< header */
+ struct dmub_cmd_enable_disp_power_gating_data power_gating; /**< payload */
};
+/**
+ * struct dmub_dig_transmitter_control_data_v1_7 - Transmitter control.
+ */
struct dmub_dig_transmitter_control_data_v1_7 {
uint8_t phyid; /**< 0=UNIPHYA, 1=UNIPHYB, 2=UNIPHYC, 3=UNIPHYD, 4=UNIPHYE, 5=UNIPHYF */
uint8_t action; /**< Defined as ATOM_TRANSMITER_ACTION_xxx */
@@ -511,118 +813,266 @@ struct dmub_dig_transmitter_control_data_v1_7 {
uint32_t reserved3[11]; /**< For future use */
};
+/**
+ * union dmub_cmd_dig1_transmitter_control_data - Transmitter control data.
+ */
union dmub_cmd_dig1_transmitter_control_data {
- struct dig_transmitter_control_parameters_v1_6 dig;
- struct dmub_dig_transmitter_control_data_v1_7 dig_v1_7;
+ struct dig_transmitter_control_parameters_v1_6 dig; /**< payload */
+ struct dmub_dig_transmitter_control_data_v1_7 dig_v1_7; /**< payload 1.7 */
};
+/**
+ * struct dmub_rb_cmd_dig1_transmitter_control - Transmitter control command.
+ */
struct dmub_rb_cmd_dig1_transmitter_control {
- struct dmub_cmd_header header;
- union dmub_cmd_dig1_transmitter_control_data transmitter_control;
+ struct dmub_cmd_header header; /**< header */
+ union dmub_cmd_dig1_transmitter_control_data transmitter_control; /**< payload */
};
+/**
+ * struct dmub_rb_cmd_dpphy_init - DPPHY init.
+ */
struct dmub_rb_cmd_dpphy_init {
- struct dmub_cmd_header header;
- uint8_t reserved[60];
+ struct dmub_cmd_header header; /**< header */
+ uint8_t reserved[60]; /**< reserved bits */
};
+/**
+ * enum dp_aux_request_action - DP AUX request command listing.
+ *
+ * 4 AUX request command bits are shifted to high nibble.
+ */
enum dp_aux_request_action {
+ /** I2C-over-AUX write request */
DP_AUX_REQ_ACTION_I2C_WRITE = 0x00,
+ /** I2C-over-AUX read request */
DP_AUX_REQ_ACTION_I2C_READ = 0x10,
+ /** I2C-over-AUX write status request */
DP_AUX_REQ_ACTION_I2C_STATUS_REQ = 0x20,
+ /** I2C-over-AUX write request with MOT=1 */
DP_AUX_REQ_ACTION_I2C_WRITE_MOT = 0x40,
+ /** I2C-over-AUX read request with MOT=1 */
DP_AUX_REQ_ACTION_I2C_READ_MOT = 0x50,
+ /** I2C-over-AUX write status request with MOT=1 */
DP_AUX_REQ_ACTION_I2C_STATUS_REQ_MOT = 0x60,
+ /** Native AUX write request */
DP_AUX_REQ_ACTION_DPCD_WRITE = 0x80,
+ /** Native AUX read request */
DP_AUX_REQ_ACTION_DPCD_READ = 0x90
};
+/**
+ * enum aux_return_code_type - DP AUX process return code listing.
+ */
enum aux_return_code_type {
+ /** AUX process succeeded */
AUX_RET_SUCCESS = 0,
+ /** AUX process failed with unknown reason */
AUX_RET_ERROR_UNKNOWN,
+ /** AUX process completed with invalid reply */
AUX_RET_ERROR_INVALID_REPLY,
+ /** AUX process timed out */
AUX_RET_ERROR_TIMEOUT,
+ /** HPD was low during AUX process */
AUX_RET_ERROR_HPD_DISCON,
+ /** Failed to acquire AUX engine */
AUX_RET_ERROR_ENGINE_ACQUIRE,
+ /** AUX request not supported */
AUX_RET_ERROR_INVALID_OPERATION,
+ /** AUX process not available */
AUX_RET_ERROR_PROTOCOL_ERROR,
};
+/**
+ * enum aux_channel_type - DP AUX channel type listing.
+ */
enum aux_channel_type {
+ /** AUX thru Legacy DP AUX */
AUX_CHANNEL_LEGACY_DDC,
+ /** AUX thru DPIA DP tunneling */
AUX_CHANNEL_DPIA
};
-/* DP AUX command */
+/**
+ * struct aux_transaction_parameters - DP AUX request transaction data
+ */
struct aux_transaction_parameters {
- uint8_t is_i2c_over_aux;
- uint8_t action;
- uint8_t length;
- uint8_t pad;
- uint32_t address;
- uint8_t data[16];
+ uint8_t is_i2c_over_aux; /**< 0=native AUX, 1=I2C-over-AUX */
+ uint8_t action; /**< enum dp_aux_request_action */
+ uint8_t length; /**< DP AUX request data length */
+ uint8_t reserved; /**< For future use */
+ uint32_t address; /**< DP AUX address */
+ uint8_t data[16]; /**< DP AUX write data */
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__DP_AUX_ACCESS command.
+ */
struct dmub_cmd_dp_aux_control_data {
- uint32_t handle;
- uint8_t instance;
- uint8_t sw_crc_enabled;
- uint16_t timeout;
- enum aux_channel_type type;
- struct aux_transaction_parameters dpaux;
+ uint8_t instance; /**< AUX instance or DPIA instance */
+ uint8_t manual_acq_rel_enable; /**< manual control for acquiring or releasing AUX channel */
+ uint8_t sw_crc_enabled; /**< Use software CRC for tunneling packet instead of hardware CRC */
+ uint8_t reserved0; /**< For future use */
+ uint16_t timeout; /**< timeout time in us */
+ uint16_t reserved1; /**< For future use */
+ enum aux_channel_type type; /**< enum aux_channel_type */
+ struct aux_transaction_parameters dpaux; /**< struct aux_transaction_parameters */
};
+/**
+ * Definition of a DMUB_CMD__DP_AUX_ACCESS command.
+ */
struct dmub_rb_cmd_dp_aux_access {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__DP_AUX_ACCESS command.
+ */
struct dmub_cmd_dp_aux_control_data aux_control;
};
+/**
+ * Definition of a DMUB_CMD__OUTBOX1_ENABLE command.
+ */
struct dmub_rb_cmd_outbox1_enable {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * enable: 0x0 -> disable outbox1 notification (default value)
+ * 0x1 -> enable outbox1 notification
+ */
uint32_t enable;
};
/* DP AUX Reply command - OutBox Cmd */
+/**
+ * Data passed to driver from FW in a DMUB_OUT_CMD__DP_AUX_REPLY command.
+ */
struct aux_reply_data {
+ /**
+ * Aux cmd
+ */
uint8_t command;
+ /**
+ * Aux reply data length (max: 16 bytes)
+ */
uint8_t length;
+ /**
+ * Alignment only
+ */
uint8_t pad[2];
+ /**
+ * Aux reply data
+ */
uint8_t data[16];
};
+/**
+ * Control Data passed to driver from FW in a DMUB_OUT_CMD__DP_AUX_REPLY command.
+ */
struct aux_reply_control_data {
+ /**
+ * Reserved for future use
+ */
uint32_t handle;
+ /**
+ * Aux Instance
+ */
uint8_t instance;
+ /**
+ * Aux transaction result: definition in enum aux_return_code_type
+ */
uint8_t result;
+ /**
+ * Alignment only
+ */
uint16_t pad;
};
+/**
+ * Definition of a DMUB_OUT_CMD__DP_AUX_REPLY command.
+ */
struct dmub_rb_cmd_dp_aux_reply {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * Control Data passed to driver from FW in a DMUB_OUT_CMD__DP_AUX_REPLY command.
+ */
struct aux_reply_control_data control;
+ /**
+ * Data passed to driver from FW in a DMUB_OUT_CMD__DP_AUX_REPLY command.
+ */
struct aux_reply_data reply_data;
};
/* DP HPD Notify command - OutBox Cmd */
+/**
+ * DP HPD Type
+ */
enum dp_hpd_type {
+ /**
+ * Normal DP HPD
+ */
DP_HPD = 0,
+ /**
+ * DP HPD short pulse
+ */
DP_IRQ
};
+/**
+ * DP HPD Status
+ */
enum dp_hpd_status {
+ /**
+ * DP_HPD status low
+ */
DP_HPD_UNPLUG = 0,
+ /**
+ * DP_HPD status high
+ */
DP_HPD_PLUG
};
+/**
+ * Data passed to driver from FW in a DMUB_OUT_CMD__DP_HPD_NOTIFY command.
+ */
struct dp_hpd_data {
+ /**
+ * DP HPD instance
+ */
uint8_t instance;
+ /**
+ * HPD type
+ */
uint8_t hpd_type;
+ /**
+ * HPD status: only for type: DP_HPD to indicate status
+ */
uint8_t hpd_status;
+ /**
+ * Alignment only
+ */
uint8_t pad;
};
+/**
+ * Definition of a DMUB_OUT_CMD__DP_HPD_NOTIFY command.
+ */
struct dmub_rb_cmd_dp_hpd_notify {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * Data passed to driver from FW in a DMUB_OUT_CMD__DP_HPD_NOTIFY command.
+ */
struct dp_hpd_data hpd_data;
};
@@ -631,270 +1081,903 @@ struct dmub_rb_cmd_dp_hpd_notify {
* Do not reuse or modify IDs.
*/
+/**
+ * PSR command sub-types.
+ */
enum dmub_cmd_psr_type {
+ /**
+ * Set PSR version support.
+ */
DMUB_CMD__PSR_SET_VERSION = 0,
+ /**
+ * Copy driver-calculated parameters to PSR state.
+ */
DMUB_CMD__PSR_COPY_SETTINGS = 1,
+ /**
+ * Enable PSR.
+ */
DMUB_CMD__PSR_ENABLE = 2,
+
+ /**
+ * Disable PSR.
+ */
DMUB_CMD__PSR_DISABLE = 3,
+
+ /**
+ * Set PSR level.
+ * PSR level is a 16-bit value dicated by driver that
+ * will enable/disable different functionality.
+ */
DMUB_CMD__PSR_SET_LEVEL = 4,
+
+ /**
+ * Forces PSR enabled until an explicit PSR disable call.
+ */
DMUB_CMD__PSR_FORCE_STATIC = 5,
};
+/**
+ * PSR versions.
+ */
enum psr_version {
+ /**
+ * PSR version 1.
+ */
PSR_VERSION_1 = 0,
+ /**
+ * PSR not supported.
+ */
PSR_VERSION_UNSUPPORTED = 0xFFFFFFFF,
};
+/**
+ * enum dmub_cmd_mall_type - MALL commands
+ */
enum dmub_cmd_mall_type {
+ /**
+ * Allows display refresh from MALL.
+ */
DMUB_CMD__MALL_ACTION_ALLOW = 0,
+ /**
+ * Disallows display refresh from MALL.
+ */
DMUB_CMD__MALL_ACTION_DISALLOW = 1,
+ /**
+ * Cursor copy for MALL.
+ */
DMUB_CMD__MALL_ACTION_COPY_CURSOR = 2,
+ /**
+ * Controls DF requests.
+ */
DMUB_CMD__MALL_ACTION_NO_DF_REQ = 3,
};
+
+/**
+ * Data passed from driver to FW in a DMUB_CMD__PSR_COPY_SETTINGS command.
+ */
struct dmub_cmd_psr_copy_settings_data {
+ /**
+ * Flags that can be set by driver to change some PSR behaviour.
+ */
union dmub_psr_debug_flags debug;
+ /**
+ * 16-bit value dicated by driver that will enable/disable different functionality.
+ */
uint16_t psr_level;
+ /**
+ * DPP HW instance.
+ */
uint8_t dpp_inst;
- /* opp_inst and mpcc_inst will not be used in dmub fw,
+ /**
+ * MPCC HW instance.
+ * Not used in dmub fw,
* dmub fw will get active opp by reading odm registers.
*/
uint8_t mpcc_inst;
+ /**
+ * OPP HW instance.
+ * Not used in dmub fw,
+ * dmub fw will get active opp by reading odm registers.
+ */
uint8_t opp_inst;
-
+ /**
+ * OTG HW instance.
+ */
uint8_t otg_inst;
+ /**
+ * DIG FE HW instance.
+ */
uint8_t digfe_inst;
+ /**
+ * DIG BE HW instance.
+ */
uint8_t digbe_inst;
+ /**
+ * DP PHY HW instance.
+ */
uint8_t dpphy_inst;
+ /**
+ * AUX HW instance.
+ */
uint8_t aux_inst;
+ /**
+ * Determines if SMU optimzations are enabled/disabled.
+ */
uint8_t smu_optimizations_en;
+ /**
+ * Unused.
+ * TODO: Remove.
+ */
uint8_t frame_delay;
+ /**
+ * If RFB setup time is greater than the total VBLANK time,
+ * it is not possible for the sink to capture the video frame
+ * in the same frame the SDP is sent. In this case,
+ * the frame capture indication bit should be set and an extra
+ * static frame should be transmitted to the sink.
+ */
uint8_t frame_cap_ind;
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
uint8_t pad[2];
+ /**
+ * Multi-display optimizations are implemented on certain ASICs.
+ */
uint8_t multi_disp_optimizations_en;
+ /**
+ * The last possible line SDP may be transmitted without violating
+ * the RFB setup time or entering the active video frame.
+ */
uint16_t init_sdp_deadline;
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
uint16_t pad2;
+ /**
+ * Length of each horizontal line in us.
+ */
uint32_t line_time_in_us;
+ /**
+ * FEC enable status in driver
+ */
+ uint8_t fec_enable_status;
+ /**
+ * FEC re-enable delay when PSR exit.
+ * unit is 100us, range form 0~255(0xFF).
+ */
+ uint8_t fec_enable_delay_in100us;
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
+ uint8_t pad3[2];
};
+/**
+ * Definition of a DMUB_CMD__PSR_COPY_SETTINGS command.
+ */
struct dmub_rb_cmd_psr_copy_settings {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__PSR_COPY_SETTINGS command.
+ */
struct dmub_cmd_psr_copy_settings_data psr_copy_settings_data;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__PSR_SET_LEVEL command.
+ */
struct dmub_cmd_psr_set_level_data {
+ /**
+ * 16-bit value dicated by driver that will enable/disable different functionality.
+ */
uint16_t psr_level;
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
uint8_t pad[2];
};
+/**
+ * Definition of a DMUB_CMD__PSR_SET_LEVEL command.
+ */
struct dmub_rb_cmd_psr_set_level {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * Definition of a DMUB_CMD__PSR_SET_LEVEL command.
+ */
struct dmub_cmd_psr_set_level_data psr_set_level_data;
};
+/**
+ * Definition of a DMUB_CMD__PSR_ENABLE command.
+ * PSR enable/disable is controlled using the sub_type.
+ */
struct dmub_rb_cmd_psr_enable {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__PSR_SET_VERSION command.
+ */
struct dmub_cmd_psr_set_version_data {
- enum psr_version version; // PSR version 1 or 2
+ /**
+ * PSR version that FW should implement.
+ */
+ enum psr_version version;
};
+/**
+ * Definition of a DMUB_CMD__PSR_SET_VERSION command.
+ */
struct dmub_rb_cmd_psr_set_version {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__PSR_SET_VERSION command.
+ */
struct dmub_cmd_psr_set_version_data psr_set_version_data;
};
+/**
+ * Definition of a DMUB_CMD__PSR_FORCE_STATIC command.
+ */
struct dmub_rb_cmd_psr_force_static {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
};
+/**
+ * Set of HW components that can be locked.
+ */
union dmub_hw_lock_flags {
+ /**
+ * Set of HW components that can be locked.
+ */
struct {
+ /**
+ * Lock/unlock OTG master update lock.
+ */
uint8_t lock_pipe : 1;
+ /**
+ * Lock/unlock cursor.
+ */
uint8_t lock_cursor : 1;
+ /**
+ * Lock/unlock global update lock.
+ */
uint8_t lock_dig : 1;
+ /**
+ * Triple buffer lock requires additional hw programming to usual OTG master lock.
+ */
uint8_t triple_buffer_lock : 1;
} bits;
+ /**
+ * Union for HW Lock flags.
+ */
uint8_t u8All;
};
+/**
+ * Instances of HW to be locked.
+ */
struct dmub_hw_lock_inst_flags {
+ /**
+ * OTG HW instance for OTG master update lock.
+ */
uint8_t otg_inst;
+ /**
+ * OPP instance for cursor lock.
+ */
uint8_t opp_inst;
+ /**
+ * OTG HW instance for global update lock.
+ * TODO: Remove, and re-use otg_inst.
+ */
uint8_t dig_inst;
+ /**
+ * Explicit pad to 4 byte boundary.
+ */
uint8_t pad;
};
+/**
+ * Clients that can acquire the HW Lock Manager.
+ */
enum hw_lock_client {
+ /**
+ * Driver is the client of HW Lock Manager.
+ */
HW_LOCK_CLIENT_DRIVER = 0,
+ /**
+ * FW is the client of HW Lock Manager.
+ */
HW_LOCK_CLIENT_FW,
+ /**
+ * Invalid client.
+ */
HW_LOCK_CLIENT_INVALID = 0xFFFFFFFF,
};
+/**
+ * Data passed to HW Lock Mgr in a DMUB_CMD__HW_LOCK command.
+ */
struct dmub_cmd_lock_hw_data {
+ /**
+ * Specifies the client accessing HW Lock Manager.
+ */
enum hw_lock_client client;
+ /**
+ * HW instances to be locked.
+ */
struct dmub_hw_lock_inst_flags inst_flags;
+ /**
+ * Which components to be locked.
+ */
union dmub_hw_lock_flags hw_locks;
+ /**
+ * Specifies lock/unlock.
+ */
uint8_t lock;
+ /**
+ * HW can be unlocked separately from releasing the HW Lock Mgr.
+ * This flag is set if the client wishes to release the object.
+ */
uint8_t should_release;
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
uint8_t pad;
};
+/**
+ * Definition of a DMUB_CMD__HW_LOCK command.
+ * Command is used by driver and FW.
+ */
struct dmub_rb_cmd_lock_hw {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+ /**
+ * Data passed to HW Lock Mgr in a DMUB_CMD__HW_LOCK command.
+ */
struct dmub_cmd_lock_hw_data lock_hw_data;
};
+/**
+ * ABM command sub-types.
+ */
enum dmub_cmd_abm_type {
+ /**
+ * Initialize parameters for ABM algorithm.
+ * Data is passed through an indirect buffer.
+ */
DMUB_CMD__ABM_INIT_CONFIG = 0,
+ /**
+ * Set OTG and panel HW instance.
+ */
DMUB_CMD__ABM_SET_PIPE = 1,
+ /**
+ * Set user requested backklight level.
+ */
DMUB_CMD__ABM_SET_BACKLIGHT = 2,
+ /**
+ * Set ABM operating/aggression level.
+ */
DMUB_CMD__ABM_SET_LEVEL = 3,
+ /**
+ * Set ambient light level.
+ */
DMUB_CMD__ABM_SET_AMBIENT_LEVEL = 4,
+ /**
+ * Enable/disable fractional duty cycle for backlight PWM.
+ */
DMUB_CMD__ABM_SET_PWM_FRAC = 5,
};
-#define NUM_AMBI_LEVEL 5
-#define NUM_AGGR_LEVEL 4
-#define NUM_POWER_FN_SEGS 8
-#define NUM_BL_CURVE_SEGS 16
-
-/*
- * Parameters for ABM2.4 algorithm.
- * Padded explicitly to 32-bit boundary.
+/**
+ * Parameters for ABM2.4 algorithm. Passed from driver to FW via an indirect buffer.
+ * Requirements:
+ * - Padded explicitly to 32-bit boundary.
+ * - Must ensure this structure matches the one on driver-side,
+ * otherwise it won't be aligned.
*/
struct abm_config_table {
- /* Parameters for crgb conversion */
+ /**
+ * Gamma curve thresholds, used for crgb conversion.
+ */
uint16_t crgb_thresh[NUM_POWER_FN_SEGS]; // 0B
+ /**
+ * Gamma curve offsets, used for crgb conversion.
+ */
uint16_t crgb_offset[NUM_POWER_FN_SEGS]; // 16B
+ /**
+ * Gamma curve slopes, used for crgb conversion.
+ */
uint16_t crgb_slope[NUM_POWER_FN_SEGS]; // 32B
-
- /* Parameters for custom curve */
+ /**
+ * Custom backlight curve thresholds.
+ */
uint16_t backlight_thresholds[NUM_BL_CURVE_SEGS]; // 48B
+ /**
+ * Custom backlight curve offsets.
+ */
uint16_t backlight_offsets[NUM_BL_CURVE_SEGS]; // 78B
-
+ /**
+ * Ambient light thresholds.
+ */
uint16_t ambient_thresholds_lux[NUM_AMBI_LEVEL]; // 112B
+ /**
+ * Minimum programmable backlight.
+ */
uint16_t min_abm_backlight; // 122B
-
+ /**
+ * Minimum reduction values.
+ */
uint8_t min_reduction[NUM_AMBI_LEVEL][NUM_AGGR_LEVEL]; // 124B
+ /**
+ * Maximum reduction values.
+ */
uint8_t max_reduction[NUM_AMBI_LEVEL][NUM_AGGR_LEVEL]; // 144B
+ /**
+ * Bright positive gain.
+ */
uint8_t bright_pos_gain[NUM_AMBI_LEVEL][NUM_AGGR_LEVEL]; // 164B
+ /**
+ * Dark negative gain.
+ */
uint8_t dark_pos_gain[NUM_AMBI_LEVEL][NUM_AGGR_LEVEL]; // 184B
+ /**
+ * Hybrid factor.
+ */
uint8_t hybrid_factor[NUM_AGGR_LEVEL]; // 204B
+ /**
+ * Contrast factor.
+ */
uint8_t contrast_factor[NUM_AGGR_LEVEL]; // 208B
+ /**
+ * Deviation gain.
+ */
uint8_t deviation_gain[NUM_AGGR_LEVEL]; // 212B
+ /**
+ * Minimum knee.
+ */
uint8_t min_knee[NUM_AGGR_LEVEL]; // 216B
+ /**
+ * Maximum knee.
+ */
uint8_t max_knee[NUM_AGGR_LEVEL]; // 220B
+ /**
+ * Unused.
+ */
uint8_t iir_curve[NUM_AMBI_LEVEL]; // 224B
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
uint8_t pad3[3]; // 229B
-
+ /**
+ * Backlight ramp reduction.
+ */
uint16_t blRampReduction[NUM_AGGR_LEVEL]; // 232B
+ /**
+ * Backlight ramp start.
+ */
uint16_t blRampStart[NUM_AGGR_LEVEL]; // 240B
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_PIPE command.
+ */
struct dmub_cmd_abm_set_pipe_data {
+ /**
+ * OTG HW instance.
+ */
uint8_t otg_inst;
+
+ /**
+ * Panel Control HW instance.
+ */
uint8_t panel_inst;
+
+ /**
+ * Controls how ABM will interpret a set pipe or set level command.
+ */
uint8_t set_pipe_option;
- uint8_t ramping_boundary; // TODO: Remove this
+
+ /**
+ * Unused.
+ * TODO: Remove.
+ */
+ uint8_t ramping_boundary;
};
+/**
+ * Definition of a DMUB_CMD__ABM_SET_PIPE command.
+ */
struct dmub_rb_cmd_abm_set_pipe {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_PIPE command.
+ */
struct dmub_cmd_abm_set_pipe_data abm_set_pipe_data;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_BACKLIGHT command.
+ */
struct dmub_cmd_abm_set_backlight_data {
+ /**
+ * Number of frames to ramp to backlight user level.
+ */
uint32_t frame_ramp;
+
+ /**
+ * Requested backlight level from user.
+ */
uint32_t backlight_user_level;
+
+ /**
+ * Backlight data version.
+ */
+ uint8_t version;
+
+ /**
+ * Panel Control HW instance mask.
+ * Bit 0 is Panel Control HW instance 0.
+ * Bit 1 is Panel Control HW instance 1.
+ */
+ uint8_t panel_mask;
+
+ /**
+ * Explicit padding to 4 byte boundary.
+ */
+ uint8_t pad[2];
};
+/**
+ * Definition of a DMUB_CMD__ABM_SET_BACKLIGHT command.
+ */
struct dmub_rb_cmd_abm_set_backlight {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_BACKLIGHT command.
+ */
struct dmub_cmd_abm_set_backlight_data abm_set_backlight_data;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_LEVEL command.
+ */
struct dmub_cmd_abm_set_level_data {
+ /**
+ * Set current ABM operating/aggression level.
+ */
uint32_t level;
};
+/**
+ * Definition of a DMUB_CMD__ABM_SET_LEVEL command.
+ */
struct dmub_rb_cmd_abm_set_level {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_LEVEL command.
+ */
struct dmub_cmd_abm_set_level_data abm_set_level_data;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_AMBIENT_LEVEL command.
+ */
struct dmub_cmd_abm_set_ambient_level_data {
+ /**
+ * Ambient light sensor reading from OS.
+ */
uint32_t ambient_lux;
};
+/**
+ * Definition of a DMUB_CMD__ABM_SET_AMBIENT_LEVEL command.
+ */
struct dmub_rb_cmd_abm_set_ambient_level {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_AMBIENT_LEVEL command.
+ */
struct dmub_cmd_abm_set_ambient_level_data abm_set_ambient_level_data;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_PWM_FRAC command.
+ */
struct dmub_cmd_abm_set_pwm_frac_data {
+ /**
+ * Enable/disable fractional duty cycle for backlight PWM.
+ * TODO: Convert to uint8_t.
+ */
uint32_t fractional_pwm;
};
+/**
+ * Definition of a DMUB_CMD__ABM_SET_PWM_FRAC command.
+ */
struct dmub_rb_cmd_abm_set_pwm_frac {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_SET_PWM_FRAC command.
+ */
struct dmub_cmd_abm_set_pwm_frac_data abm_set_pwm_frac_data;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_INIT_CONFIG command.
+ */
struct dmub_cmd_abm_init_config_data {
+ /**
+ * Location of indirect buffer used to pass init data to ABM.
+ */
union dmub_addr src;
+
+ /**
+ * Indirect buffer length.
+ */
uint16_t bytes;
};
+/**
+ * Definition of a DMUB_CMD__ABM_INIT_CONFIG command.
+ */
struct dmub_rb_cmd_abm_init_config {
+ /**
+ * Command header.
+ */
struct dmub_cmd_header header;
+
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__ABM_INIT_CONFIG command.
+ */
struct dmub_cmd_abm_init_config_data abm_init_config_data;
};
+/**
+ * Data passed from driver to FW in a DMUB_CMD__QUERY_FEATURE_CAPS command.
+ */
struct dmub_cmd_query_feature_caps_data {
- struct dmub_feature_caps feature_caps;
+ /**
+ * DMUB feature capabilities.
+ * After DMUB init, driver will query FW capabilities prior to enabling certain features.
+ */
+ struct dmub_feature_caps feature_caps;
};
+/**
+ * Definition of a DMUB_CMD__QUERY_FEATURE_CAPS command.
+ */
struct dmub_rb_cmd_query_feature_caps {
- struct dmub_cmd_header header;
- struct dmub_cmd_query_feature_caps_data query_feature_caps_data;
+ /**
+ * Command header.
+ */
+ struct dmub_cmd_header header;
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__QUERY_FEATURE_CAPS command.
+ */
+ struct dmub_cmd_query_feature_caps_data query_feature_caps_data;
+};
+
+struct dmub_optc_state {
+ uint32_t v_total_max;
+ uint32_t v_total_min;
+ uint32_t v_total_mid;
+ uint32_t v_total_mid_frame_num;
+ uint32_t tg_inst;
+ uint32_t enable_manual_trigger;
+ uint32_t clear_force_vsync;
+};
+
+struct dmub_rb_cmd_drr_update {
+ struct dmub_cmd_header header;
+ struct dmub_optc_state dmub_optc_state_req;
+};
+
+/**
+ * Data passed from driver to FW in a DMUB_CMD__VBIOS_LVTMA_CONTROL command.
+ */
+struct dmub_cmd_lvtma_control_data {
+ uint8_t uc_pwr_action; /**< LVTMA_ACTION */
+ uint8_t reserved_0[3]; /**< For future use */
+ uint8_t panel_inst; /**< LVTMA control instance */
+ uint8_t reserved_1[3]; /**< For future use */
};
- union dmub_rb_cmd {
+/**
+ * Definition of a DMUB_CMD__VBIOS_LVTMA_CONTROL command.
+ */
+struct dmub_rb_cmd_lvtma_control {
+ /**
+ * Command header.
+ */
+ struct dmub_cmd_header header;
+ /**
+ * Data passed from driver to FW in a DMUB_CMD__VBIOS_LVTMA_CONTROL command.
+ */
+ struct dmub_cmd_lvtma_control_data data;
+};
+
+/**
+ * union dmub_rb_cmd - DMUB inbox command.
+ */
+union dmub_rb_cmd {
struct dmub_rb_cmd_lock_hw lock_hw;
+ /**
+ * Elements shared with all commands.
+ */
+ struct dmub_rb_cmd_common cmd_common;
+ /**
+ * Definition of a DMUB_CMD__REG_SEQ_READ_MODIFY_WRITE command.
+ */
struct dmub_rb_cmd_read_modify_write read_modify_write;
+ /**
+ * Definition of a DMUB_CMD__REG_SEQ_FIELD_UPDATE_SEQ command.
+ */
struct dmub_rb_cmd_reg_field_update_sequence reg_field_update_seq;
+ /**
+ * Definition of a DMUB_CMD__REG_SEQ_BURST_WRITE command.
+ */
struct dmub_rb_cmd_burst_write burst_write;
+ /**
+ * Definition of a DMUB_CMD__REG_REG_WAIT command.
+ */
struct dmub_rb_cmd_reg_wait reg_wait;
- struct dmub_rb_cmd_common cmd_common;
+ /**
+ * Definition of a DMUB_CMD__VBIOS_DIGX_ENCODER_CONTROL command.
+ */
struct dmub_rb_cmd_digx_encoder_control digx_encoder_control;
+ /**
+ * Definition of a DMUB_CMD__VBIOS_SET_PIXEL_CLOCK command.
+ */
struct dmub_rb_cmd_set_pixel_clock set_pixel_clock;
+ /**
+ * Definition of a DMUB_CMD__VBIOS_ENABLE_DISP_POWER_GATING command.
+ */
struct dmub_rb_cmd_enable_disp_power_gating enable_disp_power_gating;
+ /**
+ * Definition of a DMUB_CMD__VBIOS_DPPHY_INIT command.
+ */
struct dmub_rb_cmd_dpphy_init dpphy_init;
+ /**
+ * Definition of a DMUB_CMD__VBIOS_DIG1_TRANSMITTER_CONTROL command.
+ */
struct dmub_rb_cmd_dig1_transmitter_control dig1_transmitter_control;
+ /**
+ * Definition of a DMUB_CMD__PSR_SET_VERSION command.
+ */
struct dmub_rb_cmd_psr_set_version psr_set_version;
+ /**
+ * Definition of a DMUB_CMD__PSR_COPY_SETTINGS command.
+ */
struct dmub_rb_cmd_psr_copy_settings psr_copy_settings;
+ /**
+ * Definition of a DMUB_CMD__PSR_ENABLE command.
+ */
struct dmub_rb_cmd_psr_enable psr_enable;
+ /**
+ * Definition of a DMUB_CMD__PSR_SET_LEVEL command.
+ */
struct dmub_rb_cmd_psr_set_level psr_set_level;
+ /**
+ * Definition of a DMUB_CMD__PSR_FORCE_STATIC command.
+ */
struct dmub_rb_cmd_psr_force_static psr_force_static;
+ /**
+ * Definition of a DMUB_CMD__PLAT_54186_WA command.
+ */
struct dmub_rb_cmd_PLAT_54186_wa PLAT_54186_wa;
+ /**
+ * Definition of a DMUB_CMD__MALL command.
+ */
struct dmub_rb_cmd_mall mall;
+ /**
+ * Definition of a DMUB_CMD__ABM_SET_PIPE command.
+ */
struct dmub_rb_cmd_abm_set_pipe abm_set_pipe;
+
+ /**
+ * Definition of a DMUB_CMD__ABM_SET_BACKLIGHT command.
+ */
struct dmub_rb_cmd_abm_set_backlight abm_set_backlight;
+
+ /**
+ * Definition of a DMUB_CMD__ABM_SET_LEVEL command.
+ */
struct dmub_rb_cmd_abm_set_level abm_set_level;
+
+ /**
+ * Definition of a DMUB_CMD__ABM_SET_AMBIENT_LEVEL command.
+ */
struct dmub_rb_cmd_abm_set_ambient_level abm_set_ambient_level;
+
+ /**
+ * Definition of a DMUB_CMD__ABM_SET_PWM_FRAC command.
+ */
struct dmub_rb_cmd_abm_set_pwm_frac abm_set_pwm_frac;
+
+ /**
+ * Definition of a DMUB_CMD__ABM_INIT_CONFIG command.
+ */
struct dmub_rb_cmd_abm_init_config abm_init_config;
+
+ /**
+ * Definition of a DMUB_CMD__DP_AUX_ACCESS command.
+ */
struct dmub_rb_cmd_dp_aux_access dp_aux_access;
+
+ /**
+ * Definition of a DMUB_CMD__OUTBOX1_ENABLE command.
+ */
struct dmub_rb_cmd_outbox1_enable outbox1_enable;
+
+ /**
+ * Definition of a DMUB_CMD__QUERY_FEATURE_CAPS command.
+ */
struct dmub_rb_cmd_query_feature_caps query_feature_caps;
+ struct dmub_rb_cmd_drr_update drr_update;
+ /**
+ * Definition of a DMUB_CMD__VBIOS_LVTMA_CONTROL command.
+ */
+ struct dmub_rb_cmd_lvtma_control lvtma_control;
};
+/**
+ * union dmub_rb_out_cmd - Outbox command
+ */
union dmub_rb_out_cmd {
+ /**
+ * Parameters common to every command.
+ */
struct dmub_rb_cmd_common cmd_common;
+ /**
+ * AUX reply command.
+ */
struct dmub_rb_cmd_dp_aux_reply dp_aux_reply;
+ /**
+ * HPD notify command.
+ */
struct dmub_rb_cmd_dp_hpd_notify dp_hpd_notify;
};
#pragma pack(pop)
@@ -910,31 +1993,49 @@ union dmub_rb_out_cmd {
extern "C" {
#endif
+/**
+ * struct dmub_rb_init_params - Initialization params for DMUB ringbuffer
+ */
struct dmub_rb_init_params {
- void *ctx;
- void *base_address;
- uint32_t capacity;
- uint32_t read_ptr;
- uint32_t write_ptr;
+ void *ctx; /**< Caller provided context pointer */
+ void *base_address; /**< CPU base address for ring's data */
+ uint32_t capacity; /**< Ringbuffer capacity in bytes */
+ uint32_t read_ptr; /**< Initial read pointer for consumer in bytes */
+ uint32_t write_ptr; /**< Initial write pointer for producer in bytes */
};
+/**
+ * struct dmub_rb - Inbox or outbox DMUB ringbuffer
+ */
struct dmub_rb {
- void *base_address;
- uint32_t data_count;
- uint32_t rptr;
- uint32_t wrpt;
- uint32_t capacity;
+ void *base_address; /**< CPU address for the ring's data */
+ uint32_t rptr; /**< Read pointer for consumer in bytes */
+ uint32_t wrpt; /**< Write pointer for producer in bytes */
+ uint32_t capacity; /**< Ringbuffer capacity in bytes */
- void *ctx;
- void *dmub;
+ void *ctx; /**< Caller provided context pointer */
+ void *dmub; /**< Pointer to the DMUB interface */
};
-
+/**
+ * @brief Checks if the ringbuffer is empty.
+ *
+ * @param rb DMUB Ringbuffer
+ * @return true if empty
+ * @return false otherwise
+ */
static inline bool dmub_rb_empty(struct dmub_rb *rb)
{
return (rb->wrpt == rb->rptr);
}
+/**
+ * @brief Checks if the ringbuffer is full
+ *
+ * @param rb DMUB Ringbuffer
+ * @return true if full
+ * @return false otherwise
+ */
static inline bool dmub_rb_full(struct dmub_rb *rb)
{
uint32_t data_count;
@@ -947,6 +2048,14 @@ static inline bool dmub_rb_full(struct dmub_rb *rb)
return (data_count == (rb->capacity - DMUB_RB_CMD_SIZE));
}
+/**
+ * @brief Pushes a command into the ringbuffer
+ *
+ * @param rb DMUB ringbuffer
+ * @param cmd The command to push
+ * @return true if the ringbuffer was not full
+ * @return false otherwise
+ */
static inline bool dmub_rb_push_front(struct dmub_rb *rb,
const union dmub_rb_cmd *cmd)
{
@@ -969,6 +2078,14 @@ static inline bool dmub_rb_push_front(struct dmub_rb *rb,
return true;
}
+/**
+ * @brief Pushes a command into the DMUB outbox ringbuffer
+ *
+ * @param rb DMUB outbox ringbuffer
+ * @param cmd Outbox command
+ * @return true if not full
+ * @return false otherwise
+ */
static inline bool dmub_rb_out_push_front(struct dmub_rb *rb,
const union dmub_rb_out_cmd *cmd)
{
@@ -988,6 +2105,14 @@ static inline bool dmub_rb_out_push_front(struct dmub_rb *rb,
return true;
}
+/**
+ * @brief Returns the next unprocessed command in the ringbuffer.
+ *
+ * @param rb DMUB ringbuffer
+ * @param cmd The command to return
+ * @return true if not empty
+ * @return false otherwise
+ */
static inline bool dmub_rb_front(struct dmub_rb *rb,
union dmub_rb_cmd **cmd)
{
@@ -1001,6 +2126,14 @@ static inline bool dmub_rb_front(struct dmub_rb *rb,
return true;
}
+/**
+ * @brief Returns the next unprocessed command in the outbox.
+ *
+ * @param rb DMUB outbox ringbuffer
+ * @param cmd The outbox command to return
+ * @return true if not empty
+ * @return false otherwise
+ */
static inline bool dmub_rb_out_front(struct dmub_rb *rb,
union dmub_rb_out_cmd *cmd)
{
@@ -1018,6 +2151,13 @@ static inline bool dmub_rb_out_front(struct dmub_rb *rb,
return true;
}
+/**
+ * @brief Removes the front entry in the ringbuffer.
+ *
+ * @param rb DMUB ringbuffer
+ * @return true if the command was removed
+ * @return false if there were no commands
+ */
static inline bool dmub_rb_pop_front(struct dmub_rb *rb)
{
if (dmub_rb_empty(rb))
@@ -1031,6 +2171,14 @@ static inline bool dmub_rb_pop_front(struct dmub_rb *rb)
return true;
}
+/**
+ * @brief Flushes commands in the ringbuffer to framebuffer memory.
+ *
+ * Avoids a race condition where DMCUB accesses memory while
+ * there are still writes in flight to framebuffer.
+ *
+ * @param rb DMUB ringbuffer
+ */
static inline void dmub_rb_flush_pending(const struct dmub_rb *rb)
{
uint32_t rptr = rb->rptr;
@@ -1049,6 +2197,12 @@ static inline void dmub_rb_flush_pending(const struct dmub_rb *rb)
}
}
+/**
+ * @brief Initializes a DMCUB ringbuffer
+ *
+ * @param rb DMUB ringbuffer
+ * @param init_params initial configuration for the ringbuffer
+ */
static inline void dmub_rb_init(struct dmub_rb *rb,
struct dmub_rb_init_params *init_params)
{
@@ -1058,6 +2212,12 @@ static inline void dmub_rb_init(struct dmub_rb *rb,
rb->wrpt = init_params->write_ptr;
}
+/**
+ * @brief Copies output data from in/out commands into the given command.
+ *
+ * @param rb DMUB ringbuffer
+ * @param cmd Command to copy data into
+ */
static inline void dmub_rb_get_return_data(struct dmub_rb *rb,
union dmub_rb_cmd *cmd)
{
diff --git a/drivers/gpu/drm/amd/display/dmub/inc/dmub_trace_buffer.h b/drivers/gpu/drm/amd/display/dmub/inc/dmub_trace_buffer.h
index 6b3ee42db350..8a122ceabb3a 100644
--- a/drivers/gpu/drm/amd/display/dmub/inc/dmub_trace_buffer.h
+++ b/drivers/gpu/drm/amd/display/dmub/inc/dmub_trace_buffer.h
@@ -25,7 +25,7 @@
#ifndef _DMUB_TRACE_BUFFER_H_
#define _DMUB_TRACE_BUFFER_H_
-#include "dmub_types.h"
+#include "dmub_cmd.h"
#define LOAD_DMCU_FW 1
#define LOAD_PHY_FW 2
@@ -65,5 +65,4 @@ struct dmcub_trace_buf {
struct dmcub_trace_buf_entry entries[PERF_TRACE_MAX_ENTRY];
};
-
#endif /* _DMUB_TRACE_BUFFER_H_ */
diff --git a/drivers/gpu/drm/amd/display/dmub/src/Makefile b/drivers/gpu/drm/amd/display/dmub/src/Makefile
index 945287164cf2..7495c23c73a9 100644
--- a/drivers/gpu/drm/amd/display/dmub/src/Makefile
+++ b/drivers/gpu/drm/amd/display/dmub/src/Makefile
@@ -20,7 +20,7 @@
# OTHER DEALINGS IN THE SOFTWARE.
#
-DMUB = dmub_srv.o dmub_reg.o dmub_dcn20.o dmub_dcn21.o
+DMUB = dmub_srv.o dmub_srv_stat.o dmub_reg.o dmub_dcn20.o dmub_dcn21.o
DMUB += dmub_dcn30.o dmub_dcn301.o
DMUB += dmub_dcn302.o
diff --git a/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.c b/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.c
index 8e8e65fa83c0..6934906c665e 100644
--- a/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.c
+++ b/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.c
@@ -135,6 +135,8 @@ void dmub_dcn20_reset(struct dmub_srv *dmub)
REG_UPDATE(MMHUBBUB_SOFT_RESET, DMUIF_SOFT_RESET, 1);
REG_WRITE(DMCUB_INBOX1_RPTR, 0);
REG_WRITE(DMCUB_INBOX1_WPTR, 0);
+ REG_WRITE(DMCUB_OUTBOX1_RPTR, 0);
+ REG_WRITE(DMCUB_OUTBOX1_WPTR, 0);
REG_WRITE(DMCUB_SCRATCH0, 0);
}
@@ -248,6 +250,13 @@ void dmub_dcn20_setup_windows(struct dmub_srv *dmub,
DMCUB_REGION3_CW5_TOP_ADDRESS, cw5->region.top,
DMCUB_REGION3_CW5_ENABLE, 1);
+ REG_WRITE(DMCUB_REGION5_OFFSET, offset.u.low_part);
+ REG_WRITE(DMCUB_REGION5_OFFSET_HIGH, offset.u.high_part);
+ REG_SET_2(DMCUB_REGION5_TOP_ADDRESS, 0,
+ DMCUB_REGION5_TOP_ADDRESS,
+ cw5->region.top - cw5->region.base - 1,
+ DMCUB_REGION5_ENABLE, 1);
+
dmub_dcn20_translate_addr(&cw6->offset, fb_base, fb_offset, &offset);
REG_WRITE(DMCUB_REGION3_CW6_OFFSET, offset.u.low_part);
@@ -280,6 +289,54 @@ void dmub_dcn20_set_inbox1_wptr(struct dmub_srv *dmub, uint32_t wptr_offset)
REG_WRITE(DMCUB_INBOX1_WPTR, wptr_offset);
}
+void dmub_dcn20_setup_out_mailbox(struct dmub_srv *dmub,
+ const struct dmub_region *outbox1)
+{
+ /* New firmware can support CW4 for the outbox. */
+ if (dmub_dcn20_use_cached_inbox(dmub))
+ REG_WRITE(DMCUB_OUTBOX1_BASE_ADDRESS, outbox1->base);
+ else
+ REG_WRITE(DMCUB_OUTBOX1_BASE_ADDRESS, 0x80002000);
+
+ REG_WRITE(DMCUB_OUTBOX1_SIZE, outbox1->top - outbox1->base);
+}
+
+uint32_t dmub_dcn20_get_outbox1_wptr(struct dmub_srv *dmub)
+{
+ /**
+ * outbox1 wptr register is accessed without locks (dal & dc)
+ * and to be called only by dmub_srv_stat_get_notification()
+ */
+ return REG_READ(DMCUB_OUTBOX1_WPTR);
+}
+
+void dmub_dcn20_set_outbox1_rptr(struct dmub_srv *dmub, uint32_t rptr_offset)
+{
+ /**
+ * outbox1 rptr register is accessed without locks (dal & dc)
+ * and to be called only by dmub_srv_stat_get_notification()
+ */
+ REG_WRITE(DMCUB_OUTBOX1_RPTR, rptr_offset);
+}
+
+void dmub_dcn20_setup_outbox0(struct dmub_srv *dmub,
+ const struct dmub_region *outbox0)
+{
+ REG_WRITE(DMCUB_OUTBOX0_BASE_ADDRESS, outbox0->base);
+
+ REG_WRITE(DMCUB_OUTBOX0_SIZE, outbox0->top - outbox0->base);
+}
+
+uint32_t dmub_dcn20_get_outbox0_wptr(struct dmub_srv *dmub)
+{
+ return REG_READ(DMCUB_OUTBOX0_WPTR);
+}
+
+void dmub_dcn20_set_outbox0_rptr(struct dmub_srv *dmub, uint32_t rptr_offset)
+{
+ REG_WRITE(DMCUB_OUTBOX0_RPTR, rptr_offset);
+}
+
bool dmub_dcn20_is_hw_init(struct dmub_srv *dmub)
{
uint32_t is_hw_init;
diff --git a/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.h b/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.h
index a62be9c0652e..de5351cd5abc 100644
--- a/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.h
+++ b/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn20.h
@@ -40,6 +40,14 @@ struct dmub_srv;
DMUB_SR(DMCUB_INBOX1_SIZE) \
DMUB_SR(DMCUB_INBOX1_RPTR) \
DMUB_SR(DMCUB_INBOX1_WPTR) \
+ DMUB_SR(DMCUB_OUTBOX0_BASE_ADDRESS) \
+ DMUB_SR(DMCUB_OUTBOX0_SIZE) \
+ DMUB_SR(DMCUB_OUTBOX0_RPTR) \
+ DMUB_SR(DMCUB_OUTBOX0_WPTR) \
+ DMUB_SR(DMCUB_OUTBOX1_BASE_ADDRESS) \
+ DMUB_SR(DMCUB_OUTBOX1_SIZE) \
+ DMUB_SR(DMCUB_OUTBOX1_RPTR) \
+ DMUB_SR(DMCUB_OUTBOX1_WPTR) \
DMUB_SR(DMCUB_REGION3_CW0_OFFSET) \
DMUB_SR(DMCUB_REGION3_CW1_OFFSET) \
DMUB_SR(DMCUB_REGION3_CW2_OFFSET) \
@@ -75,6 +83,9 @@ struct dmub_srv;
DMUB_SR(DMCUB_REGION4_OFFSET) \
DMUB_SR(DMCUB_REGION4_OFFSET_HIGH) \
DMUB_SR(DMCUB_REGION4_TOP_ADDRESS) \
+ DMUB_SR(DMCUB_REGION5_OFFSET) \
+ DMUB_SR(DMCUB_REGION5_OFFSET_HIGH) \
+ DMUB_SR(DMCUB_REGION5_TOP_ADDRESS) \
DMUB_SR(DMCUB_SCRATCH0) \
DMUB_SR(DMCUB_SCRATCH1) \
DMUB_SR(DMCUB_SCRATCH2) \
@@ -95,7 +106,8 @@ struct dmub_srv;
DMUB_SR(CC_DC_PIPE_DIS) \
DMUB_SR(MMHUBBUB_SOFT_RESET) \
DMUB_SR(DCN_VM_FB_LOCATION_BASE) \
- DMUB_SR(DCN_VM_FB_OFFSET)
+ DMUB_SR(DCN_VM_FB_OFFSET) \
+ DMUB_SR(DMCUB_INTERRUPT_ACK)
#define DMUB_COMMON_FIELDS() \
DMUB_SF(DMCUB_CNTL, DMCUB_ENABLE) \
@@ -123,10 +135,13 @@ struct dmub_srv;
DMUB_SF(DMCUB_REGION3_CW7_TOP_ADDRESS, DMCUB_REGION3_CW7_ENABLE) \
DMUB_SF(DMCUB_REGION4_TOP_ADDRESS, DMCUB_REGION4_TOP_ADDRESS) \
DMUB_SF(DMCUB_REGION4_TOP_ADDRESS, DMCUB_REGION4_ENABLE) \
+ DMUB_SF(DMCUB_REGION5_TOP_ADDRESS, DMCUB_REGION5_TOP_ADDRESS) \
+ DMUB_SF(DMCUB_REGION5_TOP_ADDRESS, DMCUB_REGION5_ENABLE) \
DMUB_SF(CC_DC_PIPE_DIS, DC_DMCUB_ENABLE) \
DMUB_SF(MMHUBBUB_SOFT_RESET, DMUIF_SOFT_RESET) \
DMUB_SF(DCN_VM_FB_LOCATION_BASE, FB_BASE) \
- DMUB_SF(DCN_VM_FB_OFFSET, FB_OFFSET)
+ DMUB_SF(DCN_VM_FB_OFFSET, FB_OFFSET) \
+ DMUB_SF(DMCUB_INTERRUPT_ACK, DMCUB_OUTBOX0_READY_INT_ACK)
struct dmub_srv_common_reg_offset {
#define DMUB_SR(reg) uint32_t reg;
@@ -180,6 +195,20 @@ uint32_t dmub_dcn20_get_inbox1_rptr(struct dmub_srv *dmub);
void dmub_dcn20_set_inbox1_wptr(struct dmub_srv *dmub, uint32_t wptr_offset);
+void dmub_dcn20_setup_out_mailbox(struct dmub_srv *dmub,
+ const struct dmub_region *outbox1);
+
+uint32_t dmub_dcn20_get_outbox1_wptr(struct dmub_srv *dmub);
+
+void dmub_dcn20_set_outbox1_rptr(struct dmub_srv *dmub, uint32_t rptr_offset);
+
+void dmub_dcn20_setup_outbox0(struct dmub_srv *dmub,
+ const struct dmub_region *outbox0);
+
+uint32_t dmub_dcn20_get_outbox0_wptr(struct dmub_srv *dmub);
+
+void dmub_dcn20_set_outbox0_rptr(struct dmub_srv *dmub, uint32_t rptr_offset);
+
bool dmub_dcn20_is_hw_init(struct dmub_srv *dmub);
bool dmub_dcn20_is_supported(struct dmub_srv *dmub);
@@ -200,4 +229,6 @@ union dmub_fw_boot_status dmub_dcn20_get_fw_boot_status(struct dmub_srv *dmub);
bool dmub_dcn20_use_cached_inbox(struct dmub_srv *dmub);
+bool dmub_dcn20_use_cached_trace_buffer(struct dmub_srv *dmub);
+
#endif /* _DMUB_DCN20_H_ */
diff --git a/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn30.c b/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn30.c
index b4bc0df2f14a..fb11c8d39208 100644
--- a/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn30.c
+++ b/drivers/gpu/drm/amd/display/dmub/src/dmub_dcn30.c
@@ -180,6 +180,13 @@ void dmub_dcn30_setup_windows(struct dmub_srv *dmub,
DMCUB_REGION3_CW5_TOP_ADDRESS, cw5->region.top,
DMCUB_REGION3_CW5_ENABLE, 1);
+ REG_WRITE(DMCUB_REGION5_OFFSET, offset.u.low_part);
+ REG_WRITE(DMCUB_REGION5_OFFSET_HIGH, offset.u.high_part);
+ REG_SET_2(DMCUB_REGION5_TOP_ADDRESS, 0,
+ DMCUB_REGION5_TOP_ADDRESS,
+ cw5->region.top - cw5->region.base - 1,
+ DMCUB_REGION5_ENABLE, 1);
+
offset = cw6->offset;
REG_WRITE(DMCUB_REGION3_CW6_OFFSET, offset.u.low_part);
diff --git a/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c b/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c
index 61f64a295f06..1cbb125b4063 100644
--- a/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c
+++ b/drivers/gpu/drm/amd/display/dmub/src/dmub_srv.c
@@ -46,8 +46,8 @@
/* Context size. */
#define DMUB_CONTEXT_SIZE (512 * 1024)
-/* Mailbox size */
-#define DMUB_MAILBOX_SIZE (DMUB_RB_SIZE)
+/* Mailbox size : Ring buffers are required for both inbox and outbox */
+#define DMUB_MAILBOX_SIZE ((2 * DMUB_RB_SIZE))
/* Default state size if meta is absent. */
#define DMUB_FW_STATE_SIZE (64 * 1024)
@@ -55,6 +55,7 @@
/* Default tracebuffer size if meta is absent. */
#define DMUB_TRACE_BUFFER_SIZE (64 * 1024)
+
/* Default scratch mem size. */
#define DMUB_SCRATCH_MEM_SIZE (256)
@@ -69,6 +70,8 @@
#define DMUB_CW5_BASE (0x65000000)
#define DMUB_CW6_BASE (0x66000000)
+#define DMUB_REGION5_BASE (0xA0000000)
+
static inline uint32_t dmub_align(uint32_t val, uint32_t factor)
{
return (val + factor - 1) / factor * factor;
@@ -157,6 +160,16 @@ static bool dmub_srv_hw_setup(struct dmub_srv *dmub, enum dmub_asic asic)
funcs->enable_dmub_boot_options = dmub_dcn20_enable_dmub_boot_options;
funcs->skip_dmub_panel_power_sequence = dmub_dcn20_skip_dmub_panel_power_sequence;
+ // Out mailbox register access functions for RN and above
+ funcs->setup_out_mailbox = dmub_dcn20_setup_out_mailbox;
+ funcs->get_outbox1_wptr = dmub_dcn20_get_outbox1_wptr;
+ funcs->set_outbox1_rptr = dmub_dcn20_set_outbox1_rptr;
+
+ //outbox0 call stacks
+ funcs->setup_outbox0 = dmub_dcn20_setup_outbox0;
+ funcs->get_outbox0_wptr = dmub_dcn20_get_outbox0_wptr;
+ funcs->set_outbox0_rptr = dmub_dcn20_set_outbox0_rptr;
+
if (asic == DMUB_ASIC_DCN21) {
dmub->regs = &dmub_srv_dcn21_regs;
@@ -395,13 +408,19 @@ enum dmub_status dmub_srv_hw_init(struct dmub_srv *dmub,
struct dmub_fb *fw_state_fb = params->fb[DMUB_WINDOW_6_FW_STATE];
struct dmub_fb *scratch_mem_fb = params->fb[DMUB_WINDOW_7_SCRATCH_MEM];
- struct dmub_rb_init_params rb_params;
+ struct dmub_rb_init_params rb_params, outbox0_rb_params;
struct dmub_window cw0, cw1, cw2, cw3, cw4, cw5, cw6;
- struct dmub_region inbox1;
+ struct dmub_region inbox1, outbox1, outbox0;
if (!dmub->sw_init)
return DMUB_STATUS_INVALID;
+ if (!inst_fb || !stack_fb || !data_fb || !bios_fb || !mail_fb ||
+ !tracebuff_fb || !fw_state_fb || !scratch_mem_fb) {
+ ASSERT(0);
+ return DMUB_STATUS_INVALID;
+ }
+
dmub->fb_base = params->fb_base;
dmub->fb_offset = params->fb_offset;
dmub->psp_version = params->psp_version;
@@ -409,72 +428,91 @@ enum dmub_status dmub_srv_hw_init(struct dmub_srv *dmub,
if (dmub->hw_funcs.reset)
dmub->hw_funcs.reset(dmub);
- if (inst_fb && data_fb) {
- cw0.offset.quad_part = inst_fb->gpu_addr;
- cw0.region.base = DMUB_CW0_BASE;
- cw0.region.top = cw0.region.base + inst_fb->size - 1;
-
- cw1.offset.quad_part = stack_fb->gpu_addr;
- cw1.region.base = DMUB_CW1_BASE;
- cw1.region.top = cw1.region.base + stack_fb->size - 1;
-
- if (params->load_inst_const && dmub->hw_funcs.backdoor_load) {
- /**
- * Read back all the instruction memory so we don't hang the
- * DMCUB when backdoor loading if the write from x86 hasn't been
- * flushed yet. This only occurs in backdoor loading.
- */
- dmub_flush_buffer_mem(inst_fb);
- dmub->hw_funcs.backdoor_load(dmub, &cw0, &cw1);
- }
+ cw0.offset.quad_part = inst_fb->gpu_addr;
+ cw0.region.base = DMUB_CW0_BASE;
+ cw0.region.top = cw0.region.base + inst_fb->size - 1;
+
+ cw1.offset.quad_part = stack_fb->gpu_addr;
+ cw1.region.base = DMUB_CW1_BASE;
+ cw1.region.top = cw1.region.base + stack_fb->size - 1;
+ if (params->load_inst_const && dmub->hw_funcs.backdoor_load) {
+ /**
+ * Read back all the instruction memory so we don't hang the
+ * DMCUB when backdoor loading if the write from x86 hasn't been
+ * flushed yet. This only occurs in backdoor loading.
+ */
+ dmub_flush_buffer_mem(inst_fb);
+ dmub->hw_funcs.backdoor_load(dmub, &cw0, &cw1);
}
- if (inst_fb && data_fb && bios_fb && mail_fb && tracebuff_fb &&
- fw_state_fb && scratch_mem_fb) {
- cw2.offset.quad_part = data_fb->gpu_addr;
- cw2.region.base = DMUB_CW0_BASE + inst_fb->size;
- cw2.region.top = cw2.region.base + data_fb->size;
+ cw2.offset.quad_part = data_fb->gpu_addr;
+ cw2.region.base = DMUB_CW0_BASE + inst_fb->size;
+ cw2.region.top = cw2.region.base + data_fb->size;
+
+ cw3.offset.quad_part = bios_fb->gpu_addr;
+ cw3.region.base = DMUB_CW3_BASE;
+ cw3.region.top = cw3.region.base + bios_fb->size;
+
+ cw4.offset.quad_part = mail_fb->gpu_addr;
+ cw4.region.base = DMUB_CW4_BASE;
+ cw4.region.top = cw4.region.base + mail_fb->size;
- cw3.offset.quad_part = bios_fb->gpu_addr;
- cw3.region.base = DMUB_CW3_BASE;
- cw3.region.top = cw3.region.base + bios_fb->size;
+ /**
+ * Doubled the mailbox region to accomodate inbox and outbox.
+ * Note: Currently, currently total mailbox size is 16KB. It is split
+ * equally into 8KB between inbox and outbox. If this config is
+ * changed, then uncached base address configuration of outbox1
+ * has to be updated in funcs->setup_out_mailbox.
+ */
+ inbox1.base = cw4.region.base;
+ inbox1.top = cw4.region.base + DMUB_RB_SIZE;
+ outbox1.base = inbox1.top;
+ outbox1.top = cw4.region.top;
- cw4.offset.quad_part = mail_fb->gpu_addr;
- cw4.region.base = DMUB_CW4_BASE;
- cw4.region.top = cw4.region.base + mail_fb->size;
+ cw5.offset.quad_part = tracebuff_fb->gpu_addr;
+ cw5.region.base = DMUB_CW5_BASE;
+ cw5.region.top = cw5.region.base + tracebuff_fb->size;
- inbox1.base = cw4.region.base;
- inbox1.top = cw4.region.top;
+ outbox0.base = DMUB_REGION5_BASE + TRACE_BUFFER_ENTRY_OFFSET;
+ outbox0.top = outbox0.base + tracebuff_fb->size - TRACE_BUFFER_ENTRY_OFFSET;
- cw5.offset.quad_part = tracebuff_fb->gpu_addr;
- cw5.region.base = DMUB_CW5_BASE;
- cw5.region.top = cw5.region.base + tracebuff_fb->size;
+ cw6.offset.quad_part = fw_state_fb->gpu_addr;
+ cw6.region.base = DMUB_CW6_BASE;
+ cw6.region.top = cw6.region.base + fw_state_fb->size;
- cw6.offset.quad_part = fw_state_fb->gpu_addr;
- cw6.region.base = DMUB_CW6_BASE;
- cw6.region.top = cw6.region.base + fw_state_fb->size;
+ dmub->fw_state = fw_state_fb->cpu_addr;
- dmub->fw_state = fw_state_fb->cpu_addr;
+ dmub->scratch_mem_fb = *scratch_mem_fb;
- dmub->scratch_mem_fb = *scratch_mem_fb;
+ if (dmub->hw_funcs.setup_windows)
+ dmub->hw_funcs.setup_windows(dmub, &cw2, &cw3, &cw4, &cw5, &cw6);
- if (dmub->hw_funcs.setup_windows)
- dmub->hw_funcs.setup_windows(dmub, &cw2, &cw3, &cw4,
- &cw5, &cw6);
+ if (dmub->hw_funcs.setup_outbox0)
+ dmub->hw_funcs.setup_outbox0(dmub, &outbox0);
- if (dmub->hw_funcs.setup_mailbox)
- dmub->hw_funcs.setup_mailbox(dmub, &inbox1);
- }
+ if (dmub->hw_funcs.setup_mailbox)
+ dmub->hw_funcs.setup_mailbox(dmub, &inbox1);
+ if (dmub->hw_funcs.setup_out_mailbox)
+ dmub->hw_funcs.setup_out_mailbox(dmub, &outbox1);
- if (mail_fb) {
- dmub_memset(&rb_params, 0, sizeof(rb_params));
- rb_params.ctx = dmub;
- rb_params.base_address = mail_fb->cpu_addr;
- rb_params.capacity = DMUB_RB_SIZE;
+ dmub_memset(&rb_params, 0, sizeof(rb_params));
+ rb_params.ctx = dmub;
+ rb_params.base_address = mail_fb->cpu_addr;
+ rb_params.capacity = DMUB_RB_SIZE;
+ dmub_rb_init(&dmub->inbox1_rb, &rb_params);
- dmub_rb_init(&dmub->inbox1_rb, &rb_params);
- }
+ // Initialize outbox1 ring buffer
+ rb_params.ctx = dmub;
+ rb_params.base_address = (void *) ((uint8_t *) (mail_fb->cpu_addr) + DMUB_RB_SIZE);
+ rb_params.capacity = DMUB_RB_SIZE;
+ dmub_rb_init(&dmub->outbox1_rb, &rb_params);
+
+ dmub_memset(&outbox0_rb_params, 0, sizeof(outbox0_rb_params));
+ outbox0_rb_params.ctx = dmub;
+ outbox0_rb_params.base_address = (void *)((uintptr_t)(tracebuff_fb->cpu_addr) + TRACE_BUFFER_ENTRY_OFFSET);
+ outbox0_rb_params.capacity = tracebuff_fb->size - dmub_align(TRACE_BUFFER_ENTRY_OFFSET, 64);
+ dmub_rb_init(&dmub->outbox0_rb, &outbox0_rb_params);
if (dmub->hw_funcs.reset_release)
dmub->hw_funcs.reset_release(dmub);
@@ -609,6 +647,8 @@ dmub_srv_send_gpint_command(struct dmub_srv *dmub,
dmub->hw_funcs.set_gpint(dmub, reg);
for (i = 0; i < timeout_us; ++i) {
+ udelay(1);
+
if (dmub->hw_funcs.is_gpint_acked(dmub, reg))
return DMUB_STATUS_OK;
}
@@ -674,3 +714,33 @@ enum dmub_status dmub_srv_cmd_with_reply_data(struct dmub_srv *dmub,
return status;
}
+
+static inline bool dmub_rb_out_trace_buffer_front(struct dmub_rb *rb,
+ void *entry)
+{
+ const uint64_t *src = (const uint64_t *)(rb->base_address) + rb->rptr / sizeof(uint64_t);
+ uint64_t *dst = (uint64_t *)entry;
+ uint8_t i;
+ uint8_t loop_count;
+
+ if (rb->rptr == rb->wrpt)
+ return false;
+
+ loop_count = sizeof(struct dmcub_trace_buf_entry) / sizeof(uint64_t);
+ // copying data
+ for (i = 0; i < loop_count; i++)
+ *dst++ = *src++;
+
+ rb->rptr += sizeof(struct dmcub_trace_buf_entry);
+
+ rb->rptr %= rb->capacity;
+
+ return true;
+}
+
+bool dmub_srv_get_outbox0_msg(struct dmub_srv *dmub, struct dmcub_trace_buf_entry *entry)
+{
+ dmub->outbox0_rb.wrpt = dmub->hw_funcs.get_outbox0_wptr(dmub);
+
+ return dmub_rb_out_trace_buffer_front(&dmub->outbox0_rb, (void *)entry);
+}
diff --git a/drivers/gpu/drm/amd/display/dmub/src/dmub_srv_stat.c b/drivers/gpu/drm/amd/display/dmub/src/dmub_srv_stat.c
new file mode 100644
index 000000000000..e6f3bfab33d3
--- /dev/null
+++ b/drivers/gpu/drm/amd/display/dmub/src/dmub_srv_stat.c
@@ -0,0 +1,105 @@
+/*
+ * Copyright 2019 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ * Authors: AMD
+ *
+ */
+
+#include "dmub/dmub_srv_stat.h"
+#include "dmub/inc/dmub_cmd.h"
+
+/**
+ * DOC: DMUB_SRV STAT Interface
+ *
+ * These interfaces are called without acquiring DAL and DC locks.
+ * Hence, there is limitations on whese interfaces can access. Only
+ * variables exclusively defined for these interfaces can be modified.
+ */
+
+/**
+ *****************************************************************************
+ * Function: dmub_srv_stat_get_notification
+ *
+ * @brief
+ * Retrieves a dmub outbox notification, set up dmub notification
+ * structure with message information. Also a pending bit if queue
+ * is having more notifications
+ *
+ * @param [in] dmub: dmub srv structure
+ * @param [out] pnotify: dmub notification structure to be filled up
+ *
+ * @return
+ * dmub_status
+ *****************************************************************************
+ */
+enum dmub_status dmub_srv_stat_get_notification(struct dmub_srv *dmub,
+ struct dmub_notification *notify)
+{
+ /**
+ * This function is called without dal and dc locks, so
+ * we shall not modify any dmub variables, only dmub->outbox1_rb
+ * is exempted as it is exclusively accessed by this function
+ */
+ union dmub_rb_out_cmd cmd = {0};
+
+ if (!dmub->hw_init) {
+ notify->type = DMUB_NOTIFICATION_NO_DATA;
+ notify->pending_notification = false;
+ return DMUB_STATUS_INVALID;
+ }
+
+ /* Get write pointer which is updated by dmub */
+ dmub->outbox1_rb.wrpt = dmub->hw_funcs.get_outbox1_wptr(dmub);
+
+ if (!dmub_rb_out_front(&dmub->outbox1_rb, &cmd)) {
+ notify->type = DMUB_NOTIFICATION_NO_DATA;
+ notify->pending_notification = false;
+ return DMUB_STATUS_OK;
+ }
+
+ switch (cmd.cmd_common.header.type) {
+ case DMUB_OUT_CMD__DP_AUX_REPLY:
+ notify->type = DMUB_NOTIFICATION_AUX_REPLY;
+ notify->link_index = cmd.dp_aux_reply.control.instance;
+ notify->result = cmd.dp_aux_reply.control.result;
+ dmub_memcpy((void *)&notify->aux_reply,
+ (void *)&cmd.dp_aux_reply.reply_data, sizeof(struct aux_reply_data));
+ break;
+ default:
+ notify->type = DMUB_NOTIFICATION_NO_DATA;
+ break;
+ }
+
+ /* Pop outbox1 ringbuffer and update read pointer */
+ dmub_rb_pop_front(&dmub->outbox1_rb);
+ dmub->hw_funcs.set_outbox1_rptr(dmub, dmub->outbox1_rb.rptr);
+
+ /**
+ * Notify dc whether dmub has a pending outbox message,
+ * this is to avoid one more call to dmub_srv_stat_get_notification
+ */
+ if (dmub_rb_empty(&dmub->outbox1_rb))
+ notify->pending_notification = false;
+ else
+ notify->pending_notification = true;
+
+ return DMUB_STATUS_OK;
+}
diff --git a/drivers/gpu/drm/amd/display/include/logger_types.h b/drivers/gpu/drm/amd/display/include/logger_types.h
index 21bbee17c527..571fcf23cea9 100644
--- a/drivers/gpu/drm/amd/display/include/logger_types.h
+++ b/drivers/gpu/drm/amd/display/include/logger_types.h
@@ -36,6 +36,9 @@
#define DC_LOG_DC(...) DRM_DEBUG_KMS(__VA_ARGS__)
#define DC_LOG_DTN(...) DRM_DEBUG_KMS(__VA_ARGS__)
#define DC_LOG_SURFACE(...) pr_debug("[SURFACE]:"__VA_ARGS__)
+#define DC_LOG_CURSOR(...) pr_debug("[CURSOR]:"__VA_ARGS__)
+#define DC_LOG_PFLIP(...) pr_debug("[PFLIP]:"__VA_ARGS__)
+#define DC_LOG_VBLANK(...) pr_debug("[VBLANK]:"__VA_ARGS__)
#define DC_LOG_HW_HOTPLUG(...) DRM_DEBUG_KMS(__VA_ARGS__)
#define DC_LOG_HW_LINK_TRAINING(...) pr_debug("[HW_LINK_TRAINING]:"__VA_ARGS__)
#define DC_LOG_HW_SET_MODE(...) DRM_DEBUG_KMS(__VA_ARGS__)
diff --git a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
index 5c67e12b2e55..ef742d95ef05 100644
--- a/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
+++ b/drivers/gpu/drm/amd/display/modules/color/color_gamma.c
@@ -942,7 +942,7 @@ static void hermite_spline_eetf(struct fixed31_32 input_x,
static bool build_freesync_hdr(struct pwl_float_data_ex *rgb_regamma,
uint32_t hw_points_num,
const struct hw_x_point *coordinate_x,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
struct calculate_buffer *cal_buffer)
{
uint32_t i;
@@ -2027,7 +2027,7 @@ rgb_user_alloc_fail:
static bool calculate_curve(enum dc_transfer_func_predefined trans,
struct dc_transfer_func_distributed_points *points,
struct pwl_float_data_ex *rgb_regamma,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
uint32_t sdr_ref_white_level,
struct calculate_buffer *cal_buffer)
{
@@ -2106,7 +2106,7 @@ static bool calculate_curve(enum dc_transfer_func_predefined trans,
bool mod_color_calculate_regamma_params(struct dc_transfer_func *output_tf,
const struct dc_gamma *ramp, bool mapUserRamp, bool canRomBeUsed,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
struct calculate_buffer *cal_buffer)
{
struct dc_transfer_func_distributed_points *tf_pts = &output_tf->tf_pts;
diff --git a/drivers/gpu/drm/amd/display/modules/color/color_gamma.h b/drivers/gpu/drm/amd/display/modules/color/color_gamma.h
index 7563457e2ff4..2893abf48208 100644
--- a/drivers/gpu/drm/amd/display/modules/color/color_gamma.h
+++ b/drivers/gpu/drm/amd/display/modules/color/color_gamma.h
@@ -76,7 +76,7 @@ struct regamma_lut {
};
};
-struct freesync_hdr_tf_params {
+struct hdr_tm_params {
unsigned int sdr_white_level;
unsigned int min_content; // luminance in 1/10000 nits
unsigned int max_content; // luminance in nits
@@ -108,7 +108,7 @@ void precompute_de_pq(void);
bool mod_color_calculate_regamma_params(struct dc_transfer_func *output_tf,
const struct dc_gamma *ramp, bool mapUserRamp, bool canRomBeUsed,
- const struct freesync_hdr_tf_params *fs_params,
+ const struct hdr_tm_params *fs_params,
struct calculate_buffer *cal_buffer);
bool mod_color_calculate_degamma_params(struct dc_color_caps *dc_caps,
diff --git a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
index 4762273b5bb9..3f4f44b44e6a 100644
--- a/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
+++ b/drivers/gpu/drm/amd/display/modules/freesync/freesync.c
@@ -118,7 +118,7 @@ static unsigned int calc_duration_in_us_from_v_total(
return duration_in_us;
}
-static unsigned int calc_v_total_from_refresh(
+unsigned int mod_freesync_calc_v_total_from_refresh(
const struct dc_stream_state *stream,
unsigned int refresh_in_uhz)
{
@@ -280,10 +280,10 @@ static void apply_below_the_range(struct core_freesync *core_freesync,
/* Restore FreeSync */
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
/* BTR set to "active" so engage */
} else {
@@ -442,16 +442,16 @@ static void apply_fixed_refresh(struct core_freesync *core_freesync,
if (update) {
if (in_out_vrr->fixed.fixed_active) {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(
+ mod_freesync_calc_v_total_from_refresh(
stream, in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
in_out_vrr->adjust.v_total_min;
} else {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
}
}
@@ -543,8 +543,8 @@ static void build_vrr_infopacket_data_v1(const struct mod_vrr_params *vrr,
infopacket->sb[6] |= 0x02;
/* PB6 = [Bit 2 = FreeSync Active] */
- if (vrr->state == VRR_STATE_ACTIVE_VARIABLE ||
- vrr->state == VRR_STATE_ACTIVE_FIXED)
+ if (vrr->state != VRR_STATE_DISABLED &&
+ vrr->state != VRR_STATE_UNSUPPORTED)
infopacket->sb[6] |= 0x04;
// For v1 & 2 infoframes program nominal if non-fs mode, otherwise full range
@@ -903,12 +903,31 @@ static void build_vrr_infopacket_v3(enum signal_type signal,
infopacket->valid = true;
}
+static void build_vrr_infopacket_sdp_v1_3(enum vrr_packet_type packet_type,
+ struct dc_info_packet *infopacket)
+{
+ uint8_t idx = 0, size = 0;
+
+ size = ((packet_type == PACKET_TYPE_FS_V1) ? 0x08 :
+ (packet_type == PACKET_TYPE_FS_V3) ? 0x10 :
+ 0x09);
+
+ for (idx = infopacket->hb2; idx > 1; idx--) // Data Byte Count: 0x1B
+ infopacket->sb[idx] = infopacket->sb[idx-1];
+
+ infopacket->sb[1] = size; // Length
+ infopacket->sb[0] = (infopacket->hb3 >> 2) & 0x3F;//Version
+ infopacket->hb3 = (0x13 << 2); // Header,SDP 1.3
+ infopacket->hb2 = 0x1D;
+}
+
void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync,
const struct dc_stream_state *stream,
const struct mod_vrr_params *vrr,
enum vrr_packet_type packet_type,
enum color_transfer_func app_tf,
- struct dc_info_packet *infopacket)
+ struct dc_info_packet *infopacket,
+ bool pack_sdp_v1_3)
{
/* SPD info packet for FreeSync
* VTEM info packet for HdmiVRR
@@ -941,6 +960,12 @@ void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync,
default:
build_vrr_infopacket_v1(stream->signal, vrr, infopacket);
}
+
+ if (true == pack_sdp_v1_3 &&
+ true == dc_is_dp_signal(stream->signal) &&
+ packet_type != PACKET_TYPE_VRR &&
+ packet_type != PACKET_TYPE_VTEM)
+ build_vrr_infopacket_sdp_v1_3(packet_type, infopacket);
}
void mod_freesync_build_vrr_params(struct mod_freesync *mod_freesync,
@@ -1057,10 +1082,10 @@ void mod_freesync_build_vrr_params(struct mod_freesync *mod_freesync,
refresh_range >= MIN_REFRESH_RANGE) {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
} else if (in_out_vrr->state == VRR_STATE_ACTIVE_FIXED) {
in_out_vrr->fixed.target_refresh_in_uhz =
@@ -1074,7 +1099,7 @@ void mod_freesync_build_vrr_params(struct mod_freesync *mod_freesync,
} else {
in_out_vrr->fixed.fixed_active = true;
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->fixed.target_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
in_out_vrr->adjust.v_total_min;
@@ -1181,10 +1206,10 @@ void mod_freesync_handle_v_update(struct mod_freesync *mod_freesync,
/* Restore FreeSync */
if (in_out_vrr->btr.frame_counter == 0) {
in_out_vrr->adjust.v_total_min =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->max_refresh_in_uhz);
in_out_vrr->adjust.v_total_max =
- calc_v_total_from_refresh(stream,
+ mod_freesync_calc_v_total_from_refresh(stream,
in_out_vrr->min_refresh_in_uhz);
}
}
@@ -1242,6 +1267,21 @@ unsigned long long mod_freesync_calc_nominal_field_rate(
return nominal_field_rate_in_uhz;
}
+unsigned long long mod_freesync_calc_field_rate_from_timing(
+ unsigned int vtotal, unsigned int htotal, unsigned int pix_clk)
+{
+ unsigned long long field_rate_in_uhz = 0;
+ unsigned int total = htotal * vtotal;
+
+ /* Calculate nominal field rate for stream, rounded up to nearest integer */
+ field_rate_in_uhz = pix_clk;
+ field_rate_in_uhz *= 1000000ULL;
+
+ field_rate_in_uhz = div_u64(field_rate_in_uhz, total);
+
+ return field_rate_in_uhz;
+}
+
bool mod_freesync_is_valid_range(uint32_t min_refresh_cap_in_uhz,
uint32_t max_refresh_cap_in_uhz,
uint32_t nominal_field_rate_in_uhz)
@@ -1304,4 +1344,3 @@ bool mod_freesync_is_valid_range(uint32_t min_refresh_cap_in_uhz,
return true;
}
-
diff --git a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c
index 20e554e771d1..68a6481d7f8f 100644
--- a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c
+++ b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.c
@@ -53,7 +53,7 @@ static uint8_t is_cp_desired_hdcp1(struct mod_hdcp *hdcp)
*/
for (i = 0; i < MAX_NUM_OF_DISPLAYS; i++) {
if (hdcp->displays[i].state != MOD_HDCP_DISPLAY_INACTIVE &&
- !hdcp->displays[i].adjust.disable) {
+ hdcp->displays[i].adjust.disable != MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION) {
is_auth_needed = 1;
break;
}
@@ -74,7 +74,7 @@ static uint8_t is_cp_desired_hdcp2(struct mod_hdcp *hdcp)
*/
for (i = 0; i < MAX_NUM_OF_DISPLAYS; i++) {
if (hdcp->displays[i].state != MOD_HDCP_DISPLAY_INACTIVE &&
- !hdcp->displays[i].adjust.disable) {
+ hdcp->displays[i].adjust.disable != MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION) {
is_auth_needed = 1;
break;
}
@@ -314,6 +314,9 @@ enum mod_hdcp_status mod_hdcp_add_display(struct mod_hdcp *hdcp,
goto out;
}
+ /* save current encryption states to restore after next authentication */
+ mod_hdcp_save_current_encryption_states(hdcp);
+
/* reset existing authentication status */
status = reset_authentication(hdcp, output);
if (status != MOD_HDCP_STATUS_SUCCESS)
@@ -360,6 +363,9 @@ enum mod_hdcp_status mod_hdcp_remove_display(struct mod_hdcp *hdcp,
goto out;
}
+ /* save current encryption states to restore after next authentication */
+ mod_hdcp_save_current_encryption_states(hdcp);
+
/* stop current authentication */
status = reset_authentication(hdcp, output);
if (status != MOD_HDCP_STATUS_SUCCESS)
diff --git a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h
index 5c22cf7e6118..3ce91db560d1 100644
--- a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h
+++ b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp.h
@@ -331,6 +331,8 @@ enum mod_hdcp_status mod_hdcp_add_display_to_topology(
struct mod_hdcp *hdcp, struct mod_hdcp_display *display);
enum mod_hdcp_status mod_hdcp_remove_display_from_topology(
struct mod_hdcp *hdcp, uint8_t index);
+bool mod_hdcp_is_link_encryption_enabled(struct mod_hdcp *hdcp);
+void mod_hdcp_save_current_encryption_states(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_create_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_destroy_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_validate_rx(struct mod_hdcp *hdcp);
@@ -339,8 +341,6 @@ enum mod_hdcp_status mod_hdcp_hdcp1_validate_ksvlist_vp(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_enable_dp_stream_encryption(
struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp1_link_maintenance(struct mod_hdcp *hdcp);
-enum mod_hdcp_status mod_hdcp_hdcp1_get_link_encryption_status(struct mod_hdcp *hdcp,
- enum mod_hdcp_encryption_status *encryption_status);
enum mod_hdcp_status mod_hdcp_hdcp2_create_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp2_destroy_session(struct mod_hdcp *hdcp);
enum mod_hdcp_status mod_hdcp_hdcp2_prepare_ake_init(struct mod_hdcp *hdcp);
diff --git a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c
index 73ca49f05bd3..2cbd931363bd 100644
--- a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c
+++ b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_execution.c
@@ -256,10 +256,12 @@ static enum mod_hdcp_status authenticated(struct mod_hdcp *hdcp,
goto out;
}
- if (!mod_hdcp_execute_and_set(mod_hdcp_hdcp1_link_maintenance,
+ mod_hdcp_execute_and_set(mod_hdcp_hdcp1_link_maintenance,
&input->link_maintenance, &status,
- hdcp, "link_maintenance"))
- goto out;
+ hdcp, "link_maintenance");
+
+ if (status != MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_save_current_encryption_states(hdcp);
out:
return status;
}
@@ -426,18 +428,21 @@ static enum mod_hdcp_status authenticated_dp(struct mod_hdcp *hdcp,
goto out;
}
- if (!mod_hdcp_execute_and_set(mod_hdcp_read_bstatus,
- &input->bstatus_read, &status,
- hdcp, "bstatus_read"))
- goto out;
- if (!mod_hdcp_execute_and_set(check_link_integrity_dp,
- &input->link_integrity_check, &status,
- hdcp, "link_integrity_check"))
- goto out;
- if (!mod_hdcp_execute_and_set(check_no_reauthentication_request_dp,
- &input->reauth_request_check, &status,
- hdcp, "reauth_request_check"))
- goto out;
+ if (status == MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_execute_and_set(mod_hdcp_read_bstatus,
+ &input->bstatus_read, &status,
+ hdcp, "bstatus_read");
+ if (status == MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_execute_and_set(check_link_integrity_dp,
+ &input->link_integrity_check, &status,
+ hdcp, "link_integrity_check");
+ if (status == MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_execute_and_set(check_no_reauthentication_request_dp,
+ &input->reauth_request_check, &status,
+ hdcp, "reauth_request_check");
+
+ if (status != MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_save_current_encryption_states(hdcp);
out:
return status;
}
diff --git a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_transition.c b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_transition.c
index 24ab95b093f7..3dda8c1d83fc 100644
--- a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_transition.c
+++ b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp1_transition.c
@@ -93,7 +93,7 @@ enum mod_hdcp_status mod_hdcp_hdcp1_transition(struct mod_hdcp *hdcp,
}
break;
case H1_A45_AUTHENTICATED:
- if (input->link_maintenance != PASS) {
+ if (input->link_maintenance == FAIL) {
/* 1A-07: consider invalid ri' a failure */
/* 1A-07a: consider read ri' not returned a failure */
fail_and_restart_in_ms(0, &status, output);
@@ -243,8 +243,8 @@ enum mod_hdcp_status mod_hdcp_hdcp1_dp_transition(struct mod_hdcp *hdcp,
}
break;
case D1_A4_AUTHENTICATED:
- if (input->link_integrity_check != PASS ||
- input->reauth_request_check != PASS) {
+ if (input->link_integrity_check == FAIL ||
+ input->reauth_request_check == FAIL) {
/* 1A-07: restart hdcp on a link integrity failure */
fail_and_restart_in_ms(0, &status, output);
break;
diff --git a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_execution.c b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_execution.c
index a0895a7efda2..c1331facdcb4 100644
--- a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_execution.c
+++ b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_execution.c
@@ -565,10 +565,10 @@ static enum mod_hdcp_status authenticated(struct mod_hdcp *hdcp,
goto out;
}
- if (!process_rxstatus(hdcp, event_ctx, input, &status))
- goto out;
- if (event_ctx->rx_id_list_ready)
- goto out;
+ process_rxstatus(hdcp, event_ctx, input, &status);
+
+ if (status != MOD_HDCP_STATUS_SUCCESS)
+ mod_hdcp_save_current_encryption_states(hdcp);
out:
return status;
}
diff --git a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_transition.c b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_transition.c
index e738c7ae66ec..b0306ed6d6b4 100644
--- a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_transition.c
+++ b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp2_transition.c
@@ -245,8 +245,8 @@ enum mod_hdcp_status mod_hdcp_hdcp2_transition(struct mod_hdcp *hdcp,
HDCP_FULL_DDC_TRACE(hdcp);
break;
case H2_A5_AUTHENTICATED:
- if (input->rxstatus_read != PASS ||
- input->reauth_request_check != PASS) {
+ if (input->rxstatus_read == FAIL ||
+ input->reauth_request_check == FAIL) {
fail_and_restart_in_ms(0, &status, output);
break;
} else if (event_ctx->rx_id_list_ready && conn->is_repeater) {
@@ -562,11 +562,11 @@ enum mod_hdcp_status mod_hdcp_hdcp2_dp_transition(struct mod_hdcp *hdcp,
HDCP_FULL_DDC_TRACE(hdcp);
break;
case D2_A5_AUTHENTICATED:
- if (input->rxstatus_read != PASS ||
- input->reauth_request_check != PASS) {
+ if (input->rxstatus_read == FAIL ||
+ input->reauth_request_check == FAIL) {
fail_and_restart_in_ms(0, &status, output);
break;
- } else if (input->link_integrity_check_dp != PASS) {
+ } else if (input->link_integrity_check_dp == FAIL) {
if (hdcp->connection.hdcp2_retry_count >= 1)
adjust->hdcp2.force_type = MOD_HDCP_FORCE_TYPE_0;
fail_and_restart_in_ms(0, &status, output);
diff --git a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp_psp.c b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp_psp.c
index 904ce9b88088..26f96c05e0ec 100644
--- a/drivers/gpu/drm/amd/display/modules/hdcp/hdcp_psp.c
+++ b/drivers/gpu/drm/amd/display/modules/hdcp/hdcp_psp.c
@@ -791,6 +791,8 @@ enum mod_hdcp_status mod_hdcp_hdcp2_validate_rx_id_list(struct mod_hdcp *hdcp)
TA_HDCP2_MSG_AUTHENTICATION_STATUS__RECEIVERID_REVOKED) {
hdcp->connection.is_hdcp2_revoked = 1;
status = MOD_HDCP_STATUS_HDCP2_RX_ID_LIST_REVOKED;
+ } else {
+ status = MOD_HDCP_STATUS_HDCP2_VALIDATE_RX_ID_LIST_FAILURE;
}
}
mutex_unlock(&psp->hdcp_context.mutex);
@@ -914,3 +916,13 @@ enum mod_hdcp_status mod_hdcp_hdcp2_validate_stream_ready(struct mod_hdcp *hdcp)
return status;
}
+bool mod_hdcp_is_link_encryption_enabled(struct mod_hdcp *hdcp)
+{
+ /* unsupported */
+ return true;
+}
+
+void mod_hdcp_save_current_encryption_states(struct mod_hdcp *hdcp)
+{
+ /* unsupported */
+}
diff --git a/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h b/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h
index c80fc10d732c..75a158a2514c 100644
--- a/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h
+++ b/drivers/gpu/drm/amd/display/modules/inc/mod_freesync.h
@@ -150,7 +150,8 @@ void mod_freesync_build_vrr_infopacket(struct mod_freesync *mod_freesync,
const struct mod_vrr_params *vrr,
enum vrr_packet_type packet_type,
enum color_transfer_func app_tf,
- struct dc_info_packet *infopacket);
+ struct dc_info_packet *infopacket,
+ bool pack_sdp_v1_3);
void mod_freesync_build_vrr_params(struct mod_freesync *mod_freesync,
const struct dc_stream_state *stream,
@@ -170,10 +171,15 @@ void mod_freesync_handle_v_update(struct mod_freesync *mod_freesync,
unsigned long long mod_freesync_calc_nominal_field_rate(
const struct dc_stream_state *stream);
+unsigned long long mod_freesync_calc_field_rate_from_timing(
+ unsigned int vtotal, unsigned int htotal, unsigned int pix_clk);
+
bool mod_freesync_is_valid_range(uint32_t min_refresh_cap_in_uhz,
uint32_t max_refresh_cap_in_uhz,
uint32_t nominal_field_rate_in_uhz);
-
+unsigned int mod_freesync_calc_v_total_from_refresh(
+ const struct dc_stream_state *stream,
+ unsigned int refresh_in_uhz);
#endif
diff --git a/drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h b/drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h
index d223ed3be5d3..acbeada5215b 100644
--- a/drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h
+++ b/drivers/gpu/drm/amd/display/modules/inc/mod_hdcp.h
@@ -120,6 +120,12 @@ enum mod_hdcp_display_state {
MOD_HDCP_DISPLAY_ENCRYPTION_ENABLED
};
+enum mod_hdcp_display_disable_option {
+ MOD_HDCP_DISPLAY_NOT_DISABLE = 0,
+ MOD_HDCP_DISPLAY_DISABLE_AUTHENTICATION,
+ MOD_HDCP_DISPLAY_DISABLE_ENCRYPTION,
+};
+
struct mod_hdcp_ddc {
void *handle;
struct {
@@ -149,8 +155,8 @@ struct mod_hdcp_psp {
};
struct mod_hdcp_display_adjustment {
- uint8_t disable : 1;
- uint8_t reserved : 7;
+ uint8_t disable : 2;
+ uint8_t reserved : 6;
};
struct mod_hdcp_link_adjustment_hdcp1 {
@@ -255,8 +261,6 @@ struct mod_hdcp_config {
uint8_t index;
};
-struct mod_hdcp;
-
/* dm allocates memory of mod_hdcp per dc_link on dm init based on memory size*/
size_t mod_hdcp_get_memory_size(void);
diff --git a/drivers/gpu/drm/amd/include/aldebaran_ip_offset.h b/drivers/gpu/drm/amd/include/aldebaran_ip_offset.h
new file mode 100644
index 000000000000..644ffec2b0ce
--- /dev/null
+++ b/drivers/gpu/drm/amd/include/aldebaran_ip_offset.h
@@ -0,0 +1,1738 @@
+/*
+ * Copyright (C) 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included
+ * in all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+ * OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN
+ * AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
+ */
+#ifndef _aldebaran_ip_offset_HEADER
+#define _aldebaran_ip_offset_HEADER
+
+#define MAX_INSTANCE 7
+#define MAX_SEGMENT 6
+
+struct IP_BASE_INSTANCE {
+ unsigned int segment[MAX_SEGMENT];
+};
+
+struct IP_BASE {
+ struct IP_BASE_INSTANCE instance[MAX_INSTANCE];
+};
+
+static const struct IP_BASE ATHUB_BASE = { { { { 0x00000C20, 0x02408C00, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE CLK_BASE = { { { { 0x00016C00, 0x02401800, 0, 0, 0, 0 } },
+ { { 0x00016E00, 0x02401C00, 0, 0, 0, 0 } },
+ { { 0x00017000, 0x02402000, 0, 0, 0, 0 } },
+ { { 0x00017200, 0x02402400, 0, 0, 0, 0 } },
+ { { 0x0001B000, 0x0242D800, 0, 0, 0, 0 } },
+ { { 0x0001B200, 0x0242DC00, 0, 0, 0, 0 } },
+ { { 0x00017E00, 0x0240BC00, 0, 0, 0, 0 } } } };
+static const struct IP_BASE DBGU_IO0_BASE = { { { { 0x000001E0, 0x0240B400, 0, 0, 0, 0 } },
+ { { 0x00000260, 0x02413C00, 0, 0, 0, 0 } },
+ { { 0x00000280, 0x02416000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE DF_BASE = { { { { 0x00007000, 0x0240B800, 0x07C00000, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE FUSE_BASE = { { { { 0x00017400, 0x02401400, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE GC_BASE = { { { { 0x00002000, 0x0000A000, 0x02402C00, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE HDP_BASE = { { { { 0x00000F20, 0x0240A400, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE IOAGR0_BASE = { { { { 0x02419000, 0x056C0000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE IOAPIC0_BASE = { { { { 0x00A00000, 0x0241F000, 0x050C0000, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE IOHC0_BASE = { { { { 0x00010000, 0x02406000, 0x04EC0000, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE L1IMUIOAGR0_BASE = { { { { 0x0240CC00, 0x05200000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE L1IMUPCIE0_BASE = { { { { 0x0240C800, 0x051C0000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE L2IMU0_BASE = { { { { 0x00007DC0, 0x00900000, 0x02407000, 0x04FC0000, 0x055C0000, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE MMHUB_BASE = { { { { 0x0001A000, 0x02408800, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE MP0_BASE = { { { { 0x00016000, 0x00DC0000, 0x00E00000, 0x00E40000, 0x0243FC00, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE MP1_BASE = { { { { 0x00016000, 0x00DC0000, 0x00E00000, 0x00E40000, 0x0243FC00, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE NBIO_BASE = { { { { 0x00000000, 0x00000014, 0x00000D20, 0x00010400, 0x0241B000, 0x04040000 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE OSSSYS_BASE = { { { { 0x000010A0, 0x0240A000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE PCIE0_BASE = { { { { 0x02411800, 0x04440000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE SDMA0_BASE = { { { { 0x00001260, 0x00012540, 0x0040A800, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE SDMA1_BASE = { { { { 0x00001860, 0x00012560, 0x0040AC00, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE SDMA2_BASE = { { { { 0x00013760, 0x0001E000, 0x0042EC00, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE SDMA3_BASE = { { { { 0x00013780, 0x0001E400, 0x0042F000, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE SDMA4_BASE = { { { { 0x000137A0, 0x0001E800, 0x0042F400, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE SMUIO_BASE = { { { { 0x00016800, 0x00016A00, 0x02401000, 0x03440000, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE THM_BASE = { { { { 0x00016600, 0x02400C00, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE UMC_BASE = { { { { 0x00014000, 0x00054000, 0x02425800, 0, 0, 0 } },
+ { { 0x00094000, 0x000D4000, 0x02425C00, 0, 0, 0 } },
+ { { 0x00114000, 0x00154000, 0x02426000, 0, 0, 0 } },
+ { { 0x00194000, 0x001D4000, 0x02426400, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE VCN_BASE = { { { { 0x00007800, 0x00007E00, 0x02403000, 0, 0, 0 } },
+ { { 0x00007A00, 0x00009000, 0x02445000, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE WAFL0_BASE = { { { { 0x02438000, 0x04880000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE WAFL1_BASE = { { { { 0, 0x01300000, 0x02410800, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE XGMI0_BASE = { { { { 0x02438C00, 0x04680000, 0x04940000, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE XGMI1_BASE = { { { { 0x02439000, 0x046C0000, 0x04980000, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+static const struct IP_BASE XGMI2_BASE = { { { { 0x04700000, 0x049C0000, 0, 0, 0, 0 } },
+ { { 0x04740000, 0x04A00000, 0, 0, 0, 0 } },
+ { { 0x04780000, 0x04A40000, 0, 0, 0, 0 } },
+ { { 0x047C0000, 0x04A80000, 0, 0, 0, 0 } },
+ { { 0x04800000, 0x04AC0000, 0, 0, 0, 0 } },
+ { { 0x04840000, 0x04B00000, 0, 0, 0, 0 } },
+ { { 0, 0, 0, 0, 0, 0 } } } };
+
+
+#define ATHUB_BASE__INST0_SEG0 0x00000C20
+#define ATHUB_BASE__INST0_SEG1 0x02408C00
+#define ATHUB_BASE__INST0_SEG2 0
+#define ATHUB_BASE__INST0_SEG3 0
+#define ATHUB_BASE__INST0_SEG4 0
+#define ATHUB_BASE__INST0_SEG5 0
+
+#define ATHUB_BASE__INST1_SEG0 0
+#define ATHUB_BASE__INST1_SEG1 0
+#define ATHUB_BASE__INST1_SEG2 0
+#define ATHUB_BASE__INST1_SEG3 0
+#define ATHUB_BASE__INST1_SEG4 0
+#define ATHUB_BASE__INST1_SEG5 0
+
+#define ATHUB_BASE__INST2_SEG0 0
+#define ATHUB_BASE__INST2_SEG1 0
+#define ATHUB_BASE__INST2_SEG2 0
+#define ATHUB_BASE__INST2_SEG3 0
+#define ATHUB_BASE__INST2_SEG4 0
+#define ATHUB_BASE__INST2_SEG5 0
+
+#define ATHUB_BASE__INST3_SEG0 0
+#define ATHUB_BASE__INST3_SEG1 0
+#define ATHUB_BASE__INST3_SEG2 0
+#define ATHUB_BASE__INST3_SEG3 0
+#define ATHUB_BASE__INST3_SEG4 0
+#define ATHUB_BASE__INST3_SEG5 0
+
+#define ATHUB_BASE__INST4_SEG0 0
+#define ATHUB_BASE__INST4_SEG1 0
+#define ATHUB_BASE__INST4_SEG2 0
+#define ATHUB_BASE__INST4_SEG3 0
+#define ATHUB_BASE__INST4_SEG4 0
+#define ATHUB_BASE__INST4_SEG5 0
+
+#define ATHUB_BASE__INST5_SEG0 0
+#define ATHUB_BASE__INST5_SEG1 0
+#define ATHUB_BASE__INST5_SEG2 0
+#define ATHUB_BASE__INST5_SEG3 0
+#define ATHUB_BASE__INST5_SEG4 0
+#define ATHUB_BASE__INST5_SEG5 0
+
+#define ATHUB_BASE__INST6_SEG0 0
+#define ATHUB_BASE__INST6_SEG1 0
+#define ATHUB_BASE__INST6_SEG2 0
+#define ATHUB_BASE__INST6_SEG3 0
+#define ATHUB_BASE__INST6_SEG4 0
+#define ATHUB_BASE__INST6_SEG5 0
+
+#define CLK_BASE__INST0_SEG0 0x00016C00
+#define CLK_BASE__INST0_SEG1 0x02401800
+#define CLK_BASE__INST0_SEG2 0
+#define CLK_BASE__INST0_SEG3 0
+#define CLK_BASE__INST0_SEG4 0
+#define CLK_BASE__INST0_SEG5 0
+
+#define CLK_BASE__INST1_SEG0 0x00016E00
+#define CLK_BASE__INST1_SEG1 0x02401C00
+#define CLK_BASE__INST1_SEG2 0
+#define CLK_BASE__INST1_SEG3 0
+#define CLK_BASE__INST1_SEG4 0
+#define CLK_BASE__INST1_SEG5 0
+
+#define CLK_BASE__INST2_SEG0 0x00017000
+#define CLK_BASE__INST2_SEG1 0x02402000
+#define CLK_BASE__INST2_SEG2 0
+#define CLK_BASE__INST2_SEG3 0
+#define CLK_BASE__INST2_SEG4 0
+#define CLK_BASE__INST2_SEG5 0
+
+#define CLK_BASE__INST3_SEG0 0x00017200
+#define CLK_BASE__INST3_SEG1 0x02402400
+#define CLK_BASE__INST3_SEG2 0
+#define CLK_BASE__INST3_SEG3 0
+#define CLK_BASE__INST3_SEG4 0
+#define CLK_BASE__INST3_SEG5 0
+
+#define CLK_BASE__INST4_SEG0 0x0001B000
+#define CLK_BASE__INST4_SEG1 0x0242D800
+#define CLK_BASE__INST4_SEG2 0
+#define CLK_BASE__INST4_SEG3 0
+#define CLK_BASE__INST4_SEG4 0
+#define CLK_BASE__INST4_SEG5 0
+
+#define CLK_BASE__INST5_SEG0 0x0001B200
+#define CLK_BASE__INST5_SEG1 0x0242DC00
+#define CLK_BASE__INST5_SEG2 0
+#define CLK_BASE__INST5_SEG3 0
+#define CLK_BASE__INST5_SEG4 0
+#define CLK_BASE__INST5_SEG5 0
+
+#define CLK_BASE__INST6_SEG0 0x00017E00
+#define CLK_BASE__INST6_SEG1 0x0240BC00
+#define CLK_BASE__INST6_SEG2 0
+#define CLK_BASE__INST6_SEG3 0
+#define CLK_BASE__INST6_SEG4 0
+#define CLK_BASE__INST6_SEG5 0
+
+#define DBGU_IO0_BASE__INST0_SEG0 0x000001E0
+#define DBGU_IO0_BASE__INST0_SEG1 0x0240B400
+#define DBGU_IO0_BASE__INST0_SEG2 0
+#define DBGU_IO0_BASE__INST0_SEG3 0
+#define DBGU_IO0_BASE__INST0_SEG4 0
+#define DBGU_IO0_BASE__INST0_SEG5 0
+
+#define DBGU_IO0_BASE__INST1_SEG0 0x00000260
+#define DBGU_IO0_BASE__INST1_SEG1 0x02413C00
+#define DBGU_IO0_BASE__INST1_SEG2 0
+#define DBGU_IO0_BASE__INST1_SEG3 0
+#define DBGU_IO0_BASE__INST1_SEG4 0
+#define DBGU_IO0_BASE__INST1_SEG5 0
+
+#define DBGU_IO0_BASE__INST2_SEG0 0x00000280
+#define DBGU_IO0_BASE__INST2_SEG1 0x02416000
+#define DBGU_IO0_BASE__INST2_SEG2 0
+#define DBGU_IO0_BASE__INST2_SEG3 0
+#define DBGU_IO0_BASE__INST2_SEG4 0
+#define DBGU_IO0_BASE__INST2_SEG5 0
+
+#define DBGU_IO0_BASE__INST3_SEG0 0
+#define DBGU_IO0_BASE__INST3_SEG1 0
+#define DBGU_IO0_BASE__INST3_SEG2 0
+#define DBGU_IO0_BASE__INST3_SEG3 0
+#define DBGU_IO0_BASE__INST3_SEG4 0
+#define DBGU_IO0_BASE__INST3_SEG5 0
+
+#define DBGU_IO0_BASE__INST4_SEG0 0
+#define DBGU_IO0_BASE__INST4_SEG1 0
+#define DBGU_IO0_BASE__INST4_SEG2 0
+#define DBGU_IO0_BASE__INST4_SEG3 0
+#define DBGU_IO0_BASE__INST4_SEG4 0
+#define DBGU_IO0_BASE__INST4_SEG5 0
+
+#define DBGU_IO0_BASE__INST5_SEG0 0
+#define DBGU_IO0_BASE__INST5_SEG1 0
+#define DBGU_IO0_BASE__INST5_SEG2 0
+#define DBGU_IO0_BASE__INST5_SEG3 0
+#define DBGU_IO0_BASE__INST5_SEG4 0
+#define DBGU_IO0_BASE__INST5_SEG5 0
+
+#define DBGU_IO0_BASE__INST6_SEG0 0
+#define DBGU_IO0_BASE__INST6_SEG1 0
+#define DBGU_IO0_BASE__INST6_SEG2 0
+#define DBGU_IO0_BASE__INST6_SEG3 0
+#define DBGU_IO0_BASE__INST6_SEG4 0
+#define DBGU_IO0_BASE__INST6_SEG5 0
+
+#define DF_BASE__INST0_SEG0 0x00007000
+#define DF_BASE__INST0_SEG1 0x0240B800
+#define DF_BASE__INST0_SEG2 0x07C00000
+#define DF_BASE__INST0_SEG3 0
+#define DF_BASE__INST0_SEG4 0
+#define DF_BASE__INST0_SEG5 0
+
+#define DF_BASE__INST1_SEG0 0
+#define DF_BASE__INST1_SEG1 0
+#define DF_BASE__INST1_SEG2 0
+#define DF_BASE__INST1_SEG3 0
+#define DF_BASE__INST1_SEG4 0
+#define DF_BASE__INST1_SEG5 0
+
+#define DF_BASE__INST2_SEG0 0
+#define DF_BASE__INST2_SEG1 0
+#define DF_BASE__INST2_SEG2 0
+#define DF_BASE__INST2_SEG3 0
+#define DF_BASE__INST2_SEG4 0
+#define DF_BASE__INST2_SEG5 0
+
+#define DF_BASE__INST3_SEG0 0
+#define DF_BASE__INST3_SEG1 0
+#define DF_BASE__INST3_SEG2 0
+#define DF_BASE__INST3_SEG3 0
+#define DF_BASE__INST3_SEG4 0
+#define DF_BASE__INST3_SEG5 0
+
+#define DF_BASE__INST4_SEG0 0
+#define DF_BASE__INST4_SEG1 0
+#define DF_BASE__INST4_SEG2 0
+#define DF_BASE__INST4_SEG3 0
+#define DF_BASE__INST4_SEG4 0
+#define DF_BASE__INST4_SEG5 0
+
+#define DF_BASE__INST5_SEG0 0
+#define DF_BASE__INST5_SEG1 0
+#define DF_BASE__INST5_SEG2 0
+#define DF_BASE__INST5_SEG3 0
+#define DF_BASE__INST5_SEG4 0
+#define DF_BASE__INST5_SEG5 0
+
+#define DF_BASE__INST6_SEG0 0
+#define DF_BASE__INST6_SEG1 0
+#define DF_BASE__INST6_SEG2 0
+#define DF_BASE__INST6_SEG3 0
+#define DF_BASE__INST6_SEG4 0
+#define DF_BASE__INST6_SEG5 0
+
+#define FUSE_BASE__INST0_SEG0 0x00017400
+#define FUSE_BASE__INST0_SEG1 0x02401400
+#define FUSE_BASE__INST0_SEG2 0
+#define FUSE_BASE__INST0_SEG3 0
+#define FUSE_BASE__INST0_SEG4 0
+#define FUSE_BASE__INST0_SEG5 0
+
+#define FUSE_BASE__INST1_SEG0 0
+#define FUSE_BASE__INST1_SEG1 0
+#define FUSE_BASE__INST1_SEG2 0
+#define FUSE_BASE__INST1_SEG3 0
+#define FUSE_BASE__INST1_SEG4 0
+#define FUSE_BASE__INST1_SEG5 0
+
+#define FUSE_BASE__INST2_SEG0 0
+#define FUSE_BASE__INST2_SEG1 0
+#define FUSE_BASE__INST2_SEG2 0
+#define FUSE_BASE__INST2_SEG3 0
+#define FUSE_BASE__INST2_SEG4 0
+#define FUSE_BASE__INST2_SEG5 0
+
+#define FUSE_BASE__INST3_SEG0 0
+#define FUSE_BASE__INST3_SEG1 0
+#define FUSE_BASE__INST3_SEG2 0
+#define FUSE_BASE__INST3_SEG3 0
+#define FUSE_BASE__INST3_SEG4 0
+#define FUSE_BASE__INST3_SEG5 0
+
+#define FUSE_BASE__INST4_SEG0 0
+#define FUSE_BASE__INST4_SEG1 0
+#define FUSE_BASE__INST4_SEG2 0
+#define FUSE_BASE__INST4_SEG3 0
+#define FUSE_BASE__INST4_SEG4 0
+#define FUSE_BASE__INST4_SEG5 0
+
+#define FUSE_BASE__INST5_SEG0 0
+#define FUSE_BASE__INST5_SEG1 0
+#define FUSE_BASE__INST5_SEG2 0
+#define FUSE_BASE__INST5_SEG3 0
+#define FUSE_BASE__INST5_SEG4 0
+#define FUSE_BASE__INST5_SEG5 0
+
+#define FUSE_BASE__INST6_SEG0 0
+#define FUSE_BASE__INST6_SEG1 0
+#define FUSE_BASE__INST6_SEG2 0
+#define FUSE_BASE__INST6_SEG3 0
+#define FUSE_BASE__INST6_SEG4 0
+#define FUSE_BASE__INST6_SEG5 0
+
+#define GC_BASE__INST0_SEG0 0x00002000
+#define GC_BASE__INST0_SEG1 0x0000A000
+#define GC_BASE__INST0_SEG2 0x02402C00
+#define GC_BASE__INST0_SEG3 0
+#define GC_BASE__INST0_SEG4 0
+#define GC_BASE__INST0_SEG5 0
+
+#define GC_BASE__INST1_SEG0 0
+#define GC_BASE__INST1_SEG1 0
+#define GC_BASE__INST1_SEG2 0
+#define GC_BASE__INST1_SEG3 0
+#define GC_BASE__INST1_SEG4 0
+#define GC_BASE__INST1_SEG5 0
+
+#define GC_BASE__INST2_SEG0 0
+#define GC_BASE__INST2_SEG1 0
+#define GC_BASE__INST2_SEG2 0
+#define GC_BASE__INST2_SEG3 0
+#define GC_BASE__INST2_SEG4 0
+#define GC_BASE__INST2_SEG5 0
+
+#define GC_BASE__INST3_SEG0 0
+#define GC_BASE__INST3_SEG1 0
+#define GC_BASE__INST3_SEG2 0
+#define GC_BASE__INST3_SEG3 0
+#define GC_BASE__INST3_SEG4 0
+#define GC_BASE__INST3_SEG5 0
+
+#define GC_BASE__INST4_SEG0 0
+#define GC_BASE__INST4_SEG1 0
+#define GC_BASE__INST4_SEG2 0
+#define GC_BASE__INST4_SEG3 0
+#define GC_BASE__INST4_SEG4 0
+#define GC_BASE__INST4_SEG5 0
+
+#define GC_BASE__INST5_SEG0 0
+#define GC_BASE__INST5_SEG1 0
+#define GC_BASE__INST5_SEG2 0
+#define GC_BASE__INST5_SEG3 0
+#define GC_BASE__INST5_SEG4 0
+#define GC_BASE__INST5_SEG5 0
+
+#define GC_BASE__INST6_SEG0 0
+#define GC_BASE__INST6_SEG1 0
+#define GC_BASE__INST6_SEG2 0
+#define GC_BASE__INST6_SEG3 0
+#define GC_BASE__INST6_SEG4 0
+#define GC_BASE__INST6_SEG5 0
+
+#define HDP_BASE__INST0_SEG0 0x00000F20
+#define HDP_BASE__INST0_SEG1 0x0240A400
+#define HDP_BASE__INST0_SEG2 0
+#define HDP_BASE__INST0_SEG3 0
+#define HDP_BASE__INST0_SEG4 0
+#define HDP_BASE__INST0_SEG5 0
+
+#define HDP_BASE__INST1_SEG0 0
+#define HDP_BASE__INST1_SEG1 0
+#define HDP_BASE__INST1_SEG2 0
+#define HDP_BASE__INST1_SEG3 0
+#define HDP_BASE__INST1_SEG4 0
+#define HDP_BASE__INST1_SEG5 0
+
+#define HDP_BASE__INST2_SEG0 0
+#define HDP_BASE__INST2_SEG1 0
+#define HDP_BASE__INST2_SEG2 0
+#define HDP_BASE__INST2_SEG3 0
+#define HDP_BASE__INST2_SEG4 0
+#define HDP_BASE__INST2_SEG5 0
+
+#define HDP_BASE__INST3_SEG0 0
+#define HDP_BASE__INST3_SEG1 0
+#define HDP_BASE__INST3_SEG2 0
+#define HDP_BASE__INST3_SEG3 0
+#define HDP_BASE__INST3_SEG4 0
+#define HDP_BASE__INST3_SEG5 0
+
+#define HDP_BASE__INST4_SEG0 0
+#define HDP_BASE__INST4_SEG1 0
+#define HDP_BASE__INST4_SEG2 0
+#define HDP_BASE__INST4_SEG3 0
+#define HDP_BASE__INST4_SEG4 0
+#define HDP_BASE__INST4_SEG5 0
+
+#define HDP_BASE__INST5_SEG0 0
+#define HDP_BASE__INST5_SEG1 0
+#define HDP_BASE__INST5_SEG2 0
+#define HDP_BASE__INST5_SEG3 0
+#define HDP_BASE__INST5_SEG4 0
+#define HDP_BASE__INST5_SEG5 0
+
+#define HDP_BASE__INST6_SEG0 0
+#define HDP_BASE__INST6_SEG1 0
+#define HDP_BASE__INST6_SEG2 0
+#define HDP_BASE__INST6_SEG3 0
+#define HDP_BASE__INST6_SEG4 0
+#define HDP_BASE__INST6_SEG5 0
+
+#define IOAGR0_BASE__INST0_SEG0 0x02419000
+#define IOAGR0_BASE__INST0_SEG1 0x056C0000
+#define IOAGR0_BASE__INST0_SEG2 0
+#define IOAGR0_BASE__INST0_SEG3 0
+#define IOAGR0_BASE__INST0_SEG4 0
+#define IOAGR0_BASE__INST0_SEG5 0
+
+#define IOAGR0_BASE__INST1_SEG0 0
+#define IOAGR0_BASE__INST1_SEG1 0
+#define IOAGR0_BASE__INST1_SEG2 0
+#define IOAGR0_BASE__INST1_SEG3 0
+#define IOAGR0_BASE__INST1_SEG4 0
+#define IOAGR0_BASE__INST1_SEG5 0
+
+#define IOAGR0_BASE__INST2_SEG0 0
+#define IOAGR0_BASE__INST2_SEG1 0
+#define IOAGR0_BASE__INST2_SEG2 0
+#define IOAGR0_BASE__INST2_SEG3 0
+#define IOAGR0_BASE__INST2_SEG4 0
+#define IOAGR0_BASE__INST2_SEG5 0
+
+#define IOAGR0_BASE__INST3_SEG0 0
+#define IOAGR0_BASE__INST3_SEG1 0
+#define IOAGR0_BASE__INST3_SEG2 0
+#define IOAGR0_BASE__INST3_SEG3 0
+#define IOAGR0_BASE__INST3_SEG4 0
+#define IOAGR0_BASE__INST3_SEG5 0
+
+#define IOAGR0_BASE__INST4_SEG0 0
+#define IOAGR0_BASE__INST4_SEG1 0
+#define IOAGR0_BASE__INST4_SEG2 0
+#define IOAGR0_BASE__INST4_SEG3 0
+#define IOAGR0_BASE__INST4_SEG4 0
+#define IOAGR0_BASE__INST4_SEG5 0
+
+#define IOAGR0_BASE__INST5_SEG0 0
+#define IOAGR0_BASE__INST5_SEG1 0
+#define IOAGR0_BASE__INST5_SEG2 0
+#define IOAGR0_BASE__INST5_SEG3 0
+#define IOAGR0_BASE__INST5_SEG4 0
+#define IOAGR0_BASE__INST5_SEG5 0
+
+#define IOAGR0_BASE__INST6_SEG0 0
+#define IOAGR0_BASE__INST6_SEG1 0
+#define IOAGR0_BASE__INST6_SEG2 0
+#define IOAGR0_BASE__INST6_SEG3 0
+#define IOAGR0_BASE__INST6_SEG4 0
+#define IOAGR0_BASE__INST6_SEG5 0
+
+#define IOAPIC0_BASE__INST0_SEG0 0x00A00000
+#define IOAPIC0_BASE__INST0_SEG1 0x0241F000
+#define IOAPIC0_BASE__INST0_SEG2 0x050C0000
+#define IOAPIC0_BASE__INST0_SEG3 0
+#define IOAPIC0_BASE__INST0_SEG4 0
+#define IOAPIC0_BASE__INST0_SEG5 0
+
+#define IOAPIC0_BASE__INST1_SEG0 0
+#define IOAPIC0_BASE__INST1_SEG1 0
+#define IOAPIC0_BASE__INST1_SEG2 0
+#define IOAPIC0_BASE__INST1_SEG3 0
+#define IOAPIC0_BASE__INST1_SEG4 0
+#define IOAPIC0_BASE__INST1_SEG5 0
+
+#define IOAPIC0_BASE__INST2_SEG0 0
+#define IOAPIC0_BASE__INST2_SEG1 0
+#define IOAPIC0_BASE__INST2_SEG2 0
+#define IOAPIC0_BASE__INST2_SEG3 0
+#define IOAPIC0_BASE__INST2_SEG4 0
+#define IOAPIC0_BASE__INST2_SEG5 0
+
+#define IOAPIC0_BASE__INST3_SEG0 0
+#define IOAPIC0_BASE__INST3_SEG1 0
+#define IOAPIC0_BASE__INST3_SEG2 0
+#define IOAPIC0_BASE__INST3_SEG3 0
+#define IOAPIC0_BASE__INST3_SEG4 0
+#define IOAPIC0_BASE__INST3_SEG5 0
+
+#define IOAPIC0_BASE__INST4_SEG0 0
+#define IOAPIC0_BASE__INST4_SEG1 0
+#define IOAPIC0_BASE__INST4_SEG2 0
+#define IOAPIC0_BASE__INST4_SEG3 0
+#define IOAPIC0_BASE__INST4_SEG4 0
+#define IOAPIC0_BASE__INST4_SEG5 0
+
+#define IOAPIC0_BASE__INST5_SEG0 0
+#define IOAPIC0_BASE__INST5_SEG1 0
+#define IOAPIC0_BASE__INST5_SEG2 0
+#define IOAPIC0_BASE__INST5_SEG3 0
+#define IOAPIC0_BASE__INST5_SEG4 0
+#define IOAPIC0_BASE__INST5_SEG5 0
+
+#define IOAPIC0_BASE__INST6_SEG0 0
+#define IOAPIC0_BASE__INST6_SEG1 0
+#define IOAPIC0_BASE__INST6_SEG2 0
+#define IOAPIC0_BASE__INST6_SEG3 0
+#define IOAPIC0_BASE__INST6_SEG4 0
+#define IOAPIC0_BASE__INST6_SEG5 0
+
+#define IOHC0_BASE__INST0_SEG0 0x00010000
+#define IOHC0_BASE__INST0_SEG1 0x02406000
+#define IOHC0_BASE__INST0_SEG2 0x04EC0000
+#define IOHC0_BASE__INST0_SEG3 0
+#define IOHC0_BASE__INST0_SEG4 0
+#define IOHC0_BASE__INST0_SEG5 0
+
+#define IOHC0_BASE__INST1_SEG0 0
+#define IOHC0_BASE__INST1_SEG1 0
+#define IOHC0_BASE__INST1_SEG2 0
+#define IOHC0_BASE__INST1_SEG3 0
+#define IOHC0_BASE__INST1_SEG4 0
+#define IOHC0_BASE__INST1_SEG5 0
+
+#define IOHC0_BASE__INST2_SEG0 0
+#define IOHC0_BASE__INST2_SEG1 0
+#define IOHC0_BASE__INST2_SEG2 0
+#define IOHC0_BASE__INST2_SEG3 0
+#define IOHC0_BASE__INST2_SEG4 0
+#define IOHC0_BASE__INST2_SEG5 0
+
+#define IOHC0_BASE__INST3_SEG0 0
+#define IOHC0_BASE__INST3_SEG1 0
+#define IOHC0_BASE__INST3_SEG2 0
+#define IOHC0_BASE__INST3_SEG3 0
+#define IOHC0_BASE__INST3_SEG4 0
+#define IOHC0_BASE__INST3_SEG5 0
+
+#define IOHC0_BASE__INST4_SEG0 0
+#define IOHC0_BASE__INST4_SEG1 0
+#define IOHC0_BASE__INST4_SEG2 0
+#define IOHC0_BASE__INST4_SEG3 0
+#define IOHC0_BASE__INST4_SEG4 0
+#define IOHC0_BASE__INST4_SEG5 0
+
+#define IOHC0_BASE__INST5_SEG0 0
+#define IOHC0_BASE__INST5_SEG1 0
+#define IOHC0_BASE__INST5_SEG2 0
+#define IOHC0_BASE__INST5_SEG3 0
+#define IOHC0_BASE__INST5_SEG4 0
+#define IOHC0_BASE__INST5_SEG5 0
+
+#define IOHC0_BASE__INST6_SEG0 0
+#define IOHC0_BASE__INST6_SEG1 0
+#define IOHC0_BASE__INST6_SEG2 0
+#define IOHC0_BASE__INST6_SEG3 0
+#define IOHC0_BASE__INST6_SEG4 0
+#define IOHC0_BASE__INST6_SEG5 0
+
+#define L1IMUIOAGR0_BASE__INST0_SEG0 0x0240CC00
+#define L1IMUIOAGR0_BASE__INST0_SEG1 0x05200000
+#define L1IMUIOAGR0_BASE__INST0_SEG2 0
+#define L1IMUIOAGR0_BASE__INST0_SEG3 0
+#define L1IMUIOAGR0_BASE__INST0_SEG4 0
+#define L1IMUIOAGR0_BASE__INST0_SEG5 0
+
+#define L1IMUIOAGR0_BASE__INST1_SEG0 0
+#define L1IMUIOAGR0_BASE__INST1_SEG1 0
+#define L1IMUIOAGR0_BASE__INST1_SEG2 0
+#define L1IMUIOAGR0_BASE__INST1_SEG3 0
+#define L1IMUIOAGR0_BASE__INST1_SEG4 0
+#define L1IMUIOAGR0_BASE__INST1_SEG5 0
+
+#define L1IMUIOAGR0_BASE__INST2_SEG0 0
+#define L1IMUIOAGR0_BASE__INST2_SEG1 0
+#define L1IMUIOAGR0_BASE__INST2_SEG2 0
+#define L1IMUIOAGR0_BASE__INST2_SEG3 0
+#define L1IMUIOAGR0_BASE__INST2_SEG4 0
+#define L1IMUIOAGR0_BASE__INST2_SEG5 0
+
+#define L1IMUIOAGR0_BASE__INST3_SEG0 0
+#define L1IMUIOAGR0_BASE__INST3_SEG1 0
+#define L1IMUIOAGR0_BASE__INST3_SEG2 0
+#define L1IMUIOAGR0_BASE__INST3_SEG3 0
+#define L1IMUIOAGR0_BASE__INST3_SEG4 0
+#define L1IMUIOAGR0_BASE__INST3_SEG5 0
+
+#define L1IMUIOAGR0_BASE__INST4_SEG0 0
+#define L1IMUIOAGR0_BASE__INST4_SEG1 0
+#define L1IMUIOAGR0_BASE__INST4_SEG2 0
+#define L1IMUIOAGR0_BASE__INST4_SEG3 0
+#define L1IMUIOAGR0_BASE__INST4_SEG4 0
+#define L1IMUIOAGR0_BASE__INST4_SEG5 0
+
+#define L1IMUIOAGR0_BASE__INST5_SEG0 0
+#define L1IMUIOAGR0_BASE__INST5_SEG1 0
+#define L1IMUIOAGR0_BASE__INST5_SEG2 0
+#define L1IMUIOAGR0_BASE__INST5_SEG3 0
+#define L1IMUIOAGR0_BASE__INST5_SEG4 0
+#define L1IMUIOAGR0_BASE__INST5_SEG5 0
+
+#define L1IMUIOAGR0_BASE__INST6_SEG0 0
+#define L1IMUIOAGR0_BASE__INST6_SEG1 0
+#define L1IMUIOAGR0_BASE__INST6_SEG2 0
+#define L1IMUIOAGR0_BASE__INST6_SEG3 0
+#define L1IMUIOAGR0_BASE__INST6_SEG4 0
+#define L1IMUIOAGR0_BASE__INST6_SEG5 0
+
+#define L1IMUPCIE0_BASE__INST0_SEG0 0x0240C800
+#define L1IMUPCIE0_BASE__INST0_SEG1 0x051C0000
+#define L1IMUPCIE0_BASE__INST0_SEG2 0
+#define L1IMUPCIE0_BASE__INST0_SEG3 0
+#define L1IMUPCIE0_BASE__INST0_SEG4 0
+#define L1IMUPCIE0_BASE__INST0_SEG5 0
+
+#define L1IMUPCIE0_BASE__INST1_SEG0 0
+#define L1IMUPCIE0_BASE__INST1_SEG1 0
+#define L1IMUPCIE0_BASE__INST1_SEG2 0
+#define L1IMUPCIE0_BASE__INST1_SEG3 0
+#define L1IMUPCIE0_BASE__INST1_SEG4 0
+#define L1IMUPCIE0_BASE__INST1_SEG5 0
+
+#define L1IMUPCIE0_BASE__INST2_SEG0 0
+#define L1IMUPCIE0_BASE__INST2_SEG1 0
+#define L1IMUPCIE0_BASE__INST2_SEG2 0
+#define L1IMUPCIE0_BASE__INST2_SEG3 0
+#define L1IMUPCIE0_BASE__INST2_SEG4 0
+#define L1IMUPCIE0_BASE__INST2_SEG5 0
+
+#define L1IMUPCIE0_BASE__INST3_SEG0 0
+#define L1IMUPCIE0_BASE__INST3_SEG1 0
+#define L1IMUPCIE0_BASE__INST3_SEG2 0
+#define L1IMUPCIE0_BASE__INST3_SEG3 0
+#define L1IMUPCIE0_BASE__INST3_SEG4 0
+#define L1IMUPCIE0_BASE__INST3_SEG5 0
+
+#define L1IMUPCIE0_BASE__INST4_SEG0 0
+#define L1IMUPCIE0_BASE__INST4_SEG1 0
+#define L1IMUPCIE0_BASE__INST4_SEG2 0
+#define L1IMUPCIE0_BASE__INST4_SEG3 0
+#define L1IMUPCIE0_BASE__INST4_SEG4 0
+#define L1IMUPCIE0_BASE__INST4_SEG5 0
+
+#define L1IMUPCIE0_BASE__INST5_SEG0 0
+#define L1IMUPCIE0_BASE__INST5_SEG1 0
+#define L1IMUPCIE0_BASE__INST5_SEG2 0
+#define L1IMUPCIE0_BASE__INST5_SEG3 0
+#define L1IMUPCIE0_BASE__INST5_SEG4 0
+#define L1IMUPCIE0_BASE__INST5_SEG5 0
+
+#define L1IMUPCIE0_BASE__INST6_SEG0 0
+#define L1IMUPCIE0_BASE__INST6_SEG1 0
+#define L1IMUPCIE0_BASE__INST6_SEG2 0
+#define L1IMUPCIE0_BASE__INST6_SEG3 0
+#define L1IMUPCIE0_BASE__INST6_SEG4 0
+#define L1IMUPCIE0_BASE__INST6_SEG5 0
+
+#define L2IMU0_BASE__INST0_SEG0 0x00007DC0
+#define L2IMU0_BASE__INST0_SEG1 0x00900000
+#define L2IMU0_BASE__INST0_SEG2 0x02407000
+#define L2IMU0_BASE__INST0_SEG3 0x04FC0000
+#define L2IMU0_BASE__INST0_SEG4 0x055C0000
+#define L2IMU0_BASE__INST0_SEG5 0
+
+#define L2IMU0_BASE__INST1_SEG0 0
+#define L2IMU0_BASE__INST1_SEG1 0
+#define L2IMU0_BASE__INST1_SEG2 0
+#define L2IMU0_BASE__INST1_SEG3 0
+#define L2IMU0_BASE__INST1_SEG4 0
+#define L2IMU0_BASE__INST1_SEG5 0
+
+#define L2IMU0_BASE__INST2_SEG0 0
+#define L2IMU0_BASE__INST2_SEG1 0
+#define L2IMU0_BASE__INST2_SEG2 0
+#define L2IMU0_BASE__INST2_SEG3 0
+#define L2IMU0_BASE__INST2_SEG4 0
+#define L2IMU0_BASE__INST2_SEG5 0
+
+#define L2IMU0_BASE__INST3_SEG0 0
+#define L2IMU0_BASE__INST3_SEG1 0
+#define L2IMU0_BASE__INST3_SEG2 0
+#define L2IMU0_BASE__INST3_SEG3 0
+#define L2IMU0_BASE__INST3_SEG4 0
+#define L2IMU0_BASE__INST3_SEG5 0
+
+#define L2IMU0_BASE__INST4_SEG0 0
+#define L2IMU0_BASE__INST4_SEG1 0
+#define L2IMU0_BASE__INST4_SEG2 0
+#define L2IMU0_BASE__INST4_SEG3 0
+#define L2IMU0_BASE__INST4_SEG4 0
+#define L2IMU0_BASE__INST4_SEG5 0
+
+#define L2IMU0_BASE__INST5_SEG0 0
+#define L2IMU0_BASE__INST5_SEG1 0
+#define L2IMU0_BASE__INST5_SEG2 0
+#define L2IMU0_BASE__INST5_SEG3 0
+#define L2IMU0_BASE__INST5_SEG4 0
+#define L2IMU0_BASE__INST5_SEG5 0
+
+#define L2IMU0_BASE__INST6_SEG0 0
+#define L2IMU0_BASE__INST6_SEG1 0
+#define L2IMU0_BASE__INST6_SEG2 0
+#define L2IMU0_BASE__INST6_SEG3 0
+#define L2IMU0_BASE__INST6_SEG4 0
+#define L2IMU0_BASE__INST6_SEG5 0
+
+#define MMHUB_BASE__INST0_SEG0 0x0001A000
+#define MMHUB_BASE__INST0_SEG1 0x02408800
+#define MMHUB_BASE__INST0_SEG2 0
+#define MMHUB_BASE__INST0_SEG3 0
+#define MMHUB_BASE__INST0_SEG4 0
+#define MMHUB_BASE__INST0_SEG5 0
+
+#define MMHUB_BASE__INST1_SEG0 0
+#define MMHUB_BASE__INST1_SEG1 0
+#define MMHUB_BASE__INST1_SEG2 0
+#define MMHUB_BASE__INST1_SEG3 0
+#define MMHUB_BASE__INST1_SEG4 0
+#define MMHUB_BASE__INST1_SEG5 0
+
+#define MMHUB_BASE__INST2_SEG0 0
+#define MMHUB_BASE__INST2_SEG1 0
+#define MMHUB_BASE__INST2_SEG2 0
+#define MMHUB_BASE__INST2_SEG3 0
+#define MMHUB_BASE__INST2_SEG4 0
+#define MMHUB_BASE__INST2_SEG5 0
+
+#define MMHUB_BASE__INST3_SEG0 0
+#define MMHUB_BASE__INST3_SEG1 0
+#define MMHUB_BASE__INST3_SEG2 0
+#define MMHUB_BASE__INST3_SEG3 0
+#define MMHUB_BASE__INST3_SEG4 0
+#define MMHUB_BASE__INST3_SEG5 0
+
+#define MMHUB_BASE__INST4_SEG0 0
+#define MMHUB_BASE__INST4_SEG1 0
+#define MMHUB_BASE__INST4_SEG2 0
+#define MMHUB_BASE__INST4_SEG3 0
+#define MMHUB_BASE__INST4_SEG4 0
+#define MMHUB_BASE__INST4_SEG5 0
+
+#define MMHUB_BASE__INST5_SEG0 0
+#define MMHUB_BASE__INST5_SEG1 0
+#define MMHUB_BASE__INST5_SEG2 0
+#define MMHUB_BASE__INST5_SEG3 0
+#define MMHUB_BASE__INST5_SEG4 0
+#define MMHUB_BASE__INST5_SEG5 0
+
+#define MMHUB_BASE__INST6_SEG0 0
+#define MMHUB_BASE__INST6_SEG1 0
+#define MMHUB_BASE__INST6_SEG2 0
+#define MMHUB_BASE__INST6_SEG3 0
+#define MMHUB_BASE__INST6_SEG4 0
+#define MMHUB_BASE__INST6_SEG5 0
+
+#define MP0_BASE__INST0_SEG0 0x00016000
+#define MP0_BASE__INST0_SEG1 0x00DC0000
+#define MP0_BASE__INST0_SEG2 0x00E00000
+#define MP0_BASE__INST0_SEG3 0x00E40000
+#define MP0_BASE__INST0_SEG4 0x0243FC00
+#define MP0_BASE__INST0_SEG5 0
+
+#define MP0_BASE__INST1_SEG0 0
+#define MP0_BASE__INST1_SEG1 0
+#define MP0_BASE__INST1_SEG2 0
+#define MP0_BASE__INST1_SEG3 0
+#define MP0_BASE__INST1_SEG4 0
+#define MP0_BASE__INST1_SEG5 0
+
+#define MP0_BASE__INST2_SEG0 0
+#define MP0_BASE__INST2_SEG1 0
+#define MP0_BASE__INST2_SEG2 0
+#define MP0_BASE__INST2_SEG3 0
+#define MP0_BASE__INST2_SEG4 0
+#define MP0_BASE__INST2_SEG5 0
+
+#define MP0_BASE__INST3_SEG0 0
+#define MP0_BASE__INST3_SEG1 0
+#define MP0_BASE__INST3_SEG2 0
+#define MP0_BASE__INST3_SEG3 0
+#define MP0_BASE__INST3_SEG4 0
+#define MP0_BASE__INST3_SEG5 0
+
+#define MP0_BASE__INST4_SEG0 0
+#define MP0_BASE__INST4_SEG1 0
+#define MP0_BASE__INST4_SEG2 0
+#define MP0_BASE__INST4_SEG3 0
+#define MP0_BASE__INST4_SEG4 0
+#define MP0_BASE__INST4_SEG5 0
+
+#define MP0_BASE__INST5_SEG0 0
+#define MP0_BASE__INST5_SEG1 0
+#define MP0_BASE__INST5_SEG2 0
+#define MP0_BASE__INST5_SEG3 0
+#define MP0_BASE__INST5_SEG4 0
+#define MP0_BASE__INST5_SEG5 0
+
+#define MP0_BASE__INST6_SEG0 0
+#define MP0_BASE__INST6_SEG1 0
+#define MP0_BASE__INST6_SEG2 0
+#define MP0_BASE__INST6_SEG3 0
+#define MP0_BASE__INST6_SEG4 0
+#define MP0_BASE__INST6_SEG5 0
+
+#define MP1_BASE__INST0_SEG0 0x00016000
+#define MP1_BASE__INST0_SEG1 0x00DC0000
+#define MP1_BASE__INST0_SEG2 0x00E00000
+#define MP1_BASE__INST0_SEG3 0x00E40000
+#define MP1_BASE__INST0_SEG4 0x0243FC00
+#define MP1_BASE__INST0_SEG5 0
+
+#define MP1_BASE__INST1_SEG0 0
+#define MP1_BASE__INST1_SEG1 0
+#define MP1_BASE__INST1_SEG2 0
+#define MP1_BASE__INST1_SEG3 0
+#define MP1_BASE__INST1_SEG4 0
+#define MP1_BASE__INST1_SEG5 0
+
+#define MP1_BASE__INST2_SEG0 0
+#define MP1_BASE__INST2_SEG1 0
+#define MP1_BASE__INST2_SEG2 0
+#define MP1_BASE__INST2_SEG3 0
+#define MP1_BASE__INST2_SEG4 0
+#define MP1_BASE__INST2_SEG5 0
+
+#define MP1_BASE__INST3_SEG0 0
+#define MP1_BASE__INST3_SEG1 0
+#define MP1_BASE__INST3_SEG2 0
+#define MP1_BASE__INST3_SEG3 0
+#define MP1_BASE__INST3_SEG4 0
+#define MP1_BASE__INST3_SEG5 0
+
+#define MP1_BASE__INST4_SEG0 0
+#define MP1_BASE__INST4_SEG1 0
+#define MP1_BASE__INST4_SEG2 0
+#define MP1_BASE__INST4_SEG3 0
+#define MP1_BASE__INST4_SEG4 0
+#define MP1_BASE__INST4_SEG5 0
+
+#define MP1_BASE__INST5_SEG0 0
+#define MP1_BASE__INST5_SEG1 0
+#define MP1_BASE__INST5_SEG2 0
+#define MP1_BASE__INST5_SEG3 0
+#define MP1_BASE__INST5_SEG4 0
+#define MP1_BASE__INST5_SEG5 0
+
+#define MP1_BASE__INST6_SEG0 0
+#define MP1_BASE__INST6_SEG1 0
+#define MP1_BASE__INST6_SEG2 0
+#define MP1_BASE__INST6_SEG3 0
+#define MP1_BASE__INST6_SEG4 0
+#define MP1_BASE__INST6_SEG5 0
+
+#define NBIO_BASE__INST0_SEG0 0x00000000
+#define NBIO_BASE__INST0_SEG1 0x00000014
+#define NBIO_BASE__INST0_SEG2 0x00000D20
+#define NBIO_BASE__INST0_SEG3 0x00010400
+#define NBIO_BASE__INST0_SEG4 0x0241B000
+#define NBIO_BASE__INST0_SEG5 0x04040000
+
+#define NBIO_BASE__INST1_SEG0 0
+#define NBIO_BASE__INST1_SEG1 0
+#define NBIO_BASE__INST1_SEG2 0
+#define NBIO_BASE__INST1_SEG3 0
+#define NBIO_BASE__INST1_SEG4 0
+#define NBIO_BASE__INST1_SEG5 0
+
+#define NBIO_BASE__INST2_SEG0 0
+#define NBIO_BASE__INST2_SEG1 0
+#define NBIO_BASE__INST2_SEG2 0
+#define NBIO_BASE__INST2_SEG3 0
+#define NBIO_BASE__INST2_SEG4 0
+#define NBIO_BASE__INST2_SEG5 0
+
+#define NBIO_BASE__INST3_SEG0 0
+#define NBIO_BASE__INST3_SEG1 0
+#define NBIO_BASE__INST3_SEG2 0
+#define NBIO_BASE__INST3_SEG3 0
+#define NBIO_BASE__INST3_SEG4 0
+#define NBIO_BASE__INST3_SEG5 0
+
+#define NBIO_BASE__INST4_SEG0 0
+#define NBIO_BASE__INST4_SEG1 0
+#define NBIO_BASE__INST4_SEG2 0
+#define NBIO_BASE__INST4_SEG3 0
+#define NBIO_BASE__INST4_SEG4 0
+#define NBIO_BASE__INST4_SEG5 0
+
+#define NBIO_BASE__INST5_SEG0 0
+#define NBIO_BASE__INST5_SEG1 0
+#define NBIO_BASE__INST5_SEG2 0
+#define NBIO_BASE__INST5_SEG3 0
+#define NBIO_BASE__INST5_SEG4 0
+#define NBIO_BASE__INST5_SEG5 0
+
+#define NBIO_BASE__INST6_SEG0 0
+#define NBIO_BASE__INST6_SEG1 0
+#define NBIO_BASE__INST6_SEG2 0
+#define NBIO_BASE__INST6_SEG3 0
+#define NBIO_BASE__INST6_SEG4 0
+#define NBIO_BASE__INST6_SEG5 0
+
+#define OSSSYS_BASE__INST0_SEG0 0x000010A0
+#define OSSSYS_BASE__INST0_SEG1 0x0240A000
+#define OSSSYS_BASE__INST0_SEG2 0
+#define OSSSYS_BASE__INST0_SEG3 0
+#define OSSSYS_BASE__INST0_SEG4 0
+#define OSSSYS_BASE__INST0_SEG5 0
+
+#define OSSSYS_BASE__INST1_SEG0 0
+#define OSSSYS_BASE__INST1_SEG1 0
+#define OSSSYS_BASE__INST1_SEG2 0
+#define OSSSYS_BASE__INST1_SEG3 0
+#define OSSSYS_BASE__INST1_SEG4 0
+#define OSSSYS_BASE__INST1_SEG5 0
+
+#define OSSSYS_BASE__INST2_SEG0 0
+#define OSSSYS_BASE__INST2_SEG1 0
+#define OSSSYS_BASE__INST2_SEG2 0
+#define OSSSYS_BASE__INST2_SEG3 0
+#define OSSSYS_BASE__INST2_SEG4 0
+#define OSSSYS_BASE__INST2_SEG5 0
+
+#define OSSSYS_BASE__INST3_SEG0 0
+#define OSSSYS_BASE__INST3_SEG1 0
+#define OSSSYS_BASE__INST3_SEG2 0
+#define OSSSYS_BASE__INST3_SEG3 0
+#define OSSSYS_BASE__INST3_SEG4 0
+#define OSSSYS_BASE__INST3_SEG5 0
+
+#define OSSSYS_BASE__INST4_SEG0 0
+#define OSSSYS_BASE__INST4_SEG1 0
+#define OSSSYS_BASE__INST4_SEG2 0
+#define OSSSYS_BASE__INST4_SEG3 0
+#define OSSSYS_BASE__INST4_SEG4 0
+#define OSSSYS_BASE__INST4_SEG5 0
+
+#define OSSSYS_BASE__INST5_SEG0 0
+#define OSSSYS_BASE__INST5_SEG1 0
+#define OSSSYS_BASE__INST5_SEG2 0
+#define OSSSYS_BASE__INST5_SEG3 0
+#define OSSSYS_BASE__INST5_SEG4 0
+#define OSSSYS_BASE__INST5_SEG5 0
+
+#define OSSSYS_BASE__INST6_SEG0 0
+#define OSSSYS_BASE__INST6_SEG1 0
+#define OSSSYS_BASE__INST6_SEG2 0
+#define OSSSYS_BASE__INST6_SEG3 0
+#define OSSSYS_BASE__INST6_SEG4 0
+#define OSSSYS_BASE__INST6_SEG5 0
+
+#define PCIE0_BASE__INST0_SEG0 0x02411800
+#define PCIE0_BASE__INST0_SEG1 0x04440000
+#define PCIE0_BASE__INST0_SEG2 0
+#define PCIE0_BASE__INST0_SEG3 0
+#define PCIE0_BASE__INST0_SEG4 0
+#define PCIE0_BASE__INST0_SEG5 0
+
+#define PCIE0_BASE__INST1_SEG0 0
+#define PCIE0_BASE__INST1_SEG1 0
+#define PCIE0_BASE__INST1_SEG2 0
+#define PCIE0_BASE__INST1_SEG3 0
+#define PCIE0_BASE__INST1_SEG4 0
+#define PCIE0_BASE__INST1_SEG5 0
+
+#define PCIE0_BASE__INST2_SEG0 0
+#define PCIE0_BASE__INST2_SEG1 0
+#define PCIE0_BASE__INST2_SEG2 0
+#define PCIE0_BASE__INST2_SEG3 0
+#define PCIE0_BASE__INST2_SEG4 0
+#define PCIE0_BASE__INST2_SEG5 0
+
+#define PCIE0_BASE__INST3_SEG0 0
+#define PCIE0_BASE__INST3_SEG1 0
+#define PCIE0_BASE__INST3_SEG2 0
+#define PCIE0_BASE__INST3_SEG3 0
+#define PCIE0_BASE__INST3_SEG4 0
+#define PCIE0_BASE__INST3_SEG5 0
+
+#define PCIE0_BASE__INST4_SEG0 0
+#define PCIE0_BASE__INST4_SEG1 0
+#define PCIE0_BASE__INST4_SEG2 0
+#define PCIE0_BASE__INST4_SEG3 0
+#define PCIE0_BASE__INST4_SEG4 0
+#define PCIE0_BASE__INST4_SEG5 0
+
+#define PCIE0_BASE__INST5_SEG0 0
+#define PCIE0_BASE__INST5_SEG1 0
+#define PCIE0_BASE__INST5_SEG2 0
+#define PCIE0_BASE__INST5_SEG3 0
+#define PCIE0_BASE__INST5_SEG4 0
+#define PCIE0_BASE__INST5_SEG5 0
+
+#define PCIE0_BASE__INST6_SEG0 0
+#define PCIE0_BASE__INST6_SEG1 0
+#define PCIE0_BASE__INST6_SEG2 0
+#define PCIE0_BASE__INST6_SEG3 0
+#define PCIE0_BASE__INST6_SEG4 0
+#define PCIE0_BASE__INST6_SEG5 0
+
+#define SDMA0_BASE__INST0_SEG0 0x00001260
+#define SDMA0_BASE__INST0_SEG1 0x02445400
+#define SDMA0_BASE__INST0_SEG2 0
+#define SDMA0_BASE__INST0_SEG3 0
+#define SDMA0_BASE__INST0_SEG4 0
+#define SDMA0_BASE__INST0_SEG5 0
+
+#define SDMA0_BASE__INST1_SEG0 0
+#define SDMA0_BASE__INST1_SEG1 0
+#define SDMA0_BASE__INST1_SEG2 0
+#define SDMA0_BASE__INST1_SEG3 0
+#define SDMA0_BASE__INST1_SEG4 0
+#define SDMA0_BASE__INST1_SEG5 0
+
+#define SDMA0_BASE__INST2_SEG0 0
+#define SDMA0_BASE__INST2_SEG1 0
+#define SDMA0_BASE__INST2_SEG2 0
+#define SDMA0_BASE__INST2_SEG3 0
+#define SDMA0_BASE__INST2_SEG4 0
+#define SDMA0_BASE__INST2_SEG5 0
+
+#define SDMA0_BASE__INST3_SEG0 0
+#define SDMA0_BASE__INST3_SEG1 0
+#define SDMA0_BASE__INST3_SEG2 0
+#define SDMA0_BASE__INST3_SEG3 0
+#define SDMA0_BASE__INST3_SEG4 0
+#define SDMA0_BASE__INST3_SEG5 0
+
+#define SDMA0_BASE__INST4_SEG0 0
+#define SDMA0_BASE__INST4_SEG1 0
+#define SDMA0_BASE__INST4_SEG2 0
+#define SDMA0_BASE__INST4_SEG3 0
+#define SDMA0_BASE__INST4_SEG4 0
+#define SDMA0_BASE__INST4_SEG5 0
+
+#define SDMA0_BASE__INST5_SEG0 0
+#define SDMA0_BASE__INST5_SEG1 0
+#define SDMA0_BASE__INST5_SEG2 0
+#define SDMA0_BASE__INST5_SEG3 0
+#define SDMA0_BASE__INST5_SEG4 0
+#define SDMA0_BASE__INST5_SEG5 0
+
+#define SDMA0_BASE__INST6_SEG0 0
+#define SDMA0_BASE__INST6_SEG1 0
+#define SDMA0_BASE__INST6_SEG2 0
+#define SDMA0_BASE__INST6_SEG3 0
+#define SDMA0_BASE__INST6_SEG4 0
+#define SDMA0_BASE__INST6_SEG5 0
+
+#define SDMA1_BASE__INST0_SEG0 0x00001860
+#define SDMA1_BASE__INST0_SEG1 0x02445800
+#define SDMA1_BASE__INST0_SEG2 0
+#define SDMA1_BASE__INST0_SEG3 0
+#define SDMA1_BASE__INST0_SEG4 0
+#define SDMA1_BASE__INST0_SEG5 0
+
+#define SDMA1_BASE__INST1_SEG0 0x0001E000
+#define SDMA1_BASE__INST1_SEG1 0x02446400
+#define SDMA1_BASE__INST1_SEG2 0
+#define SDMA1_BASE__INST1_SEG3 0
+#define SDMA1_BASE__INST1_SEG4 0
+#define SDMA1_BASE__INST1_SEG5 0
+
+#define SDMA1_BASE__INST2_SEG0 0x0001E400
+#define SDMA1_BASE__INST2_SEG1 0x02446800
+#define SDMA1_BASE__INST2_SEG2 0
+#define SDMA1_BASE__INST2_SEG3 0
+#define SDMA1_BASE__INST2_SEG4 0
+#define SDMA1_BASE__INST2_SEG5 0
+
+#define SDMA1_BASE__INST3_SEG0 0x0001E800
+#define SDMA1_BASE__INST3_SEG1 0x02446C00
+#define SDMA1_BASE__INST3_SEG2 0
+#define SDMA1_BASE__INST3_SEG3 0
+#define SDMA1_BASE__INST3_SEG4 0
+#define SDMA1_BASE__INST3_SEG5 0
+
+#define SDMA1_BASE__INST4_SEG0 0
+#define SDMA1_BASE__INST4_SEG1 0
+#define SDMA1_BASE__INST4_SEG2 0
+#define SDMA1_BASE__INST4_SEG3 0
+#define SDMA1_BASE__INST4_SEG4 0
+#define SDMA1_BASE__INST4_SEG5 0
+
+#define SDMA1_BASE__INST5_SEG0 0
+#define SDMA1_BASE__INST5_SEG1 0
+#define SDMA1_BASE__INST5_SEG2 0
+#define SDMA1_BASE__INST5_SEG3 0
+#define SDMA1_BASE__INST5_SEG4 0
+#define SDMA1_BASE__INST5_SEG5 0
+
+#define SDMA1_BASE__INST6_SEG0 0
+#define SDMA1_BASE__INST6_SEG1 0
+#define SDMA1_BASE__INST6_SEG2 0
+#define SDMA1_BASE__INST6_SEG3 0
+#define SDMA1_BASE__INST6_SEG4 0
+#define SDMA1_BASE__INST6_SEG5 0
+
+#define SMUIO_BASE__INST0_SEG0 0x00016800
+#define SMUIO_BASE__INST0_SEG1 0x00016A00
+#define SMUIO_BASE__INST0_SEG2 0x02401000
+#define SMUIO_BASE__INST0_SEG3 0x03440000
+#define SMUIO_BASE__INST0_SEG4 0
+#define SMUIO_BASE__INST0_SEG5 0
+
+#define SMUIO_BASE__INST1_SEG0 0
+#define SMUIO_BASE__INST1_SEG1 0
+#define SMUIO_BASE__INST1_SEG2 0
+#define SMUIO_BASE__INST1_SEG3 0
+#define SMUIO_BASE__INST1_SEG4 0
+#define SMUIO_BASE__INST1_SEG5 0
+
+#define SMUIO_BASE__INST2_SEG0 0
+#define SMUIO_BASE__INST2_SEG1 0
+#define SMUIO_BASE__INST2_SEG2 0
+#define SMUIO_BASE__INST2_SEG3 0
+#define SMUIO_BASE__INST2_SEG4 0
+#define SMUIO_BASE__INST2_SEG5 0
+
+#define SMUIO_BASE__INST3_SEG0 0
+#define SMUIO_BASE__INST3_SEG1 0
+#define SMUIO_BASE__INST3_SEG2 0
+#define SMUIO_BASE__INST3_SEG3 0
+#define SMUIO_BASE__INST3_SEG4 0
+#define SMUIO_BASE__INST3_SEG5 0
+
+#define SMUIO_BASE__INST4_SEG0 0
+#define SMUIO_BASE__INST4_SEG1 0
+#define SMUIO_BASE__INST4_SEG2 0
+#define SMUIO_BASE__INST4_SEG3 0
+#define SMUIO_BASE__INST4_SEG4 0
+#define SMUIO_BASE__INST4_SEG5 0
+
+#define SMUIO_BASE__INST5_SEG0 0
+#define SMUIO_BASE__INST5_SEG1 0
+#define SMUIO_BASE__INST5_SEG2 0
+#define SMUIO_BASE__INST5_SEG3 0
+#define SMUIO_BASE__INST5_SEG4 0
+#define SMUIO_BASE__INST5_SEG5 0
+
+#define SMUIO_BASE__INST6_SEG0 0
+#define SMUIO_BASE__INST6_SEG1 0
+#define SMUIO_BASE__INST6_SEG2 0
+#define SMUIO_BASE__INST6_SEG3 0
+#define SMUIO_BASE__INST6_SEG4 0
+#define SMUIO_BASE__INST6_SEG5 0
+
+#define THM_BASE__INST0_SEG0 0x00016600
+#define THM_BASE__INST0_SEG1 0x02400C00
+#define THM_BASE__INST0_SEG2 0
+#define THM_BASE__INST0_SEG3 0
+#define THM_BASE__INST0_SEG4 0
+#define THM_BASE__INST0_SEG5 0
+
+#define THM_BASE__INST1_SEG0 0
+#define THM_BASE__INST1_SEG1 0
+#define THM_BASE__INST1_SEG2 0
+#define THM_BASE__INST1_SEG3 0
+#define THM_BASE__INST1_SEG4 0
+#define THM_BASE__INST1_SEG5 0
+
+#define THM_BASE__INST2_SEG0 0
+#define THM_BASE__INST2_SEG1 0
+#define THM_BASE__INST2_SEG2 0
+#define THM_BASE__INST2_SEG3 0
+#define THM_BASE__INST2_SEG4 0
+#define THM_BASE__INST2_SEG5 0
+
+#define THM_BASE__INST3_SEG0 0
+#define THM_BASE__INST3_SEG1 0
+#define THM_BASE__INST3_SEG2 0
+#define THM_BASE__INST3_SEG3 0
+#define THM_BASE__INST3_SEG4 0
+#define THM_BASE__INST3_SEG5 0
+
+#define THM_BASE__INST4_SEG0 0
+#define THM_BASE__INST4_SEG1 0
+#define THM_BASE__INST4_SEG2 0
+#define THM_BASE__INST4_SEG3 0
+#define THM_BASE__INST4_SEG4 0
+#define THM_BASE__INST4_SEG5 0
+
+#define THM_BASE__INST5_SEG0 0
+#define THM_BASE__INST5_SEG1 0
+#define THM_BASE__INST5_SEG2 0
+#define THM_BASE__INST5_SEG3 0
+#define THM_BASE__INST5_SEG4 0
+#define THM_BASE__INST5_SEG5 0
+
+#define THM_BASE__INST6_SEG0 0
+#define THM_BASE__INST6_SEG1 0
+#define THM_BASE__INST6_SEG2 0
+#define THM_BASE__INST6_SEG3 0
+#define THM_BASE__INST6_SEG4 0
+#define THM_BASE__INST6_SEG5 0
+
+#define UMC_BASE__INST0_SEG0 0x00014000
+#define UMC_BASE__INST0_SEG1 0x00054000
+#define UMC_BASE__INST0_SEG2 0x02425800
+#define UMC_BASE__INST0_SEG3 0
+#define UMC_BASE__INST0_SEG4 0
+#define UMC_BASE__INST0_SEG5 0
+
+#define UMC_BASE__INST1_SEG0 0x00094000
+#define UMC_BASE__INST1_SEG1 0x000D4000
+#define UMC_BASE__INST1_SEG2 0x02425C00
+#define UMC_BASE__INST1_SEG3 0
+#define UMC_BASE__INST1_SEG4 0
+#define UMC_BASE__INST1_SEG5 0
+
+#define UMC_BASE__INST2_SEG0 0x00114000
+#define UMC_BASE__INST2_SEG1 0x00154000
+#define UMC_BASE__INST2_SEG2 0x02426000
+#define UMC_BASE__INST2_SEG3 0
+#define UMC_BASE__INST2_SEG4 0
+#define UMC_BASE__INST2_SEG5 0
+
+#define UMC_BASE__INST3_SEG0 0x00194000
+#define UMC_BASE__INST3_SEG1 0x001D4000
+#define UMC_BASE__INST3_SEG2 0x02426400
+#define UMC_BASE__INST3_SEG3 0
+#define UMC_BASE__INST3_SEG4 0
+#define UMC_BASE__INST3_SEG5 0
+
+#define UMC_BASE__INST4_SEG0 0
+#define UMC_BASE__INST4_SEG1 0
+#define UMC_BASE__INST4_SEG2 0
+#define UMC_BASE__INST4_SEG3 0
+#define UMC_BASE__INST4_SEG4 0
+#define UMC_BASE__INST4_SEG5 0
+
+#define UMC_BASE__INST5_SEG0 0
+#define UMC_BASE__INST5_SEG1 0
+#define UMC_BASE__INST5_SEG2 0
+#define UMC_BASE__INST5_SEG3 0
+#define UMC_BASE__INST5_SEG4 0
+#define UMC_BASE__INST5_SEG5 0
+
+#define UMC_BASE__INST6_SEG0 0
+#define UMC_BASE__INST6_SEG1 0
+#define UMC_BASE__INST6_SEG2 0
+#define UMC_BASE__INST6_SEG3 0
+#define UMC_BASE__INST6_SEG4 0
+#define UMC_BASE__INST6_SEG5 0
+
+#define VCN_BASE__INST0_SEG0 0x00007800
+#define VCN_BASE__INST0_SEG1 0x00007E00
+#define VCN_BASE__INST0_SEG2 0x02403000
+#define VCN_BASE__INST0_SEG3 0
+#define VCN_BASE__INST0_SEG4 0
+#define VCN_BASE__INST0_SEG5 0
+
+#define VCN_BASE__INST1_SEG0 0x00007A00
+#define VCN_BASE__INST1_SEG1 0x00009000
+#define VCN_BASE__INST1_SEG2 0x02445000
+#define VCN_BASE__INST1_SEG3 0
+#define VCN_BASE__INST1_SEG4 0
+#define VCN_BASE__INST1_SEG5 0
+
+#define VCN_BASE__INST2_SEG0 0
+#define VCN_BASE__INST2_SEG1 0
+#define VCN_BASE__INST2_SEG2 0
+#define VCN_BASE__INST2_SEG3 0
+#define VCN_BASE__INST2_SEG4 0
+#define VCN_BASE__INST2_SEG5 0
+
+#define VCN_BASE__INST3_SEG0 0
+#define VCN_BASE__INST3_SEG1 0
+#define VCN_BASE__INST3_SEG2 0
+#define VCN_BASE__INST3_SEG3 0
+#define VCN_BASE__INST3_SEG4 0
+#define VCN_BASE__INST3_SEG5 0
+
+#define VCN_BASE__INST4_SEG0 0
+#define VCN_BASE__INST4_SEG1 0
+#define VCN_BASE__INST4_SEG2 0
+#define VCN_BASE__INST4_SEG3 0
+#define VCN_BASE__INST4_SEG4 0
+#define VCN_BASE__INST4_SEG5 0
+
+#define VCN_BASE__INST5_SEG0 0
+#define VCN_BASE__INST5_SEG1 0
+#define VCN_BASE__INST5_SEG2 0
+#define VCN_BASE__INST5_SEG3 0
+#define VCN_BASE__INST5_SEG4 0
+#define VCN_BASE__INST5_SEG5 0
+
+#define VCN_BASE__INST6_SEG0 0
+#define VCN_BASE__INST6_SEG1 0
+#define VCN_BASE__INST6_SEG2 0
+#define VCN_BASE__INST6_SEG3 0
+#define VCN_BASE__INST6_SEG4 0
+#define VCN_BASE__INST6_SEG5 0
+
+#define WAFL0_BASE__INST0_SEG0 0x02438000
+#define WAFL0_BASE__INST0_SEG1 0x04880000
+#define WAFL0_BASE__INST0_SEG2 0
+#define WAFL0_BASE__INST0_SEG3 0
+#define WAFL0_BASE__INST0_SEG4 0
+#define WAFL0_BASE__INST0_SEG5 0
+
+#define WAFL0_BASE__INST1_SEG0 0
+#define WAFL0_BASE__INST1_SEG1 0
+#define WAFL0_BASE__INST1_SEG2 0
+#define WAFL0_BASE__INST1_SEG3 0
+#define WAFL0_BASE__INST1_SEG4 0
+#define WAFL0_BASE__INST1_SEG5 0
+
+#define WAFL0_BASE__INST2_SEG0 0
+#define WAFL0_BASE__INST2_SEG1 0
+#define WAFL0_BASE__INST2_SEG2 0
+#define WAFL0_BASE__INST2_SEG3 0
+#define WAFL0_BASE__INST2_SEG4 0
+#define WAFL0_BASE__INST2_SEG5 0
+
+#define WAFL0_BASE__INST3_SEG0 0
+#define WAFL0_BASE__INST3_SEG1 0
+#define WAFL0_BASE__INST3_SEG2 0
+#define WAFL0_BASE__INST3_SEG3 0
+#define WAFL0_BASE__INST3_SEG4 0
+#define WAFL0_BASE__INST3_SEG5 0
+
+#define WAFL0_BASE__INST4_SEG0 0
+#define WAFL0_BASE__INST4_SEG1 0
+#define WAFL0_BASE__INST4_SEG2 0
+#define WAFL0_BASE__INST4_SEG3 0
+#define WAFL0_BASE__INST4_SEG4 0
+#define WAFL0_BASE__INST4_SEG5 0
+
+#define WAFL0_BASE__INST5_SEG0 0
+#define WAFL0_BASE__INST5_SEG1 0
+#define WAFL0_BASE__INST5_SEG2 0
+#define WAFL0_BASE__INST5_SEG3 0
+#define WAFL0_BASE__INST5_SEG4 0
+#define WAFL0_BASE__INST5_SEG5 0
+
+#define WAFL0_BASE__INST6_SEG0 0
+#define WAFL0_BASE__INST6_SEG1 0
+#define WAFL0_BASE__INST6_SEG2 0
+#define WAFL0_BASE__INST6_SEG3 0
+#define WAFL0_BASE__INST6_SEG4 0
+#define WAFL0_BASE__INST6_SEG5 0
+
+#define WAFL1_BASE__INST0_SEG0 0
+#define WAFL1_BASE__INST0_SEG1 0x01300000
+#define WAFL1_BASE__INST0_SEG2 0x02410800
+#define WAFL1_BASE__INST0_SEG3 0
+#define WAFL1_BASE__INST0_SEG4 0
+#define WAFL1_BASE__INST0_SEG5 0
+
+#define WAFL1_BASE__INST1_SEG0 0
+#define WAFL1_BASE__INST1_SEG1 0
+#define WAFL1_BASE__INST1_SEG2 0
+#define WAFL1_BASE__INST1_SEG3 0
+#define WAFL1_BASE__INST1_SEG4 0
+#define WAFL1_BASE__INST1_SEG5 0
+
+#define WAFL1_BASE__INST2_SEG0 0
+#define WAFL1_BASE__INST2_SEG1 0
+#define WAFL1_BASE__INST2_SEG2 0
+#define WAFL1_BASE__INST2_SEG3 0
+#define WAFL1_BASE__INST2_SEG4 0
+#define WAFL1_BASE__INST2_SEG5 0
+
+#define WAFL1_BASE__INST3_SEG0 0
+#define WAFL1_BASE__INST3_SEG1 0
+#define WAFL1_BASE__INST3_SEG2 0
+#define WAFL1_BASE__INST3_SEG3 0
+#define WAFL1_BASE__INST3_SEG4 0
+#define WAFL1_BASE__INST3_SEG5 0
+
+#define WAFL1_BASE__INST4_SEG0 0
+#define WAFL1_BASE__INST4_SEG1 0
+#define WAFL1_BASE__INST4_SEG2 0
+#define WAFL1_BASE__INST4_SEG3 0
+#define WAFL1_BASE__INST4_SEG4 0
+#define WAFL1_BASE__INST4_SEG5 0
+
+#define WAFL1_BASE__INST5_SEG0 0
+#define WAFL1_BASE__INST5_SEG1 0
+#define WAFL1_BASE__INST5_SEG2 0
+#define WAFL1_BASE__INST5_SEG3 0
+#define WAFL1_BASE__INST5_SEG4 0
+#define WAFL1_BASE__INST5_SEG5 0
+
+#define WAFL1_BASE__INST6_SEG0 0
+#define WAFL1_BASE__INST6_SEG1 0
+#define WAFL1_BASE__INST6_SEG2 0
+#define WAFL1_BASE__INST6_SEG3 0
+#define WAFL1_BASE__INST6_SEG4 0
+#define WAFL1_BASE__INST6_SEG5 0
+
+#define XGMI0_BASE__INST0_SEG0 0x02438C00
+#define XGMI0_BASE__INST0_SEG1 0x04680000
+#define XGMI0_BASE__INST0_SEG2 0x04940000
+#define XGMI0_BASE__INST0_SEG3 0
+#define XGMI0_BASE__INST0_SEG4 0
+#define XGMI0_BASE__INST0_SEG5 0
+
+#define XGMI0_BASE__INST1_SEG0 0
+#define XGMI0_BASE__INST1_SEG1 0
+#define XGMI0_BASE__INST1_SEG2 0
+#define XGMI0_BASE__INST1_SEG3 0
+#define XGMI0_BASE__INST1_SEG4 0
+#define XGMI0_BASE__INST1_SEG5 0
+
+#define XGMI0_BASE__INST2_SEG0 0
+#define XGMI0_BASE__INST2_SEG1 0
+#define XGMI0_BASE__INST2_SEG2 0
+#define XGMI0_BASE__INST2_SEG3 0
+#define XGMI0_BASE__INST2_SEG4 0
+#define XGMI0_BASE__INST2_SEG5 0
+
+#define XGMI0_BASE__INST3_SEG0 0
+#define XGMI0_BASE__INST3_SEG1 0
+#define XGMI0_BASE__INST3_SEG2 0
+#define XGMI0_BASE__INST3_SEG3 0
+#define XGMI0_BASE__INST3_SEG4 0
+#define XGMI0_BASE__INST3_SEG5 0
+
+#define XGMI0_BASE__INST4_SEG0 0
+#define XGMI0_BASE__INST4_SEG1 0
+#define XGMI0_BASE__INST4_SEG2 0
+#define XGMI0_BASE__INST4_SEG3 0
+#define XGMI0_BASE__INST4_SEG4 0
+#define XGMI0_BASE__INST4_SEG5 0
+
+#define XGMI0_BASE__INST5_SEG0 0
+#define XGMI0_BASE__INST5_SEG1 0
+#define XGMI0_BASE__INST5_SEG2 0
+#define XGMI0_BASE__INST5_SEG3 0
+#define XGMI0_BASE__INST5_SEG4 0
+#define XGMI0_BASE__INST5_SEG5 0
+
+#define XGMI0_BASE__INST6_SEG0 0
+#define XGMI0_BASE__INST6_SEG1 0
+#define XGMI0_BASE__INST6_SEG2 0
+#define XGMI0_BASE__INST6_SEG3 0
+#define XGMI0_BASE__INST6_SEG4 0
+#define XGMI0_BASE__INST6_SEG5 0
+
+#define XGMI1_BASE__INST0_SEG0 0x02439000
+#define XGMI1_BASE__INST0_SEG1 0x046C0000
+#define XGMI1_BASE__INST0_SEG2 0x04980000
+#define XGMI1_BASE__INST0_SEG3 0
+#define XGMI1_BASE__INST0_SEG4 0
+#define XGMI1_BASE__INST0_SEG5 0
+
+#define XGMI1_BASE__INST1_SEG0 0
+#define XGMI1_BASE__INST1_SEG1 0
+#define XGMI1_BASE__INST1_SEG2 0
+#define XGMI1_BASE__INST1_SEG3 0
+#define XGMI1_BASE__INST1_SEG4 0
+#define XGMI1_BASE__INST1_SEG5 0
+
+#define XGMI1_BASE__INST2_SEG0 0
+#define XGMI1_BASE__INST2_SEG1 0
+#define XGMI1_BASE__INST2_SEG2 0
+#define XGMI1_BASE__INST2_SEG3 0
+#define XGMI1_BASE__INST2_SEG4 0
+#define XGMI1_BASE__INST2_SEG5 0
+
+#define XGMI1_BASE__INST3_SEG0 0
+#define XGMI1_BASE__INST3_SEG1 0
+#define XGMI1_BASE__INST3_SEG2 0
+#define XGMI1_BASE__INST3_SEG3 0
+#define XGMI1_BASE__INST3_SEG4 0
+#define XGMI1_BASE__INST3_SEG5 0
+
+#define XGMI1_BASE__INST4_SEG0 0
+#define XGMI1_BASE__INST4_SEG1 0
+#define XGMI1_BASE__INST4_SEG2 0
+#define XGMI1_BASE__INST4_SEG3 0
+#define XGMI1_BASE__INST4_SEG4 0
+#define XGMI1_BASE__INST4_SEG5 0
+
+#define XGMI1_BASE__INST5_SEG0 0
+#define XGMI1_BASE__INST5_SEG1 0
+#define XGMI1_BASE__INST5_SEG2 0
+#define XGMI1_BASE__INST5_SEG3 0
+#define XGMI1_BASE__INST5_SEG4 0
+#define XGMI1_BASE__INST5_SEG5 0
+
+#define XGMI1_BASE__INST6_SEG0 0
+#define XGMI1_BASE__INST6_SEG1 0
+#define XGMI1_BASE__INST6_SEG2 0
+#define XGMI1_BASE__INST6_SEG3 0
+#define XGMI1_BASE__INST6_SEG4 0
+#define XGMI1_BASE__INST6_SEG5 0
+
+#define XGMI2_BASE__INST0_SEG0 0x04700000
+#define XGMI2_BASE__INST0_SEG1 0x049C0000
+#define XGMI2_BASE__INST0_SEG2 0
+#define XGMI2_BASE__INST0_SEG3 0
+#define XGMI2_BASE__INST0_SEG4 0
+#define XGMI2_BASE__INST0_SEG5 0
+
+#define XGMI2_BASE__INST1_SEG0 0x04740000
+#define XGMI2_BASE__INST1_SEG1 0x04A00000
+#define XGMI2_BASE__INST1_SEG2 0
+#define XGMI2_BASE__INST1_SEG3 0
+#define XGMI2_BASE__INST1_SEG4 0
+#define XGMI2_BASE__INST1_SEG5 0
+
+#define XGMI2_BASE__INST2_SEG0 0x04780000
+#define XGMI2_BASE__INST2_SEG1 0x04A40000
+#define XGMI2_BASE__INST2_SEG2 0
+#define XGMI2_BASE__INST2_SEG3 0
+#define XGMI2_BASE__INST2_SEG4 0
+#define XGMI2_BASE__INST2_SEG5 0
+
+#define XGMI2_BASE__INST3_SEG0 0x047C0000
+#define XGMI2_BASE__INST3_SEG1 0x04A80000
+#define XGMI2_BASE__INST3_SEG2 0
+#define XGMI2_BASE__INST3_SEG3 0
+#define XGMI2_BASE__INST3_SEG4 0
+#define XGMI2_BASE__INST3_SEG5 0
+
+#define XGMI2_BASE__INST4_SEG0 0x04800000
+#define XGMI2_BASE__INST4_SEG1 0x04AC0000
+#define XGMI2_BASE__INST4_SEG2 0
+#define XGMI2_BASE__INST4_SEG3 0
+#define XGMI2_BASE__INST4_SEG4 0
+#define XGMI2_BASE__INST4_SEG5 0
+
+#define XGMI2_BASE__INST5_SEG0 0x04840000
+#define XGMI2_BASE__INST5_SEG1 0x04B00000
+#define XGMI2_BASE__INST5_SEG2 0
+#define XGMI2_BASE__INST5_SEG3 0
+#define XGMI2_BASE__INST5_SEG4 0
+#define XGMI2_BASE__INST5_SEG5 0
+
+#define XGMI2_BASE__INST6_SEG0 0
+#define XGMI2_BASE__INST6_SEG1 0
+#define XGMI2_BASE__INST6_SEG2 0
+#define XGMI2_BASE__INST6_SEG3 0
+#define XGMI2_BASE__INST6_SEG4 0
+#define XGMI2_BASE__INST6_SEG5 0
+
+#endif
diff --git a/drivers/gpu/drm/amd/include/amd_shared.h b/drivers/gpu/drm/amd/include/amd_shared.h
index 43ed6291b2b8..9ab706cd07ff 100644
--- a/drivers/gpu/drm/amd/include/amd_shared.h
+++ b/drivers/gpu/drm/amd/include/amd_shared.h
@@ -216,6 +216,12 @@ enum PP_FEATURE_MASK {
PP_GFX_DCS_MASK = 0x80000,
};
+enum amd_harvest_ip_mask {
+ AMD_HARVEST_IP_VCN_MASK = 0x1,
+ AMD_HARVEST_IP_JPEG_MASK = 0x2,
+ AMD_HARVEST_IP_DMU_MASK = 0x4,
+};
+
enum DC_FEATURE_MASK {
DC_FBC_MASK = 0x1,
DC_MULTI_MON_PP_MCLK_SWITCH_MASK = 0x2,
diff --git a/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h
index cf166b591bc5..483769fb1736 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/dcn/dcn_3_0_0_offset.h
@@ -8922,7 +8922,7 @@
#define mmOTG1_OTG_CONTROL 0x1bc1
#define mmOTG1_OTG_CONTROL_BASE_IDX 2
#define mmOTG1_OTG_BLANK_CONTROL 0x1bc2
-#define mmOTG1_OTG_BLANK_CONTROL_BASE_IDX
+#define mmOTG1_OTG_BLANK_CONTROL_BASE_IDX 2
#define mmOTG1_OTG_INTERLACE_CONTROL 0x1bc4
#define mmOTG1_OTG_INTERLACE_CONTROL_BASE_IDX 2
#define mmOTG1_OTG_INTERLACE_STATUS 0x1bc5
diff --git a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_offset.h b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_offset.h
index 0102487a2c5f..f21554a1c86c 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_offset.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_offset.h
@@ -6955,6 +6955,12 @@
#define mmCP_CE_IB2_BASE_HI_BASE_IDX 1
#define mmCP_CE_IB2_BUFSZ 0x20cb
#define mmCP_CE_IB2_BUFSZ_BASE_IDX 1
+#define mmCP_IB1_BASE_LO 0x20cc
+#define mmCP_IB1_BASE_LO_BASE_IDX 1
+#define mmCP_IB1_BASE_HI 0x20cd
+#define mmCP_IB1_BASE_HI_BASE_IDX 1
+#define mmCP_IB1_BUFSZ 0x20ce
+#define mmCP_IB1_BUFSZ_BASE_IDX 1
#define mmCP_IB2_BASE_LO 0x20cf
#define mmCP_IB2_BASE_LO_BASE_IDX 1
#define mmCP_IB2_BASE_HI 0x20d0
diff --git a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_sh_mask.h
index 4d2a1432c121..a827b0ff8905 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_sh_mask.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_10_3_0_sh_mask.h
@@ -25818,6 +25818,15 @@
//CP_CE_IB2_BUFSZ
#define CP_CE_IB2_BUFSZ__IB2_BUFSZ__SHIFT 0x0
#define CP_CE_IB2_BUFSZ__IB2_BUFSZ_MASK 0x000FFFFFL
+//CP_IB1_BASE_LO
+#define CP_IB1_BASE_LO__IB1_BASE_LO__SHIFT 0x2
+#define CP_IB1_BASE_LO__IB1_BASE_LO_MASK 0xFFFFFFFCL
+//CP_IB1_BASE_HI
+#define CP_IB1_BASE_HI__IB1_BASE_HI__SHIFT 0x0
+#define CP_IB1_BASE_HI__IB1_BASE_HI_MASK 0x0000FFFFL
+//CP_IB1_BUFSZ
+#define CP_IB1_BUFSZ__IB1_BUFSZ__SHIFT 0x0
+#define CP_IB1_BUFSZ__IB1_BUFSZ_MASK 0x000FFFFFL
//CP_IB2_BASE_LO
#define CP_IB2_BASE_LO__IB2_BASE_LO__SHIFT 0x2
#define CP_IB2_BASE_LO__IB2_BASE_LO_MASK 0xFFFFFFFCL
diff --git a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_1_sh_mask.h b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_1_sh_mask.h
index 4089cfa081f5..849450caca15 100644
--- a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_1_sh_mask.h
+++ b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_1_sh_mask.h
@@ -617,6 +617,22 @@
#define GCEA_EDC_CNT3__MAM_A3MEM_SEC_COUNT_MASK 0x30000000L
#define GCEA_EDC_CNT3__MAM_A3MEM_DED_COUNT_MASK 0xC0000000L
+//GCEA_ERR_STATUS
+#define GCEA_ERR_STATUS__SDP_RDRSP_STATUS__SHIFT 0x0
+#define GCEA_ERR_STATUS__SDP_WRRSP_STATUS__SHIFT 0x4
+#define GCEA_ERR_STATUS__SDP_RDRSP_DATASTATUS__SHIFT 0x8
+#define GCEA_ERR_STATUS__SDP_RDRSP_DATAPARITY_ERROR__SHIFT 0xa
+#define GCEA_ERR_STATUS__CLEAR_ERROR_STATUS__SHIFT 0xb
+#define GCEA_ERR_STATUS__BUSY_ON_ERROR__SHIFT 0xc
+#define GCEA_ERR_STATUS__FUE_FLAG__SHIFT 0xd
+#define GCEA_ERR_STATUS__SDP_RDRSP_STATUS_MASK 0x0000000FL
+#define GCEA_ERR_STATUS__SDP_WRRSP_STATUS_MASK 0x000000F0L
+#define GCEA_ERR_STATUS__SDP_RDRSP_DATASTATUS_MASK 0x00000300L
+#define GCEA_ERR_STATUS__SDP_RDRSP_DATAPARITY_ERROR_MASK 0x00000400L
+#define GCEA_ERR_STATUS__CLEAR_ERROR_STATUS_MASK 0x00000800L
+#define GCEA_ERR_STATUS__BUSY_ON_ERROR_MASK 0x00001000L
+#define GCEA_ERR_STATUS__FUE_FLAG_MASK 0x00002000L
+
// addressBlock: gc_gfxudec
//GRBM_GFX_INDEX
#define GRBM_GFX_INDEX__INSTANCE_INDEX__SHIFT 0x0
diff --git a/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_2_offset.h b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_2_offset.h
new file mode 100644
index 000000000000..a9ad00e017a5
--- /dev/null
+++ b/drivers/gpu/drm/amd/include/asic_reg/gc/gc_9_4_2_offset.h
@@ -0,0 +1,7683 @@
+/*
+ * Copyright 2020 Advanced Micro Devices, Inc.
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE COPYRIGHT HOLDER(S) OR AUTHOR(S) BE LIABLE FOR ANY CLAIM, DAMAGES OR
+ * OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
+ * ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
+ * OTHER DEALINGS IN THE SOFTWARE.
+ *
+ */
+#ifndef _gc_9_4_2_OFFSET_HEADER
+#define _gc_9_4_2_OFFSET_HEADER
+
+
+
+// addressBlock: didtind
+// base address: 0x0
+#define ixDIDT_SQ_CTRL0 0x0000
+#define ixDIDT_SQ_CTRL2 0x0002
+#define ixDIDT_SQ_STALL_CTRL 0x0004
+#define ixDIDT_SQ_TUNING_CTRL 0x0005
+#define ixDIDT_SQ_STALL_AUTO_RELEASE_CTRL 0x0006
+#define ixDIDT_SQ_CTRL3 0x0007
+#define ixDIDT_SQ_STALL_PATTERN_1_2 0x0008
+#define ixDIDT_SQ_STALL_PATTERN_3_4 0x0009
+#define ixDIDT_SQ_STALL_PATTERN_5_6 0x000a
+#define ixDIDT_SQ_STALL_PATTERN_7 0x000b
+#define ixDIDT_SQ_MPD_SCALE_FACTOR 0x000c
+#define ixDIDT_SQ_THROTTLE_CNTL0 0x000d
+#define ixDIDT_SQ_THROTTLE_CNTL1 0x000e
+#define ixDIDT_SQ_THROTTLE_CNTL_STATUS 0x000f
+#define ixDIDT_SQ_WEIGHT0_3 0x0010
+#define ixDIDT_SQ_WEIGHT4_7 0x0011
+#define ixDIDT_SQ_WEIGHT8_11 0x0012
+#define ixDIDT_SQ_EDC_CTRL 0x0013
+#define ixDIDT_SQ_THROTTLE_CTRL 0x0014
+#define ixDIDT_SQ_EDC_STALL_PATTERN_1_2 0x0015
+#define ixDIDT_SQ_EDC_STALL_PATTERN_3_4 0x0016
+#define ixDIDT_SQ_EDC_STALL_PATTERN_5_6 0x0017
+#define ixDIDT_SQ_EDC_STALL_PATTERN_7 0x0018
+#define ixDIDT_SQ_EDC_STATUS 0x0019
+#define ixDIDT_SQ_EDC_STALL_DELAY_1 0x001a
+#define ixDIDT_SQ_EDC_STALL_DELAY_2 0x001b
+#define ixDIDT_SQ_EDC_STALL_DELAY_3 0x001c
+#define ixDIDT_SQ_EDC_STALL_DELAY_4 0x001d
+#define ixDIDT_SQ_EDC_OVERFLOW 0x001e
+#define ixDIDT_SQ_EDC_ROLLING_POWER_DELTA 0x001f
+#define ixDIDT_DB_CTRL0 0x0020
+#define ixDIDT_DB_CTRL2 0x0022
+#define ixDIDT_DB_STALL_CTRL 0x0024
+#define ixDIDT_DB_TUNING_CTRL 0x0025
+#define ixDIDT_DB_STALL_AUTO_RELEASE_CTRL 0x0026
+#define ixDIDT_DB_CTRL3 0x0027
+#define ixDIDT_DB_STALL_PATTERN_1_2 0x0028
+#define ixDIDT_DB_STALL_PATTERN_3_4 0x0029
+#define ixDIDT_DB_STALL_PATTERN_5_6 0x002a
+#define ixDIDT_DB_STALL_PATTERN_7 0x002b
+#define ixDIDT_DB_MPD_SCALE_FACTOR 0x002c
+#define ixDIDT_DB_THROTTLE_CNTL0 0x002d
+#define ixDIDT_DB_THROTTLE_CNTL1 0x002e
+#define ixDIDT_DB_THROTTLE_CNTL_STATUS 0x002f
+#define ixDIDT_DB_WEIGHT0_3 0x0030
+#define ixDIDT_DB_WEIGHT4_7 0x0031
+#define ixDIDT_DB_WEIGHT8_11 0x0032
+#define ixDIDT_DB_EDC_CTRL 0x0033
+#define ixDIDT_DB_THROTTLE_CTRL 0x0034
+#define ixDIDT_DB_EDC_STALL_PATTERN_1_2 0x0035
+#define ixDIDT_DB_EDC_STALL_PATTERN_3_4 0x0036
+#define ixDIDT_DB_EDC_STALL_PATTERN_5_6 0x0037
+#define ixDIDT_DB_EDC_STALL_PATTERN_7 0x0038
+#define ixDIDT_DB_EDC_STATUS 0x0039
+#define ixDIDT_DB_EDC_STALL_DELAY_1 0x003a
+#define ixDIDT_DB_EDC_OVERFLOW 0x003e
+#define ixDIDT_DB_EDC_ROLLING_POWER_DELTA 0x003f
+#define ixDIDT_TD_CTRL0 0x0040
+#define ixDIDT_TD_CTRL2 0x0042
+#define ixDIDT_TD_STALL_CTRL 0x0044
+#define ixDIDT_TD_TUNING_CTRL 0x0045
+#define ixDIDT_TD_STALL_AUTO_RELEASE_CTRL 0x0046
+#define ixDIDT_TD_CTRL3 0x0047
+#define ixDIDT_TD_STALL_PATTERN_1_2 0x0048
+#define ixDIDT_TD_STALL_PATTERN_3_4 0x0049
+#define ixDIDT_TD_STALL_PATTERN_5_6 0x004a
+#define ixDIDT_TD_STALL_PATTERN_7 0x004b
+#define ixDIDT_TD_MPD_SCALE_FACTOR 0x004c
+#define ixDIDT_TD_THROTTLE_CNTL0 0x004d
+#define ixDIDT_TD_THROTTLE_CNTL1 0x004e
+#define ixDIDT_TD_THROTTLE_CNTL_STATUS 0x004f
+#define ixDIDT_TD_WEIGHT0_3 0x0050
+#define ixDIDT_TD_WEIGHT4_7 0x0051
+#define ixDIDT_TD_WEIGHT8_11 0x0052
+#define ixDIDT_TD_EDC_CTRL 0x0053
+#define ixDIDT_TD_THROTTLE_CTRL 0x0054
+#define ixDIDT_TD_EDC_STALL_PATTERN_1_2 0x0055
+#define ixDIDT_TD_EDC_STALL_PATTERN_3_4 0x0056
+#define ixDIDT_TD_EDC_STALL_PATTERN_5_6 0x0057
+#define ixDIDT_TD_EDC_STALL_PATTERN_7 0x0058
+#define ixDIDT_TD_EDC_STATUS 0x0059
+#define ixDIDT_TD_EDC_STALL_DELAY_1 0x005a
+#define ixDIDT_TD_EDC_STALL_DELAY_2 0x005b
+#define ixDIDT_TD_EDC_STALL_DELAY_3 0x005c
+#define ixDIDT_TD_EDC_STALL_DELAY_4 0x005d
+#define ixDIDT_TD_EDC_OVERFLOW 0x005e
+#define ixDIDT_TD_EDC_ROLLING_POWER_DELTA 0x005f
+#define ixDIDT_TCP_CTRL0 0x0060
+#define ixDIDT_TCP_CTRL2 0x0062
+#define ixDIDT_TCP_STALL_CTRL 0x0064
+#define ixDIDT_TCP_TUNING_CTRL 0x0065
+#define ixDIDT_TCP_STALL_AUTO_RELEASE_CTRL 0x0066
+#define ixDIDT_TCP_CTRL3 0x0067
+#define ixDIDT_TCP_STALL_PATTERN_1_2 0x0068
+#define ixDIDT_TCP_STALL_PATTERN_3_4 0x0069
+#define ixDIDT_TCP_STALL_PATTERN_5_6 0x006a
+#define ixDIDT_TCP_STALL_PATTERN_7 0x006b
+#define ixDIDT_TCP_MPD_SCALE_FACTOR 0x006c
+#define ixDIDT_TCP_THROTTLE_CNTL0 0x006d
+#define ixDIDT_TCP_THROTTLE_CNTL1 0x006e
+#define ixDIDT_TCP_THROTTLE_CNTL_STATUS 0x006f
+#define ixDIDT_TCP_WEIGHT0_3 0x0070
+#define ixDIDT_TCP_WEIGHT4_7 0x0071
+#define ixDIDT_TCP_WEIGHT8_11 0x0072
+#define ixDIDT_TCP_EDC_CTRL 0x0073
+#define ixDIDT_TCP_THROTTLE_CTRL 0x0074
+#define ixDIDT_TCP_EDC_STALL_PATTERN_1_2 0x0075
+#define ixDIDT_TCP_EDC_STALL_PATTERN_3_4 0x0076
+#define ixDIDT_TCP_EDC_STALL_PATTERN_5_6 0x0077
+#define ixDIDT_TCP_EDC_STALL_PATTERN_7 0x0078
+#define ixDIDT_TCP_EDC_STATUS 0x0079
+#define ixDIDT_TCP_EDC_STALL_DELAY_1 0x007a
+#define ixDIDT_TCP_EDC_STALL_DELAY_2 0x007b
+#define ixDIDT_TCP_EDC_STALL_DELAY_3 0x007c
+#define ixDIDT_TCP_EDC_STALL_DELAY_4 0x007d
+#define ixDIDT_TCP_EDC_OVERFLOW 0x007e
+#define ixDIDT_TCP_EDC_ROLLING_POWER_DELTA 0x007f
+#define ixDIDT_SQ_STALL_EVENT_COUNTER 0x00a0
+#define ixDIDT_DB_STALL_EVENT_COUNTER 0x00a1
+#define ixDIDT_TD_STALL_EVENT_COUNTER 0x00a2
+#define ixDIDT_TCP_STALL_EVENT_COUNTER 0x00a3
+#define ixDIDT_DBR_STALL_EVENT_COUNTER 0x00a4
+#define ixDIDT_SQ_EDC_PCC_PERF_COUNTER 0x00a5
+#define ixDIDT_TD_EDC_PCC_PERF_COUNTER 0x00a6
+#define ixDIDT_TCP_EDC_PCC_PERF_COUNTER 0x00a7
+#define ixDIDT_DB_EDC_PCC_PERF_COUNTER 0x00a8
+#define ixDIDT_DBR_EDC_PCC_PERF_COUNTER 0x00a9
+#define ixDIDT_SQ_CTRL1 0x00b0
+#define ixDIDT_SQ_EDC_THRESHOLD 0x00b1
+#define ixDIDT_DB_CTRL1 0x00b2
+#define ixDIDT_DB_EDC_THRESHOLD 0x00b3
+#define ixDIDT_TD_CTRL1 0x00b4
+#define ixDIDT_TD_EDC_THRESHOLD 0x00b5
+#define ixDIDT_TCP_CTRL1 0x00b6
+#define ixDIDT_TCP_EDC_THRESHOLD 0x00b7
+
+
+// addressBlock: gc_cpdec
+// base address: 0x8200
+#define regCP_CPC_STATUS 0x0084
+#define regCP_CPC_STATUS_BASE_IDX 0
+#define regCP_CPC_BUSY_STAT 0x0085
+#define regCP_CPC_BUSY_STAT_BASE_IDX 0
+#define regCP_CPC_STALLED_STAT1 0x0086
+#define regCP_CPC_STALLED_STAT1_BASE_IDX 0
+#define regCP_CPF_STATUS 0x0087
+#define regCP_CPF_STATUS_BASE_IDX 0
+#define regCP_CPF_BUSY_STAT 0x0088
+#define regCP_CPF_BUSY_STAT_BASE_IDX 0
+#define regCP_CPF_STALLED_STAT1 0x0089
+#define regCP_CPF_STALLED_STAT1_BASE_IDX 0
+#define regCP_CPC_GRBM_FREE_COUNT 0x008b
+#define regCP_CPC_GRBM_FREE_COUNT_BASE_IDX 0
+#define regCP_CPC_PRIV_VIOLATION_ADDR 0x008c
+#define regCP_CPC_PRIV_VIOLATION_ADDR_BASE_IDX 0
+#define regCP_MEC_CNTL 0x008d
+#define regCP_MEC_CNTL_BASE_IDX 0
+#define regCP_MEC_ME1_HEADER_DUMP 0x008e
+#define regCP_MEC_ME1_HEADER_DUMP_BASE_IDX 0
+#define regCP_MEC_ME2_HEADER_DUMP 0x008f
+#define regCP_MEC_ME2_HEADER_DUMP_BASE_IDX 0
+#define regCP_CPC_SCRATCH_INDEX 0x0090
+#define regCP_CPC_SCRATCH_INDEX_BASE_IDX 0
+#define regCP_CPC_SCRATCH_DATA 0x0091
+#define regCP_CPC_SCRATCH_DATA_BASE_IDX 0
+#define regCP_CPF_GRBM_FREE_COUNT 0x0092
+#define regCP_CPF_GRBM_FREE_COUNT_BASE_IDX 0
+#define regCP_CPC_HALT_HYST_COUNT 0x00a7
+#define regCP_CPC_HALT_HYST_COUNT_BASE_IDX 0
+#define regCP_CE_COMPARE_COUNT 0x00c0
+#define regCP_CE_COMPARE_COUNT_BASE_IDX 0
+#define regCP_CE_DE_COUNT 0x00c1
+#define regCP_CE_DE_COUNT_BASE_IDX 0
+#define regCP_DE_CE_COUNT 0x00c2
+#define regCP_DE_CE_COUNT_BASE_IDX 0
+#define regCP_DE_LAST_INVAL_COUNT 0x00c3
+#define regCP_DE_LAST_INVAL_COUNT_BASE_IDX 0
+#define regCP_DE_DE_COUNT 0x00c4
+#define regCP_DE_DE_COUNT_BASE_IDX 0
+#define regCP_STALLED_STAT3 0x019c
+#define regCP_STALLED_STAT3_BASE_IDX 0
+#define regCP_STALLED_STAT1 0x019d
+#define regCP_STALLED_STAT1_BASE_IDX 0
+#define regCP_STALLED_STAT2 0x019e
+#define regCP_STALLED_STAT2_BASE_IDX 0
+#define regCP_BUSY_STAT 0x019f
+#define regCP_BUSY_STAT_BASE_IDX 0
+#define regCP_STAT 0x01a0
+#define regCP_STAT_BASE_IDX 0
+#define regCP_ME_HEADER_DUMP 0x01a1
+#define regCP_ME_HEADER_DUMP_BASE_IDX 0
+#define regCP_PFP_HEADER_DUMP 0x01a2
+#define regCP_PFP_HEADER_DUMP_BASE_IDX 0
+#define regCP_GRBM_FREE_COUNT 0x01a3
+#define regCP_GRBM_FREE_COUNT_BASE_IDX 0
+#define regCP_CE_HEADER_DUMP 0x01a4
+#define regCP_CE_HEADER_DUMP_BASE_IDX 0
+#define regCP_PFP_INSTR_PNTR 0x01a5
+#define regCP_PFP_INSTR_PNTR_BASE_IDX 0
+#define regCP_ME_INSTR_PNTR 0x01a6
+#define regCP_ME_INSTR_PNTR_BASE_IDX 0
+#define regCP_CE_INSTR_PNTR 0x01a7
+#define regCP_CE_INSTR_PNTR_BASE_IDX 0
+#define regCP_MEC1_INSTR_PNTR 0x01a8
+#define regCP_MEC1_INSTR_PNTR_BASE_IDX 0
+#define regCP_MEC2_INSTR_PNTR 0x01a9
+#define regCP_MEC2_INSTR_PNTR_BASE_IDX 0
+#define regCP_CSF_STAT 0x01b4
+#define regCP_CSF_STAT_BASE_IDX 0
+#define regCP_ME_CNTL 0x01b6
+#define regCP_ME_CNTL_BASE_IDX 0
+#define regCP_CNTX_STAT 0x01b8
+#define regCP_CNTX_STAT_BASE_IDX 0
+#define regCP_ME_PREEMPTION 0x01b9
+#define regCP_ME_PREEMPTION_BASE_IDX 0
+#define regCP_ROQ_THRESHOLDS 0x01bc
+#define regCP_ROQ_THRESHOLDS_BASE_IDX 0
+#define regCP_MEQ_STQ_THRESHOLD 0x01bd
+#define regCP_MEQ_STQ_THRESHOLD_BASE_IDX 0
+#define regCP_RB2_RPTR 0x01be
+#define regCP_RB2_RPTR_BASE_IDX 0
+#define regCP_RB1_RPTR 0x01bf
+#define regCP_RB1_RPTR_BASE_IDX 0
+#define regCP_RB0_RPTR 0x01c0
+#define regCP_RB0_RPTR_BASE_IDX 0
+#define regCP_RB_RPTR 0x01c0
+#define regCP_RB_RPTR_BASE_IDX 0
+#define regCP_RB_WPTR_DELAY 0x01c1
+#define regCP_RB_WPTR_DELAY_BASE_IDX 0
+#define regCP_RB_WPTR_POLL_CNTL 0x01c2
+#define regCP_RB_WPTR_POLL_CNTL_BASE_IDX 0
+#define regCP_ROQ1_THRESHOLDS 0x01d5
+#define regCP_ROQ1_THRESHOLDS_BASE_IDX 0
+#define regCP_ROQ2_THRESHOLDS 0x01d6
+#define regCP_ROQ2_THRESHOLDS_BASE_IDX 0
+#define regCP_STQ_THRESHOLDS 0x01d7
+#define regCP_STQ_THRESHOLDS_BASE_IDX 0
+#define regCP_QUEUE_THRESHOLDS 0x01d8
+#define regCP_QUEUE_THRESHOLDS_BASE_IDX 0
+#define regCP_MEQ_THRESHOLDS 0x01d9
+#define regCP_MEQ_THRESHOLDS_BASE_IDX 0
+#define regCP_ROQ_AVAIL 0x01da
+#define regCP_ROQ_AVAIL_BASE_IDX 0
+#define regCP_STQ_AVAIL 0x01db
+#define regCP_STQ_AVAIL_BASE_IDX 0
+#define regCP_ROQ2_AVAIL 0x01dc
+#define regCP_ROQ2_AVAIL_BASE_IDX 0
+#define regCP_MEQ_AVAIL 0x01dd
+#define regCP_MEQ_AVAIL_BASE_IDX 0
+#define regCP_CMD_INDEX 0x01de
+#define regCP_CMD_INDEX_BASE_IDX 0
+#define regCP_CMD_DATA 0x01df
+#define regCP_CMD_DATA_BASE_IDX 0
+#define regCP_ROQ_RB_STAT 0x01e0
+#define regCP_ROQ_RB_STAT_BASE_IDX 0
+#define regCP_ROQ_IB1_STAT 0x01e1
+#define regCP_ROQ_IB1_STAT_BASE_IDX 0
+#define regCP_ROQ_IB2_STAT 0x01e2
+#define regCP_ROQ_IB2_STAT_BASE_IDX 0
+#define regCP_STQ_STAT 0x01e3
+#define regCP_STQ_STAT_BASE_IDX 0
+#define regCP_STQ_WR_STAT 0x01e4
+#define regCP_STQ_WR_STAT_BASE_IDX 0
+#define regCP_MEQ_STAT 0x01e5
+#define regCP_MEQ_STAT_BASE_IDX 0
+#define regCP_CEQ1_AVAIL 0x01e6
+#define regCP_CEQ1_AVAIL_BASE_IDX 0
+#define regCP_CEQ2_AVAIL 0x01e7
+#define regCP_CEQ2_AVAIL_BASE_IDX 0
+#define regCP_CE_ROQ_RB_STAT 0x01e8
+#define regCP_CE_ROQ_RB_STAT_BASE_IDX 0
+#define regCP_CE_ROQ_IB1_STAT 0x01e9
+#define regCP_CE_ROQ_IB1_STAT_BASE_IDX 0
+#define regCP_CE_ROQ_IB2_STAT 0x01ea
+#define regCP_CE_ROQ_IB2_STAT_BASE_IDX 0
+#define regCP_PRIV_VIOLATION_ADDR 0x01fa
+#define regCP_PRIV_VIOLATION_ADDR_BASE_IDX 0
+
+
+// addressBlock: gc_cppdec
+// base address: 0xc080
+#define regCP_EOPQ_WAIT_TIME 0x1035
+#define regCP_EOPQ_WAIT_TIME_BASE_IDX 0
+#define regCP_CPC_MGCG_SYNC_CNTL 0x1036
+#define regCP_CPC_MGCG_SYNC_CNTL_BASE_IDX 0
+#define regCPC_INT_INFO 0x1037
+#define regCPC_INT_INFO_BASE_IDX 0
+#define regCP_VIRT_STATUS 0x1038
+#define regCP_VIRT_STATUS_BASE_IDX 0
+#define regCPC_INT_ADDR 0x1039
+#define regCPC_INT_ADDR_BASE_IDX 0
+#define regCPC_INT_PASID 0x103a
+#define regCPC_INT_PASID_BASE_IDX 0
+#define regCP_GFX_ERROR 0x103b
+#define regCP_GFX_ERROR_BASE_IDX 0
+#define regCPG_UTCL1_CNTL 0x103c
+#define regCPG_UTCL1_CNTL_BASE_IDX 0
+#define regCPC_UTCL1_CNTL 0x103d
+#define regCPC_UTCL1_CNTL_BASE_IDX 0
+#define regCPF_UTCL1_CNTL 0x103e
+#define regCPF_UTCL1_CNTL_BASE_IDX 0
+#define regCP_AQL_SMM_STATUS 0x103f
+#define regCP_AQL_SMM_STATUS_BASE_IDX 0
+#define regCP_RB0_BASE 0x1040
+#define regCP_RB0_BASE_BASE_IDX 0
+#define regCP_RB_BASE 0x1040
+#define regCP_RB_BASE_BASE_IDX 0
+#define regCP_RB0_CNTL 0x1041
+#define regCP_RB0_CNTL_BASE_IDX 0
+#define regCP_RB_CNTL 0x1041
+#define regCP_RB_CNTL_BASE_IDX 0
+#define regCP_RB_RPTR_WR 0x1042
+#define regCP_RB_RPTR_WR_BASE_IDX 0
+#define regCP_RB0_RPTR_ADDR 0x1043
+#define regCP_RB0_RPTR_ADDR_BASE_IDX 0
+#define regCP_RB_RPTR_ADDR 0x1043
+#define regCP_RB_RPTR_ADDR_BASE_IDX 0
+#define regCP_RB0_RPTR_ADDR_HI 0x1044
+#define regCP_RB0_RPTR_ADDR_HI_BASE_IDX 0
+#define regCP_RB_RPTR_ADDR_HI 0x1044
+#define regCP_RB_RPTR_ADDR_HI_BASE_IDX 0
+#define regCP_RB0_BUFSZ_MASK 0x1045
+#define regCP_RB0_BUFSZ_MASK_BASE_IDX 0
+#define regCP_RB_BUFSZ_MASK 0x1045
+#define regCP_RB_BUFSZ_MASK_BASE_IDX 0
+#define regCP_RB_WPTR_POLL_ADDR_LO 0x1046
+#define regCP_RB_WPTR_POLL_ADDR_LO_BASE_IDX 0
+#define regCP_RB_WPTR_POLL_ADDR_HI 0x1047
+#define regCP_RB_WPTR_POLL_ADDR_HI_BASE_IDX 0
+#define regCP_INT_CNTL 0x1049
+#define regCP_INT_CNTL_BASE_IDX 0
+#define regCP_INT_STATUS 0x104a
+#define regCP_INT_STATUS_BASE_IDX 0
+#define regCP_DEVICE_ID 0x104b
+#define regCP_DEVICE_ID_BASE_IDX 0
+#define regCP_ME0_PIPE_PRIORITY_CNTS 0x104c
+#define regCP_ME0_PIPE_PRIORITY_CNTS_BASE_IDX 0
+#define regCP_RING_PRIORITY_CNTS 0x104c
+#define regCP_RING_PRIORITY_CNTS_BASE_IDX 0
+#define regCP_ME0_PIPE0_PRIORITY 0x104d
+#define regCP_ME0_PIPE0_PRIORITY_BASE_IDX 0
+#define regCP_RING0_PRIORITY 0x104d
+#define regCP_RING0_PRIORITY_BASE_IDX 0
+#define regCP_ME0_PIPE1_PRIORITY 0x104e
+#define regCP_ME0_PIPE1_PRIORITY_BASE_IDX 0
+#define regCP_RING1_PRIORITY 0x104e
+#define regCP_RING1_PRIORITY_BASE_IDX 0
+#define regCP_ME0_PIPE2_PRIORITY 0x104f
+#define regCP_ME0_PIPE2_PRIORITY_BASE_IDX 0
+#define regCP_RING2_PRIORITY 0x104f
+#define regCP_RING2_PRIORITY_BASE_IDX 0
+#define regCP_FATAL_ERROR 0x1050
+#define regCP_FATAL_ERROR_BASE_IDX 0
+#define regCP_RB_VMID 0x1051
+#define regCP_RB_VMID_BASE_IDX 0
+#define regCP_ME0_PIPE0_VMID 0x1052
+#define regCP_ME0_PIPE0_VMID_BASE_IDX 0
+#define regCP_ME0_PIPE1_VMID 0x1053
+#define regCP_ME0_PIPE1_VMID_BASE_IDX 0
+#define regCP_RB0_WPTR 0x1054
+#define regCP_RB0_WPTR_BASE_IDX 0
+#define regCP_RB_WPTR 0x1054
+#define regCP_RB_WPTR_BASE_IDX 0
+#define regCP_RB0_WPTR_HI 0x1055
+#define regCP_RB0_WPTR_HI_BASE_IDX 0
+#define regCP_RB_WPTR_HI 0x1055
+#define regCP_RB_WPTR_HI_BASE_IDX 0
+#define regCP_RB1_WPTR 0x1056
+#define regCP_RB1_WPTR_BASE_IDX 0
+#define regCP_RB1_WPTR_HI 0x1057
+#define regCP_RB1_WPTR_HI_BASE_IDX 0
+#define regCP_RB2_WPTR 0x1058
+#define regCP_RB2_WPTR_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL 0x1059
+#define regCP_RB_DOORBELL_CONTROL_BASE_IDX 0
+#define regCP_RB_DOORBELL_RANGE_LOWER 0x105a
+#define regCP_RB_DOORBELL_RANGE_LOWER_BASE_IDX 0
+#define regCP_RB_DOORBELL_RANGE_UPPER 0x105b
+#define regCP_RB_DOORBELL_RANGE_UPPER_BASE_IDX 0
+#define regCP_MEC_DOORBELL_RANGE_LOWER 0x105c
+#define regCP_MEC_DOORBELL_RANGE_LOWER_BASE_IDX 0
+#define regCP_MEC_DOORBELL_RANGE_UPPER 0x105d
+#define regCP_MEC_DOORBELL_RANGE_UPPER_BASE_IDX 0
+#define regCPG_UTCL1_ERROR 0x105e
+#define regCPG_UTCL1_ERROR_BASE_IDX 0
+#define regCPC_UTCL1_ERROR 0x105f
+#define regCPC_UTCL1_ERROR_BASE_IDX 0
+#define regCP_RB1_BASE 0x1060
+#define regCP_RB1_BASE_BASE_IDX 0
+#define regCP_RB1_CNTL 0x1061
+#define regCP_RB1_CNTL_BASE_IDX 0
+#define regCP_RB1_RPTR_ADDR 0x1062
+#define regCP_RB1_RPTR_ADDR_BASE_IDX 0
+#define regCP_RB1_RPTR_ADDR_HI 0x1063
+#define regCP_RB1_RPTR_ADDR_HI_BASE_IDX 0
+#define regCP_RB2_BASE 0x1065
+#define regCP_RB2_BASE_BASE_IDX 0
+#define regCP_RB2_CNTL 0x1066
+#define regCP_RB2_CNTL_BASE_IDX 0
+#define regCP_RB2_RPTR_ADDR 0x1067
+#define regCP_RB2_RPTR_ADDR_BASE_IDX 0
+#define regCP_RB2_RPTR_ADDR_HI 0x1068
+#define regCP_RB2_RPTR_ADDR_HI_BASE_IDX 0
+#define regCP_RB0_ACTIVE 0x1069
+#define regCP_RB0_ACTIVE_BASE_IDX 0
+#define regCP_RB_ACTIVE 0x1069
+#define regCP_RB_ACTIVE_BASE_IDX 0
+#define regCP_INT_CNTL_RING0 0x106a
+#define regCP_INT_CNTL_RING0_BASE_IDX 0
+#define regCP_INT_CNTL_RING1 0x106b
+#define regCP_INT_CNTL_RING1_BASE_IDX 0
+#define regCP_INT_CNTL_RING2 0x106c
+#define regCP_INT_CNTL_RING2_BASE_IDX 0
+#define regCP_INT_STATUS_RING0 0x106d
+#define regCP_INT_STATUS_RING0_BASE_IDX 0
+#define regCP_INT_STATUS_RING1 0x106e
+#define regCP_INT_STATUS_RING1_BASE_IDX 0
+#define regCP_INT_STATUS_RING2 0x106f
+#define regCP_INT_STATUS_RING2_BASE_IDX 0
+#define regCP_ME_F32_INTERRUPT 0x1073
+#define regCP_ME_F32_INTERRUPT_BASE_IDX 0
+#define regCP_PFP_F32_INTERRUPT 0x1074
+#define regCP_PFP_F32_INTERRUPT_BASE_IDX 0
+#define regCP_CE_F32_INTERRUPT 0x1075
+#define regCP_CE_F32_INTERRUPT_BASE_IDX 0
+#define regCP_MEC1_F32_INTERRUPT 0x1076
+#define regCP_MEC1_F32_INTERRUPT_BASE_IDX 0
+#define regCP_MEC2_F32_INTERRUPT 0x1077
+#define regCP_MEC2_F32_INTERRUPT_BASE_IDX 0
+#define regCP_PWR_CNTL 0x1078
+#define regCP_PWR_CNTL_BASE_IDX 0
+#define regCP_MEM_SLP_CNTL 0x1079
+#define regCP_MEM_SLP_CNTL_BASE_IDX 0
+#define regCP_ECC_DMA_FIRST_OCCURRENCE 0x107a
+#define regCP_ECC_DMA_FIRST_OCCURRENCE_BASE_IDX 0
+#define regCP_ECC_FIRSTOCCURRENCE 0x107a
+#define regCP_ECC_FIRSTOCCURRENCE_BASE_IDX 0
+#define regCP_ECC_FIRSTOCCURRENCE_RING0 0x107b
+#define regCP_ECC_FIRSTOCCURRENCE_RING0_BASE_IDX 0
+#define regCP_ECC_FIRSTOCCURRENCE_RING1 0x107c
+#define regCP_ECC_FIRSTOCCURRENCE_RING1_BASE_IDX 0
+#define regCP_ECC_FIRSTOCCURRENCE_RING2 0x107d
+#define regCP_ECC_FIRSTOCCURRENCE_RING2_BASE_IDX 0
+#define regGB_EDC_MODE 0x107e
+#define regGB_EDC_MODE_BASE_IDX 0
+#define regCP_PQ_WPTR_POLL_CNTL 0x1083
+#define regCP_PQ_WPTR_POLL_CNTL_BASE_IDX 0
+#define regCP_PQ_WPTR_POLL_CNTL1 0x1084
+#define regCP_PQ_WPTR_POLL_CNTL1_BASE_IDX 0
+#define regCP_ME1_PIPE0_INT_CNTL 0x1085
+#define regCP_ME1_PIPE0_INT_CNTL_BASE_IDX 0
+#define regCP_ME1_PIPE1_INT_CNTL 0x1086
+#define regCP_ME1_PIPE1_INT_CNTL_BASE_IDX 0
+#define regCP_ME1_PIPE2_INT_CNTL 0x1087
+#define regCP_ME1_PIPE2_INT_CNTL_BASE_IDX 0
+#define regCP_ME1_PIPE3_INT_CNTL 0x1088
+#define regCP_ME1_PIPE3_INT_CNTL_BASE_IDX 0
+#define regCP_ME2_PIPE0_INT_CNTL 0x1089
+#define regCP_ME2_PIPE0_INT_CNTL_BASE_IDX 0
+#define regCP_ME2_PIPE1_INT_CNTL 0x108a
+#define regCP_ME2_PIPE1_INT_CNTL_BASE_IDX 0
+#define regCP_ME2_PIPE2_INT_CNTL 0x108b
+#define regCP_ME2_PIPE2_INT_CNTL_BASE_IDX 0
+#define regCP_ME2_PIPE3_INT_CNTL 0x108c
+#define regCP_ME2_PIPE3_INT_CNTL_BASE_IDX 0
+#define regCP_ME1_PIPE0_INT_STATUS 0x108d
+#define regCP_ME1_PIPE0_INT_STATUS_BASE_IDX 0
+#define regCP_ME1_PIPE1_INT_STATUS 0x108e
+#define regCP_ME1_PIPE1_INT_STATUS_BASE_IDX 0
+#define regCP_ME1_PIPE2_INT_STATUS 0x108f
+#define regCP_ME1_PIPE2_INT_STATUS_BASE_IDX 0
+#define regCP_ME1_PIPE3_INT_STATUS 0x1090
+#define regCP_ME1_PIPE3_INT_STATUS_BASE_IDX 0
+#define regCP_ME2_PIPE0_INT_STATUS 0x1091
+#define regCP_ME2_PIPE0_INT_STATUS_BASE_IDX 0
+#define regCP_ME2_PIPE1_INT_STATUS 0x1092
+#define regCP_ME2_PIPE1_INT_STATUS_BASE_IDX 0
+#define regCP_ME2_PIPE2_INT_STATUS 0x1093
+#define regCP_ME2_PIPE2_INT_STATUS_BASE_IDX 0
+#define regCP_ME2_PIPE3_INT_STATUS 0x1094
+#define regCP_ME2_PIPE3_INT_STATUS_BASE_IDX 0
+#define regCC_GC_EDC_CONFIG 0x1098
+#define regCC_GC_EDC_CONFIG_BASE_IDX 0
+#define regCP_ME1_PIPE_PRIORITY_CNTS 0x1099
+#define regCP_ME1_PIPE_PRIORITY_CNTS_BASE_IDX 0
+#define regCP_ME1_PIPE0_PRIORITY 0x109a
+#define regCP_ME1_PIPE0_PRIORITY_BASE_IDX 0
+#define regCP_ME1_PIPE1_PRIORITY 0x109b
+#define regCP_ME1_PIPE1_PRIORITY_BASE_IDX 0
+#define regCP_ME1_PIPE2_PRIORITY 0x109c
+#define regCP_ME1_PIPE2_PRIORITY_BASE_IDX 0
+#define regCP_ME1_PIPE3_PRIORITY 0x109d
+#define regCP_ME1_PIPE3_PRIORITY_BASE_IDX 0
+#define regCP_ME2_PIPE_PRIORITY_CNTS 0x109e
+#define regCP_ME2_PIPE_PRIORITY_CNTS_BASE_IDX 0
+#define regCP_ME2_PIPE0_PRIORITY 0x109f
+#define regCP_ME2_PIPE0_PRIORITY_BASE_IDX 0
+#define regCP_ME2_PIPE1_PRIORITY 0x10a0
+#define regCP_ME2_PIPE1_PRIORITY_BASE_IDX 0
+#define regCP_ME2_PIPE2_PRIORITY 0x10a1
+#define regCP_ME2_PIPE2_PRIORITY_BASE_IDX 0
+#define regCP_ME2_PIPE3_PRIORITY 0x10a2
+#define regCP_ME2_PIPE3_PRIORITY_BASE_IDX 0
+#define regCP_CE_PRGRM_CNTR_START 0x10a3
+#define regCP_CE_PRGRM_CNTR_START_BASE_IDX 0
+#define regCP_PFP_PRGRM_CNTR_START 0x10a4
+#define regCP_PFP_PRGRM_CNTR_START_BASE_IDX 0
+#define regCP_ME_PRGRM_CNTR_START 0x10a5
+#define regCP_ME_PRGRM_CNTR_START_BASE_IDX 0
+#define regCP_MEC1_PRGRM_CNTR_START 0x10a6
+#define regCP_MEC1_PRGRM_CNTR_START_BASE_IDX 0
+#define regCP_MEC2_PRGRM_CNTR_START 0x10a7
+#define regCP_MEC2_PRGRM_CNTR_START_BASE_IDX 0
+#define regCP_CE_INTR_ROUTINE_START 0x10a8
+#define regCP_CE_INTR_ROUTINE_START_BASE_IDX 0
+#define regCP_PFP_INTR_ROUTINE_START 0x10a9
+#define regCP_PFP_INTR_ROUTINE_START_BASE_IDX 0
+#define regCP_ME_INTR_ROUTINE_START 0x10aa
+#define regCP_ME_INTR_ROUTINE_START_BASE_IDX 0
+#define regCP_MEC1_INTR_ROUTINE_START 0x10ab
+#define regCP_MEC1_INTR_ROUTINE_START_BASE_IDX 0
+#define regCP_MEC2_INTR_ROUTINE_START 0x10ac
+#define regCP_MEC2_INTR_ROUTINE_START_BASE_IDX 0
+#define regCP_CONTEXT_CNTL 0x10ad
+#define regCP_CONTEXT_CNTL_BASE_IDX 0
+#define regCP_MAX_CONTEXT 0x10ae
+#define regCP_MAX_CONTEXT_BASE_IDX 0
+#define regCP_IQ_WAIT_TIME1 0x10af
+#define regCP_IQ_WAIT_TIME1_BASE_IDX 0
+#define regCP_IQ_WAIT_TIME2 0x10b0
+#define regCP_IQ_WAIT_TIME2_BASE_IDX 0
+#define regCP_RB0_BASE_HI 0x10b1
+#define regCP_RB0_BASE_HI_BASE_IDX 0
+#define regCP_RB1_BASE_HI 0x10b2
+#define regCP_RB1_BASE_HI_BASE_IDX 0
+#define regCP_VMID_RESET 0x10b3
+#define regCP_VMID_RESET_BASE_IDX 0
+#define regCPC_INT_CNTL 0x10b4
+#define regCPC_INT_CNTL_BASE_IDX 0
+#define regCPC_INT_STATUS 0x10b5
+#define regCPC_INT_STATUS_BASE_IDX 0
+#define regCP_VMID_PREEMPT 0x10b6
+#define regCP_VMID_PREEMPT_BASE_IDX 0
+#define regCPC_INT_CNTX_ID 0x10b7
+#define regCPC_INT_CNTX_ID_BASE_IDX 0
+#define regCP_PQ_STATUS 0x10b8
+#define regCP_PQ_STATUS_BASE_IDX 0
+#define regCP_CPC_IC_BASE_LO 0x10b9
+#define regCP_CPC_IC_BASE_LO_BASE_IDX 0
+#define regCP_CPC_IC_BASE_HI 0x10ba
+#define regCP_CPC_IC_BASE_HI_BASE_IDX 0
+#define regCP_CPC_IC_BASE_CNTL 0x10bb
+#define regCP_CPC_IC_BASE_CNTL_BASE_IDX 0
+#define regCP_CPC_IC_OP_CNTL 0x10bc
+#define regCP_CPC_IC_OP_CNTL_BASE_IDX 0
+#define regCP_MEC1_F32_INT_DIS 0x10bd
+#define regCP_MEC1_F32_INT_DIS_BASE_IDX 0
+#define regCP_MEC2_F32_INT_DIS 0x10be
+#define regCP_MEC2_F32_INT_DIS_BASE_IDX 0
+#define regCP_VMID_STATUS 0x10bf
+#define regCP_VMID_STATUS_BASE_IDX 0
+
+
+// addressBlock: gc_cppdec2
+// base address: 0xc600
+#define regCP_RB_DOORBELL_CONTROL_SCH_0 0x1180
+#define regCP_RB_DOORBELL_CONTROL_SCH_0_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL_SCH_1 0x1181
+#define regCP_RB_DOORBELL_CONTROL_SCH_1_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL_SCH_2 0x1182
+#define regCP_RB_DOORBELL_CONTROL_SCH_2_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL_SCH_3 0x1183
+#define regCP_RB_DOORBELL_CONTROL_SCH_3_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL_SCH_4 0x1184
+#define regCP_RB_DOORBELL_CONTROL_SCH_4_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL_SCH_5 0x1185
+#define regCP_RB_DOORBELL_CONTROL_SCH_5_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL_SCH_6 0x1186
+#define regCP_RB_DOORBELL_CONTROL_SCH_6_BASE_IDX 0
+#define regCP_RB_DOORBELL_CONTROL_SCH_7 0x1187
+#define regCP_RB_DOORBELL_CONTROL_SCH_7_BASE_IDX 0
+#define regCP_RB_DOORBELL_CLEAR 0x1188
+#define regCP_RB_DOORBELL_CLEAR_BASE_IDX 0
+#define regCPF_EDC_TAG_CNT 0x1189
+#define regCPF_EDC_TAG_CNT_BASE_IDX 0
+#define regCPF_EDC_ROQ_CNT 0x118a
+#define regCPF_EDC_ROQ_CNT_BASE_IDX 0
+#define regCPG_EDC_TAG_CNT 0x118b
+#define regCPG_EDC_TAG_CNT_BASE_IDX 0
+#define regCPG_EDC_DMA_CNT 0x118d
+#define regCPG_EDC_DMA_CNT_BASE_IDX 0
+#define regCPC_EDC_SCRATCH_CNT 0x118e
+#define regCPC_EDC_SCRATCH_CNT_BASE_IDX 0
+#define regCPC_EDC_UCODE_CNT 0x118f
+#define regCPC_EDC_UCODE_CNT_BASE_IDX 0
+#define regDC_EDC_STATE_CNT 0x1191
+#define regDC_EDC_STATE_CNT_BASE_IDX 0
+#define regDC_EDC_CSINVOC_CNT 0x1192
+#define regDC_EDC_CSINVOC_CNT_BASE_IDX 0
+#define regDC_EDC_RESTORE_CNT 0x1193
+#define regDC_EDC_RESTORE_CNT_BASE_IDX 0
+#define regCP_CPF_DSM_CNTL 0x1194
+#define regCP_CPF_DSM_CNTL_BASE_IDX 0
+#define regCP_CPG_DSM_CNTL 0x1195
+#define regCP_CPG_DSM_CNTL_BASE_IDX 0
+#define regCP_CPC_DSM_CNTL 0x1196
+#define regCP_CPC_DSM_CNTL_BASE_IDX 0
+#define regCP_CPF_DSM_CNTL2 0x1197
+#define regCP_CPF_DSM_CNTL2_BASE_IDX 0
+#define regCP_CPG_DSM_CNTL2 0x1198
+#define regCP_CPG_DSM_CNTL2_BASE_IDX 0
+#define regCP_CPC_DSM_CNTL2 0x1199
+#define regCP_CPC_DSM_CNTL2_BASE_IDX 0
+#define regCP_CPF_DSM_CNTL2A 0x119a
+#define regCP_CPF_DSM_CNTL2A_BASE_IDX 0
+#define regCP_CPG_DSM_CNTL2A 0x119b
+#define regCP_CPG_DSM_CNTL2A_BASE_IDX 0
+#define regCP_CPC_DSM_CNTL2A 0x119c
+#define regCP_CPC_DSM_CNTL2A_BASE_IDX 0
+#define regCP_EDC_FUE_CNTL 0x119d
+#define regCP_EDC_FUE_CNTL_BASE_IDX 0
+#define regCP_GFX_MQD_CONTROL 0x11a0
+#define regCP_GFX_MQD_CONTROL_BASE_IDX 0
+#define regCP_GFX_MQD_BASE_ADDR 0x11a1
+#define regCP_GFX_MQD_BASE_ADDR_BASE_IDX 0
+#define regCP_GFX_MQD_BASE_ADDR_HI 0x11a2
+#define regCP_GFX_MQD_BASE_ADDR_HI_BASE_IDX 0
+#define regCP_RB_STATUS 0x11a3
+#define regCP_RB_STATUS_BASE_IDX 0
+#define regCPG_UTCL1_STATUS 0x11b4
+#define regCPG_UTCL1_STATUS_BASE_IDX 0
+#define regCPC_UTCL1_STATUS 0x11b5
+#define regCPC_UTCL1_STATUS_BASE_IDX 0
+#define regCPF_UTCL1_STATUS 0x11b6
+#define regCPF_UTCL1_STATUS_BASE_IDX 0
+#define regCP_SD_CNTL 0x11b7
+#define regCP_SD_CNTL_BASE_IDX 0
+#define regCP_SOFT_RESET_CNTL 0x11b9
+#define regCP_SOFT_RESET_CNTL_BASE_IDX 0
+#define regCP_CPC_GFX_CNTL 0x11ba
+#define regCP_CPC_GFX_CNTL_BASE_IDX 0
+
+
+// addressBlock: gc_cpphqddec
+// base address: 0xc800
+#define regCP_HQD_GFX_CONTROL 0x123e
+#define regCP_HQD_GFX_CONTROL_BASE_IDX 0
+#define regCP_HQD_GFX_STATUS 0x123f
+#define regCP_HQD_GFX_STATUS_BASE_IDX 0
+#define regCP_HPD_ROQ_OFFSETS 0x1240
+#define regCP_HPD_ROQ_OFFSETS_BASE_IDX 0
+#define regCP_HPD_STATUS0 0x1241
+#define regCP_HPD_STATUS0_BASE_IDX 0
+#define regCP_HPD_UTCL1_CNTL 0x1242
+#define regCP_HPD_UTCL1_CNTL_BASE_IDX 0
+#define regCP_HPD_UTCL1_ERROR 0x1243
+#define regCP_HPD_UTCL1_ERROR_BASE_IDX 0
+#define regCP_HPD_UTCL1_ERROR_ADDR 0x1244
+#define regCP_HPD_UTCL1_ERROR_ADDR_BASE_IDX 0
+#define regCP_MQD_BASE_ADDR 0x1245
+#define regCP_MQD_BASE_ADDR_BASE_IDX 0
+#define regCP_MQD_BASE_ADDR_HI 0x1246
+#define regCP_MQD_BASE_ADDR_HI_BASE_IDX 0
+#define regCP_HQD_ACTIVE 0x1247
+#define regCP_HQD_ACTIVE_BASE_IDX 0
+#define regCP_HQD_VMID 0x1248
+#define regCP_HQD_VMID_BASE_IDX 0
+#define regCP_HQD_PERSISTENT_STATE 0x1249
+#define regCP_HQD_PERSISTENT_STATE_BASE_IDX 0
+#define regCP_HQD_PIPE_PRIORITY 0x124a
+#define regCP_HQD_PIPE_PRIORITY_BASE_IDX 0
+#define regCP_HQD_QUEUE_PRIORITY 0x124b
+#define regCP_HQD_QUEUE_PRIORITY_BASE_IDX 0
+#define regCP_HQD_QUANTUM 0x124c
+#define regCP_HQD_QUANTUM_BASE_IDX 0
+#define regCP_HQD_PQ_BASE 0x124d
+#define regCP_HQD_PQ_BASE_BASE_IDX 0
+#define regCP_HQD_PQ_BASE_HI 0x124e
+#define regCP_HQD_PQ_BASE_HI_BASE_IDX 0
+#define regCP_HQD_PQ_RPTR 0x124f
+#define regCP_HQD_PQ_RPTR_BASE_IDX 0
+#define regCP_HQD_PQ_RPTR_REPORT_ADDR 0x1250
+#define regCP_HQD_PQ_RPTR_REPORT_ADDR_BASE_IDX 0
+#define regCP_HQD_PQ_RPTR_REPORT_ADDR_HI 0x1251
+#define regCP_HQD_PQ_RPTR_REPORT_ADDR_HI_BASE_IDX 0
+#define regCP_HQD_PQ_WPTR_POLL_ADDR 0x1252
+#define regCP_HQD_PQ_WPTR_POLL_ADDR_BASE_IDX 0
+#define regCP_HQD_PQ_WPTR_POLL_ADDR_HI 0x1253
+#define regCP_HQD_PQ_WPTR_POLL_ADDR_HI_BASE_IDX 0
+#define regCP_HQD_PQ_DOORBELL_CONTROL 0x1254
+#define regCP_HQD_PQ_DOORBELL_CONTROL_BASE_IDX 0
+#define regCP_HQD_PQ_CONTROL 0x1256
+#define regCP_HQD_PQ_CONTROL_BASE_IDX 0
+#define regCP_HQD_IB_BASE_ADDR 0x1257
+#define regCP_HQD_IB_BASE_ADDR_BASE_IDX 0
+#define regCP_HQD_IB_BASE_ADDR_HI 0x1258
+#define regCP_HQD_IB_BASE_ADDR_HI_BASE_IDX 0
+#define regCP_HQD_IB_RPTR 0x1259
+#define regCP_HQD_IB_RPTR_BASE_IDX 0
+#define regCP_HQD_IB_CONTROL 0x125a
+#define regCP_HQD_IB_CONTROL_BASE_IDX 0
+#define regCP_HQD_IQ_TIMER 0x125b
+#define regCP_HQD_IQ_TIMER_BASE_IDX 0
+#define regCP_HQD_IQ_RPTR 0x125c
+#define regCP_HQD_IQ_RPTR_BASE_IDX 0
+#define regCP_HQD_DEQUEUE_REQUEST 0x125d
+#define regCP_HQD_DEQUEUE_REQUEST_BASE_IDX 0
+#define regCP_HQD_DMA_OFFLOAD 0x125e
+#define regCP_HQD_DMA_OFFLOAD_BASE_IDX 0
+#define regCP_HQD_OFFLOAD 0x125e
+#define regCP_HQD_OFFLOAD_BASE_IDX 0
+#define regCP_HQD_SEMA_CMD 0x125f
+#define regCP_HQD_SEMA_CMD_BASE_IDX 0
+#define regCP_HQD_MSG_TYPE 0x1260
+#define regCP_HQD_MSG_TYPE_BASE_IDX 0
+#define regCP_HQD_ATOMIC0_PREOP_LO 0x1261
+#define regCP_HQD_ATOMIC0_PREOP_LO_BASE_IDX 0
+#define regCP_HQD_ATOMIC0_PREOP_HI 0x1262
+#define regCP_HQD_ATOMIC0_PREOP_HI_BASE_IDX 0
+#define regCP_HQD_ATOMIC1_PREOP_LO 0x1263
+#define regCP_HQD_ATOMIC1_PREOP_LO_BASE_IDX 0
+#define regCP_HQD_ATOMIC1_PREOP_HI 0x1264
+#define regCP_HQD_ATOMIC1_PREOP_HI_BASE_IDX 0
+#define regCP_HQD_HQ_SCHEDULER0 0x1265
+#define regCP_HQD_HQ_SCHEDULER0_BASE_IDX 0
+#define regCP_HQD_HQ_STATUS0 0x1265
+#define regCP_HQD_HQ_STATUS0_BASE_IDX 0
+#define regCP_HQD_HQ_CONTROL0 0x1266
+#define regCP_HQD_HQ_CONTROL0_BASE_IDX 0
+#define regCP_HQD_HQ_SCHEDULER1 0x1266
+#define regCP_HQD_HQ_SCHEDULER1_BASE_IDX 0
+#define regCP_MQD_CONTROL 0x1267
+#define regCP_MQD_CONTROL_BASE_IDX 0
+#define regCP_HQD_HQ_STATUS1 0x1268
+#define regCP_HQD_HQ_STATUS1_BASE_IDX 0
+#define regCP_HQD_HQ_CONTROL1 0x1269
+#define regCP_HQD_HQ_CONTROL1_BASE_IDX 0
+#define regCP_HQD_EOP_BASE_ADDR 0x126a
+#define regCP_HQD_EOP_BASE_ADDR_BASE_IDX 0
+#define regCP_HQD_EOP_BASE_ADDR_HI 0x126b
+#define regCP_HQD_EOP_BASE_ADDR_HI_BASE_IDX 0
+#define regCP_HQD_EOP_CONTROL 0x126c
+#define regCP_HQD_EOP_CONTROL_BASE_IDX 0
+#define regCP_HQD_EOP_RPTR 0x126d
+#define regCP_HQD_EOP_RPTR_BASE_IDX 0
+#define regCP_HQD_EOP_WPTR 0x126e
+#define regCP_HQD_EOP_WPTR_BASE_IDX 0
+#define regCP_HQD_EOP_EVENTS 0x126f
+#define regCP_HQD_EOP_EVENTS_BASE_IDX 0
+#define regCP_HQD_CTX_SAVE_BASE_ADDR_LO 0x1270
+#define regCP_HQD_CTX_SAVE_BASE_ADDR_LO_BASE_IDX 0
+#define regCP_HQD_CTX_SAVE_BASE_ADDR_HI 0x1271
+#define regCP_HQD_CTX_SAVE_BASE_ADDR_HI_BASE_IDX 0
+#define regCP_HQD_CTX_SAVE_CONTROL 0x1272
+#define regCP_HQD_CTX_SAVE_CONTROL_BASE_IDX 0
+#define regCP_HQD_CNTL_STACK_OFFSET 0x1273
+#define regCP_HQD_CNTL_STACK_OFFSET_BASE_IDX 0
+#define regCP_HQD_CNTL_STACK_SIZE 0x1274
+#define regCP_HQD_CNTL_STACK_SIZE_BASE_IDX 0
+#define regCP_HQD_WG_STATE_OFFSET 0x1275
+#define regCP_HQD_WG_STATE_OFFSET_BASE_IDX 0
+#define regCP_HQD_CTX_SAVE_SIZE 0x1276
+#define regCP_HQD_CTX_SAVE_SIZE_BASE_IDX 0
+#define regCP_HQD_GDS_RESOURCE_STATE 0x1277
+#define regCP_HQD_GDS_RESOURCE_STATE_BASE_IDX 0
+#define regCP_HQD_ERROR 0x1278
+#define regCP_HQD_ERROR_BASE_IDX 0
+#define regCP_HQD_EOP_WPTR_MEM 0x1279
+#define regCP_HQD_EOP_WPTR_MEM_BASE_IDX 0
+#define regCP_HQD_AQL_CONTROL 0x127a
+#define regCP_HQD_AQL_CONTROL_BASE_IDX 0
+#define regCP_HQD_PQ_WPTR_LO 0x127b
+#define regCP_HQD_PQ_WPTR_LO_BASE_IDX 0
+#define regCP_HQD_PQ_WPTR_HI 0x127c
+#define regCP_HQD_PQ_WPTR_HI_BASE_IDX 0
+
+
+// addressBlock: gc_didtdec
+// base address: 0xca00
+#define regDIDT_IND_INDEX 0x1280
+#define regDIDT_IND_INDEX_BASE_IDX 0
+#define regDIDT_IND_DATA 0x1281
+#define regDIDT_IND_DATA_BASE_IDX 0
+#define regDIDT_INDEX_AUTO_INCR_EN 0x1282
+#define regDIDT_INDEX_AUTO_INCR_EN_BASE_IDX 0
+
+
+// addressBlock: gc_ea_gceadec
+// base address: 0xa800
+#define regGCEA_DRAM_RD_CLI2GRP_MAP0 0x0a00
+#define regGCEA_DRAM_RD_CLI2GRP_MAP0_BASE_IDX 0
+#define regGCEA_DRAM_RD_CLI2GRP_MAP1 0x0a01
+#define regGCEA_DRAM_RD_CLI2GRP_MAP1_BASE_IDX 0
+#define regGCEA_DRAM_WR_CLI2GRP_MAP0 0x0a02
+#define regGCEA_DRAM_WR_CLI2GRP_MAP0_BASE_IDX 0
+#define regGCEA_DRAM_WR_CLI2GRP_MAP1 0x0a03
+#define regGCEA_DRAM_WR_CLI2GRP_MAP1_BASE_IDX 0
+#define regGCEA_DRAM_RD_GRP2VC_MAP 0x0a04
+#define regGCEA_DRAM_RD_GRP2VC_MAP_BASE_IDX 0
+#define regGCEA_DRAM_WR_GRP2VC_MAP 0x0a05
+#define regGCEA_DRAM_WR_GRP2VC_MAP_BASE_IDX 0
+#define regGCEA_DRAM_RD_LAZY 0x0a06
+#define regGCEA_DRAM_RD_LAZY_BASE_IDX 0
+#define regGCEA_DRAM_WR_LAZY 0x0a07
+#define regGCEA_DRAM_WR_LAZY_BASE_IDX 0
+#define regGCEA_DRAM_RD_CAM_CNTL 0x0a08
+#define regGCEA_DRAM_RD_CAM_CNTL_BASE_IDX 0
+#define regGCEA_DRAM_WR_CAM_CNTL 0x0a09
+#define regGCEA_DRAM_WR_CAM_CNTL_BASE_IDX 0
+#define regGCEA_DRAM_PAGE_BURST 0x0a0a
+#define regGCEA_DRAM_PAGE_BURST_BASE_IDX 0
+#define regGCEA_DRAM_RD_PRI_AGE 0x0a0b
+#define regGCEA_DRAM_RD_PRI_AGE_BASE_IDX 0
+#define regGCEA_DRAM_WR_PRI_AGE 0x0a0c
+#define regGCEA_DRAM_WR_PRI_AGE_BASE_IDX 0
+#define regGCEA_DRAM_RD_PRI_QUEUING 0x0a0d
+#define regGCEA_DRAM_RD_PRI_QUEUING_BASE_IDX 0
+#define regGCEA_DRAM_WR_PRI_QUEUING 0x0a0e
+#define regGCEA_DRAM_WR_PRI_QUEUING_BASE_IDX 0
+#define regGCEA_DRAM_RD_PRI_FIXED 0x0a0f
+#define regGCEA_DRAM_RD_PRI_FIXED_BASE_IDX 0
+#define regGCEA_DRAM_WR_PRI_FIXED 0x0a10
+#define regGCEA_DRAM_WR_PRI_FIXED_BASE_IDX 0
+#define regGCEA_DRAM_RD_PRI_URGENCY 0x0a11
+#define regGCEA_DRAM_RD_PRI_URGENCY_BASE_IDX 0
+#define regGCEA_DRAM_WR_PRI_URGENCY 0x0a12
+#define regGCEA_DRAM_WR_PRI_URGENCY_BASE_IDX 0
+#define regGCEA_DRAM_RD_PRI_QUANT_PRI1 0x0a13
+#define regGCEA_DRAM_RD_PRI_QUANT_PRI1_BASE_IDX 0
+#define regGCEA_DRAM_RD_PRI_QUANT_PRI2 0x0a14
+#define regGCEA_DRAM_RD_PRI_QUANT_PRI2_BASE_IDX 0
+#define regGCEA_DRAM_RD_PRI_QUANT_PRI3 0x0a15
+#define regGCEA_DRAM_RD_PRI_QUANT_PRI3_BASE_IDX 0
+#define regGCEA_DRAM_WR_PRI_QUANT_PRI1 0x0a16
+#define regGCEA_DRAM_WR_PRI_QUANT_PRI1_BASE_IDX 0
+#define regGCEA_DRAM_WR_PRI_QUANT_PRI2 0x0a17
+#define regGCEA_DRAM_WR_PRI_QUANT_PRI2_BASE_IDX 0
+#define regGCEA_DRAM_WR_PRI_QUANT_PRI3 0x0a18
+#define regGCEA_DRAM_WR_PRI_QUANT_PRI3_BASE_IDX 0
+#define regGCEA_ADDRNORM_BASE_ADDR0 0x0a34
+#define regGCEA_ADDRNORM_BASE_ADDR0_BASE_IDX 0
+#define regGCEA_ADDRNORM_LIMIT_ADDR0 0x0a35
+#define regGCEA_ADDRNORM_LIMIT_ADDR0_BASE_IDX 0
+#define regGCEA_ADDRNORM_BASE_ADDR1 0x0a36
+#define regGCEA_ADDRNORM_BASE_ADDR1_BASE_IDX 0
+#define regGCEA_ADDRNORM_LIMIT_ADDR1 0x0a37
+#define regGCEA_ADDRNORM_LIMIT_ADDR1_BASE_IDX 0
+#define regGCEA_ADDRNORM_OFFSET_ADDR1 0x0a38
+#define regGCEA_ADDRNORM_OFFSET_ADDR1_BASE_IDX 0
+#define regGCEA_ADDRNORM_BASE_ADDR2 0x0a39
+#define regGCEA_ADDRNORM_BASE_ADDR2_BASE_IDX 0
+#define regGCEA_ADDRNORM_LIMIT_ADDR2 0x0a3a
+#define regGCEA_ADDRNORM_LIMIT_ADDR2_BASE_IDX 0
+#define regGCEA_ADDRNORM_BASE_ADDR3 0x0a3b
+#define regGCEA_ADDRNORM_BASE_ADDR3_BASE_IDX 0
+#define regGCEA_ADDRNORM_LIMIT_ADDR3 0x0a3c
+#define regGCEA_ADDRNORM_LIMIT_ADDR3_BASE_IDX 0
+#define regGCEA_ADDRNORM_OFFSET_ADDR3 0x0a3d
+#define regGCEA_ADDRNORM_OFFSET_ADDR3_BASE_IDX 0
+#define regGCEA_ADDRNORM_MEGABASE_ADDR0 0x0a3e
+#define regGCEA_ADDRNORM_MEGABASE_ADDR0_BASE_IDX 0
+#define regGCEA_ADDRNORM_MEGALIMIT_ADDR0 0x0a3f
+#define regGCEA_ADDRNORM_MEGALIMIT_ADDR0_BASE_IDX 0
+#define regGCEA_ADDRNORM_MEGABASE_ADDR1 0x0a40
+#define regGCEA_ADDRNORM_MEGABASE_ADDR1_BASE_IDX 0
+#define regGCEA_ADDRNORM_MEGALIMIT_ADDR1 0x0a41
+#define regGCEA_ADDRNORM_MEGALIMIT_ADDR1_BASE_IDX 0
+#define regGCEA_ADDRNORMDRAM_HOLE_CNTL 0x0a43
+#define regGCEA_ADDRNORMDRAM_HOLE_CNTL_BASE_IDX 0
+#define regGCEA_ADDRNORMGMI_HOLE_CNTL 0x0a44
+#define regGCEA_ADDRNORMGMI_HOLE_CNTL_BASE_IDX 0
+#define regGCEA_ADDRNORMDRAM_NP2_CHANNEL_CFG 0x0a45
+#define regGCEA_ADDRNORMDRAM_NP2_CHANNEL_CFG_BASE_IDX 0
+#define regGCEA_ADDRNORMGMI_NP2_CHANNEL_CFG 0x0a46
+#define regGCEA_ADDRNORMGMI_NP2_CHANNEL_CFG_BASE_IDX 0
+#define regGCEA_ADDRDEC_BANK_CFG 0x0a47
+#define regGCEA_ADDRDEC_BANK_CFG_BASE_IDX 0
+#define regGCEA_ADDRDEC_MISC_CFG 0x0a48
+#define regGCEA_ADDRDEC_MISC_CFG_BASE_IDX 0
+#define regGCEA_ADDRDECDRAM_HARVEST_ENABLE 0x0a53
+#define regGCEA_ADDRDECDRAM_HARVEST_ENABLE_BASE_IDX 0
+#define regGCEA_ADDRDECGMI_HARVEST_ENABLE 0x0a5e
+#define regGCEA_ADDRDECGMI_HARVEST_ENABLE_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS0 0x0a5f
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS0_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS1 0x0a60
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS1_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS2 0x0a61
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS2_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS3 0x0a62
+#define regGCEA_ADDRDEC0_BASE_ADDR_CS3_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS0 0x0a63
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS0_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS1 0x0a64
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS1_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS2 0x0a65
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS2_BASE_IDX 0
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS3 0x0a66
+#define regGCEA_ADDRDEC0_BASE_ADDR_SECCS3_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_MASK_CS01 0x0a67
+#define regGCEA_ADDRDEC0_ADDR_MASK_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_MASK_CS23 0x0a68
+#define regGCEA_ADDRDEC0_ADDR_MASK_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_MASK_SECCS01 0x0a69
+#define regGCEA_ADDRDEC0_ADDR_MASK_SECCS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_MASK_SECCS23 0x0a6a
+#define regGCEA_ADDRDEC0_ADDR_MASK_SECCS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_CFG_CS01 0x0a6b
+#define regGCEA_ADDRDEC0_ADDR_CFG_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_CFG_CS23 0x0a6c
+#define regGCEA_ADDRDEC0_ADDR_CFG_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_SEL_CS01 0x0a6d
+#define regGCEA_ADDRDEC0_ADDR_SEL_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_SEL_CS23 0x0a6e
+#define regGCEA_ADDRDEC0_ADDR_SEL_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_SEL2_CS01 0x0a6f
+#define regGCEA_ADDRDEC0_ADDR_SEL2_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_ADDR_SEL2_CS23 0x0a70
+#define regGCEA_ADDRDEC0_ADDR_SEL2_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_COL_SEL_LO_CS01 0x0a71
+#define regGCEA_ADDRDEC0_COL_SEL_LO_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_COL_SEL_LO_CS23 0x0a72
+#define regGCEA_ADDRDEC0_COL_SEL_LO_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_COL_SEL_HI_CS01 0x0a73
+#define regGCEA_ADDRDEC0_COL_SEL_HI_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_COL_SEL_HI_CS23 0x0a74
+#define regGCEA_ADDRDEC0_COL_SEL_HI_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_RM_SEL_CS01 0x0a75
+#define regGCEA_ADDRDEC0_RM_SEL_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_RM_SEL_CS23 0x0a76
+#define regGCEA_ADDRDEC0_RM_SEL_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC0_RM_SEL_SECCS01 0x0a77
+#define regGCEA_ADDRDEC0_RM_SEL_SECCS01_BASE_IDX 0
+#define regGCEA_ADDRDEC0_RM_SEL_SECCS23 0x0a78
+#define regGCEA_ADDRDEC0_RM_SEL_SECCS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS0 0x0a79
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS0_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS1 0x0a7a
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS1_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS2 0x0a7b
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS2_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS3 0x0a7c
+#define regGCEA_ADDRDEC1_BASE_ADDR_CS3_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS0 0x0a7d
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS0_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS1 0x0a7e
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS1_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS2 0x0a7f
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS2_BASE_IDX 0
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS3 0x0a80
+#define regGCEA_ADDRDEC1_BASE_ADDR_SECCS3_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_MASK_CS01 0x0a81
+#define regGCEA_ADDRDEC1_ADDR_MASK_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_MASK_CS23 0x0a82
+#define regGCEA_ADDRDEC1_ADDR_MASK_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_MASK_SECCS01 0x0a83
+#define regGCEA_ADDRDEC1_ADDR_MASK_SECCS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_MASK_SECCS23 0x0a84
+#define regGCEA_ADDRDEC1_ADDR_MASK_SECCS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_CFG_CS01 0x0a85
+#define regGCEA_ADDRDEC1_ADDR_CFG_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_CFG_CS23 0x0a86
+#define regGCEA_ADDRDEC1_ADDR_CFG_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_SEL_CS01 0x0a87
+#define regGCEA_ADDRDEC1_ADDR_SEL_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_SEL_CS23 0x0a88
+#define regGCEA_ADDRDEC1_ADDR_SEL_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_SEL2_CS01 0x0a89
+#define regGCEA_ADDRDEC1_ADDR_SEL2_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_ADDR_SEL2_CS23 0x0a8a
+#define regGCEA_ADDRDEC1_ADDR_SEL2_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_COL_SEL_LO_CS01 0x0a8b
+#define regGCEA_ADDRDEC1_COL_SEL_LO_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_COL_SEL_LO_CS23 0x0a8c
+#define regGCEA_ADDRDEC1_COL_SEL_LO_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_COL_SEL_HI_CS01 0x0a8d
+#define regGCEA_ADDRDEC1_COL_SEL_HI_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_COL_SEL_HI_CS23 0x0a8e
+#define regGCEA_ADDRDEC1_COL_SEL_HI_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_RM_SEL_CS01 0x0a8f
+#define regGCEA_ADDRDEC1_RM_SEL_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_RM_SEL_CS23 0x0a90
+#define regGCEA_ADDRDEC1_RM_SEL_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC1_RM_SEL_SECCS01 0x0a91
+#define regGCEA_ADDRDEC1_RM_SEL_SECCS01_BASE_IDX 0
+#define regGCEA_ADDRDEC1_RM_SEL_SECCS23 0x0a92
+#define regGCEA_ADDRDEC1_RM_SEL_SECCS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS0 0x0a93
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS0_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS1 0x0a94
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS1_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS2 0x0a95
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS2_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS3 0x0a96
+#define regGCEA_ADDRDEC2_BASE_ADDR_CS3_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS0 0x0a97
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS0_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS1 0x0a98
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS1_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS2 0x0a99
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS2_BASE_IDX 0
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS3 0x0a9a
+#define regGCEA_ADDRDEC2_BASE_ADDR_SECCS3_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_MASK_CS01 0x0a9b
+#define regGCEA_ADDRDEC2_ADDR_MASK_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_MASK_CS23 0x0a9c
+#define regGCEA_ADDRDEC2_ADDR_MASK_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_MASK_SECCS01 0x0a9d
+#define regGCEA_ADDRDEC2_ADDR_MASK_SECCS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_MASK_SECCS23 0x0a9e
+#define regGCEA_ADDRDEC2_ADDR_MASK_SECCS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_CFG_CS01 0x0a9f
+#define regGCEA_ADDRDEC2_ADDR_CFG_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_CFG_CS23 0x0aa0
+#define regGCEA_ADDRDEC2_ADDR_CFG_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_SEL_CS01 0x0aa1
+#define regGCEA_ADDRDEC2_ADDR_SEL_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_SEL_CS23 0x0aa2
+#define regGCEA_ADDRDEC2_ADDR_SEL_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_SEL2_CS01 0x0aa3
+#define regGCEA_ADDRDEC2_ADDR_SEL2_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_ADDR_SEL2_CS23 0x0aa4
+#define regGCEA_ADDRDEC2_ADDR_SEL2_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_COL_SEL_LO_CS01 0x0aa5
+#define regGCEA_ADDRDEC2_COL_SEL_LO_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_COL_SEL_LO_CS23 0x0aa6
+#define regGCEA_ADDRDEC2_COL_SEL_LO_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_COL_SEL_HI_CS01 0x0aa7
+#define regGCEA_ADDRDEC2_COL_SEL_HI_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_COL_SEL_HI_CS23 0x0aa8
+#define regGCEA_ADDRDEC2_COL_SEL_HI_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_RM_SEL_CS01 0x0aa9
+#define regGCEA_ADDRDEC2_RM_SEL_CS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_RM_SEL_CS23 0x0aaa
+#define regGCEA_ADDRDEC2_RM_SEL_CS23_BASE_IDX 0
+#define regGCEA_ADDRDEC2_RM_SEL_SECCS01 0x0aab
+#define regGCEA_ADDRDEC2_RM_SEL_SECCS01_BASE_IDX 0
+#define regGCEA_ADDRDEC2_RM_SEL_SECCS23 0x0aac
+#define regGCEA_ADDRDEC2_RM_SEL_SECCS23_BASE_IDX 0
+#define regGCEA_ADDRNORMDRAM_GLOBAL_CNTL 0x0aad
+#define regGCEA_ADDRNORMDRAM_GLOBAL_CNTL_BASE_IDX 0
+#define regGCEA_ADDRNORMGMI_GLOBAL_CNTL 0x0aae
+#define regGCEA_ADDRNORMGMI_GLOBAL_CNTL_BASE_IDX 0
+#define regGCEA_ADDRNORM_MEGACONTROL_ADDR0 0x0ad1
+#define regGCEA_ADDRNORM_MEGACONTROL_ADDR0_BASE_IDX 0
+#define regGCEA_ADDRNORM_MEGACONTROL_ADDR1 0x0ad2
+#define regGCEA_ADDRNORM_MEGACONTROL_ADDR1_BASE_IDX 0
+#define regGCEA_ADDRNORMDRAM_MASKING 0x0ad3
+#define regGCEA_ADDRNORMDRAM_MASKING_BASE_IDX 0
+#define regGCEA_ADDRNORMGMI_MASKING 0x0ad4
+#define regGCEA_ADDRNORMGMI_MASKING_BASE_IDX 0
+#define regGCEA_IO_RD_CLI2GRP_MAP0 0x0ad5
+#define regGCEA_IO_RD_CLI2GRP_MAP0_BASE_IDX 0
+#define regGCEA_IO_RD_CLI2GRP_MAP1 0x0ad6
+#define regGCEA_IO_RD_CLI2GRP_MAP1_BASE_IDX 0
+#define regGCEA_IO_WR_CLI2GRP_MAP0 0x0ad7
+#define regGCEA_IO_WR_CLI2GRP_MAP0_BASE_IDX 0
+#define regGCEA_IO_WR_CLI2GRP_MAP1 0x0ad8
+#define regGCEA_IO_WR_CLI2GRP_MAP1_BASE_IDX 0
+#define regGCEA_IO_RD_COMBINE_FLUSH 0x0ad9
+#define regGCEA_IO_RD_COMBINE_FLUSH_BASE_IDX 0
+#define regGCEA_IO_WR_COMBINE_FLUSH 0x0ada
+#define regGCEA_IO_WR_COMBINE_FLUSH_BASE_IDX 0
+#define regGCEA_IO_GROUP_BURST 0x0adb
+#define regGCEA_IO_GROUP_BURST_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_AGE 0x0adc
+#define regGCEA_IO_RD_PRI_AGE_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_AGE 0x0add
+#define regGCEA_IO_WR_PRI_AGE_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_QUEUING 0x0ade
+#define regGCEA_IO_RD_PRI_QUEUING_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_QUEUING 0x0adf
+#define regGCEA_IO_WR_PRI_QUEUING_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_FIXED 0x0ae0
+#define regGCEA_IO_RD_PRI_FIXED_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_FIXED 0x0ae1
+#define regGCEA_IO_WR_PRI_FIXED_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_URGENCY 0x0ae2
+#define regGCEA_IO_RD_PRI_URGENCY_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_URGENCY 0x0ae3
+#define regGCEA_IO_WR_PRI_URGENCY_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_URGENCY_MASKING 0x0ae4
+#define regGCEA_IO_RD_PRI_URGENCY_MASKING_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_URGENCY_MASKING 0x0ae5
+#define regGCEA_IO_WR_PRI_URGENCY_MASKING_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_QUANT_PRI1 0x0ae6
+#define regGCEA_IO_RD_PRI_QUANT_PRI1_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_QUANT_PRI2 0x0ae7
+#define regGCEA_IO_RD_PRI_QUANT_PRI2_BASE_IDX 0
+#define regGCEA_IO_RD_PRI_QUANT_PRI3 0x0ae8
+#define regGCEA_IO_RD_PRI_QUANT_PRI3_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_QUANT_PRI1 0x0ae9
+#define regGCEA_IO_WR_PRI_QUANT_PRI1_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_QUANT_PRI2 0x0aea
+#define regGCEA_IO_WR_PRI_QUANT_PRI2_BASE_IDX 0
+#define regGCEA_IO_WR_PRI_QUANT_PRI3 0x0aeb
+#define regGCEA_IO_WR_PRI_QUANT_PRI3_BASE_IDX 0
+#define regGCEA_MISC 0x0afa
+#define regGCEA_MISC_BASE_IDX 0
+#define regGCEA_LATENCY_SAMPLING 0x0afb
+#define regGCEA_LATENCY_SAMPLING_BASE_IDX 0
+#define regGCEA_PERFCOUNTER_LO 0x0afc
+#define regGCEA_PERFCOUNTER_LO_BASE_IDX 0
+#define regGCEA_PERFCOUNTER_HI 0x0afd
+#define regGCEA_PERFCOUNTER_HI_BASE_IDX 0
+#define regGCEA_PERFCOUNTER0_CFG 0x0afe
+#define regGCEA_PERFCOUNTER0_CFG_BASE_IDX 0
+#define regGCEA_PERFCOUNTER1_CFG 0x0aff
+#define regGCEA_PERFCOUNTER1_CFG_BASE_IDX 0
+
+
+// addressBlock: gc_ea_gceadec2
+// base address: 0x9c00
+#define regGCEA_PERFCOUNTER_RSLT_CNTL 0x0700
+#define regGCEA_PERFCOUNTER_RSLT_CNTL_BASE_IDX 0
+#define regGCEA_EDC_CNT 0x0706
+#define regGCEA_EDC_CNT_BASE_IDX 0
+#define regGCEA_EDC_CNT2 0x0707
+#define regGCEA_EDC_CNT2_BASE_IDX 0
+#define regGCEA_DSM_CNTL 0x0708
+#define regGCEA_DSM_CNTL_BASE_IDX 0
+#define regGCEA_DSM_CNTLA 0x0709
+#define regGCEA_DSM_CNTLA_BASE_IDX 0
+#define regGCEA_DSM_CNTLB 0x070a
+#define regGCEA_DSM_CNTLB_BASE_IDX 0
+#define regGCEA_DSM_CNTL2 0x070b
+#define regGCEA_DSM_CNTL2_BASE_IDX 0
+#define regGCEA_DSM_CNTL2A 0x070c
+#define regGCEA_DSM_CNTL2A_BASE_IDX 0
+#define regGCEA_DSM_CNTL2B 0x070d
+#define regGCEA_DSM_CNTL2B_BASE_IDX 0
+#define regGCEA_TCC_XBR_CREDITS 0x070e
+#define regGCEA_TCC_XBR_CREDITS_BASE_IDX 0
+#define regGCEA_TCC_XBR_MAXBURST 0x070f
+#define regGCEA_TCC_XBR_MAXBURST_BASE_IDX 0
+#define regGCEA_PROBE_CNTL 0x0710
+#define regGCEA_PROBE_CNTL_BASE_IDX 0
+#define regGCEA_PROBE_MAP 0x0711
+#define regGCEA_PROBE_MAP_BASE_IDX 0
+#define regGCEA_ERR_STATUS 0x0712
+#define regGCEA_ERR_STATUS_BASE_IDX 0
+#define regGCEA_MISC2 0x0713
+#define regGCEA_MISC2_BASE_IDX 0
+#define regGCEA_DRAM_BANK_ARB 0x0714
+#define regGCEA_DRAM_BANK_ARB_BASE_IDX 0
+#define regGCEA_ADDRDEC_SELECT 0x071a
+#define regGCEA_ADDRDEC_SELECT_BASE_IDX 0
+#define regGCEA_EDC_CNT3 0x071b
+#define regGCEA_EDC_CNT3_BASE_IDX 0
+
+// addressBlock: gc_ea_pwrdec
+// base address: 0x3c000
+#define regGCEA_CGTT_CLK_CTRL 0x50c4
+#define regGCEA_CGTT_CLK_CTRL_BASE_IDX 1
+
+
+// addressBlock: gc_gccacdec
+// base address: 0xca10
+#define regGC_CAC_CTRL_1 0x1284
+#define regGC_CAC_CTRL_1_BASE_IDX 0
+#define regGC_CAC_CTRL_2 0x1285
+#define regGC_CAC_CTRL_2_BASE_IDX 0
+#define regGC_CAC_INDEX_AUTO_INCR_EN 0x1286
+#define regGC_CAC_INDEX_AUTO_INCR_EN_BASE_IDX 0
+#define regGC_CAC_AGGR_LOWER 0x1287
+#define regGC_CAC_AGGR_LOWER_BASE_IDX 0
+#define regGC_CAC_AGGR_UPPER 0x1288
+#define regGC_CAC_AGGR_UPPER_BASE_IDX 0
+#define regGC_EDC_PERF_COUNTER 0x1289
+#define regGC_EDC_PERF_COUNTER_BASE_IDX 0
+#define regPCC_PERF_COUNTER 0x128a
+#define regPCC_PERF_COUNTER_BASE_IDX 0
+#define regGC_CAC_SOFT_CTRL 0x128d
+#define regGC_CAC_SOFT_CTRL_BASE_IDX 0
+#define regGC_DIDT_CTRL0 0x128e
+#define regGC_DIDT_CTRL0_BASE_IDX 0
+#define regGC_DIDT_CTRL1 0x128f
+#define regGC_DIDT_CTRL1_BASE_IDX 0
+#define regGC_DIDT_CTRL2 0x1290
+#define regGC_DIDT_CTRL2_BASE_IDX 0
+#define regGC_DIDT_WEIGHT 0x1291
+#define regGC_DIDT_WEIGHT_BASE_IDX 0
+#define regGC_THROTTLE_CTRL1 0x1292
+#define regGC_THROTTLE_CTRL1_BASE_IDX 0
+#define regGC_EDC_CTRL 0x1293
+#define regGC_EDC_CTRL_BASE_IDX 0
+#define regGC_EDC_THRESHOLD 0x1294
+#define regGC_EDC_THRESHOLD_BASE_IDX 0
+#define regGC_EDC_STATUS 0x1295
+#define regGC_EDC_STATUS_BASE_IDX 0
+#define regGC_EDC_OVERFLOW 0x1296
+#define regGC_EDC_OVERFLOW_BASE_IDX 0
+#define regGC_EDC_ROLLING_POWER_DELTA 0x1297
+#define regGC_EDC_ROLLING_POWER_DELTA_BASE_IDX 0
+#define regGC_EDC_CTRL1 0x1298
+#define regGC_EDC_CTRL1_BASE_IDX 0
+#define regGC_THROTTLE_CTRL2 0x1299
+#define regGC_THROTTLE_CTRL2_BASE_IDX 0
+#define regPWRBRK_PERF_COUNTER 0x129a
+#define regPWRBRK_PERF_COUNTER_BASE_IDX 0
+#define regGC_THROTTLE_CTRL 0x129b
+#define regGC_THROTTLE_CTRL_BASE_IDX 0
+#define regGC_CAC_IND_INDEX 0x129c
+#define regGC_CAC_IND_INDEX_BASE_IDX 0
+#define regGC_CAC_IND_DATA 0x129d
+#define regGC_CAC_IND_DATA_BASE_IDX 0
+#define regSE_CAC_IND_INDEX 0x129e
+#define regSE_CAC_IND_INDEX_BASE_IDX 0
+#define regSE_CAC_IND_DATA 0x129f
+#define regSE_CAC_IND_DATA_BASE_IDX 0
+
+
+// addressBlock: gc_gdsdec
+// base address: 0x9700
+#define regGDS_CONFIG 0x05c0
+#define regGDS_CONFIG_BASE_IDX 0
+#define regGDS_CNTL_STATUS 0x05c1
+#define regGDS_CNTL_STATUS_BASE_IDX 0
+#define regGDS_ENHANCE2 0x05c2
+#define regGDS_ENHANCE2_BASE_IDX 0
+#define regGDS_PROTECTION_FAULT 0x05c3
+#define regGDS_PROTECTION_FAULT_BASE_IDX 0
+#define regGDS_VM_PROTECTION_FAULT 0x05c4
+#define regGDS_VM_PROTECTION_FAULT_BASE_IDX 0
+#define regGDS_EDC_CNT 0x05c5
+#define regGDS_EDC_CNT_BASE_IDX 0
+#define regGDS_EDC_GRBM_CNT 0x05c6
+#define regGDS_EDC_GRBM_CNT_BASE_IDX 0
+#define regGDS_EDC_OA_DED 0x05c7
+#define regGDS_EDC_OA_DED_BASE_IDX 0
+#define regGDS_DSM_CNTL 0x05ca
+#define regGDS_DSM_CNTL_BASE_IDX 0
+#define regGDS_EDC_OA_PHY_CNT 0x05cb
+#define regGDS_EDC_OA_PHY_CNT_BASE_IDX 0
+#define regGDS_EDC_OA_PIPE_CNT 0x05cc
+#define regGDS_EDC_OA_PIPE_CNT_BASE_IDX 0
+#define regGDS_DSM_CNTL2 0x05cd
+#define regGDS_DSM_CNTL2_BASE_IDX 0
+#define regGDS_WD_GDS_CSB 0x05ce
+#define regGDS_WD_GDS_CSB_BASE_IDX 0
+
+
+// addressBlock: gc_gdspdec
+// base address: 0xcc00
+#define regGDS_VMID0_BASE 0x1300
+#define regGDS_VMID0_BASE_BASE_IDX 0
+#define regGDS_VMID0_SIZE 0x1301
+#define regGDS_VMID0_SIZE_BASE_IDX 0
+#define regGDS_VMID1_BASE 0x1302
+#define regGDS_VMID1_BASE_BASE_IDX 0
+#define regGDS_VMID1_SIZE 0x1303
+#define regGDS_VMID1_SIZE_BASE_IDX 0
+#define regGDS_VMID2_BASE 0x1304
+#define regGDS_VMID2_BASE_BASE_IDX 0
+#define regGDS_VMID2_SIZE 0x1305
+#define regGDS_VMID2_SIZE_BASE_IDX 0
+#define regGDS_VMID3_BASE 0x1306
+#define regGDS_VMID3_BASE_BASE_IDX 0
+#define regGDS_VMID3_SIZE 0x1307
+#define regGDS_VMID3_SIZE_BASE_IDX 0
+#define regGDS_VMID4_BASE 0x1308
+#define regGDS_VMID4_BASE_BASE_IDX 0
+#define regGDS_VMID4_SIZE 0x1309
+#define regGDS_VMID4_SIZE_BASE_IDX 0
+#define regGDS_VMID5_BASE 0x130a
+#define regGDS_VMID5_BASE_BASE_IDX 0
+#define regGDS_VMID5_SIZE 0x130b
+#define regGDS_VMID5_SIZE_BASE_IDX 0
+#define regGDS_VMID6_BASE 0x130c
+#define regGDS_VMID6_BASE_BASE_IDX 0
+#define regGDS_VMID6_SIZE 0x130d
+#define regGDS_VMID6_SIZE_BASE_IDX 0
+#define regGDS_VMID7_BASE 0x130e
+#define regGDS_VMID7_BASE_BASE_IDX 0
+#define regGDS_VMID7_SIZE 0x130f
+#define regGDS_VMID7_SIZE_BASE_IDX 0
+#define regGDS_VMID8_BASE 0x1310
+#define regGDS_VMID8_BASE_BASE_IDX 0
+#define regGDS_VMID8_SIZE 0x1311
+#define regGDS_VMID8_SIZE_BASE_IDX 0
+#define regGDS_VMID9_BASE 0x1312
+#define regGDS_VMID9_BASE_BASE_IDX 0
+#define regGDS_VMID9_SIZE 0x1313
+#define regGDS_VMID9_SIZE_BASE_IDX 0
+#define regGDS_VMID10_BASE 0x1314
+#define regGDS_VMID10_BASE_BASE_IDX 0
+#define regGDS_VMID10_SIZE 0x1315
+#define regGDS_VMID10_SIZE_BASE_IDX 0
+#define regGDS_VMID11_BASE 0x1316
+#define regGDS_VMID11_BASE_BASE_IDX 0
+#define regGDS_VMID11_SIZE 0x1317
+#define regGDS_VMID11_SIZE_BASE_IDX 0
+#define regGDS_VMID12_BASE 0x1318
+#define regGDS_VMID12_BASE_BASE_IDX 0
+#define regGDS_VMID12_SIZE 0x1319
+#define regGDS_VMID12_SIZE_BASE_IDX 0
+#define regGDS_VMID13_BASE 0x131a
+#define regGDS_VMID13_BASE_BASE_IDX 0
+#define regGDS_VMID13_SIZE 0x131b
+#define regGDS_VMID13_SIZE_BASE_IDX 0
+#define regGDS_VMID14_BASE 0x131c
+#define regGDS_VMID14_BASE_BASE_IDX 0
+#define regGDS_VMID14_SIZE 0x131d
+#define regGDS_VMID14_SIZE_BASE_IDX 0
+#define regGDS_VMID15_BASE 0x131e
+#define regGDS_VMID15_BASE_BASE_IDX 0
+#define regGDS_VMID15_SIZE 0x131f
+#define regGDS_VMID15_SIZE_BASE_IDX 0
+#define regGDS_GWS_VMID0 0x1320
+#define regGDS_GWS_VMID0_BASE_IDX 0
+#define regGDS_GWS_VMID1 0x1321
+#define regGDS_GWS_VMID1_BASE_IDX 0
+#define regGDS_GWS_VMID2 0x1322
+#define regGDS_GWS_VMID2_BASE_IDX 0
+#define regGDS_GWS_VMID3 0x1323
+#define regGDS_GWS_VMID3_BASE_IDX 0
+#define regGDS_GWS_VMID4 0x1324
+#define regGDS_GWS_VMID4_BASE_IDX 0
+#define regGDS_GWS_VMID5 0x1325
+#define regGDS_GWS_VMID5_BASE_IDX 0
+#define regGDS_GWS_VMID6 0x1326
+#define regGDS_GWS_VMID6_BASE_IDX 0
+#define regGDS_GWS_VMID7 0x1327
+#define regGDS_GWS_VMID7_BASE_IDX 0
+#define regGDS_GWS_VMID8 0x1328
+#define regGDS_GWS_VMID8_BASE_IDX 0
+#define regGDS_GWS_VMID9 0x1329
+#define regGDS_GWS_VMID9_BASE_IDX 0
+#define regGDS_GWS_VMID10 0x132a
+#define regGDS_GWS_VMID10_BASE_IDX 0
+#define regGDS_GWS_VMID11 0x132b
+#define regGDS_GWS_VMID11_BASE_IDX 0
+#define regGDS_GWS_VMID12 0x132c
+#define regGDS_GWS_VMID12_BASE_IDX 0
+#define regGDS_GWS_VMID13 0x132d
+#define regGDS_GWS_VMID13_BASE_IDX 0
+#define regGDS_GWS_VMID14 0x132e
+#define regGDS_GWS_VMID14_BASE_IDX 0
+#define regGDS_GWS_VMID15 0x132f
+#define regGDS_GWS_VMID15_BASE_IDX 0
+#define regGDS_OA_VMID0 0x1330
+#define regGDS_OA_VMID0_BASE_IDX 0
+#define regGDS_OA_VMID1 0x1331
+#define regGDS_OA_VMID1_BASE_IDX 0
+#define regGDS_OA_VMID2 0x1332
+#define regGDS_OA_VMID2_BASE_IDX 0
+#define regGDS_OA_VMID3 0x1333
+#define regGDS_OA_VMID3_BASE_IDX 0
+#define regGDS_OA_VMID4 0x1334
+#define regGDS_OA_VMID4_BASE_IDX 0
+#define regGDS_OA_VMID5 0x1335
+#define regGDS_OA_VMID5_BASE_IDX 0
+#define regGDS_OA_VMID6 0x1336
+#define regGDS_OA_VMID6_BASE_IDX 0
+#define regGDS_OA_VMID7 0x1337
+#define regGDS_OA_VMID7_BASE_IDX 0
+#define regGDS_OA_VMID8 0x1338
+#define regGDS_OA_VMID8_BASE_IDX 0
+#define regGDS_OA_VMID9 0x1339
+#define regGDS_OA_VMID9_BASE_IDX 0
+#define regGDS_OA_VMID10 0x133a
+#define regGDS_OA_VMID10_BASE_IDX 0
+#define regGDS_OA_VMID11 0x133b
+#define regGDS_OA_VMID11_BASE_IDX 0
+#define regGDS_OA_VMID12 0x133c
+#define regGDS_OA_VMID12_BASE_IDX 0
+#define regGDS_OA_VMID13 0x133d
+#define regGDS_OA_VMID13_BASE_IDX 0
+#define regGDS_OA_VMID14 0x133e
+#define regGDS_OA_VMID14_BASE_IDX 0
+#define regGDS_OA_VMID15 0x133f
+#define regGDS_OA_VMID15_BASE_IDX 0
+#define regGDS_GWS_RESET0 0x1344
+#define regGDS_GWS_RESET0_BASE_IDX 0
+#define regGDS_GWS_RESET1 0x1345
+#define regGDS_GWS_RESET1_BASE_IDX 0
+#define regGDS_GWS_RESOURCE_RESET 0x1346
+#define regGDS_GWS_RESOURCE_RESET_BASE_IDX 0
+#define regGDS_COMPUTE_MAX_WAVE_ID 0x1348
+#define regGDS_COMPUTE_MAX_WAVE_ID_BASE_IDX 0
+#define regGDS_OA_RESET_MASK 0x1349
+#define regGDS_OA_RESET_MASK_BASE_IDX 0
+#define regGDS_OA_RESET 0x134a
+#define regGDS_OA_RESET_BASE_IDX 0
+#define regGDS_ENHANCE 0x134b
+#define regGDS_ENHANCE_BASE_IDX 0
+#define regGDS_OA_CGPG_RESTORE 0x134c
+#define regGDS_OA_CGPG_RESTORE_BASE_IDX 0
+#define regGDS_CS_CTXSW_STATUS 0x134d
+#define regGDS_CS_CTXSW_STATUS_BASE_IDX 0
+#define regGDS_CS_CTXSW_CNT0 0x134e
+#define regGDS_CS_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_CS_CTXSW_CNT1 0x134f
+#define regGDS_CS_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_CS_CTXSW_CNT2 0x1350
+#define regGDS_CS_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_CS_CTXSW_CNT3 0x1351
+#define regGDS_CS_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_GFX_CTXSW_STATUS 0x1352
+#define regGDS_GFX_CTXSW_STATUS_BASE_IDX 0
+#define regGDS_VS_CTXSW_CNT0 0x1353
+#define regGDS_VS_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_VS_CTXSW_CNT1 0x1354
+#define regGDS_VS_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_VS_CTXSW_CNT2 0x1355
+#define regGDS_VS_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_VS_CTXSW_CNT3 0x1356
+#define regGDS_VS_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS0_CTXSW_CNT0 0x1357
+#define regGDS_PS0_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS0_CTXSW_CNT1 0x1358
+#define regGDS_PS0_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS0_CTXSW_CNT2 0x1359
+#define regGDS_PS0_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS0_CTXSW_CNT3 0x135a
+#define regGDS_PS0_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS1_CTXSW_CNT0 0x135b
+#define regGDS_PS1_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS1_CTXSW_CNT1 0x135c
+#define regGDS_PS1_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS1_CTXSW_CNT2 0x135d
+#define regGDS_PS1_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS1_CTXSW_CNT3 0x135e
+#define regGDS_PS1_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS2_CTXSW_CNT0 0x135f
+#define regGDS_PS2_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS2_CTXSW_CNT1 0x1360
+#define regGDS_PS2_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS2_CTXSW_CNT2 0x1361
+#define regGDS_PS2_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS2_CTXSW_CNT3 0x1362
+#define regGDS_PS2_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS3_CTXSW_CNT0 0x1363
+#define regGDS_PS3_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS3_CTXSW_CNT1 0x1364
+#define regGDS_PS3_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS3_CTXSW_CNT2 0x1365
+#define regGDS_PS3_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS3_CTXSW_CNT3 0x1366
+#define regGDS_PS3_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS4_CTXSW_CNT0 0x1367
+#define regGDS_PS4_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS4_CTXSW_CNT1 0x1368
+#define regGDS_PS4_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS4_CTXSW_CNT2 0x1369
+#define regGDS_PS4_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS4_CTXSW_CNT3 0x136a
+#define regGDS_PS4_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS5_CTXSW_CNT0 0x136b
+#define regGDS_PS5_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS5_CTXSW_CNT1 0x136c
+#define regGDS_PS5_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS5_CTXSW_CNT2 0x136d
+#define regGDS_PS5_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS5_CTXSW_CNT3 0x136e
+#define regGDS_PS5_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS6_CTXSW_CNT0 0x136f
+#define regGDS_PS6_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS6_CTXSW_CNT1 0x1370
+#define regGDS_PS6_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS6_CTXSW_CNT2 0x1371
+#define regGDS_PS6_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS6_CTXSW_CNT3 0x1372
+#define regGDS_PS6_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_PS7_CTXSW_CNT0 0x1373
+#define regGDS_PS7_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_PS7_CTXSW_CNT1 0x1374
+#define regGDS_PS7_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_PS7_CTXSW_CNT2 0x1375
+#define regGDS_PS7_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_PS7_CTXSW_CNT3 0x1376
+#define regGDS_PS7_CTXSW_CNT3_BASE_IDX 0
+#define regGDS_GS_CTXSW_CNT0 0x1377
+#define regGDS_GS_CTXSW_CNT0_BASE_IDX 0
+#define regGDS_GS_CTXSW_CNT1 0x1378
+#define regGDS_GS_CTXSW_CNT1_BASE_IDX 0
+#define regGDS_GS_CTXSW_CNT2 0x1379
+#define regGDS_GS_CTXSW_CNT2_BASE_IDX 0
+#define regGDS_GS_CTXSW_CNT3 0x137a
+#define regGDS_GS_CTXSW_CNT3_BASE_IDX 0
+
+
+// addressBlock: gc_gfxdec0
+// base address: 0x28000
+#define regDB_RENDER_CONTROL 0x0000
+#define regDB_RENDER_CONTROL_BASE_IDX 1
+#define regDB_COUNT_CONTROL 0x0001
+#define regDB_COUNT_CONTROL_BASE_IDX 1
+#define regDB_DEPTH_VIEW 0x0002
+#define regDB_DEPTH_VIEW_BASE_IDX 1
+#define regDB_RENDER_OVERRIDE 0x0003
+#define regDB_RENDER_OVERRIDE_BASE_IDX 1
+#define regDB_RENDER_OVERRIDE2 0x0004
+#define regDB_RENDER_OVERRIDE2_BASE_IDX 1
+#define regDB_HTILE_DATA_BASE 0x0005
+#define regDB_HTILE_DATA_BASE_BASE_IDX 1
+#define regDB_HTILE_DATA_BASE_HI 0x0006
+#define regDB_HTILE_DATA_BASE_HI_BASE_IDX 1
+#define regDB_DEPTH_SIZE 0x0007
+#define regDB_DEPTH_SIZE_BASE_IDX 1
+#define regDB_DEPTH_BOUNDS_MIN 0x0008
+#define regDB_DEPTH_BOUNDS_MIN_BASE_IDX 1
+#define regDB_DEPTH_BOUNDS_MAX 0x0009
+#define regDB_DEPTH_BOUNDS_MAX_BASE_IDX 1
+#define regDB_STENCIL_CLEAR 0x000a
+#define regDB_STENCIL_CLEAR_BASE_IDX 1
+#define regDB_DEPTH_CLEAR 0x000b
+#define regDB_DEPTH_CLEAR_BASE_IDX 1
+#define regPA_SC_SCREEN_SCISSOR_TL 0x000c
+#define regPA_SC_SCREEN_SCISSOR_TL_BASE_IDX 1
+#define regPA_SC_SCREEN_SCISSOR_BR 0x000d
+#define regPA_SC_SCREEN_SCISSOR_BR_BASE_IDX 1
+#define regDB_Z_INFO 0x000e
+#define regDB_Z_INFO_BASE_IDX 1
+#define regDB_STENCIL_INFO 0x000f
+#define regDB_STENCIL_INFO_BASE_IDX 1
+#define regDB_Z_READ_BASE 0x0010
+#define regDB_Z_READ_BASE_BASE_IDX 1
+#define regDB_Z_READ_BASE_HI 0x0011
+#define regDB_Z_READ_BASE_HI_BASE_IDX 1
+#define regDB_STENCIL_READ_BASE 0x0012
+#define regDB_STENCIL_READ_BASE_BASE_IDX 1
+#define regDB_STENCIL_READ_BASE_HI 0x0013
+#define regDB_STENCIL_READ_BASE_HI_BASE_IDX 1
+#define regDB_Z_WRITE_BASE 0x0014
+#define regDB_Z_WRITE_BASE_BASE_IDX 1
+#define regDB_Z_WRITE_BASE_HI 0x0015
+#define regDB_Z_WRITE_BASE_HI_BASE_IDX 1
+#define regDB_STENCIL_WRITE_BASE 0x0016
+#define regDB_STENCIL_WRITE_BASE_BASE_IDX 1
+#define regDB_STENCIL_WRITE_BASE_HI 0x0017
+#define regDB_STENCIL_WRITE_BASE_HI_BASE_IDX 1
+#define regDB_DFSM_CONTROL 0x0018
+#define regDB_DFSM_CONTROL_BASE_IDX 1
+#define regDB_Z_INFO2 0x001a
+#define regDB_Z_INFO2_BASE_IDX 1
+#define regDB_STENCIL_INFO2 0x001b
+#define regDB_STENCIL_INFO2_BASE_IDX 1
+#define regCOHER_DEST_BASE_HI_0 0x007a
+#define regCOHER_DEST_BASE_HI_0_BASE_IDX 1
+#define regCOHER_DEST_BASE_HI_1 0x007b
+#define regCOHER_DEST_BASE_HI_1_BASE_IDX 1
+#define regCOHER_DEST_BASE_HI_2 0x007c
+#define regCOHER_DEST_BASE_HI_2_BASE_IDX 1
+#define regCOHER_DEST_BASE_HI_3 0x007d
+#define regCOHER_DEST_BASE_HI_3_BASE_IDX 1
+#define regCOHER_DEST_BASE_2 0x007e
+#define regCOHER_DEST_BASE_2_BASE_IDX 1
+#define regCOHER_DEST_BASE_3 0x007f
+#define regCOHER_DEST_BASE_3_BASE_IDX 1
+#define regPA_SC_WINDOW_OFFSET 0x0080
+#define regPA_SC_WINDOW_OFFSET_BASE_IDX 1
+#define regPA_SC_WINDOW_SCISSOR_TL 0x0081
+#define regPA_SC_WINDOW_SCISSOR_TL_BASE_IDX 1
+#define regPA_SC_WINDOW_SCISSOR_BR 0x0082
+#define regPA_SC_WINDOW_SCISSOR_BR_BASE_IDX 1
+#define regPA_SC_CLIPRECT_RULE 0x0083
+#define regPA_SC_CLIPRECT_RULE_BASE_IDX 1
+#define regPA_SC_CLIPRECT_0_TL 0x0084
+#define regPA_SC_CLIPRECT_0_TL_BASE_IDX 1
+#define regPA_SC_CLIPRECT_0_BR 0x0085
+#define regPA_SC_CLIPRECT_0_BR_BASE_IDX 1
+#define regPA_SC_CLIPRECT_1_TL 0x0086
+#define regPA_SC_CLIPRECT_1_TL_BASE_IDX 1
+#define regPA_SC_CLIPRECT_1_BR 0x0087
+#define regPA_SC_CLIPRECT_1_BR_BASE_IDX 1
+#define regPA_SC_CLIPRECT_2_TL 0x0088
+#define regPA_SC_CLIPRECT_2_TL_BASE_IDX 1
+#define regPA_SC_CLIPRECT_2_BR 0x0089
+#define regPA_SC_CLIPRECT_2_BR_BASE_IDX 1
+#define regPA_SC_CLIPRECT_3_TL 0x008a
+#define regPA_SC_CLIPRECT_3_TL_BASE_IDX 1
+#define regPA_SC_CLIPRECT_3_BR 0x008b
+#define regPA_SC_CLIPRECT_3_BR_BASE_IDX 1
+#define regPA_SC_EDGERULE 0x008c
+#define regPA_SC_EDGERULE_BASE_IDX 1
+#define regPA_SU_HARDWARE_SCREEN_OFFSET 0x008d
+#define regPA_SU_HARDWARE_SCREEN_OFFSET_BASE_IDX 1
+#define regCB_TARGET_MASK 0x008e
+#define regCB_TARGET_MASK_BASE_IDX 1
+#define regCB_SHADER_MASK 0x008f
+#define regCB_SHADER_MASK_BASE_IDX 1
+#define regPA_SC_GENERIC_SCISSOR_TL 0x0090
+#define regPA_SC_GENERIC_SCISSOR_TL_BASE_IDX 1
+#define regPA_SC_GENERIC_SCISSOR_BR 0x0091
+#define regPA_SC_GENERIC_SCISSOR_BR_BASE_IDX 1
+#define regCOHER_DEST_BASE_0 0x0092
+#define regCOHER_DEST_BASE_0_BASE_IDX 1
+#define regCOHER_DEST_BASE_1 0x0093
+#define regCOHER_DEST_BASE_1_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_0_TL 0x0094
+#define regPA_SC_VPORT_SCISSOR_0_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_0_BR 0x0095
+#define regPA_SC_VPORT_SCISSOR_0_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_1_TL 0x0096
+#define regPA_SC_VPORT_SCISSOR_1_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_1_BR 0x0097
+#define regPA_SC_VPORT_SCISSOR_1_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_2_TL 0x0098
+#define regPA_SC_VPORT_SCISSOR_2_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_2_BR 0x0099
+#define regPA_SC_VPORT_SCISSOR_2_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_3_TL 0x009a
+#define regPA_SC_VPORT_SCISSOR_3_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_3_BR 0x009b
+#define regPA_SC_VPORT_SCISSOR_3_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_4_TL 0x009c
+#define regPA_SC_VPORT_SCISSOR_4_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_4_BR 0x009d
+#define regPA_SC_VPORT_SCISSOR_4_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_5_TL 0x009e
+#define regPA_SC_VPORT_SCISSOR_5_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_5_BR 0x009f
+#define regPA_SC_VPORT_SCISSOR_5_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_6_TL 0x00a0
+#define regPA_SC_VPORT_SCISSOR_6_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_6_BR 0x00a1
+#define regPA_SC_VPORT_SCISSOR_6_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_7_TL 0x00a2
+#define regPA_SC_VPORT_SCISSOR_7_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_7_BR 0x00a3
+#define regPA_SC_VPORT_SCISSOR_7_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_8_TL 0x00a4
+#define regPA_SC_VPORT_SCISSOR_8_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_8_BR 0x00a5
+#define regPA_SC_VPORT_SCISSOR_8_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_9_TL 0x00a6
+#define regPA_SC_VPORT_SCISSOR_9_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_9_BR 0x00a7
+#define regPA_SC_VPORT_SCISSOR_9_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_10_TL 0x00a8
+#define regPA_SC_VPORT_SCISSOR_10_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_10_BR 0x00a9
+#define regPA_SC_VPORT_SCISSOR_10_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_11_TL 0x00aa
+#define regPA_SC_VPORT_SCISSOR_11_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_11_BR 0x00ab
+#define regPA_SC_VPORT_SCISSOR_11_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_12_TL 0x00ac
+#define regPA_SC_VPORT_SCISSOR_12_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_12_BR 0x00ad
+#define regPA_SC_VPORT_SCISSOR_12_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_13_TL 0x00ae
+#define regPA_SC_VPORT_SCISSOR_13_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_13_BR 0x00af
+#define regPA_SC_VPORT_SCISSOR_13_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_14_TL 0x00b0
+#define regPA_SC_VPORT_SCISSOR_14_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_14_BR 0x00b1
+#define regPA_SC_VPORT_SCISSOR_14_BR_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_15_TL 0x00b2
+#define regPA_SC_VPORT_SCISSOR_15_TL_BASE_IDX 1
+#define regPA_SC_VPORT_SCISSOR_15_BR 0x00b3
+#define regPA_SC_VPORT_SCISSOR_15_BR_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_0 0x00b4
+#define regPA_SC_VPORT_ZMIN_0_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_0 0x00b5
+#define regPA_SC_VPORT_ZMAX_0_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_1 0x00b6
+#define regPA_SC_VPORT_ZMIN_1_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_1 0x00b7
+#define regPA_SC_VPORT_ZMAX_1_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_2 0x00b8
+#define regPA_SC_VPORT_ZMIN_2_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_2 0x00b9
+#define regPA_SC_VPORT_ZMAX_2_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_3 0x00ba
+#define regPA_SC_VPORT_ZMIN_3_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_3 0x00bb
+#define regPA_SC_VPORT_ZMAX_3_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_4 0x00bc
+#define regPA_SC_VPORT_ZMIN_4_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_4 0x00bd
+#define regPA_SC_VPORT_ZMAX_4_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_5 0x00be
+#define regPA_SC_VPORT_ZMIN_5_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_5 0x00bf
+#define regPA_SC_VPORT_ZMAX_5_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_6 0x00c0
+#define regPA_SC_VPORT_ZMIN_6_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_6 0x00c1
+#define regPA_SC_VPORT_ZMAX_6_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_7 0x00c2
+#define regPA_SC_VPORT_ZMIN_7_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_7 0x00c3
+#define regPA_SC_VPORT_ZMAX_7_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_8 0x00c4
+#define regPA_SC_VPORT_ZMIN_8_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_8 0x00c5
+#define regPA_SC_VPORT_ZMAX_8_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_9 0x00c6
+#define regPA_SC_VPORT_ZMIN_9_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_9 0x00c7
+#define regPA_SC_VPORT_ZMAX_9_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_10 0x00c8
+#define regPA_SC_VPORT_ZMIN_10_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_10 0x00c9
+#define regPA_SC_VPORT_ZMAX_10_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_11 0x00ca
+#define regPA_SC_VPORT_ZMIN_11_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_11 0x00cb
+#define regPA_SC_VPORT_ZMAX_11_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_12 0x00cc
+#define regPA_SC_VPORT_ZMIN_12_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_12 0x00cd
+#define regPA_SC_VPORT_ZMAX_12_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_13 0x00ce
+#define regPA_SC_VPORT_ZMIN_13_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_13 0x00cf
+#define regPA_SC_VPORT_ZMAX_13_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_14 0x00d0
+#define regPA_SC_VPORT_ZMIN_14_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_14 0x00d1
+#define regPA_SC_VPORT_ZMAX_14_BASE_IDX 1
+#define regPA_SC_VPORT_ZMIN_15 0x00d2
+#define regPA_SC_VPORT_ZMIN_15_BASE_IDX 1
+#define regPA_SC_VPORT_ZMAX_15 0x00d3
+#define regPA_SC_VPORT_ZMAX_15_BASE_IDX 1
+#define regPA_SC_RASTER_CONFIG 0x00d4
+#define regPA_SC_RASTER_CONFIG_BASE_IDX 1
+#define regPA_SC_RASTER_CONFIG_1 0x00d5
+#define regPA_SC_RASTER_CONFIG_1_BASE_IDX 1
+#define regPA_SC_SCREEN_EXTENT_CONTROL 0x00d6
+#define regPA_SC_SCREEN_EXTENT_CONTROL_BASE_IDX 1
+#define regPA_SC_TILE_STEERING_OVERRIDE 0x00d7
+#define regPA_SC_TILE_STEERING_OVERRIDE_BASE_IDX 1
+#define regCP_PERFMON_CNTX_CNTL 0x00d8
+#define regCP_PERFMON_CNTX_CNTL_BASE_IDX 1
+#define regCP_PIPEID 0x00d9
+#define regCP_PIPEID_BASE_IDX 1
+#define regCP_RINGID 0x00d9
+#define regCP_RINGID_BASE_IDX 1
+#define regCP_VMID 0x00da
+#define regCP_VMID_BASE_IDX 1
+#define regPA_SC_RIGHT_VERT_GRID 0x00e8
+#define regPA_SC_RIGHT_VERT_GRID_BASE_IDX 1
+#define regPA_SC_LEFT_VERT_GRID 0x00e9
+#define regPA_SC_LEFT_VERT_GRID_BASE_IDX 1
+#define regPA_SC_HORIZ_GRID 0x00ea
+#define regPA_SC_HORIZ_GRID_BASE_IDX 1
+#define regVGT_MULTI_PRIM_IB_RESET_INDX 0x0103
+#define regVGT_MULTI_PRIM_IB_RESET_INDX_BASE_IDX 1
+#define regCB_BLEND_RED 0x0105
+#define regCB_BLEND_RED_BASE_IDX 1
+#define regCB_BLEND_GREEN 0x0106
+#define regCB_BLEND_GREEN_BASE_IDX 1
+#define regCB_BLEND_BLUE 0x0107
+#define regCB_BLEND_BLUE_BASE_IDX 1
+#define regCB_BLEND_ALPHA 0x0108
+#define regCB_BLEND_ALPHA_BASE_IDX 1
+#define regCB_DCC_CONTROL 0x0109
+#define regCB_DCC_CONTROL_BASE_IDX 1
+#define regDB_STENCIL_CONTROL 0x010b
+#define regDB_STENCIL_CONTROL_BASE_IDX 1
+#define regDB_STENCILREFMASK 0x010c
+#define regDB_STENCILREFMASK_BASE_IDX 1
+#define regDB_STENCILREFMASK_BF 0x010d
+#define regDB_STENCILREFMASK_BF_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE 0x010f
+#define regPA_CL_VPORT_XSCALE_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET 0x0110
+#define regPA_CL_VPORT_XOFFSET_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE 0x0111
+#define regPA_CL_VPORT_YSCALE_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET 0x0112
+#define regPA_CL_VPORT_YOFFSET_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE 0x0113
+#define regPA_CL_VPORT_ZSCALE_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET 0x0114
+#define regPA_CL_VPORT_ZOFFSET_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_1 0x0115
+#define regPA_CL_VPORT_XSCALE_1_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_1 0x0116
+#define regPA_CL_VPORT_XOFFSET_1_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_1 0x0117
+#define regPA_CL_VPORT_YSCALE_1_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_1 0x0118
+#define regPA_CL_VPORT_YOFFSET_1_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_1 0x0119
+#define regPA_CL_VPORT_ZSCALE_1_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_1 0x011a
+#define regPA_CL_VPORT_ZOFFSET_1_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_2 0x011b
+#define regPA_CL_VPORT_XSCALE_2_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_2 0x011c
+#define regPA_CL_VPORT_XOFFSET_2_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_2 0x011d
+#define regPA_CL_VPORT_YSCALE_2_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_2 0x011e
+#define regPA_CL_VPORT_YOFFSET_2_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_2 0x011f
+#define regPA_CL_VPORT_ZSCALE_2_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_2 0x0120
+#define regPA_CL_VPORT_ZOFFSET_2_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_3 0x0121
+#define regPA_CL_VPORT_XSCALE_3_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_3 0x0122
+#define regPA_CL_VPORT_XOFFSET_3_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_3 0x0123
+#define regPA_CL_VPORT_YSCALE_3_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_3 0x0124
+#define regPA_CL_VPORT_YOFFSET_3_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_3 0x0125
+#define regPA_CL_VPORT_ZSCALE_3_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_3 0x0126
+#define regPA_CL_VPORT_ZOFFSET_3_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_4 0x0127
+#define regPA_CL_VPORT_XSCALE_4_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_4 0x0128
+#define regPA_CL_VPORT_XOFFSET_4_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_4 0x0129
+#define regPA_CL_VPORT_YSCALE_4_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_4 0x012a
+#define regPA_CL_VPORT_YOFFSET_4_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_4 0x012b
+#define regPA_CL_VPORT_ZSCALE_4_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_4 0x012c
+#define regPA_CL_VPORT_ZOFFSET_4_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_5 0x012d
+#define regPA_CL_VPORT_XSCALE_5_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_5 0x012e
+#define regPA_CL_VPORT_XOFFSET_5_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_5 0x012f
+#define regPA_CL_VPORT_YSCALE_5_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_5 0x0130
+#define regPA_CL_VPORT_YOFFSET_5_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_5 0x0131
+#define regPA_CL_VPORT_ZSCALE_5_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_5 0x0132
+#define regPA_CL_VPORT_ZOFFSET_5_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_6 0x0133
+#define regPA_CL_VPORT_XSCALE_6_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_6 0x0134
+#define regPA_CL_VPORT_XOFFSET_6_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_6 0x0135
+#define regPA_CL_VPORT_YSCALE_6_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_6 0x0136
+#define regPA_CL_VPORT_YOFFSET_6_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_6 0x0137
+#define regPA_CL_VPORT_ZSCALE_6_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_6 0x0138
+#define regPA_CL_VPORT_ZOFFSET_6_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_7 0x0139
+#define regPA_CL_VPORT_XSCALE_7_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_7 0x013a
+#define regPA_CL_VPORT_XOFFSET_7_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_7 0x013b
+#define regPA_CL_VPORT_YSCALE_7_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_7 0x013c
+#define regPA_CL_VPORT_YOFFSET_7_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_7 0x013d
+#define regPA_CL_VPORT_ZSCALE_7_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_7 0x013e
+#define regPA_CL_VPORT_ZOFFSET_7_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_8 0x013f
+#define regPA_CL_VPORT_XSCALE_8_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_8 0x0140
+#define regPA_CL_VPORT_XOFFSET_8_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_8 0x0141
+#define regPA_CL_VPORT_YSCALE_8_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_8 0x0142
+#define regPA_CL_VPORT_YOFFSET_8_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_8 0x0143
+#define regPA_CL_VPORT_ZSCALE_8_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_8 0x0144
+#define regPA_CL_VPORT_ZOFFSET_8_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_9 0x0145
+#define regPA_CL_VPORT_XSCALE_9_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_9 0x0146
+#define regPA_CL_VPORT_XOFFSET_9_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_9 0x0147
+#define regPA_CL_VPORT_YSCALE_9_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_9 0x0148
+#define regPA_CL_VPORT_YOFFSET_9_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_9 0x0149
+#define regPA_CL_VPORT_ZSCALE_9_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_9 0x014a
+#define regPA_CL_VPORT_ZOFFSET_9_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_10 0x014b
+#define regPA_CL_VPORT_XSCALE_10_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_10 0x014c
+#define regPA_CL_VPORT_XOFFSET_10_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_10 0x014d
+#define regPA_CL_VPORT_YSCALE_10_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_10 0x014e
+#define regPA_CL_VPORT_YOFFSET_10_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_10 0x014f
+#define regPA_CL_VPORT_ZSCALE_10_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_10 0x0150
+#define regPA_CL_VPORT_ZOFFSET_10_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_11 0x0151
+#define regPA_CL_VPORT_XSCALE_11_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_11 0x0152
+#define regPA_CL_VPORT_XOFFSET_11_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_11 0x0153
+#define regPA_CL_VPORT_YSCALE_11_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_11 0x0154
+#define regPA_CL_VPORT_YOFFSET_11_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_11 0x0155
+#define regPA_CL_VPORT_ZSCALE_11_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_11 0x0156
+#define regPA_CL_VPORT_ZOFFSET_11_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_12 0x0157
+#define regPA_CL_VPORT_XSCALE_12_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_12 0x0158
+#define regPA_CL_VPORT_XOFFSET_12_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_12 0x0159
+#define regPA_CL_VPORT_YSCALE_12_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_12 0x015a
+#define regPA_CL_VPORT_YOFFSET_12_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_12 0x015b
+#define regPA_CL_VPORT_ZSCALE_12_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_12 0x015c
+#define regPA_CL_VPORT_ZOFFSET_12_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_13 0x015d
+#define regPA_CL_VPORT_XSCALE_13_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_13 0x015e
+#define regPA_CL_VPORT_XOFFSET_13_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_13 0x015f
+#define regPA_CL_VPORT_YSCALE_13_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_13 0x0160
+#define regPA_CL_VPORT_YOFFSET_13_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_13 0x0161
+#define regPA_CL_VPORT_ZSCALE_13_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_13 0x0162
+#define regPA_CL_VPORT_ZOFFSET_13_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_14 0x0163
+#define regPA_CL_VPORT_XSCALE_14_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_14 0x0164
+#define regPA_CL_VPORT_XOFFSET_14_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_14 0x0165
+#define regPA_CL_VPORT_YSCALE_14_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_14 0x0166
+#define regPA_CL_VPORT_YOFFSET_14_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_14 0x0167
+#define regPA_CL_VPORT_ZSCALE_14_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_14 0x0168
+#define regPA_CL_VPORT_ZOFFSET_14_BASE_IDX 1
+#define regPA_CL_VPORT_XSCALE_15 0x0169
+#define regPA_CL_VPORT_XSCALE_15_BASE_IDX 1
+#define regPA_CL_VPORT_XOFFSET_15 0x016a
+#define regPA_CL_VPORT_XOFFSET_15_BASE_IDX 1
+#define regPA_CL_VPORT_YSCALE_15 0x016b
+#define regPA_CL_VPORT_YSCALE_15_BASE_IDX 1
+#define regPA_CL_VPORT_YOFFSET_15 0x016c
+#define regPA_CL_VPORT_YOFFSET_15_BASE_IDX 1
+#define regPA_CL_VPORT_ZSCALE_15 0x016d
+#define regPA_CL_VPORT_ZSCALE_15_BASE_IDX 1
+#define regPA_CL_VPORT_ZOFFSET_15 0x016e
+#define regPA_CL_VPORT_ZOFFSET_15_BASE_IDX 1
+#define regPA_CL_UCP_0_X 0x016f
+#define regPA_CL_UCP_0_X_BASE_IDX 1
+#define regPA_CL_UCP_0_Y 0x0170
+#define regPA_CL_UCP_0_Y_BASE_IDX 1
+#define regPA_CL_UCP_0_Z 0x0171
+#define regPA_CL_UCP_0_Z_BASE_IDX 1
+#define regPA_CL_UCP_0_W 0x0172
+#define regPA_CL_UCP_0_W_BASE_IDX 1
+#define regPA_CL_UCP_1_X 0x0173
+#define regPA_CL_UCP_1_X_BASE_IDX 1
+#define regPA_CL_UCP_1_Y 0x0174
+#define regPA_CL_UCP_1_Y_BASE_IDX 1
+#define regPA_CL_UCP_1_Z 0x0175
+#define regPA_CL_UCP_1_Z_BASE_IDX 1
+#define regPA_CL_UCP_1_W 0x0176
+#define regPA_CL_UCP_1_W_BASE_IDX 1
+#define regPA_CL_UCP_2_X 0x0177
+#define regPA_CL_UCP_2_X_BASE_IDX 1
+#define regPA_CL_UCP_2_Y 0x0178
+#define regPA_CL_UCP_2_Y_BASE_IDX 1
+#define regPA_CL_UCP_2_Z 0x0179
+#define regPA_CL_UCP_2_Z_BASE_IDX 1
+#define regPA_CL_UCP_2_W 0x017a
+#define regPA_CL_UCP_2_W_BASE_IDX 1
+#define regPA_CL_UCP_3_X 0x017b
+#define regPA_CL_UCP_3_X_BASE_IDX 1
+#define regPA_CL_UCP_3_Y 0x017c
+#define regPA_CL_UCP_3_Y_BASE_IDX 1
+#define regPA_CL_UCP_3_Z 0x017d
+#define regPA_CL_UCP_3_Z_BASE_IDX 1
+#define regPA_CL_UCP_3_W 0x017e
+#define regPA_CL_UCP_3_W_BASE_IDX 1
+#define regPA_CL_UCP_4_X 0x017f
+#define regPA_CL_UCP_4_X_BASE_IDX 1
+#define regPA_CL_UCP_4_Y 0x0180
+#define regPA_CL_UCP_4_Y_BASE_IDX 1
+#define regPA_CL_UCP_4_Z 0x0181
+#define regPA_CL_UCP_4_Z_BASE_IDX 1
+#define regPA_CL_UCP_4_W 0x0182
+#define regPA_CL_UCP_4_W_BASE_IDX 1
+#define regPA_CL_UCP_5_X 0x0183
+#define regPA_CL_UCP_5_X_BASE_IDX 1
+#define regPA_CL_UCP_5_Y 0x0184
+#define regPA_CL_UCP_5_Y_BASE_IDX 1
+#define regPA_CL_UCP_5_Z 0x0185
+#define regPA_CL_UCP_5_Z_BASE_IDX 1
+#define regPA_CL_UCP_5_W 0x0186
+#define regPA_CL_UCP_5_W_BASE_IDX 1
+#define regPA_CL_PROG_NEAR_CLIP_Z 0x0187
+#define regPA_CL_PROG_NEAR_CLIP_Z_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_0 0x0191
+#define regSPI_PS_INPUT_CNTL_0_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_1 0x0192
+#define regSPI_PS_INPUT_CNTL_1_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_2 0x0193
+#define regSPI_PS_INPUT_CNTL_2_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_3 0x0194
+#define regSPI_PS_INPUT_CNTL_3_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_4 0x0195
+#define regSPI_PS_INPUT_CNTL_4_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_5 0x0196
+#define regSPI_PS_INPUT_CNTL_5_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_6 0x0197
+#define regSPI_PS_INPUT_CNTL_6_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_7 0x0198
+#define regSPI_PS_INPUT_CNTL_7_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_8 0x0199
+#define regSPI_PS_INPUT_CNTL_8_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_9 0x019a
+#define regSPI_PS_INPUT_CNTL_9_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_10 0x019b
+#define regSPI_PS_INPUT_CNTL_10_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_11 0x019c
+#define regSPI_PS_INPUT_CNTL_11_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_12 0x019d
+#define regSPI_PS_INPUT_CNTL_12_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_13 0x019e
+#define regSPI_PS_INPUT_CNTL_13_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_14 0x019f
+#define regSPI_PS_INPUT_CNTL_14_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_15 0x01a0
+#define regSPI_PS_INPUT_CNTL_15_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_16 0x01a1
+#define regSPI_PS_INPUT_CNTL_16_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_17 0x01a2
+#define regSPI_PS_INPUT_CNTL_17_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_18 0x01a3
+#define regSPI_PS_INPUT_CNTL_18_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_19 0x01a4
+#define regSPI_PS_INPUT_CNTL_19_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_20 0x01a5
+#define regSPI_PS_INPUT_CNTL_20_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_21 0x01a6
+#define regSPI_PS_INPUT_CNTL_21_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_22 0x01a7
+#define regSPI_PS_INPUT_CNTL_22_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_23 0x01a8
+#define regSPI_PS_INPUT_CNTL_23_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_24 0x01a9
+#define regSPI_PS_INPUT_CNTL_24_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_25 0x01aa
+#define regSPI_PS_INPUT_CNTL_25_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_26 0x01ab
+#define regSPI_PS_INPUT_CNTL_26_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_27 0x01ac
+#define regSPI_PS_INPUT_CNTL_27_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_28 0x01ad
+#define regSPI_PS_INPUT_CNTL_28_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_29 0x01ae
+#define regSPI_PS_INPUT_CNTL_29_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_30 0x01af
+#define regSPI_PS_INPUT_CNTL_30_BASE_IDX 1
+#define regSPI_PS_INPUT_CNTL_31 0x01b0
+#define regSPI_PS_INPUT_CNTL_31_BASE_IDX 1
+#define regSPI_VS_OUT_CONFIG 0x01b1
+#define regSPI_VS_OUT_CONFIG_BASE_IDX 1
+#define regSPI_PS_INPUT_ENA 0x01b3
+#define regSPI_PS_INPUT_ENA_BASE_IDX 1
+#define regSPI_PS_INPUT_ADDR 0x01b4
+#define regSPI_PS_INPUT_ADDR_BASE_IDX 1
+#define regSPI_INTERP_CONTROL_0 0x01b5
+#define regSPI_INTERP_CONTROL_0_BASE_IDX 1
+#define regSPI_PS_IN_CONTROL 0x01b6
+#define regSPI_PS_IN_CONTROL_BASE_IDX 1
+#define regSPI_BARYC_CNTL 0x01b8
+#define regSPI_BARYC_CNTL_BASE_IDX 1
+#define regSPI_TMPRING_SIZE 0x01ba
+#define regSPI_TMPRING_SIZE_BASE_IDX 1
+#define regSPI_SHADER_POS_FORMAT 0x01c3
+#define regSPI_SHADER_POS_FORMAT_BASE_IDX 1
+#define regSPI_SHADER_Z_FORMAT 0x01c4
+#define regSPI_SHADER_Z_FORMAT_BASE_IDX 1
+#define regSPI_SHADER_COL_FORMAT 0x01c5
+#define regSPI_SHADER_COL_FORMAT_BASE_IDX 1
+#define regSX_PS_DOWNCONVERT 0x01d5
+#define regSX_PS_DOWNCONVERT_BASE_IDX 1
+#define regSX_BLEND_OPT_EPSILON 0x01d6
+#define regSX_BLEND_OPT_EPSILON_BASE_IDX 1
+#define regSX_BLEND_OPT_CONTROL 0x01d7
+#define regSX_BLEND_OPT_CONTROL_BASE_IDX 1
+#define regSX_MRT0_BLEND_OPT 0x01d8
+#define regSX_MRT0_BLEND_OPT_BASE_IDX 1
+#define regSX_MRT1_BLEND_OPT 0x01d9
+#define regSX_MRT1_BLEND_OPT_BASE_IDX 1
+#define regSX_MRT2_BLEND_OPT 0x01da
+#define regSX_MRT2_BLEND_OPT_BASE_IDX 1
+#define regSX_MRT3_BLEND_OPT 0x01db
+#define regSX_MRT3_BLEND_OPT_BASE_IDX 1
+#define regSX_MRT4_BLEND_OPT 0x01dc
+#define regSX_MRT4_BLEND_OPT_BASE_IDX 1
+#define regSX_MRT5_BLEND_OPT 0x01dd
+#define regSX_MRT5_BLEND_OPT_BASE_IDX 1
+#define regSX_MRT6_BLEND_OPT 0x01de
+#define regSX_MRT6_BLEND_OPT_BASE_IDX 1
+#define regSX_MRT7_BLEND_OPT 0x01df
+#define regSX_MRT7_BLEND_OPT_BASE_IDX 1
+#define regCB_BLEND0_CONTROL 0x01e0
+#define regCB_BLEND0_CONTROL_BASE_IDX 1
+#define regCB_BLEND1_CONTROL 0x01e1
+#define regCB_BLEND1_CONTROL_BASE_IDX 1
+#define regCB_BLEND2_CONTROL 0x01e2
+#define regCB_BLEND2_CONTROL_BASE_IDX 1
+#define regCB_BLEND3_CONTROL 0x01e3
+#define regCB_BLEND3_CONTROL_BASE_IDX 1
+#define regCB_BLEND4_CONTROL 0x01e4
+#define regCB_BLEND4_CONTROL_BASE_IDX 1
+#define regCB_BLEND5_CONTROL 0x01e5
+#define regCB_BLEND5_CONTROL_BASE_IDX 1
+#define regCB_BLEND6_CONTROL 0x01e6
+#define regCB_BLEND6_CONTROL_BASE_IDX 1
+#define regCB_BLEND7_CONTROL 0x01e7
+#define regCB_BLEND7_CONTROL_BASE_IDX 1
+#define regCB_MRT0_EPITCH 0x01e8
+#define regCB_MRT0_EPITCH_BASE_IDX 1
+#define regCB_MRT1_EPITCH 0x01e9
+#define regCB_MRT1_EPITCH_BASE_IDX 1
+#define regCB_MRT2_EPITCH 0x01ea
+#define regCB_MRT2_EPITCH_BASE_IDX 1
+#define regCB_MRT3_EPITCH 0x01eb
+#define regCB_MRT3_EPITCH_BASE_IDX 1
+#define regCB_MRT4_EPITCH 0x01ec
+#define regCB_MRT4_EPITCH_BASE_IDX 1
+#define regCB_MRT5_EPITCH 0x01ed
+#define regCB_MRT5_EPITCH_BASE_IDX 1
+#define regCB_MRT6_EPITCH 0x01ee
+#define regCB_MRT6_EPITCH_BASE_IDX 1
+#define regCB_MRT7_EPITCH 0x01ef
+#define regCB_MRT7_EPITCH_BASE_IDX 1
+#define regCS_COPY_STATE 0x01f3
+#define regCS_COPY_STATE_BASE_IDX 1
+#define regGFX_COPY_STATE 0x01f4
+#define regGFX_COPY_STATE_BASE_IDX 1
+#define regPA_CL_POINT_X_RAD 0x01f5
+#define regPA_CL_POINT_X_RAD_BASE_IDX 1
+#define regPA_CL_POINT_Y_RAD 0x01f6
+#define regPA_CL_POINT_Y_RAD_BASE_IDX 1
+#define regPA_CL_POINT_SIZE 0x01f7
+#define regPA_CL_POINT_SIZE_BASE_IDX 1
+#define regPA_CL_POINT_CULL_RAD 0x01f8
+#define regPA_CL_POINT_CULL_RAD_BASE_IDX 1
+#define regVGT_DMA_BASE_HI 0x01f9
+#define regVGT_DMA_BASE_HI_BASE_IDX 1
+#define regVGT_DMA_BASE 0x01fa
+#define regVGT_DMA_BASE_BASE_IDX 1
+#define regVGT_DRAW_INITIATOR 0x01fc
+#define regVGT_DRAW_INITIATOR_BASE_IDX 1
+#define regVGT_IMMED_DATA 0x01fd
+#define regVGT_IMMED_DATA_BASE_IDX 1
+#define regVGT_EVENT_ADDRESS_REG 0x01fe
+#define regVGT_EVENT_ADDRESS_REG_BASE_IDX 1
+#define regDB_DEPTH_CONTROL 0x0200
+#define regDB_DEPTH_CONTROL_BASE_IDX 1
+#define regDB_EQAA 0x0201
+#define regDB_EQAA_BASE_IDX 1
+#define regCB_COLOR_CONTROL 0x0202
+#define regCB_COLOR_CONTROL_BASE_IDX 1
+#define regDB_SHADER_CONTROL 0x0203
+#define regDB_SHADER_CONTROL_BASE_IDX 1
+#define regPA_CL_CLIP_CNTL 0x0204
+#define regPA_CL_CLIP_CNTL_BASE_IDX 1
+#define regPA_SU_SC_MODE_CNTL 0x0205
+#define regPA_SU_SC_MODE_CNTL_BASE_IDX 1
+#define regPA_CL_VTE_CNTL 0x0206
+#define regPA_CL_VTE_CNTL_BASE_IDX 1
+#define regPA_CL_VS_OUT_CNTL 0x0207
+#define regPA_CL_VS_OUT_CNTL_BASE_IDX 1
+#define regPA_CL_NANINF_CNTL 0x0208
+#define regPA_CL_NANINF_CNTL_BASE_IDX 1
+#define regPA_SU_LINE_STIPPLE_CNTL 0x0209
+#define regPA_SU_LINE_STIPPLE_CNTL_BASE_IDX 1
+#define regPA_SU_LINE_STIPPLE_SCALE 0x020a
+#define regPA_SU_LINE_STIPPLE_SCALE_BASE_IDX 1
+#define regPA_SU_PRIM_FILTER_CNTL 0x020b
+#define regPA_SU_PRIM_FILTER_CNTL_BASE_IDX 1
+#define regPA_SU_SMALL_PRIM_FILTER_CNTL 0x020c
+#define regPA_SU_SMALL_PRIM_FILTER_CNTL_BASE_IDX 1
+#define regPA_CL_OBJPRIM_ID_CNTL 0x020d
+#define regPA_CL_OBJPRIM_ID_CNTL_BASE_IDX 1
+#define regPA_CL_NGG_CNTL 0x020e
+#define regPA_CL_NGG_CNTL_BASE_IDX 1
+#define regPA_SU_OVER_RASTERIZATION_CNTL 0x020f
+#define regPA_SU_OVER_RASTERIZATION_CNTL_BASE_IDX 1
+#define regPA_STEREO_CNTL 0x0210
+#define regPA_STEREO_CNTL_BASE_IDX 1
+#define regPA_SU_POINT_SIZE 0x0280
+#define regPA_SU_POINT_SIZE_BASE_IDX 1
+#define regPA_SU_POINT_MINMAX 0x0281
+#define regPA_SU_POINT_MINMAX_BASE_IDX 1
+#define regPA_SU_LINE_CNTL 0x0282
+#define regPA_SU_LINE_CNTL_BASE_IDX 1
+#define regPA_SC_LINE_STIPPLE 0x0283
+#define regPA_SC_LINE_STIPPLE_BASE_IDX 1
+#define regVGT_OUTPUT_PATH_CNTL 0x0284
+#define regVGT_OUTPUT_PATH_CNTL_BASE_IDX 1
+#define regVGT_HOS_CNTL 0x0285
+#define regVGT_HOS_CNTL_BASE_IDX 1
+#define regVGT_HOS_MAX_TESS_LEVEL 0x0286
+#define regVGT_HOS_MAX_TESS_LEVEL_BASE_IDX 1
+#define regVGT_HOS_MIN_TESS_LEVEL 0x0287
+#define regVGT_HOS_MIN_TESS_LEVEL_BASE_IDX 1
+#define regVGT_HOS_REUSE_DEPTH 0x0288
+#define regVGT_HOS_REUSE_DEPTH_BASE_IDX 1
+#define regVGT_GROUP_PRIM_TYPE 0x0289
+#define regVGT_GROUP_PRIM_TYPE_BASE_IDX 1
+#define regVGT_GROUP_FIRST_DECR 0x028a
+#define regVGT_GROUP_FIRST_DECR_BASE_IDX 1
+#define regVGT_GROUP_DECR 0x028b
+#define regVGT_GROUP_DECR_BASE_IDX 1
+#define regVGT_GROUP_VECT_0_CNTL 0x028c
+#define regVGT_GROUP_VECT_0_CNTL_BASE_IDX 1
+#define regVGT_GROUP_VECT_1_CNTL 0x028d
+#define regVGT_GROUP_VECT_1_CNTL_BASE_IDX 1
+#define regVGT_GROUP_VECT_0_FMT_CNTL 0x028e
+#define regVGT_GROUP_VECT_0_FMT_CNTL_BASE_IDX 1
+#define regVGT_GROUP_VECT_1_FMT_CNTL 0x028f
+#define regVGT_GROUP_VECT_1_FMT_CNTL_BASE_IDX 1
+#define regVGT_GS_MODE 0x0290
+#define regVGT_GS_MODE_BASE_IDX 1
+#define regVGT_GS_ONCHIP_CNTL 0x0291
+#define regVGT_GS_ONCHIP_CNTL_BASE_IDX 1
+#define regPA_SC_MODE_CNTL_0 0x0292
+#define regPA_SC_MODE_CNTL_0_BASE_IDX 1
+#define regPA_SC_MODE_CNTL_1 0x0293
+#define regPA_SC_MODE_CNTL_1_BASE_IDX 1
+#define regVGT_ENHANCE 0x0294
+#define regVGT_ENHANCE_BASE_IDX 1
+#define regVGT_GS_PER_ES 0x0295
+#define regVGT_GS_PER_ES_BASE_IDX 1
+#define regVGT_ES_PER_GS 0x0296
+#define regVGT_ES_PER_GS_BASE_IDX 1
+#define regVGT_GS_PER_VS 0x0297
+#define regVGT_GS_PER_VS_BASE_IDX 1
+#define regVGT_GSVS_RING_OFFSET_1 0x0298
+#define regVGT_GSVS_RING_OFFSET_1_BASE_IDX 1
+#define regVGT_GSVS_RING_OFFSET_2 0x0299
+#define regVGT_GSVS_RING_OFFSET_2_BASE_IDX 1
+#define regVGT_GSVS_RING_OFFSET_3 0x029a
+#define regVGT_GSVS_RING_OFFSET_3_BASE_IDX 1
+#define regVGT_GS_OUT_PRIM_TYPE 0x029b
+#define regVGT_GS_OUT_PRIM_TYPE_BASE_IDX 1
+#define regIA_ENHANCE 0x029c
+#define regIA_ENHANCE_BASE_IDX 1
+#define regVGT_DMA_SIZE 0x029d
+#define regVGT_DMA_SIZE_BASE_IDX 1
+#define regVGT_DMA_MAX_SIZE 0x029e
+#define regVGT_DMA_MAX_SIZE_BASE_IDX 1
+#define regVGT_DMA_INDEX_TYPE 0x029f
+#define regVGT_DMA_INDEX_TYPE_BASE_IDX 1
+#define regWD_ENHANCE 0x02a0
+#define regWD_ENHANCE_BASE_IDX 1
+#define regVGT_PRIMITIVEID_EN 0x02a1
+#define regVGT_PRIMITIVEID_EN_BASE_IDX 1
+#define regVGT_DMA_NUM_INSTANCES 0x02a2
+#define regVGT_DMA_NUM_INSTANCES_BASE_IDX 1
+#define regVGT_PRIMITIVEID_RESET 0x02a3
+#define regVGT_PRIMITIVEID_RESET_BASE_IDX 1
+#define regVGT_EVENT_INITIATOR 0x02a4
+#define regVGT_EVENT_INITIATOR_BASE_IDX 1
+#define regVGT_GS_MAX_PRIMS_PER_SUBGROUP 0x02a5
+#define regVGT_GS_MAX_PRIMS_PER_SUBGROUP_BASE_IDX 1
+#define regVGT_DRAW_PAYLOAD_CNTL 0x02a6
+#define regVGT_DRAW_PAYLOAD_CNTL_BASE_IDX 1
+#define regVGT_INSTANCE_STEP_RATE_0 0x02a8
+#define regVGT_INSTANCE_STEP_RATE_0_BASE_IDX 1
+#define regVGT_INSTANCE_STEP_RATE_1 0x02a9
+#define regVGT_INSTANCE_STEP_RATE_1_BASE_IDX 1
+#define regIA_MULTI_VGT_PARAM_BC 0x02aa
+#define regIA_MULTI_VGT_PARAM_BC_BASE_IDX 1
+#define regVGT_ESGS_RING_ITEMSIZE 0x02ab
+#define regVGT_ESGS_RING_ITEMSIZE_BASE_IDX 1
+#define regVGT_GSVS_RING_ITEMSIZE 0x02ac
+#define regVGT_GSVS_RING_ITEMSIZE_BASE_IDX 1
+#define regVGT_REUSE_OFF 0x02ad
+#define regVGT_REUSE_OFF_BASE_IDX 1
+#define regVGT_VTX_CNT_EN 0x02ae
+#define regVGT_VTX_CNT_EN_BASE_IDX 1
+#define regDB_HTILE_SURFACE 0x02af
+#define regDB_HTILE_SURFACE_BASE_IDX 1
+#define regDB_SRESULTS_COMPARE_STATE0 0x02b0
+#define regDB_SRESULTS_COMPARE_STATE0_BASE_IDX 1
+#define regDB_SRESULTS_COMPARE_STATE1 0x02b1
+#define regDB_SRESULTS_COMPARE_STATE1_BASE_IDX 1
+#define regDB_PRELOAD_CONTROL 0x02b2
+#define regDB_PRELOAD_CONTROL_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_SIZE_0 0x02b4
+#define regVGT_STRMOUT_BUFFER_SIZE_0_BASE_IDX 1
+#define regVGT_STRMOUT_VTX_STRIDE_0 0x02b5
+#define regVGT_STRMOUT_VTX_STRIDE_0_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_OFFSET_0 0x02b7
+#define regVGT_STRMOUT_BUFFER_OFFSET_0_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_SIZE_1 0x02b8
+#define regVGT_STRMOUT_BUFFER_SIZE_1_BASE_IDX 1
+#define regVGT_STRMOUT_VTX_STRIDE_1 0x02b9
+#define regVGT_STRMOUT_VTX_STRIDE_1_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_OFFSET_1 0x02bb
+#define regVGT_STRMOUT_BUFFER_OFFSET_1_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_SIZE_2 0x02bc
+#define regVGT_STRMOUT_BUFFER_SIZE_2_BASE_IDX 1
+#define regVGT_STRMOUT_VTX_STRIDE_2 0x02bd
+#define regVGT_STRMOUT_VTX_STRIDE_2_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_OFFSET_2 0x02bf
+#define regVGT_STRMOUT_BUFFER_OFFSET_2_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_SIZE_3 0x02c0
+#define regVGT_STRMOUT_BUFFER_SIZE_3_BASE_IDX 1
+#define regVGT_STRMOUT_VTX_STRIDE_3 0x02c1
+#define regVGT_STRMOUT_VTX_STRIDE_3_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_OFFSET_3 0x02c3
+#define regVGT_STRMOUT_BUFFER_OFFSET_3_BASE_IDX 1
+#define regVGT_STRMOUT_DRAW_OPAQUE_OFFSET 0x02ca
+#define regVGT_STRMOUT_DRAW_OPAQUE_OFFSET_BASE_IDX 1
+#define regVGT_STRMOUT_DRAW_OPAQUE_BUFFER_FILLED_SIZE 0x02cb
+#define regVGT_STRMOUT_DRAW_OPAQUE_BUFFER_FILLED_SIZE_BASE_IDX 1
+#define regVGT_STRMOUT_DRAW_OPAQUE_VERTEX_STRIDE 0x02cc
+#define regVGT_STRMOUT_DRAW_OPAQUE_VERTEX_STRIDE_BASE_IDX 1
+#define regVGT_GS_MAX_VERT_OUT 0x02ce
+#define regVGT_GS_MAX_VERT_OUT_BASE_IDX 1
+#define regVGT_TESS_DISTRIBUTION 0x02d4
+#define regVGT_TESS_DISTRIBUTION_BASE_IDX 1
+#define regVGT_SHADER_STAGES_EN 0x02d5
+#define regVGT_SHADER_STAGES_EN_BASE_IDX 1
+#define regVGT_LS_HS_CONFIG 0x02d6
+#define regVGT_LS_HS_CONFIG_BASE_IDX 1
+#define regVGT_GS_VERT_ITEMSIZE 0x02d7
+#define regVGT_GS_VERT_ITEMSIZE_BASE_IDX 1
+#define regVGT_GS_VERT_ITEMSIZE_1 0x02d8
+#define regVGT_GS_VERT_ITEMSIZE_1_BASE_IDX 1
+#define regVGT_GS_VERT_ITEMSIZE_2 0x02d9
+#define regVGT_GS_VERT_ITEMSIZE_2_BASE_IDX 1
+#define regVGT_GS_VERT_ITEMSIZE_3 0x02da
+#define regVGT_GS_VERT_ITEMSIZE_3_BASE_IDX 1
+#define regVGT_TF_PARAM 0x02db
+#define regVGT_TF_PARAM_BASE_IDX 1
+#define regDB_ALPHA_TO_MASK 0x02dc
+#define regDB_ALPHA_TO_MASK_BASE_IDX 1
+#define regVGT_DISPATCH_DRAW_INDEX 0x02dd
+#define regVGT_DISPATCH_DRAW_INDEX_BASE_IDX 1
+#define regPA_SU_POLY_OFFSET_DB_FMT_CNTL 0x02de
+#define regPA_SU_POLY_OFFSET_DB_FMT_CNTL_BASE_IDX 1
+#define regPA_SU_POLY_OFFSET_CLAMP 0x02df
+#define regPA_SU_POLY_OFFSET_CLAMP_BASE_IDX 1
+#define regPA_SU_POLY_OFFSET_FRONT_SCALE 0x02e0
+#define regPA_SU_POLY_OFFSET_FRONT_SCALE_BASE_IDX 1
+#define regPA_SU_POLY_OFFSET_FRONT_OFFSET 0x02e1
+#define regPA_SU_POLY_OFFSET_FRONT_OFFSET_BASE_IDX 1
+#define regPA_SU_POLY_OFFSET_BACK_SCALE 0x02e2
+#define regPA_SU_POLY_OFFSET_BACK_SCALE_BASE_IDX 1
+#define regPA_SU_POLY_OFFSET_BACK_OFFSET 0x02e3
+#define regPA_SU_POLY_OFFSET_BACK_OFFSET_BASE_IDX 1
+#define regVGT_GS_INSTANCE_CNT 0x02e4
+#define regVGT_GS_INSTANCE_CNT_BASE_IDX 1
+#define regVGT_STRMOUT_CONFIG 0x02e5
+#define regVGT_STRMOUT_CONFIG_BASE_IDX 1
+#define regVGT_STRMOUT_BUFFER_CONFIG 0x02e6
+#define regVGT_STRMOUT_BUFFER_CONFIG_BASE_IDX 1
+#define regVGT_DMA_EVENT_INITIATOR 0x02e7
+#define regVGT_DMA_EVENT_INITIATOR_BASE_IDX 1
+#define regPA_SC_CENTROID_PRIORITY_0 0x02f5
+#define regPA_SC_CENTROID_PRIORITY_0_BASE_IDX 1
+#define regPA_SC_CENTROID_PRIORITY_1 0x02f6
+#define regPA_SC_CENTROID_PRIORITY_1_BASE_IDX 1
+#define regPA_SC_LINE_CNTL 0x02f7
+#define regPA_SC_LINE_CNTL_BASE_IDX 1
+#define regPA_SC_AA_CONFIG 0x02f8
+#define regPA_SC_AA_CONFIG_BASE_IDX 1
+#define regPA_SU_VTX_CNTL 0x02f9
+#define regPA_SU_VTX_CNTL_BASE_IDX 1
+#define regPA_CL_GB_VERT_CLIP_ADJ 0x02fa
+#define regPA_CL_GB_VERT_CLIP_ADJ_BASE_IDX 1
+#define regPA_CL_GB_VERT_DISC_ADJ 0x02fb
+#define regPA_CL_GB_VERT_DISC_ADJ_BASE_IDX 1
+#define regPA_CL_GB_HORZ_CLIP_ADJ 0x02fc
+#define regPA_CL_GB_HORZ_CLIP_ADJ_BASE_IDX 1
+#define regPA_CL_GB_HORZ_DISC_ADJ 0x02fd
+#define regPA_CL_GB_HORZ_DISC_ADJ_BASE_IDX 1
+#define regPA_SC_AA_SAMPLE_LOCS_PIXEL_X0Y0_0 0x02fe
+#define regPA_SC_AA_SAMPLE_LOCS_PIXEL_X0Y0_0_BASE_IDX 1