summaryrefslogtreecommitdiff
AgeCommit message (Collapse)Author
2020-01-20scsi: ufs: Use UFS device indicated maximum LU numberBean Huo
According to Jedec standard UFS 3.0 and UFS 2.1 Spec, Maximum number of logical units supported by the UFS device is indicated by parameter bMaxNumberLU in Geometry Descriptor. This patch is to delete current hard code macro definition of UFS_UPIU_MAX_GENERAL_LUN, and switch to use device indicated number instead. Link: https://lore.kernel.org/r/20200120130820.1737-9-huobean@gmail.com Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: Add max_lu_supported in struct ufs_dev_infoBean Huo
Add one new parameter max_lu_supported in struct ufs_dev_info, which will be used to express exactly how many general LUs being supported by UFS device, and initialize it during booting stage. This patch also adds a new function ufshcd_device_geo_params_init() for initialization of UFS device geometry descriptor related parameters. Link: https://lore.kernel.org/r/20200120130820.1737-8-huobean@gmail.com Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: Delete is_init_prefetch from struct ufs_hbaBean Huo
Without variable is_init_prefetch, the current logic can guarantee ufshcd_init_icc_levels() will execute only once, delete it now. Link: https://lore.kernel.org/r/20200120130820.1737-7-huobean@gmail.com Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: Inline two functions into their callersBean Huo
Delete ufshcd_read_power_desc() and ufshcd_read_device_desc(), directly inline ufshcd_read_desc() into its callers. Link: https://lore.kernel.org/r/20200120130820.1737-6-huobean@gmail.com Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: Move ufshcd_get_max_pwr_mode() to ufshcd_device_params_init()Bean Huo
ufshcd_get_max_pwr_mode() only need to be called once while booting, take it out from ufshcd_probe_hba() and inline into ufshcd_device_params_init(). Link: https://lore.kernel.org/r/20200120130820.1737-5-huobean@gmail.com Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: Split ufshcd_probe_hba() based on its called flowBean Huo
This patch has two major non-functionality changes: 1. Take scanning host if-statement out from ufshcd_probe_hba(), and move into a new added function ufshcd_add_lus(). In this new function ufshcd_add_lus(), the main functionalitis include: ICC initialization, add well-known LUs, devfreq initialization, UFS bsg probe and scsi host scan. The reason for this change is that these functionalities only being called during booting stage flow ufshcd_init()->ufshcd_async_scan(). In the processes of error handling and power management ufshcd_suspend(), ufshcd_resume(), ufshcd_probe_hba() being called, but these functionalitis above metioned are not hit. 2. Move context of initialization of parameters associated with the UFS device to a new added function ufshcd_device_params_init(). The reason of this change is that all of these parameters are used by driver, but only need to be initialized once when booting. Combine them into an integral function, make them easier maintain. Link: https://lore.kernel.org/r/20200120130820.1737-4-huobean@gmail.com Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: Delete struct ufs_dev_descBean Huo
In consideration of UFS host driver uses parameters of struct ufs_dev_desc, move its parameters to struct ufs_dev_info, delete struct ufs_dev_desc. Link: https://lore.kernel.org/r/20200120130820.1737-3-huobean@gmail.com Reviewed-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: Fix ufshcd_probe_hba() reture value in case ↵Bean Huo
ufshcd_scsi_add_wlus() fails A non-zero error value likely being returned by ufshcd_scsi_add_wlus() in case of failure of adding the WLs, but ufshcd_probe_hba() doesn't use this value, and doesn't report this failure to upper caller. This patch is to fix this issue. Fixes: 2a8fa600445c ("ufs: manually add well known logical units") Link: https://lore.kernel.org/r/20200120130820.1737-2-huobean@gmail.com Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Reviewed-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Bean Huo <beanhuo@micron.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs-mediatek: enable low-power mode for hibern8 stateStanley Chu
In MediaTek Chipsets, UniPro link and ufshci can enter proprietary low-power mode while link is in hibern8 state. Link: https://lore.kernel.org/r/20200117035108.19699-4-stanley.chu@mediatek.com Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs: export some functions for vendor usageStanley Chu
Export below functions for vendor usage: int ufshcd_hba_enable(struct ufs_hba *hba); int ufshcd_make_hba_operational(struct ufs_hba *hba); int ufshcd_uic_hibern8_exit(struct ufs_hba *hba); Link: https://lore.kernel.org/r/20200117035108.19699-3-stanley.chu@mediatek.com Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Reviewed-by: Asutosh Das <asutoshd@codeaurora.org> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-20scsi: ufs-mediatek: add dbg_register_dump implementationStanley Chu
Add dbg_register_dump variant vendor implementation in MediaTek UFS driver. Link: https://lore.kernel.org/r/20200117035108.19699-2-stanley.chu@mediatek.com Reviewed-by: Alim Akhtar <alim.akhtar@samsung.com> Signed-off-by: Stanley Chu <stanley.chu@mediatek.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2020-01-21drm/exynos: Rename Exynos to lowercaseKrzysztof Kozlowski
Fix up inconsistent usage of upper and lowercase letters in "Exynos" name. "EXYNOS" is not an abbreviation but a regular trademarked name. Therefore it should be written with lowercase letters starting with capital letter. The lowercase "Exynos" name is promoted by its manufacturer Samsung Electronics Co., Ltd., in advertisement materials and on website. Signed-off-by: Krzysztof Kozlowski <krzk@kernel.org> Signed-off-by: Inki Dae <inki.dae@samsung.com>
2020-01-21drm/exynos: change callback namesInki Dae
This patch changes Exynos specific 'disable' and 'enable' callback names to 'atomic_disable/enable' for the consistency. Signed-off-by: Inki Dae <inki.dae@samsung.com>
2020-01-20io_uring: use labeled array init in io_op_defsPavel Begunkov
Don't rely on implicit ordering of IORING_OP_ and explicitly place them at a right place in io_op_defs. Now former comments are now a part of the code and won't ever outdate. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: optimise sqe-to-req flags translationPavel Begunkov
For each IOSQE_* flag there is a corresponding REQ_F_* flag. And there is a repetitive pattern of their translation: e.g. if (sqe->flags & SQE_FLAG*) req->flags |= REQ_F_FLAG* Use same numeric values/bits for them and copy instead of manual handling. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: remove REQ_F_IO_DRAINEDPavel Begunkov
A request can get into the defer list only once, there is no need for marking it as drained, so remove it. This probably was left after extracting __need_defer() for use in timeouts. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: file switch work needs to get flushed on exitJens Axboe
We currently flush early, but if we have something in progress and a new switch is scheduled, we need to ensure to flush after our teardown as well. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: hide uring_fd in ctxPavel Begunkov
req->ring_fd and req->ring_file are used only during the prep stage during submission, which is is protected by mutex. There is no need to store them per-request, place them in ctx. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: remove extra check in __io_commit_cqringPavel Begunkov
__io_commit_cqring() is almost always called when there is a change in the rings, so the check is rather pessimising. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: optimise use of ctx->drain_nextPavel Begunkov
Move setting ctx->drain_next to the only place it could be set, when it got linked non-head requests. The same for checking it, it's interesting only for a head of a link or a non-linked request. No functional changes here. This removes some code from the common path and also removes REQ_F_DRAIN_LINK flag, as it doesn't need it anymore. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add support for probing opcodesJens Axboe
The application currently has no way of knowing if a given opcode is supported or not without having to try and issue one and see if we get -EINVAL or not. And even this approach is fraught with peril, as maybe we're getting -EINVAL due to some fields being missing, or maybe it's just not that easy to issue that particular command without doing some other leg work in terms of setup first. This adds IORING_REGISTER_PROBE, which fills in a structure with info on what it supported or not. This will work even with sparse opcode fields, which may happen in the future or even today if someone backports specific features to older kernels. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: account fixed file references correctly in batchJens Axboe
We can't assume that the whole batch has fixed files in it. If it's a mix, or none at all, then we can end up doing a ref put that either messes up accounting, or causes an oops if we have no fixed files at all. Also ensure we free requests properly between inflight accounted and normal requests. Fixes: 82c721577011 ("io_uring: extend batch freeing to cover more cases") Reported-by: Dmitrii Dolgov <9erthalion6@gmail.com> Reported-by: Pavel Begunkov <asml.silence@gmail.com> Tested-by: Dmitrii Dolgov <9erthalion6@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add opcode to issue trace eventJens Axboe
For some test apps at least, user_data is just zeroes. So it's not a good way to tell what the command actually is. Add the opcode to the issue trace point. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add support for IORING_OP_OPENAT2Jens Axboe
Add support for the new openat2(2) system call. It's trivial to do, as we can have openat(2) just be wrapped around it. Suggested-by: Stefan Metzmacher <metze@samba.org> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: remove 'fname' from io_open structureJens Axboe
We only use it internally in the prep functions for both statx and openat, so we don't need it to be persistent across the request. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add 'struct open_how' to the openat request contextJens Axboe
We'll need this for openat2(2) support, remove flags and mode from the existing io_open struct. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: enable option to only trigger eventfd for async completionsJens Axboe
If an application is using eventfd notifications with poll to know when new SQEs can be issued, it's expecting the following read/writes to complete inline. And with that, it knows that there are events available, and don't want spurious wakeups on the eventfd for those requests. This adds IORING_REGISTER_EVENTFD_ASYNC, which works just like IORING_REGISTER_EVENTFD, except it only triggers notifications for events that happen from async completions (IRQ, or io-wq worker completions). Any completions inline from the submission itself will not trigger notifications. Suggested-by: Mark Papadakis <markuspapadakis@icloud.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: change io_ring_ctx bool fields into bit fieldsJens Axboe
In preparation for adding another one, which would make us spill into another long (and hence bump the size of the ctx), change them to bit fields. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: file set registration should use interruptible waitsJens Axboe
If an application attempts to register a set with unbounded requests pending, we can be stuck here forever if they don't complete. We can make this wait interruptible, and just abort if we get signaled. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: Remove unnecessary null checkYueHaibing
Null check kfree is redundant, so remove it. This is detected by coccinelle. Signed-off-by: YueHaibing <yuehaibing@huawei.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add support for send(2) and recv(2)Jens Axboe
This adds IORING_OP_SEND for send(2) support, and IORING_OP_RECV for recv(2) support. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: remove extra io_wq_current_is_worker()Pavel Begunkov
io_wq workers use io_issue_sqe() to forward sqes and never io_queue_sqe(). Remove extra check for io_wq_current_is_worker() Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: optimise commit_sqring() for common casePavel Begunkov
It should be pretty rare to not submitting anything when there is something in the ring. No need to keep heuristics for this case. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: optimise head checks in io_get_sqring()Pavel Begunkov
A user may ask to submit more than there is in the ring, and then io_uring will submit as much as it can. However, in the last iteration it will allocate an io_kiocb and immediately free it. It could do better and adjust @to_submit to what is in the ring. And since the ring's head is already checked here, there is no need to do it in the loop, spamming with smp_load_acquire()'s barriers Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: clamp to_submit in io_submit_sqes()Pavel Begunkov
Make io_submit_sqes() to clamp @to_submit itself. It removes duplicated code and prepares for following changes. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add support for IORING_SETUP_CLAMPJens Axboe
Some applications like to start small in terms of ring size, and then ramp up as needed. This is a bit tricky to do currently, since we don't advertise the max ring size. This adds IORING_SETUP_CLAMP. If set, and the values for SQ or CQ ring size exceed what we support, then clamp them at the max values instead of returning -EINVAL. Since we return the chosen ring sizes after setup, no further changes are needed on the application side. io_uring already changes the ring sizes if the application doesn't ask for power-of-two sizes, for example. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: extend batch freeing to cover more casesJens Axboe
Currently we only batch free if fixed files are used, no links, no aux data, etc. This extends the batch freeing to only exclude the linked case and fallback case, and make io_free_req_many() handle the other cases just fine. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: wrap multi-req freeing in struct req_batchJens Axboe
This cleans up the code a bit, and it allows us to build on top of the multi-req freeing. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: batch getting pcpu referencesPavel Begunkov
percpu_ref_tryget() has its own overhead. Instead getting a reference for each request, grab a bunch once per io_submit_sqes(). ~5% throughput boost for a "submit and wait 128 nops" benchmark. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> __io_req_free_empty() -> __io_req_do_free() Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20pcpu_ref: add percpu_ref_tryget_many()Pavel Begunkov
Add percpu_ref_tryget_many(), which works the same way as percpu_ref_tryget(), but grabs specified number of refs. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Acked-by: Tejun Heo <tj@kernel.org> Acked-by: Dennis Zhou <dennis@kernel.org> Cc: Christoph Lameter <cl@linux.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add IORING_OP_MADVISEJens Axboe
This adds support for doing madvise(2) through io_uring. We assume that any operation can block, and hence punt everything async. This could be improved, but hard to make bullet proof. The async punt ensures it's safe. Reviewed-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20mm: make do_madvise() available internallyJens Axboe
This is in preparation for enabling this functionality through io_uring. Add a helper that is just exporting what sys_madvise() does, and have the system call use it. No functional changes in this patch. Reviewed-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add IORING_OP_FADVISEJens Axboe
This adds support for doing fadvise through io_uring. We assume that WILLNEED doesn't block, but that DONTNEED may block. Reviewed-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: allow use of offset == -1 to mean file positionJens Axboe
This behaves like preadv2/pwritev2 with offset == -1, it'll use (and update) the current file position. This obviously comes with the caveat that if the application has multiple read/writes in flight, then the end result will not be as expected. This is similar to threads sharing a file descriptor and doing IO using the current file position. Since this feature isn't easily detectable by doing a read or write, add a feature flags, IORING_FEAT_RW_CUR_POS, to allow applications to detect presence of this feature. Reported-by: 李通洲 <carter.li@eoitek.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add non-vectored read/write commandsJens Axboe
For uses cases that don't already naturally have an iovec, it's easier (or more convenient) to just use a buffer address + length. This is particular true if the use case is from languages that want to create a memory safe abstraction on top of io_uring, and where introducing the need for the iovec may impose an ownership issue. For those cases, they currently need an indirection buffer, which means allocating data just for this purpose. Add basic read/write that don't require the iovec. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: improve poll completion performanceJens Axboe
For busy IORING_OP_POLL_ADD workloads, we can have enough contention on the completion lock that we fail the inline completion path quite often as we fail the trylock on that lock. Add a list for deferred completions that we can use in that case. This helps reduce the number of async offloads we have to do, as if we get multiple completions in a row, we'll piggy back on to the poll_llist instead of having to queue our own offload. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: split overflow state into SQ and CQ sideJens Axboe
We currently check ->cq_overflow_list from both SQ and CQ context, which causes some bouncing of that cache line. Add separate bits of state for this instead, so that the SQ side can check using its own state, and likewise for the CQ side. This adds ->sq_check_overflow with the SQ state, and ->cq_check_overflow with the CQ state. If we hit an overflow condition, both of these bits are set. Likewise for overflow flush clear, we clear both bits. For the fast path of just checking if there's an overflow condition on either the SQ or CQ side, we can use our own private bit for this. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: add lookup table for various opcode needsJens Axboe
We currently have various switch statements that check if an opcode needs a file, mm, etc. These are hard to keep in sync as opcodes are added. Add a struct io_op_def that holds all of this information, so we have just one spot to update when opcodes are added. This also enables us to NOT allocate req->io if a deferred command doesn't need it, and corrects some mistakes we had in terms of what commands need mm context. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: remove two unnecessary function declarationsJens Axboe
__io_free_req() and io_double_put_req() aren't used before they are defined, so we can kill these two forwards. Signed-off-by: Jens Axboe <axboe@kernel.dk>
2020-01-20io_uring: move *queue_link_head() from common pathPavel Begunkov
Move io_queue_link_head() to links handling code in io_submit_sqe(), so it wouldn't need extra checks and would have better data locality. Signed-off-by: Pavel Begunkov <asml.silence@gmail.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
generated by cgit (git 2.34.1) at 2025-09-15 11:23:49 +0000