summaryrefslogtreecommitdiff
path: root/drivers/target/target_core_file.h
AgeCommit message (Collapse)Author
2018-05-14scsi: target: target/file: Add support of direct and async I/OAndrei Vagin
There are two advantages: * Direct I/O allows to avoid the write-back cache, so it reduces affects to other processes in the system. * Async I/O allows to handle a few commands concurrently. DIO + AIO shows a better perfomance for random write operations: Mode: O_DSYNC Async: 1 $ ./fio --bs=4K --direct=1 --rw=randwrite --ioengine=libaio --iodepth=64 --name=/dev/sda --runtime=20 --numjobs=2 WRITE: bw=45.9MiB/s (48.1MB/s), 21.9MiB/s-23.0MiB/s (22.0MB/s-25.2MB/s), io=919MiB (963MB), run=20002-20020msec Mode: O_DSYNC Async: 0 $ ./fio --bs=4K --direct=1 --rw=randwrite --ioengine=libaio --iodepth=64 --name=/dev/sdb --runtime=20 --numjobs=2 WRITE: bw=1607KiB/s (1645kB/s), 802KiB/s-805KiB/s (821kB/s-824kB/s), io=31.8MiB (33.4MB), run=20280-20295msec Known issue: DIF (PI) emulation doesn't work when a target uses async I/O, because DIF metadata is saved in a separate file, and it is another non-trivial task how to synchronize writing in two files, so that a following read operation always returns a consisten metadata for a specified block. Cc: "Nicholas A. Bellinger" <nab@linux-iscsi.org> Cc: Christoph Hellwig <hch@infradead.org> Cc: Bryant G. Ly <bryantly@linux.vnet.ibm.com> Tested-by: Bryant G. Ly <bryantly@linux.vnet.ibm.com> Signed-off-by: Andrei Vagin <avagin@openvz.org> Reviewed-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Bryant G. Ly <bryantly@linux.vnet.ibm.com> Reviewed-by: Mike Christie <mchristi@redhat.com> Signed-off-by: Martin K. Petersen <martin.petersen@oracle.com>
2017-11-02License cleanup: add SPDX GPL-2.0 license identifier to files with no licenseGreg Kroah-Hartman
Many source files in the tree are missing licensing information, which makes it harder for compliance tools to determine the correct license. By default all files without license information are under the default license of the kernel, which is GPL version 2. Update the files which contain no license information with the 'GPL-2.0' SPDX license identifier. The SPDX identifier is a legally binding shorthand, which can be used instead of the full boiler plate text. This patch is based on work done by Thomas Gleixner and Kate Stewart and Philippe Ombredanne. How this work was done: Patches were generated and checked against linux-4.14-rc6 for a subset of the use cases: - file had no licensing information it it. - file was a */uapi/* one with no licensing information in it, - file was a */uapi/* one with existing licensing information, Further patches will be generated in subsequent months to fix up cases where non-standard license headers were used, and references to license had to be inferred by heuristics based on keywords. The analysis to determine which SPDX License Identifier to be applied to a file was done in a spreadsheet of side by side results from of the output of two independent scanners (ScanCode & Windriver) producing SPDX tag:value files created by Philippe Ombredanne. Philippe prepared the base worksheet, and did an initial spot review of a few 1000 files. The 4.13 kernel was the starting point of the analysis with 60,537 files assessed. Kate Stewart did a file by file comparison of the scanner results in the spreadsheet to determine which SPDX license identifier(s) to be applied to the file. She confirmed any determination that was not immediately clear with lawyers working with the Linux Foundation. Criteria used to select files for SPDX license identifier tagging was: - Files considered eligible had to be source code files. - Make and config files were included as candidates if they contained >5 lines of source - File already had some variant of a license header in it (even if <5 lines). All documentation files were explicitly excluded. The following heuristics were used to determine which SPDX license identifiers to apply. - when both scanners couldn't find any license traces, file was considered to have no license information in it, and the top level COPYING file license applied. For non */uapi/* files that summary was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 11139 and resulted in the first patch in this series. If that file was a */uapi/* path one, it was "GPL-2.0 WITH Linux-syscall-note" otherwise it was "GPL-2.0". Results of that was: SPDX license identifier # files ---------------------------------------------------|------- GPL-2.0 WITH Linux-syscall-note 930 and resulted in the second patch in this series. - if a file had some form of licensing information in it, and was one of the */uapi/* ones, it was denoted with the Linux-syscall-note if any GPL family license was found in the file or had no licensing in it (per prior point). Results summary: SPDX license identifier # files ---------------------------------------------------|------ GPL-2.0 WITH Linux-syscall-note 270 GPL-2.0+ WITH Linux-syscall-note 169 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-2-Clause) 21 ((GPL-2.0 WITH Linux-syscall-note) OR BSD-3-Clause) 17 LGPL-2.1+ WITH Linux-syscall-note 15 GPL-1.0+ WITH Linux-syscall-note 14 ((GPL-2.0+ WITH Linux-syscall-note) OR BSD-3-Clause) 5 LGPL-2.0+ WITH Linux-syscall-note 4 LGPL-2.1 WITH Linux-syscall-note 3 ((GPL-2.0 WITH Linux-syscall-note) OR MIT) 3 ((GPL-2.0 WITH Linux-syscall-note) AND MIT) 1 and that resulted in the third patch in this series. - when the two scanners agreed on the detected license(s), that became the concluded license(s). - when there was disagreement between the two scanners (one detected a license but the other didn't, or they both detected different licenses) a manual inspection of the file occurred. - In most cases a manual inspection of the information in the file resulted in a clear resolution of the license that should apply (and which scanner probably needed to revisit its heuristics). - When it was not immediately clear, the license identifier was confirmed with lawyers working with the Linux Foundation. - If there was any question as to the appropriate license identifier, the file was flagged for further research and to be revisited later in time. In total, over 70 hours of logged manual review was done on the spreadsheet to determine the SPDX license identifiers to apply to the source files by Kate, Philippe, Thomas and, in some cases, confirmation by lawyers working with the Linux Foundation. Kate also obtained a third independent scan of the 4.13 code base from FOSSology, and compared selected files where the other two scanners disagreed against that SPDX file, to see if there was new insights. The Windriver scanner is based on an older version of FOSSology in part, so they are related. Thomas did random spot checks in about 500 files from the spreadsheets for the uapi headers and agreed with SPDX license identifier in the files he inspected. For the non-uapi files Thomas did random spot checks in about 15000 files. In initial set of patches against 4.14-rc6, 3 files were found to have copy/paste license identifier errors, and have been fixed to reflect the correct identifier. Additionally Philippe spent 10 hours this week doing a detailed manual inspection and review of the 12,461 patched files from the initial patch version early this week with: - a full scancode scan run, collecting the matched texts, detected license ids and scores - reviewing anything where there was a license detected (about 500+ files) to ensure that the applied SPDX license was correct - reviewing anything where there was no detection but the patch license was not GPL-2.0 WITH Linux-syscall-note to ensure that the applied SPDX license was correct This produced a worksheet with 20 files needing minor correction. This worksheet was then exported into 3 different .csv files for the different types of files to be modified. These .csv files were then reviewed by Greg. Thomas wrote a script to parse the csv files and add the proper SPDX tag to the file, in the format that the file expected. This script was further refined by Greg based on the output to detect more types of files automatically and to distinguish between header and source .c files (which need different comment types.) Finally Greg ran the script using the .csv files to generate the patches. Reviewed-by: Kate Stewart <kstewart@linuxfoundation.org> Reviewed-by: Philippe Ombredanne <pombredanne@nexb.com> Reviewed-by: Thomas Gleixner <tglx@linutronix.de> Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2016-12-09target: Minimize #include directivesBart Van Assche
Remove superfluous #include directives from the include/target/*.h files. Add missing #include directives to other *.h and *.c files. Use forward declarations for structures where possible. This change reduces the build time for make M=drivers/target on my laptop from 27.1s to 18.7s or by about 30%. Signed-off-by: Bart Van Assche <bart.vanassche@sandisk.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Bryant G. Ly <bryantly@linux.vnet.ibm.com>
2015-05-30target/file: Remove fd_prot bounce bufferSagi Grimberg
The reason this bounce buffer exists is to allow code reuse between rd_mcp and fileio in DIF mode. But the fact is, that this bounce is really not needed at all, we can simply call sbc_dif_verify on cmd->t_prot_sg and use it for file IO. This also removes fd_do_prot_rw as fd_do_rw was generalised to receive file pointer, block size (8 bytes for DIF data) and total data length. (Fix apply breakage from commit c836777 - nab) Tested-by: Akinobu Mita <akinobu.mita@gmail.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2014-01-31Merge branch 'for-next' of ↵Linus Torvalds
git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending Pull SCSI target updates from Nicholas Bellinger: "The highlights this round include: - add support for SCSI Referrals (Hannes) - add support for T10 DIF into target core (nab + mkp) - add support for T10 DIF emulation in FILEIO + RAMDISK backends (Sagi + nab) - add support for T10 DIF -> bio_integrity passthrough in IBLOCK backend (nab) - prep changes to iser-target for >= v3.15 T10 DIF support (Sagi) - add support for qla2xxx N_Port ID Virtualization - NPIV (Saurav + Quinn) - allow percpu_ida_alloc() to receive task state bitmask (Kent) - fix >= v3.12 iscsi-target session reset hung task regression (nab) - fix >= v3.13 percpu_ref se_lun->lun_ref_active race (nab) - fix a long-standing network portal creation race (Andy)" * 'for-next' of git://git.kernel.org/pub/scm/linux/kernel/git/nab/target-pending: (51 commits) target: Fix percpu_ref_put race in transport_lun_remove_cmd target/iscsi: Fix network portal creation race target: Report bad sector in sense data for DIF errors iscsi-target: Convert gfp_t parameter to task state bitmask iscsi-target: Fix connection reset hang with percpu_ida_alloc percpu_ida: Make percpu_ida_alloc + callers accept task state bitmask iscsi-target: Pre-allocate more tags to avoid ack starvation qla2xxx: Configure NPIV fc_vport via tcm_qla2xxx_npiv_make_lport qla2xxx: Enhancements to enable NPIV support for QLOGIC ISPs with TCM/LIO. qla2xxx: Fix scsi_host leak on qlt_lport_register callback failure IB/isert: pass scatterlist instead of cmd to fast_reg_mr routine IB/isert: Move fastreg descriptor creation to a function IB/isert: Avoid frwr notation, user fastreg IB/isert: seperate connection protection domains and dma MRs tcm_loop: Enable DIF/DIX modes in SCSI host LLD target/rd: Add DIF protection into rd_execute_rw target/rd: Add support for protection SGL setup + release target/rd: Refactor rd_build_device_space + rd_release_device_space target/file: Add DIF protection support to fd_execute_rw target/file: Add DIF protection init/format support ...
2014-01-19target/file: Add DIF protection support to fd_execute_rwNicholas Bellinger
This patch adds support for DIF protection into fd_execute_rw() code for WRITE/READ I/O using sbc_dif_verify_[write,read]() logic. It adds fd_do_prot_rw() for handling interface with FILEIO PI, and uses a locally allocated fd_prot->prot_buf + fd_prot->prot_sg for interacting with SBC DIF verify emulation code. Cc: Martin K. Petersen <martin.petersen@oracle.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Sagi Grimberg <sagig@mellanox.com> Cc: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2014-01-19target/file: Add DIF protection init/format supportNicholas Bellinger
This patch adds support for DIF protection init/format support into the FILEIO backend. It involves using a seperate $FILE.protection for storing PI that is opened via fd_init_prot() using the common pi_prot_type attribute. The actual formatting of the protection is done via fd_format_prot() using the common pi_prot_format attribute, that will populate the initial PI data based upon the currently configured pi_prot_type. Based on original FILEIO code from Sagi. v1 changes: - Fix sparse warnings in fd_init_format_buf (Fengguang) Cc: Martin K. Petersen <martin.petersen@oracle.com> Cc: Christoph Hellwig <hch@lst.de> Cc: Hannes Reinecke <hare@suse.de> Cc: Sagi Grimberg <sagig@mellanox.com> Cc: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-12-19target/file: Update hw_max_sectors based on current block_sizeNicholas Bellinger
This patch allows FILEIO to update hw_max_sectors based on the current max_bytes_per_io. This is required because vfs_[writev,readv]() can accept a maximum of 2048 iovecs per call, so the enforced hw_max_sectors really needs to be calculated based on block_size. This addresses a >= v3.5 bug where block_size=512 was rejecting > 1M sized I/O requests, because FD_MAX_SECTORS was hardcoded to 2048 for the block_size=4096 case. (v2: Use max_bytes_per_io instead of ->update_hw_max_sectors) Reported-by: Henrik Goldman <hg@x-formation.com> Cc: <stable@vger.kernel.org> #3.5+ Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2013-03-19target/file: Bump FD_MAX_SECTORS to 2048 to handle 1M sized I/OsNicholas Bellinger
This patch bumps the default FILEIO backend FD_MAX_SECTORS value from 1024 -> 2048 in order to allow block_size=512 to handle 1M sized I/Os. The current default rejects I/Os larger than 512K in sbc_parse_cdb(): [12015.915146] SCSI OP 2ah with too big sectors 1347 exceeds backend hw_max_sectors: 1024 [12015.977744] SCSI OP 2ah with too big sectors 2048 exceeds backend hw_max_sectors: 1024 This issue is present in >= v3.5 based kernels, introduced after the removal of se_task logic. Reported-by: Viljami Ilola <azmulx@netikka.fi> Cc: <stable@vger.kernel.org> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2012-11-06target: kill struct se_subsystem_devChristoph Hellwig
Simplify the code a lot by killing the superflous struct se_subsystem_dev. Instead se_device is allocated early on by the backend driver, which allocates it as part of its own per-device structure, borrowing the scheme that is for example used for inode allocation. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2012-10-02target/file: Re-enable optional fd_buffered_io=1 operationNicholas Bellinger
This patch re-adds the ability to optionally run in buffered FILEIO mode (eg: w/o O_DSYNC) for device backends in order to once again use the Linux buffered cache as a write-back storage mechanism. This logic was originally dropped with mainline v3.5-rc commit: commit a4dff3043c231d57f982af635c9d2192ee40e5ae Author: Nicholas Bellinger <nab@linux-iscsi.org> Date: Wed May 30 16:25:41 2012 -0700 target/file: Use O_DSYNC by default for FILEIO backends This difference with this patch is that fd_create_virtdevice() now forces the explicit setting of emulate_write_cache=1 when buffered FILEIO operation has been enabled. (v2: Switch to FDBD_HAS_BUFFERED_IO_WCE + add more detailed comment as requested by hch) Reported-by: Ferry <iscsitmp@bananateam.nl> Cc: Christoph Hellwig <hch@lst.de> Cc: <stable@vger.kernel.org> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2012-06-02target/file: Use O_DSYNC by default for FILEIO backendsNicholas Bellinger
Convert to use O_DSYNC for all cases at FILEIO backend creation time to avoid the extra syncing of pure timestamp updates with legacy O_SYNC during default operation as recommended by hch. Continue to do this independently of Write Cache Enable (WCE) bit, as WCE=0 is currently the default for all backend devices and enabled by user on per device basis via attrib/emulate_write_cache. This patch drops the now unnecessary fd_buffered_io= token usage that was originally signalling when to explictly disable O_SYNC at backend creation time for buffered I/O operation. This can end up being dangerous for a number of reasons during physical node failure, so go ahead and drop this option for now when O_DSYNC is used as the default. Also allow explict FUA WRITEs -> vfs_fsync_range() call to function in fd_execute_cmd() independently of WCE bit setting. Reported-by: Christoph Hellwig <hch@lst.de> Cc: Linus Torvalds <torvalds@linux-foundation.org> Cc: <stable@vger.kernel.org> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2012-05-06target: remove struct se_taskChristoph Hellwig
We can use struct se_cmd for everything it did. Make sure to pass the S/G list and data direction to the execution function to ease adding back BIDI support later on. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2011-10-24target: make the ->get_cdb method optionalChristoph Hellwig
The most commonly used file, iblock and rd backends have no use for a per-task CDB and thus don't need a method to copy it into their otherwise unused CDB fields. Signed-off-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2011-07-22target: Follow up core updates from AGrover and HCH (round 4)Andy Grover
This patch contains the squashed version of forth round series cleanups from Andy and Christoph following the post heavy lifting in the preceeding: 'Eliminate usage of struct se_mem' and 'Make all control CDBs scatter-gather' changes. This also includes a conversion of target core and the v3.0 mainline fabric modules (loopback and tcm_fc) to use pr_debug and the CONFIG_DYNAMIC_DEBUG infrastructure! These have been squashed into this third and final round for v3.1. target: Remove ifdeffed code in t_g_process_write target: Remove direct ramdisk code target: Rename task_sg_num to task_sg_nents target: Remove custom debug macros for pr_debug. Use pr_err(). target: Remove custom debug macros in mainline fabrics target: Set WSNZ=1 in block limits VPD. Abort if WRITE_SAME sectors = 0 target: Remove transport do_se_mem_map callback target: Further simplify transport_free_pages target: Redo task allocation return value handling target: Remove extra parentheses target: change alloc_task call to take *cdb, not *cmd (nab: Fix bogus struct file assignments in fd_do_readv and fd_do_writev) Signed-off-by: Andy Grover <agrover@redhat.com> Reviewed-by: Christoph Hellwig <hch@lst.de> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2011-07-22target: Core cleanups from AGrover (round 1)Andy Grover
This patch contains the squashed version of a number of cleanups and minor fixes from Andy's initial series (round 1) for target core this past spring. The condensed log looks like: target: use errno values instead of returning -1 for everything target: Rename transport_calc_sg_num to transport_init_task_sg target: Fix leak in error path in transport_init_task_sg target/pscsi: Remove pscsi_get_sh() usage target: Make two runtime checks into WARN_ONs target: Remove hba queue depth and convert to spin_lock_irq usage target: dev->dev_status_queue_obj is unused target: Make struct se_queue_req.cmd type struct se_cmd * target: Remove __transport_get_qr_from_queue() target: Rename se_dev->g_se_dev_list to se_dev_node target: Remove struct se_global target: Simplify scsi mib index table code target: Make dev_queue_obj a member of se_device instead of a pointer target: remove extraneous returns at end of void functions target: Ensure transport_dump_vpd_ident_type returns null-terminated str target: Function pointers don't need to use '&' to be assigned target: Fix comment in __transport_execute_tasks() target: Misc style cleanups target: rename struct pr_reservation_template to pr_reservation target: Remove #defines that just perform indirection target: Inline transport_get_task_from_execute_queue() target: Minor header comment fixes Signed-off-by: Andy Grover <agrover@redhat.com> Signed-off-by: Nicholas Bellinger <nab@linux-iscsi.org>
2011-01-14[SCSI] target: Add LIO target core v4.0.0-rc6Nicholas Bellinger
LIO target is a full featured in-kernel target framework with the following feature set: High-performance, non-blocking, multithreaded architecture with SIMD support. Advanced SCSI feature set: * Persistent Reservations (PRs) * Asymmetric Logical Unit Assignment (ALUA) * Protocol and intra-nexus multiplexing, load-balancing and failover (MC/S) * Full Error Recovery (ERL=0,1,2) * Active/active task migration and session continuation (ERL=2) * Thin LUN provisioning (UNMAP and WRITE_SAMExx) Multiprotocol target plugins Storage media independence: * Virtualization of all storage media; transparent mapping of IO to LUNs * No hard limits on number of LUNs per Target; maximum LUN size ~750 TB * Backstores: SATA, SAS, SCSI, BluRay, DVD, FLASH, USB, ramdisk, etc. Standards compliance: * Full compliance with IETF (RFC 3720) * Full implementation of SPC-4 PRs and ALUA Significant code cleanups done by Christoph Hellwig. [jejb: fix up for new block bdev exclusive interface. Minor fixes from Randy Dunlap and Dan Carpenter.] Signed-off-by: Nicholas A. Bellinger <nab@linux-iscsi.org> Signed-off-by: James Bottomley <James.Bottomley@suse.de>