Fio - Revision history

Leo at 17:49, 18 August 2022

2022-08-18T17:49:32Z

← Older revision		Revision as of 17:49, 18 August 2022
Line 140:		Line 140:
	fio_plot is a python based tool to help run various fio benchmarks and then visualizing the results in various graphs.		fio_plot is a python based tool to help run various fio benchmarks and then visualizing the results in various graphs.

			=== Installation ===
			To install fio-plot system-wide, run: <code>pip3 install fio-plot</code>. If you only want to install it on a user account, use: <code>pip3 install --user fio-plot</code>.

			=== Usage ===
	Create a <code>benchmark.ini</code> file with the following contents. Modify the target to a location that you want to test.		Create a <code>benchmark.ini</code> file with the following contents. Modify the target to a location that you want to test.
	{{Highlight		{{Highlight

Leo at 17:01, 4 August 2022

2022-08-04T17:01:30Z

@@ Line 1: / Line 1: @@
-The flexible IO (fio) utility can be used to quickly benchmark the filesystem IO performance. A related project that uses fio is fio_plot which runs multiple benchmarks with various options and generates graphs.
+The flexible IO (fio) tester is an open source disk benchmark utility. It includes a number of features including a wide range of IO engines (for testing different aspects of IO on a system), threaded jobs, IO depth, and much more than what something like dd can offer.
 * fio project website: https://github.com/axboe/fio
@@ Line 5: / Line 7: @@
 == Quick start ==
-To simply test the filesystem performance with fio (direct IO, no buffers), run:{{Highlight
 | code = # fio \
    --ioengine=libaio \
@@ Line 48: / Line 82: @@
 |<code>--numjobs=1</code>
 |Number of parallel jobs that fio will spawn for the test.
 |-
 |<code>--randrepeat=0</code>
@@ Line 98: / Line 135: @@
 * slat - submission latency. This is the time it took to submit the I/O. This could be useful if you need to determine if you need to tune the IO scheduler (eg. from spinning disk to SSD) or if there's a network issue for network based filesystems.
 * clat - completion latency. This is the time between submission and completion.
 === Using fio_plot ===
@@ Line 123: / Line 161: @@
 Then run the benchmark with <code>bench-fio benchmark.ini</code>
-Generate graphs with fio-plot
+Generate graphs with fio-plot. Something like the following script will generate a bunch of graphs.
 == Other notes ==
 === Drop caches ===
-You might want to drop caches before doing a test to avoid skewed results by running: <code>echo 1 > /proc/sys/vm/drop_caches</code>
+You might want to drop caches before doing a test to avoid skewed results by running: <code>echo 3 > /proc/sys/vm/drop_caches</code>
 === Change the output format ===

Leo at 15:43, 4 August 2022

2022-08-04T15:43:18Z

← Older revision		Revision as of 15:43, 4 August 2022
Line 2:		Line 2:

	* fio project website: https://github.com/axboe/fio		* fio project website: https://github.com/axboe/fio
	* fio man page: https://linux.die.net/man/1/fio
	* fio_plot project website: https://github.com/louwrentius/fio-plot		* fio_plot project website: https://github.com/louwrentius/fio-plot

Line 133:		Line 132:
	=== Change the output format ===		=== Change the output format ===
	Use the --output-format=json option to change the output format to json.		Use the --output-format=json option to change the output format to json.

			== See also ==

			* Documentation - https://fio.readthedocs.io/en/latest/fio_doc.html
			* fio man page: https://linux.die.net/man/1/fio

Leo at 18:58, 2 August 2022

2022-08-02T18:58:48Z

← Older revision		Revision as of 18:58, 2 August 2022
Line 27:		Line 27:
	!Description		!Description
	\|-		\|-
	\| --rw=read		\| <code>--rw=read</code>
	\|Do a sequential read test. Alternatively, change this to one of the following other options:		\|Do a sequential read test. Alternatively, change this to one of the following other options:

	* read (sequential)		* <code>read</code> (sequential)
	* write (sequential)		* <code>write</code> (sequential)
	* randread		* <code>randread</code>
	* randwrite		* <code>randwrite</code>
	* readwrite (sequential)		* <code>readwrite</code> (sequential)
	* randrw		* <code>randrw</code>
	\|-		\|-
	\| --size=1GB		\| <code>--size=1GB</code>
	\|Use a 1GB test file size. Must be a multiple of 1MB		\|Use a 1GB test file size. Must be a multiple of 1MB
	\|-		\|-
	\| --bs=1M		\| <code>--bs=1M</code>
	\|Use a 1MB block size. Defaults to 4k		\|Use a 1MB block size. Defaults to 4k
	\|-		\|-
	\| --iodepth=1		\| <code>--iodepth=1</code>
	\|For asynchronous read/writes, the OS will immediately return and fio does not need to wait for the IO operation to complete. This option will set how many IO operations are 'in flight', or running in the background by the OS.		\|For asynchronous read/writes, the OS will immediately return and fio does not need to wait for the IO operation to complete. This option will set how many IO operations are 'in flight', or running in the background by the OS.
	This setting requires direct=1 and the use of a asynchronous IO engine such as libaio.		This setting requires direct=1 and the use of a asynchronous IO engine such as libaio.
	\|-		\|-
	\|<~~nowiki~~>--numjobs=1</~~nowiki~~>		\|<code>--numjobs=1</code>
	\|Number of parallel jobs that fio will spawn for the test.		\|Number of parallel jobs that fio will spawn for the test.
	\|-		\|-
	\| --randrepeat=0 \		\|<code>--randrepeat=0</code>

	--refill_buffers \		<code>--refill_buffers</code>

	--end_fsync=1 \		<code>--end_fsync=1</code>
	\|randrepeat reseeds the random generator differently for each run		\|randrepeat reseeds the random generator differently for each run
	refill buffers ensures that buffers are rewritten and not cached		refill buffers ensures that buffers are rewritten and not cached
Line 60:		Line 60:
	end fsync ensures the file contents are synced before the job exits		end fsync ensures the file contents are synced before the job exits
	\|-		\|-
	\| --direct=1		\| <code>--direct=1</code>
	\|use non-buffered I/O (usually O_DIRECT).		\|use non-buffered I/O (usually O_DIRECT).
	\|}		\|}
Line 99:		Line 99:
	* slat - submission latency. This is the time it took to submit the I/O. This could be useful if you need to determine if you need to tune the IO scheduler (eg. from spinning disk to SSD) or if there's a network issue for network based filesystems.		* slat - submission latency. This is the time it took to submit the I/O. This could be useful if you need to determine if you need to tune the IO scheduler (eg. from spinning disk to SSD) or if there's a network issue for network based filesystems.
	* clat - completion latency. This is the time between submission and completion.		* clat - completion latency. This is the time between submission and completion.

			=== Using fio_plot ===
			fio_plot is a python based tool to help run various fio benchmarks and then visualizing the results in various graphs.

			Create a <code>benchmark.ini</code> file with the following contents. Modify the target to a location that you want to test.
			{{Highlight
			\| code = [benchfio]
			target = /mnt/target-filesystem
			output = test-results
			type = directory
			mode = read,write,randread,randwrite
			size = 10G
			iodepth = 1,2,4,8,16,32,64
			numjobs = 1,2,4,8,16,32,64
			direct = 1
			engine = libaio
			precondition = False
			precondition_repeat = False
			runtime = 60
			destructive = True
			block_size = 1024k
			\| lang = yaml
			}}
			Then run the benchmark with <code>bench-fio benchmark.ini</code>

			Generate graphs with fio-plot

	== Other notes ==		== Other notes ==

Leo at 18:36, 2 August 2022

2022-08-02T18:36:10Z

← Older revision		Revision as of 18:36, 2 August 2022
Line 24:		Line 24:
	}}More information on each of the options are listed below.		}}More information on each of the options are listed below.
	{\| class="wikitable"		{\| class="wikitable"
	!Option		! width="15%" \| Option
	!Description		!Description
	\|-		\|-

Leo at 18:32, 2 August 2022

2022-08-02T18:32:51Z

@@ Line 1: / Line 1: @@
-The flexible IO (fio) utility can be used to quickly benchmark the filesystem IO performance.
+The flexible IO (fio) utility can be used to quickly benchmark the filesystem IO performance. A related project that uses fio is fio_plot which runs multiple benchmarks with various options and generates graphs.
 == Quick start ==
-Select one of the following commands to run.
+To simply test the filesystem performance with fio (direct IO, no buffers), run:{{Highlight
 | code = # fio \
    --ioengine=libaio \
@@ Line 15: / Line 13: @@
    --refill_buffers \
    --end_fsync=1 \
-   --filename=/home/lleung/.kdiskmark.tmp \
+   --filename=$HOME/.fiotest \
-   --name=sequential-read-8-queues-1-thread \
+   --name=fio-read-test \
    --rw=read \
-   --size=1073741824 \
+   --size=1GB \
-   --bs=1024k \
+   --bs=1M \
    --numjobs=1 \
    --iodepth=8 \
-   --runtime=5
+   --runtime=60
 | lang = terminal
-}}
+}}More information on each of the options are listed below.
 |}

Leo: Created page with "The flexible IO (fio) utility can be used to quickly benchmark the filesystem IO performance. == Quick start == Select one of the following commands to run. {| class="wikitable" !Description !Command |- |Sequential read test, 1MB block size, 1 thread, 1 queue |{{Highlight | code = # fio \ --ioengine=libaio \ --direct=1 \ --randrepeat=0 \ --refill_buffers \ --end_fsync=1 \ --filename=/home/lleung/.kdiskmark.tmp \ --name=sequential-read-8-queues-1-thread \..."

2022-07-28T21:58:16Z

Created page with "The flexible IO (fio) utility can be used to quickly benchmark the filesystem IO performance. == Quick start == Select one of the following commands to run. {| class="wikitable" !Description !Command |- |Sequential read test, 1MB block size, 1 thread, 1 queue |{{Highlight | code = # fio \ --ioengine=libaio \ --direct=1 \ --randrepeat=0 \ --refill_buffers \ --end_fsync=1 \ --filename=/home/lleung/.kdiskmark.tmp \ --name=sequential-read-8-queues-1-thread \..."

New page

The flexible IO (fio) utility can be used to quickly benchmark the filesystem IO performance.

== Quick start ==
Select one of the following commands to run.
{| class="wikitable"
!Description
!Command
|-
|Sequential read test, 1MB block size, 1 thread, 1 queue
|{{Highlight
| code = # fio \
--ioengine=libaio \
--direct=1 \
--randrepeat=0 \
--refill_buffers \
--end_fsync=1 \
--filename=/home/lleung/.kdiskmark.tmp \
--name=sequential-read-8-queues-1-thread \
--rw=read \
--size=1073741824 \
--bs=1024k \
--iodepth=1 \
--numjobs=1 \
--iodepth=8 \
--runtime=5
| lang = terminal
}}
|}

=== Understanding the results ===
Results you get after running fio looks like something below:
{{Highlight
| code = Starting 1 process
sequential-read-8-queues-1-thread: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=241MiB/s][r=241 IOPS][eta 00m:00s]
sequential-read-8-queues-1-thread: (groupid=0, jobs=1): err= 0: pid=137965: Thu Jul 28 15:41:32 2022
read: IOPS=190, BW=191MiB/s (200MB/s)(957MiB/5014msec)
slat (usec): min=105, max=881, avg=222.28, stdev=54.72
clat (msec): min=3, max=426, avg=41.63, stdev=63.38
lat (msec): min=3, max=427, avg=41.86, stdev=63.38
clat percentiles (msec):
| 1.00th=[ 6], 5.00th=[ 10], 10.00th=[ 12], 20.00th=[ 17],
| 30.00th=[ 20], 40.00th=[ 23], 50.00th=[ 27], 60.00th=[ 29],
| 70.00th=[ 33], 80.00th=[ 39], 90.00th=[ 55], 95.00th=[ 134],
| 99.00th=[ 338], 99.50th=[ 355], 99.90th=[ 426], 99.95th=[ 426],
| 99.99th=[ 426]
bw ( KiB/s): min=51200, max=323584, per=99.55%, avg=194560.00, stdev=90833.15, samples=10
iops : min= 50, max= 316, avg=190.00, stdev=88.70, samples=10
lat (msec) : 4=0.31%, 10=5.85%, 20=26.23%, 50=55.80%, 100=4.91%
lat (msec) : 250=2.72%, 500=4.18%
cpu : usr=0.08%, sys=4.39%, ctx=881, majf=0, minf=526
IO depths : 1=0.1%, 2=0.2%, 4=0.4%, 8=99.3%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=99.9%, 8=0.1%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=957,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=8

Run status group 0 (all jobs):
READ: bw=191MiB/s (200MB/s), 191MiB/s-191MiB/s (200MB/s-200MB/s), io=957MiB (1003MB), run=5014-5014msec
| lang = text
}}

* slat - submission latency. This is the time it took to submit the I/O. This could be useful if you need to determine if you need to tune the IO scheduler (eg. from spinning disk to SSD) or if there's a network issue for network based filesystems.
* clat - completion latency. This is the time between submission and completion.

== Other notes ==

=== Drop caches ===
You might want to drop caches before doing a test to avoid skewed results by running: <code>echo 1 > /proc/sys/vm/drop_caches</code>

=== Change the output format ===
Use the --output-format=json option to change the output format to json.