Fio: Difference between revisions

From Leo's Notes
This page was last edited on 2 August 2022, at 18:58.
← Older editNewer edit →
VisualWikitext
mNo edit summary
No edit summary
Line 27: Line 27:
!Description
!Description
|-
|-
| --rw=read
| <code>--rw=read</code>
|Do a sequential read test. Alternatively, change this to one of the following other options:
|Do a sequential read test. Alternatively, change this to one of the following other options:


* read (sequential)
* <code>read</code> (sequential)
* write (sequential)
* <code>write</code> (sequential)
* randread
* <code>randread</code>
* randwrite
* <code>randwrite</code>
* readwrite (sequential)
* <code>readwrite</code> (sequential)
* randrw
* <code>randrw</code>
|-
|-
| --size=1GB
| <code>--size=1GB</code>
|Use a 1GB test file size. Must be a multiple of 1MB
|Use a 1GB test file size. Must be a multiple of 1MB
|-
|-
| --bs=1M
| <code>--bs=1M</code>
|Use a 1MB block size. Defaults to 4k
|Use a 1MB block size. Defaults to 4k
|-
|-
| --iodepth=1
| <code>--iodepth=1</code>
|For asynchronous read/writes, the OS will immediately return and fio does not need to wait for the IO operation to complete. This option will set how many IO operations are 'in flight', or running in the background by the OS.
|For asynchronous read/writes, the OS will immediately return and fio does not need to wait for the IO operation to complete. This option will set how many IO operations are 'in flight', or running in the background by the OS.
This setting requires direct=1 and the use of a asynchronous IO engine such as libaio.
This setting requires direct=1 and the use of a asynchronous IO engine such as libaio.
|-
|-
|<nowiki>--numjobs=1</nowiki>
|<code>--numjobs=1</code>
|Number of parallel jobs that fio will spawn for the test.
|Number of parallel jobs that fio will spawn for the test.
|-
|-
|  --randrepeat=0 \
|<code>--randrepeat=0</code>


  --refill_buffers \
<code>--refill_buffers</code>


  --end_fsync=1 \
<code>--end_fsync=1</code>
|randrepeat reseeds the random generator differently for each run
|randrepeat reseeds the random generator differently for each run
refill buffers ensures that buffers are rewritten and not cached
refill buffers ensures that buffers are rewritten and not cached
Line 60: Line 60:
end fsync ensures the file contents are synced before the job exits
end fsync ensures the file contents are synced before the job exits
|-
|-
| --direct=1
| <code>--direct=1</code>
|use non-buffered I/O (usually O_DIRECT).  
|use non-buffered I/O (usually O_DIRECT).  
|}
|}
Line 99: Line 99:
* slat - submission latency. This is the time it took to submit the I/O. This could be useful if you need to determine if you need to tune the IO scheduler (eg. from spinning disk to SSD) or if there's a network issue for network based filesystems.
* slat - submission latency. This is the time it took to submit the I/O. This could be useful if you need to determine if you need to tune the IO scheduler (eg. from spinning disk to SSD) or if there's a network issue for network based filesystems.
* clat - completion latency. This is the time between submission and completion.
* clat - completion latency. This is the time between submission and completion.
=== Using fio_plot ===
fio_plot is a python based tool to help run various fio benchmarks and then visualizing the results in various graphs.
Create a <code>benchmark.ini</code> file with the following contents. Modify the target to a location that you want to test.
{{Highlight
| code = [benchfio]
target = /mnt/target-filesystem
output = test-results
type = directory
mode = read,write,randread,randwrite
size = 10G
iodepth = 1,2,4,8,16,32,64
numjobs = 1,2,4,8,16,32,64
direct = 1
engine = libaio
precondition = False
precondition_repeat = False
runtime = 60
destructive = True
block_size = 1024k
| lang = yaml
}}
Then run the benchmark with <code>bench-fio benchmark.ini</code>
Generate graphs with fio-plot


== Other notes ==
== Other notes ==

Revision as of 18:58, 2 August 2022

The flexible IO (fio) utility can be used to quickly benchmark the filesystem IO performance. A related project that uses fio is fio_plot which runs multiple benchmarks with various options and generates graphs.

Quick start

To simply test the filesystem performance with fio (direct IO, no buffers), run:

# fio \
  --ioengine=libaio \
  --direct=1 \
  --randrepeat=0 \
  --refill_buffers \
  --end_fsync=1 \
  --filename=$HOME/.fiotest \
  --name=fio-read-test \
  --rw=read \
  --size=1GB \
  --bs=1M \
  --numjobs=1 \
  --iodepth=8 \
  --runtime=60

More information on each of the options are listed below.

Option Description
--rw=read Do a sequential read test. Alternatively, change this to one of the following other options:
  • read (sequential)
  • write (sequential)
  • randread
  • randwrite
  • readwrite (sequential)
  • randrw
--size=1GB Use a 1GB test file size. Must be a multiple of 1MB
--bs=1M Use a 1MB block size. Defaults to 4k
--iodepth=1 For asynchronous read/writes, the OS will immediately return and fio does not need to wait for the IO operation to complete. This option will set how many IO operations are 'in flight', or running in the background by the OS.

This setting requires direct=1 and the use of a asynchronous IO engine such as libaio.

--numjobs=1 Number of parallel jobs that fio will spawn for the test.
--randrepeat=0

--refill_buffers

--end_fsync=1

randrepeat reseeds the random generator differently for each run

refill buffers ensures that buffers are rewritten and not cached

end fsync ensures the file contents are synced before the job exits

--direct=1 use non-buffered I/O (usually O_DIRECT).

Understanding the results

Results you get after running fio looks like something below: 

Starting 1 process
sequential-read-8-queues-1-thread: Laying out IO file (1 file / 1024MiB)
Jobs: 1 (f=1): [R(1)][100.0%][r=241MiB/s][r=241 IOPS][eta 00m:00s]
sequential-read-8-queues-1-thread: (groupid=0, jobs=1): err= 0: pid=137965: Thu Jul 28 15:41:32 2022
  read: IOPS=190, BW=191MiB/s (200MB/s)(957MiB/5014msec)
    slat (usec): min=105, max=881, avg=222.28, stdev=54.72
    clat (msec): min=3, max=426, avg=41.63, stdev=63.38
     lat (msec): min=3, max=427, avg=41.86, stdev=63.38
    clat percentiles (msec):
  • slat - submission latency. This is the time it took to submit the I/O. This could be useful if you need to determine if you need to tune the IO scheduler (eg. from spinning disk to SSD) or if there's a network issue for network based filesystems.
  • clat - completion latency. This is the time between submission and completion.

Using fio_plot

fio_plot is a python based tool to help run various fio benchmarks and then visualizing the results in various graphs.

Create a benchmark.ini file with the following contents. Modify the target to a location that you want to test. 

[benchfio]
target = /mnt/target-filesystem
output = test-results
type = directory
mode = read,write,randread,randwrite
size = 10G
iodepth = 1,2,4,8,16,32,64
numjobs = 1,2,4,8,16,32,64
direct = 1
engine = libaio
precondition = False
precondition_repeat = False
runtime = 60
destructive = True
block_size = 1024k

Then run the benchmark with bench-fio benchmark.ini

Generate graphs with fio-plot

Other notes

Drop caches

You might want to drop caches before doing a test to avoid skewed results by running: echo 1 > /proc/sys/vm/drop_caches

Change the output format

Use the --output-format=json option to change the output format to json.