Difference between revisions of "Filebench"
(→Running non-interactively) |
|||
| Line 155: | Line 155: | ||
=== Running non-interactively === | === Running non-interactively === | ||
| + | |||
| + | <span style="color:#0f0">XXX: describe -f option here</span> | ||
== Workloads == | == Workloads == | ||
Revision as of 22:53, 31 May 2011
Contents
Introduction
Filebench is a file system and storage benchmark that allows to generate a large variety of workloads. Unlike typical benchmarks it is very flexible and allows to minutely specify (any) applications' behaviour using extensive Workload Model Language (WML). Filebench uses loadable workload personalities to allow easy emulation of complex applications (mail servers, web servers, database server, etc.). Filebench is quick to set up and use compared to a real bulky applications. It is also a handy tool for micro-benchmarking.
Features
Filebench includes many features to facilitate file system benchmarking:
- Multiple workload types support via loadable personalities
- Includes pre-defined mail, web, file and database server personalities (as well as many others)
- Easy to add new personalities using Workload Model Language (WML)
- Reach WML language for workload specification (even includes synchronisation primitives for Database-like workloads)
- Multi-threaded and multi-process workload support
- Integrated statistics for throughput, latency, and CPU cycle counts per system call
- Fractal directory hierarchies with depth, width, and file sizes set to a given statistical distributions
- Tested on Linux, FreeBSD, and Solaris platforms and should work for any POSIX-compliant Operating System
- XXX: think anbout most relevant features
News
Filebench continues to evolve.
XXX: update with what hapened recently and what is in the plans. Releases.
Support
- If you have problems with using Filebench, use our Help Forum
- If you want to discuss some more generic questions about file system and storage benchmarking, use our Open Discussion Forum
- If you think that you found a BUG, please report it to our Bug Tracking System
- XXX: mention FAQ
Installing
Getting and Installing Filebench
Filebench is distributed as a compressed tar-ball with sources. Download the newest version of Filebench [here]. Uncompress it and follow the regular procedure for building UNIX sources:
-
./configure -
make -
sudo make install
If all of these steps complete successfully, you're ready to run Filebench. If you see any errors or warnings, please report them to Bug Tracking System.
Filebench does not have any mandatory program/library dependencies except libc. If you want command line auto-completion to work in Filebench you need to install libtecla prior to Filebench.
Files in the Installation
On Linux, package typically installs single filebench binary to /usr/local/bin/ and a number of workload personalities to /usr/local/share/filebench/workloads/.
These default directories might vary from distribution to distribution and you can set them to other values during installation. Workload personalities are stored in files with .f extension. More than 40 personalities are already included in Filebench package:
user@host$ ls -1 /usr/local/share/filebench/workloads/ compflow_demo.f filemicro_rwritedsync.f filemicro_statfile.f multistreamwritedirect.f randomrw.f tpcso.f copyfiles.f filemicro_rwrite.f filemicro_writefsync.f multistreamwrite.f randomwrite.f varmail.f createfiles.f filemicro_rwritefsync.f fileserver.f netsfs.f ratelimcopyfiles.f videoserver.f filemicro_create.f filemicro_seqread.f listdirs.f networkfs.f removedirs.f webproxy.f filemicro_createfiles.f filemicro_seqwrite.f makedirs.f oltp.f singlestreamreaddirect.f webserver.f filemicro_createrand.f filemicro_seqwriterand.f mongo.f openfiles.f singlestreamread.f filemicro_delete.f filemicro_seqwriterandvargam.f multistreamreaddirect.f randomfileaccess.f singlestreamwritedirect.f filemicro_rread.f filemicro_seqwriterandvartab.f multistreamread.f randomread.f singlestreamwrite.f
Running
Filebench generates I/O operations by executing a workload personality, which defines the workload to apply to the system, and might provide various tunables to customize the workload. As it was mentioned earlier, Filebench is shipped with a library of these personalities, ready to use. Below we describe a use case when one wants to use a pre-defined workload. If you want to create a new user-defined personality and run it, read Adding new personalities section.
Running interactively
If the directory where the filebench binary was installed is in the shell PATH, you can start Filebench by executing filebench command.
Filebench prompt will appear after that:
user@host$ filebench Filebench Version 1.4.9 IMPORTANT: Virtual address space randomization is enabled on this machine! It is highly recommended to disable randomization to provide stable Filebench runs. Echo 0 to /proc/sys/kernel/randomize_va_space file to disable the randomization. WARNING: Could not open /proc/sys/kernel/shmmax file! It means that you probably ran Filebench not as a root. Filebench will not increase shared region limits in this case, which can lead to the failures on certain workloads. 11431: 0.000: Allocated 170MB of shared memory filebench>
You can type Filebench commands now. Type quit to exit the prompt.
One can see two warnings above:
- A lot of Linux distributions enable address space randomization. This prevents Filebench from mapping shared memory region to the same address between several processes. Disable address space randomization (
echo 0 > /proc/sys/kernel/randomize_va_space) for stable operation of multi-process workloads. - Second warning informs that Filebench was not able to increase shared memory region size. You can either:
- Run Filebench as root
- Increase shared memory region size to 256MB (
echo 268435456 > /proc/sys/kernel/randomize_va_space) and ignore this warning
If one disables address space randomization and runs Filebench as root, the output looks much cleaner:
user@host$ sudo su [sudo] password for user: root@host# echo 0 > /proc/sys/kernel/randomize_va_space root@host# go_filebench Filebench Version 1.4.9 12102: 0.000: Allocated 170MB of shared memory filebench>
Now one can load and run individual workload personalities with full control over their parameters. The following example demonstrates interactively running the fileserver workload personality.
root@user# go_filebench Filebench Version 1.4.9 12324: 0.000: Allocated 170MB of shared memory filebench> load fileserver 12462: 2.869: FileServer Version 2.2 personality successfully loaded 12462: 2.869: Usage: set $dir=<dir> 12462: 2.869: set $meanfilesize=<size> defaults to 131072 12462: 2.869: set $nfiles=<value> defaults to 10000 12462: 2.869: set $nthreads=<value> defaults to 50 12462: 2.869: set $meanappendsize=<value> defaults to 16384 12462: 2.869: set $iosize=<size> defaults to 1048576 12462: 2.869: set $meandirwidth=<size> defaults to 20 12462: 2.869: (sets mean dir width and dir depth is calculated as log (width, nfiles) 12462: 2.869: 12462: 2.869: run runtime (e.g. run 60) filebench> run 60 12462: 4.909: Creating/pre-allocating files and filesets 12462: 4.918: Fileset bigfileset: 10000 files, 0 leafdirs, avg dir width = 20, avg dir depth = 3.1, 1240.757MB 12462: 5.280: Removed any existing fileset bigfileset in 1 seconds 12462: 5.280: making tree for filset /tmp/bigfileset 12462: 5.290: Creating fileset bigfileset... 12462: 6.080: Preallocated 7979 of 10000 of fileset bigfileset in 1 seconds 12462: 6.080: waiting for fileset pre-allocation to finish 12466: 6.080: Starting 1 filereader instances 12467: 6.081: Starting 50 filereaderthread threads 12462: 7.137: Running... 12462: 67.142: Run took 60 seconds... 12462: 67.145: Per-Operation Breakdown statfile1 128311ops 2138ops/s 0.0mb/s 0.0ms/op 2320us/op-cpu [0ms - 0ms] deletefile1 128316ops 2138ops/s 0.0mb/s 0.2ms/op 2535us/op-cpu [0ms - 458ms] closefile3 128323ops 2139ops/s 0.0mb/s 0.0ms/op 2328us/op-cpu [0ms - 0ms] readfile1 128327ops 2139ops/s 283.8mb/s 0.1ms/op 2460us/op-cpu [0ms - 267ms] openfile2 128329ops 2139ops/s 0.0mb/s 0.0ms/op 2332us/op-cpu [0ms - 2ms] closefile2 128332ops 2139ops/s 0.0mb/s 0.0ms/op 2332us/op-cpu [0ms - 0ms] appendfilerand1 128337ops 2139ops/s 16.6mb/s 0.1ms/op 2377us/op-cpu [0ms - 559ms] openfile1 128343ops 2139ops/s 0.0mb/s 0.0ms/op 2353us/op-cpu [0ms - 2ms] closefile1 128349ops 2139ops/s 0.0mb/s 0.0ms/op 2317us/op-cpu [0ms - 1ms] wrtfile1 128352ops 2139ops/s 265.2mb/s 0.1ms/op 2601us/op-cpu [0ms - 268ms] createfile1 128358ops 2139ops/s 0.0mb/s 0.1ms/op 2396us/op-cpu [0ms - 267ms] 12462: 67.145: IO Summary: 1411677 ops, 23526.347 ops/s, (2139/4278 r/w), 565.7mb/s, 393us cpu/op, 0.2ms latency 12462: 67.145: Shutting down processes root@user
As you can see, we first loaded fileserver personality using load command. Filebench located corresponding .f in the directory with pre-defined workloads. After that tunables of the workload personality can be set. We change the benchmark directory to /mnt where presumably the file system we want to benchmark is mounted. To start workload for 60 seconds we execute run 60 command.
In response, Filebench first created file system tree with properties defined in the personality file. After that all required processes and threads are spanned and benchmark starts. After 60 seconds of the run the statistics is printed and Filebench exits.
Running non-interactively
XXX: describe -f option here
Workloads
Workload personalities define the workload to apply, and the default tunables for scaling that workload to the system.
Workloads are stored in workload definition files which Filebench loads, and can be easily customized from either the Filebench command line interface or by editing the files themselves (or copies of the files).
Filebench is shipped with a library of different workloads, which are described in the following sections.
Application Emulation
Application-Emulation Workloads Currently Available in Filebench package
varmail
A /var/mail NFS mail server emaulation, following the workload of postmark, but multi-threaded. The workload consists of a multi-threaded set of open/read/close, open/append/close and deletes in a single directory.
fileserver
A file system workload, similar to SPECsfs. This workload performs a sequence of creates, deletes, appends, reads, writes and attribute operations on the file system. A configurable hierarchical directory structure is used for the file set.
oltp
A database emulator. This workload performance transactions into a filesystem using an I/O model from Oracle 9i. This workload tests for the performance of small random reads & writes, and is sensitive to the latency of moderate (128k+) synchronous writes as in the log file. It by default launches 200 reader processes, 10 processes for asynchronous writing, and a log writer. The emulation includes use of ISM shared memory as per Oracle, Sybase etc which is critical to I/O efficiency (as_lock optimizations).
dss
DSS Database: this is future work TBD.
webserver
A mix of open/read/close of multiple files in a directory tree, plus a file append (to simulate the web log). 100 threads are used by default. 16k is appended to the weblog for every 10 reads.
webproxy
A mix of create/write/close, open/read/close, delete of multiple files in a directory tree, plus a file append (to simulate the proxy log). 100 threads are used by default. 16k is appended to the log for every 10 read/writes.
Micro-benchmark
copyfiles
A copy of a large directory tree. This workload creates a hierarchical directory tree, then measures the rate at which files can be copied from the source tree to a new tree. A single thread is used by default, although this is configurable.
createfiles
Create a directory tree and fill files. A populate of files of specified sizes a directory tree. File sizes are chosen according to a gamma distribution of 1.5, with a mean size of 16k.
randomread
A multi-threaded read of a single large file, defaulting to 8k reads. A single thread is used by default, although configurable by $nthreads.
randomwrite
A multi-threaded write of a single large file, defaulting to 8k writes. A single thread is used by default, although configurable by $nthreads.
singlestreamread
A sequential read of a large file. 1MB reads are used by default.
singlestreamwrite
A sequential write of a large file. 1MB writes are used by default.
multistreamwrite
A sequential read of 4 large files, each with their own reader thread. 1MB reads are used by default.
multistreamwrite
A sequential write of 4 large files, each with their own writer thread. 1MB writes are used by default.
Recommended parameters
The following recommmended parameters have been provided as suggestions; customize as appropriate.
Small Configurations
These may be suitable for servers with one or two disks.
Worklads Suggested Parameters - fileserver
- bringover
- createfiles
- deletefiles
- varmail
- webproxy
- webserver
set $nfiles=50000 - randomread
- singlestreamread
- multistreamread
- singlestreamwrite
- multistreamwrite
set $filesize=1g - oltp
set $filesize=1g
Large Configurations
These may be suitable for servers with 20+ disks.
Worklads Suggested Parameters - fileserver
- bringover
- createfiles
- deletefiles
- mongo
- varmail
- webproxy
- webserver
set $nfiles=100000 - randomread
- randomwrite
set $filesize=5g
set $nthreads=256
- filemicro_createrand
- filemicro_rread
- filemicro_rwritefsync
- singlestreamread
- multistreamread
- singlestreamwrite
- multistreamwrite
set $filesize=5g - oltp
set $filesize=5g - filemicro_create
- filemicro_delete
- filemicro_seqwrite
- filemicro_seqwriterand
- filemicro_writefsync
set $count= 100000
Filebench Batch Run Harness
The filebench program calls go_filebench to automatically run multiple configurations. Each config section corresponds to one workload file (.f)
The command is as follows:
# /bin/benchmarks/filebench/bin/filebench <profile name>
The statistics results are saved in the directory specified in the DEFAULT section.
Workload Model Language
The Workload Model Language defines the workload personalities. It is a concise text based language that has the capability to simulate a variety of complex workload patterns. Filebench is shipped with numerous examples in the workloads directory. These workload files have a .f extension.
If you don't find a appropriate workload model among those supplied with Filebench, you can write your own. See the Writing Workload Models page for a tutorial on how to write Filebench workload models, and the Workload Model Language page for a description of the language syntax. As Filebench is a work in progress, this is not yet a stable interface.
The following are examples of what the language currently looks like.
singlestreamwritedirect.f
The following is the most basic example of a workload file. This workload creates a file and writes to it from a single process and (by default) a single thread.
#
# Copyright 2005 Sun Microsystems, Inc. All rights reserved.
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License.
# See the file LICENSING in this distribution for details.
#
set $dir=/tmp
set $nthreads=1
set $iosize=1m
define file name=largefile1,path=$dir
define process name=seqwrite,instances=1
{
thread name=seqwrite,memsize=10m,instances=$nthreads
{
flowop write name=seqwrite,filename=largefile1,iosize=$iosize,directio
flowop bwlimit name=limit
}
}
echo "Single Stream Write Version 1.7 2005/06/21 21:18:53 personality successfully loaded"
usage "Usage: set \$dir=<dir>"
usage " set \$filesize=<size> defaults to $filesize"
usage " set \$nthreads=<value> defaults to $nthreads"
usage " set \$iosize=<value> defaults to $iosize"
usage " "
usage " run runtime (e.g. run 60)"
randomread.f
This is the definition file for a random read workload.
varmail.f
The following workload file simulates /var/mail activity on a mail server. Note the variety of file operations that can be performed, such as readwholefile and appendfilerand.
#
# Copyright 2005 Sun Microsystems, Inc. All rights reserved
#
# The contents of this file are subject to the terms of the
# Common Development and Distribution License.
# See the file LICENSING in this distribution for details
#
set $dir=/tmp
set $nfiles=1000
set $meandirwidth=1000000
set $filesize=16k
set $nthreads=16
set $meaniosize=16k
define fileset name=bigfileset,path=$dir,size=$filesize,entries=$nfiles,dirwidth=$meandirwidth,prealloc=80
define process name=filereader,instances=1
{
thread name=filereaderthread,memsize=10m,instances=$nthreads
{
flowop deletefile name=deletefile1,filesetname=bigfileset
flowop createfile name=createfile2,filesetname=bigfileset,fd=1
flowop appendfilerand name=appendfilerand2,iosize=$meaniosize,fd=1
flowop fsync name=fsyncfile2,fd=1
flowop closefile name=closefile2,fd=1
flowop openfile name=openfile3,filesetname=bigfileset,fd=1
flowop readwholefile name=readfile3,fd=1
flowop appendfilerand name=appendfilerand3,iosize=$meaniosize,fd=1
flowop fsync name=fsyncfile3,fd=1
flowop closefile name=closefile3,fd=1
flowop openfile name=openfile4,filesetname=bigfileset,fd=1
flowop readwholefile name=readfile4,fd=1
flowop closefile name=closefile4,fd=1
}
}
echo "Varmail Version 1.24 2005/06/22 08:08:30 personality successfully loaded"
usage "Usage: set \$dir=<dir>"
usage " set \$filesize=<size> defaults to $filesize"
usage " set \$nfiles=<value> defaults to $nfiles"
usage " set \$nthreads=<value> defaults to $nthreads"
usage " set \$meaniosize=<value> defaults to $meaniosize"
usage " set \$meandirwidth=<size> defaults to $meandirwidth"
usage "(sets mean dir width and dir depth is calculated as log (width, nfiles)"
usage " dirdepth therefore defaults to dir depth of 1 as in postmark"
usage " set $meandir lower to increase depth beyond 1 if desired)"
usage " "
usage " run runtime (e.g. run 60)"
List of Flowops
Here you can find the List of Flowops.
List of Attributes
Here you can find the List of Attributes.
Filebench for Programmers
See Filebench for Programmers for details on the Filebench code.
If you want to participate in Filebench development in any way, join Filebench Developers Mailing List
