Difference between revisions of "Filebench"

Revision as of 21:19, 1 June 2011

Introduction

Filebench is a file system and storage benchmark that allows to generate a large variety of workloads. Unlike typical benchmarks it is very flexible and allows to minutely specify (any) applications' behaviour using extensive Workload Model Language (WML). Filebench uses loadable workload personalities to allow easy emulation of complex applications (e.g., mail, web, file, and database servers). Filebench is quick to set up and easy to use compared to deploying real applications. It is also a handy tool for micro-benchmarking.

Features

Filebench includes many features to facilitate file system benchmarking:

• Multiple workload types support via loadable personalities
• Ships with more than 40 pre-defined personalities, including the one that describe mail, web, file, and database servers behaviour
• Easy to add new personalities using reach Workload Model Language (WML)
• Multi-process and multi-thread workload support
• Configurable directory hierarchies with depth, width, and file sizes set to given statistical distributions
• Support of asynchronous I/O and process synchronization primitives
• Integrated statistics for throughput, latency, and CPU cycle counts per system call
• Tested on Linux, FreeBSD, and Solaris platforms (should work for any POSIX-compliant Operating System)

News

• 06/02/2011 Tentative date for Filebench 1.4.9 release
• 05/26/2011 Starting from version 1.4.9, Filebench is maintained, supported, and developed by File system and Storage Lab (FSL) located at Stony Brook University. We were maintaining Filebench Linux/FreeBSD fork before and now we were graciously allowed to manage the mainline. We are in the process of updating all Filebench-related services (wiki, mailing lists, forums, bug tracker, etc.).

Support

• If you have problems with using Filebench, use our Help Forum
• If you want to discuss some more generic questions about file system and storage benchmarking, use our Open Discussion Forum
• If you think that you found a BUG, please report it to our Bug Tracking System

Installation

Getting and Installing Filebench

Filebench is distributed as a compressed tar-ball with sources. Download the newest version of Filebench [here]. Uncompress it and follow the regular procedure for building UNIX sources:

• ./configure
• make
• sudo make install

If all of these steps complete successfully, you're ready to run Filebench. If you see any errors or warnings, please report them to Bug Tracking System.

Filebench does not have any mandatory program/library dependencies except libc. If you want command line auto-completion to work in Filebench you need to install libtecla prior to Filebench.

Files in the Installation

On Linux, package typically installs single filebench binary to /usr/local/bin/ and a number of workload personalities to /usr/local/share/filebench/workloads/. These default directories might vary from distribution to distribution and you can set them to other values during installation. Workload personalities are stored in files with .f extension. More than 40 personalities are already included in Filebench package.

Running

Filebench generates I/O operations by executing a workload personality, which defines the workload to apply to the system, and might provide various tunables to customize the workload. As it was mentioned earlier, Filebench is shipped with a library of these personalities, ready to use. Below we describe a use case when one wants to use a pre-defined workload. If you want to create a new user-defined personality and run it, read Adding new personalities section.

Interactively

If the directory where the filebench binary was installed is in the shell PATH, you can start Filebench by executing filebench command. Filebench prompt will appear after that:

user@host$ filebench
Filebench Version 1.4.9
IMPORTANT: Virtual address space randomization is enabled on this machine!
It is highly recommended to disable randomization to provide stable Filebench runs.
Echo 0 to /proc/sys/kernel/randomize_va_space file to disable the randomization.
WARNING: Could not open /proc/sys/kernel/shmmax file!
It means that you probably ran Filebench not as a root. Filebench will not increase shared
region limits in this case, which can lead to the failures on certain workloads.
11431: 0.000: Allocated 170MB of shared memory
filebench> 

You can type Filebench commands now. Type quit to exit the prompt.

One can see two warnings above:

• A lot of Linux distributions enable address space randomization. This prevents Filebench from mapping shared memory region to the same address between several processes. Disable address space randomization (echo 0 > /proc/sys/kernel/randomize_va_space) for stable operation of multi-process workloads.
• Second warning informs that Filebench was not able to increase shared memory region size. You can either:
  • Run Filebench as root
  • Increase shared memory region size to 256MB (echo 268435456 > /proc/sys/kernel/randomize_va_space) and ignore this warning

If one disables address space randomization and runs Filebench as root, the output looks much cleaner:

user@host$ sudo su
[sudo] password for user:
root@host# echo 0 > /proc/sys/kernel/randomize_va_space
root@host# go_filebench 
Filebench Version 1.4.9
12102: 0.000: Allocated 170MB of shared memory
filebench> 

Now one can load and run individual workload personalities with full control over their parameters. The following example demonstrates interactively running the fileserver workload personality.

root@user# go_filebench 
Filebench Version 1.4.9
12324: 0.000: Allocated 170MB of shared memory
filebench> load fileserver
12462: 2.869: FileServer Version 2.2 personality successfully loaded
12462: 2.869: Usage: set $dir=<dir>
12462: 2.869:        set $meanfilesize=<size>     defaults to 131072
12462: 2.869:        set $nfiles=<value>      defaults to 10000
12462: 2.869:        set $nthreads=<value>    defaults to 50
12462: 2.869:        set $meanappendsize=<value>  defaults to 16384
12462: 2.869:        set $iosize=<size>  defaults to 1048576
12462: 2.869:        set $meandirwidth=<size> defaults to 20
12462: 2.869: (sets mean dir width and dir depth is calculated as log (width, nfiles)
12462: 2.869:  
12462: 2.869:        run runtime (e.g. run 60)
filebench> run 60
12462: 4.909: Creating/pre-allocating files and filesets
12462: 4.918: Fileset bigfileset: 10000 files, avg dir width = 20, avg dir depth = 3.1, 1240.757MB
12462: 5.280: Removed any existing fileset bigfileset in 1 seconds
12462: 5.280: making tree for filset /tmp/bigfileset
12462: 5.290: Creating fileset bigfileset...
12462: 6.080: Preallocated 7979 of 10000 of fileset bigfileset in 1 seconds
12462: 6.080: waiting for fileset pre-allocation to finish
12466: 6.080: Starting 1 filereader instances
12467: 6.081: Starting 50 filereaderthread threads
12462: 7.137: Running...
12462: 67.142: Run took 60 seconds...
12462: 67.145: Per-Operation Breakdown
statfile1            128311ops     2138ops/s   0.0mb/s      0.0ms/op     2320us/op-cpu [0ms - 0ms]
deletefile1          128316ops     2138ops/s   0.0mb/s      0.2ms/op     2535us/op-cpu [0ms - 458ms]
closefile3           128323ops     2139ops/s   0.0mb/s      0.0ms/op     2328us/op-cpu [0ms - 0ms]
readfile1            128327ops     2139ops/s 283.8mb/s      0.1ms/op     2460us/op-cpu [0ms - 267ms]
openfile2            128329ops     2139ops/s   0.0mb/s      0.0ms/op     2332us/op-cpu [0ms - 2ms]
closefile2           128332ops     2139ops/s   0.0mb/s      0.0ms/op     2332us/op-cpu [0ms - 0ms]
appendfilerand1      128337ops     2139ops/s  16.6mb/s      0.1ms/op     2377us/op-cpu [0ms - 559ms]
openfile1            128343ops     2139ops/s   0.0mb/s      0.0ms/op     2353us/op-cpu [0ms - 2ms]
closefile1           128349ops     2139ops/s   0.0mb/s      0.0ms/op     2317us/op-cpu [0ms - 1ms]
wrtfile1             128352ops     2139ops/s 265.2mb/s      0.1ms/op     2601us/op-cpu [0ms - 268ms]
createfile1          128358ops     2139ops/s   0.0mb/s      0.1ms/op     2396us/op-cpu [0ms - 267ms]
12462: 67.145: IO Summary: 1411677 ops, 23526 ops/s, (2139/4278 r/w), 565mb/s, 393us cpu/op, 0.2ms latency
12462: 67.145: Shutting down processes
root@user# 

As you can see, we first loaded fileserver personality using load command. Filebench located corresponding .f in the directory with pre-defined workloads. After that tunables of the workload personality can be set. We change the benchmark directory to /mnt where presumably the file system we want to benchmark is mounted. To start workload for 60 seconds we execute run 60 command.

In response, Filebench first created file system tree with properties defined in the personality file. After that all required processes and threads are spanned and benchmark starts. After 60 seconds of the run the statistics is printed and Filebench exits.

Non-interactively

XXX: describe -f option here

Personalities

Pre-defined personalities

Workload Model Language

The Workload Model Language defines the workload personalities. It is a concise text based language that has the capability to simulate a variety of complex workload patterns. Filebench is shipped with multiple examples in the workloads directory. These workload files have a .f extension.

If you don't find an appropriate workload model among those supplied with Filebench, you can write your own. See the Writing Workload Models page for a tutorial on how to write Filebench workload models, and the Workload Model Language page for a full description of the language syntax.

Filebench for Programmers

Git repository for Filebench sources is git://filebench.git.sourceforge.net/gitroot/filebench/filebench

You can also browse the sources on the web

See Filebench for Programmers for information on Filebench internal design.

If you want to participate in Filebench development in any way, join Filebench Developers Mailing List