Roadmaps

Filebench 1.5 Goals

• Import of a file system tree
• Custom variables
• Non-interactive runs
• Performance improvements (not that much statstics by default)
• Grammer clean-up
• More code reductions

Meeting with George on 03/03

• Added George's Sourceforge account to Filebench developer's list

• Agreed with George on most of the things discussed
  • grammer and functionality reductions to stabilize FB
  • commit per single logical change (even a small one)
  • kernel coding style
  • use autoconf-2.63-5.1.el6.noarch and automake-1.11.1-4.el6.noarch from CentOS 6.3
  • add custom variables along with random variables to Filebench-1.5
  • add import functionality from George to Filebench-1.5
  • ensure non-interactive runs in Filebench-1.5
  • remove all of the "expression evaluation"-like, except one case which keep temorarly: path=$dir/path in filesets. Later work on proper fileset evaluation.

• Think more on grammar from functionality split. Grammer parsing should be separate and responsible for grammar validation mainly. The rest should be separate.

• See incompatible will be the old workload files with new Filebench

• Add performance improvements: collect less statistics by default

• George will submit his smaller scattered patches to Vasily as soon as he can. During this time-window Vasily won't change the code much, not to cause many conflicts. After the bulk of George's small changes are in - we both will continue to work on code together.

• Vasily will work on the wiki and testing platform meanwhile
  • ramdisk with many threads

• To decide later
  • Better git-based development process, e.g., branch-per developer. Think more.
  • Filebench and workload profiles versioning
  • Expression evaluation
  • MMAP_FIXED
  • Expression evaluation
  • Proper raw devices support
  • Remove boolean_t
  • More clear grammar vs. functionality separation
  • Replace random variables by custom variables in the future?

Other assorted things to fix

- Raw device functionality is broken

- Remove boolean_t - tabular distribution, percents as long

- filesizegamma parameter is not needed as we have custom/random variables with

 gamma distribution.

- Move shm_threadflow_lock to the procflow structure where it belongs to.

- make compilation independent from parser_gram.y (so that when we touch it...)

- USAGE2 macros redefine

- add posset align attribute

- FINISH IOPRIO in a better way.

- fix compilation on SUN compilers (reported by Drew Schatt)

- fix multiple run invocations and part-by-part Filebench executione (create-filset, create-proce, stat_snap, etc.) (reported by Drew Schatt)

- Testing system on calvin: VMs with different OSes.

- random file reading with seed, not /dev/urandom (Rick req.)

- fix multiple run invocations and part-by-part Filebench executione (create-filset, create-proce, stat_snap, etc.) (Drew Schatt req.)

- Testing system: VMs with different OSes.

- On Solaris we get bzero() implicit declaration!

- On FreeBSD a lot of mutex attributes are not supported: 46604: 0.001: cannot set mutex attr PROCESS_SHARED on this platform on line 1 46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1 46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1 46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1 The main problem iS SHARED, that means if we have multiple porcesses (not threads) we'll have issues. So, check it and fix if possible.

- usec/cpu-op is incorrect for oltp workload. OLTP in general works

   weird. Figure it out, do it only after process model is enabled.

- Check what happens if pthreads (and others, like lm) are not available

- Then do validation of thread vs process model (NFS Webproxy was different(?), e.g.)

- Look more carefully through Solaris-specific functions and data structures

- -D_REENTRANT, do we need it?

- FreeBSD, analog for /proc/stat or libkstat and /proc/<pid>/stat?

- leafdirs attributes for filesets does not work correctly: entry names

 for files collide with entry names for leafdirs.

- Ifdefs for all header files

- print all defines, sizeofs, functions used, or something like that

- reenable fscheck command later

- reenable multi_client_sync.c later

- Filebench does not support aio_read?

- HAVE_CFTIME -> HAVE_STRFTIME

- Auto-completio db is outdated

- Enforce configuration option:

       #ifdef HAVE_
               ....
       #elif HAVE_
       #else
               ERROR
       #fi

       but not just if else that can easily fail.

- eemove config.h on make distclean

- modify filebench so that it supports reuse properly (even for writes)

 (then we can use dd of pre-stored  images - but how to store them?...)

- Sometimes on white we hang right after thread creation: 927: 7.180: Starting 100 filereaderthread threads wchan shows: futex_wait_queue_meroot()

- add fb_ prefix to all function in order not to pollute namespace

- rename files properly

- Use __function__ constant.

- move random generation from posset.c to fb_random.c

- make a policy about abd_t usage.