Difference between revisions of "Roadmaps"

From Filebench
Jump to: navigation, search
(Created page with '== Filebench 1.5 == Big changes: - Import of a file system tree - Custom variables - Non-interactive runs Others are reductions only! - Grammer clean-up - Clean-up & reductions …')
 
(→‎Other things to fix)
Line 61: Line 61:
 
== Other things to fix ==
 
== Other things to fix ==
 
- Raw device functionality is broken
 
- Raw device functionality is broken
 +
 
- Remove boolean_t
 
- Remove boolean_t
 +
- tabular distribution, percents as long                                                                                                                                                                     
 +
 +
- filesizegamma parameter is not needed as we have custom/random variables with
 +
  gamma distribution.
 +
 +
- Move shm_threadflow_lock to the procflow structure where it belongs to.
 +
 +
- make compilation independent from  parser_gram.y (so that when we touch it...)
 +
 +
- USAGE2 macros redefine
 +
 +
- add posset align attribute
 +
 +
- FINISH IOPRIO in a better way.
 +
 +
- fix compilation on SUN compilers (reported by Drew Schatt)
 +
 +
- fix multiple run invocations and part-by-part Filebench executione
 +
(create-filset, create-proce, stat_snap, etc.) (reported by Drew Schatt)
 +
 +
- Testing system on calvin: VMs with different OSes.
 +
 +
- random file reading with seed, not /dev/urandom (Rick req.)
 +
 +
- fix multiple run invocations and part-by-part Filebench executione
 +
(create-filset, create-proce, stat_snap, etc.) (Drew Schatt req.)
 +
 +
- Testing system: VMs with different OSes.
 +
 +
- On Solaris we get bzero() implicit declaration!
 +
 +
- On FreeBSD a lot of mutex attributes are not supported:
 +
46604: 0.001: cannot set mutex attr PROCESS_SHARED on this platform on line 1
 +
46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1
 +
46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1
 +
46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1
 +
The main problem iS SHARED, that means if we have multiple
 +
porcesses (not threads) we'll have issues. So, check it and fix if possible.
 +
 +
- usec/cpu-op is incorrect for oltp workload. OLTP in general works
 +
    weird. Figure it out, do it only after process model is enabled.
 +
 +
- Check what happens if pthreads (and others, like lm) are not available
 +
 +
- Then do validation of thread vs process model (NFS Webproxy was different(?), e.g.)
 +
 +
- Look more carefully through Solaris-specific functions and data structures
 +
 +
- -D_REENTRANT, do we need it?
 +
 +
- FreeBSD, analog for /proc/stat or libkstat and /proc/<pid>/stat?
 +
 +
- leafdirs attributes for filesets does not work correctly: entry names
 +
  for files collide with entry names for leafdirs.
 +
 +
- Ifdefs for all header files
 +
 +
- print all defines, sizeofs, functions used, or something like that
 +
 +
- reenable fscheck command later
 +
 +
- reenable multi_client_sync.c later
 +
 +
- Filebench does not support aio_read?
 +
 +
- HAVE_CFTIME -> HAVE_STRFTIME
 +
 +
- Auto-completio db is outdated
 +
 +
- Enforce configuration option:
 +
        #ifdef HAVE_
 +
                ....
 +
        #elif HAVE_
 +
        #else
 +
                ERROR
 +
        #fi
 +
 +
        but not just if else that can easily fail.
 +
- eemove config.h on make distclean
 +
 +
- modify filebench so that it supports reuse properly (even for writes)
 +
  (then we can use dd of pre-stored  images - but how to store them?...)
 +
 +
- Sometimes on white we hang right after thread creation:
 +
927: 7.180: Starting 100 filereaderthread threads
 +
wchan shows: futex_wait_queue_meroot()
 +
 +
- add fb_ prefix to all function in order not to pollute namespace
 +
 +
- rename files properly
 +
 +
- Use __function__ constant.
 +
 +
- move random generation from posset.c to fb_random.c
 +
 +
- make a policy about abd_t usage.                                                                                                                                                                           
 +
                                                                                                                                                                                            69,1          Bo

Revision as of 16:42, 3 March 2014

Filebench 1.5

Big changes: - Import of a file system tree - Custom variables - Non-interactive runs

Others are reductions only! - Grammer clean-up - Clean-up & reductions - Performance improvements

Filebench Future

- Testing platform

Various

To discuss with George

- Separate commit for every logical (even small) change

- Kernel coding conventions. E.g., (avd == NULL) => !avd

int function()

commits should pass checkpatch.pl

- No code that is not used by anything. Should be used by at least something.

- configure.ac Makefile.am changes cause changes in configure/Makefile and others. Use CentOS 6.4 with specific autotools versions. Have a git-verify hook for this?

- No new grammar rules, only reductions in release 1.5. I would really like to avoid addition to grammar at this stage. I'm fine with import functionality though.

- mmap functionality

- Versioning 1.5 or 1.5.0?... Workload files versions

- I'm not a big fan of large comments. Code should explain itself. But, where necessary, the comments should present.

- Testing on varilous platforms: set of tests - both perfomance and 

 functionality. Different systems. May be on ramdisk?... With many threads?

- Replace random variables by custom variables?


Specifics:

- change in avd_get_int - shouldn't that change avd_get_* every - I don't like randvar printing reintroduction (.seed, .gamma, .type) - Path changes: what if user specifies same directory?... - I like *.fb extension


Plan

- There are many useful changes from George. I don't want to change code until he has sent all of the in a proper way. - Then the plan can be that: 1) George sends all small scattered patches to me in the coming weeks. We do ping-pong 2) I meanwhile work only on the wiki & testing - When we're done with the bul of Gerge's small patches - we do regular procedure, but my freeze is over. - For now I think sending patches is the best option. Git push-pull seems to be not easy at this point.


Other things to fix

- Raw device functionality is broken

- Remove boolean_t - tabular distribution, percents as long

- filesizegamma parameter is not needed as we have custom/random variables with 

 gamma distribution.

- Move shm_threadflow_lock to the procflow structure where it belongs to.

- make compilation independent from parser_gram.y (so that when we touch it...)

- USAGE2 macros redefine

- add posset align attribute

- FINISH IOPRIO in a better way.

- fix compilation on SUN compilers (reported by Drew Schatt)

- fix multiple run invocations and part-by-part Filebench executione (create-filset, create-proce, stat_snap, etc.) (reported by Drew Schatt)

- Testing system on calvin: VMs with different OSes.

- random file reading with seed, not /dev/urandom (Rick req.)

- fix multiple run invocations and part-by-part Filebench executione (create-filset, create-proce, stat_snap, etc.) (Drew Schatt req.)

- Testing system: VMs with different OSes.

- On Solaris we get bzero() implicit declaration!

- On FreeBSD a lot of mutex attributes are not supported: 46604: 0.001: cannot set mutex attr PROCESS_SHARED on this platform on line 1 46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1 46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1 46604: 0.002: cannot set mutex attr PROCESS_SHARED on this platform on line 1 The main problem iS SHARED, that means if we have multiple porcesses (not threads) we'll have issues. So, check it and fix if possible.

- usec/cpu-op is incorrect for oltp workload. OLTP in general works 

   weird. Figure it out, do it only after process model is enabled.

- Check what happens if pthreads (and others, like lm) are not available

- Then do validation of thread vs process model (NFS Webproxy was different(?), e.g.)

- Look more carefully through Solaris-specific functions and data structures

- -D_REENTRANT, do we need it?

- FreeBSD, analog for /proc/stat or libkstat and /proc/<pid>/stat?

- leafdirs attributes for filesets does not work correctly: entry names 

 for files collide with entry names for leafdirs.

- Ifdefs for all header files

- print all defines, sizeofs, functions used, or something like that

- reenable fscheck command later

- reenable multi_client_sync.c later

- Filebench does not support aio_read?

- HAVE_CFTIME -> HAVE_STRFTIME

- Auto-completio db is outdated

- Enforce configuration option: 

       #ifdef HAVE_
               ....
       #elif HAVE_
       #else
               ERROR
       #fi

       but not just if else that can easily fail.

- eemove config.h on make distclean

- modify filebench so that it supports reuse properly (even for writes) 

 (then we can use dd of pre-stored  images - but how to store them?...)

- Sometimes on white we hang right after thread creation: 927: 7.180: Starting 100 filereaderthread threads wchan shows: futex_wait_queue_meroot()

- add fb_ prefix to all function in order not to pollute namespace

- rename files properly

- Use __function__ constant.

- move random generation from posset.c to fb_random.c

- make a policy about abd_t usage. 

                                                                                                                                                                                            69,1          Bo
Retrieved from "http://filebench.sourceforge.net/wiki/index.php?title=Roadmaps&oldid=202"