About REPPAR

The workshop is concerned with experimental practices in parallel computing research. We are interested in research works that address the statistically rigorous analysis of experimental data and visualization techniques of these data. We also encourage researchers to state best practices to conduct experiments and papers that report experiences obtained when trying to reproduce or repeat experiments of others. The workshop also welcomes papers on new tools for experimental computational sciences, e.g., tools to archive large experimental data sets and the source code that generated them. This includes (1) workflow systems for defining the experimental structure of experiments and their automated execution as well as (2) experimental testbeds, which may serve as underlying framework for experimental workflows, e.g., deploying personalized operating system images on clusters.

Scope / Topics of Interest

• Experimental design
  • correct experimental design, e.g., factorial designs
  • best practices how (often) to measure execution times
• Experiences with experimentation
  • What is/was needed to reproduce/replicate/repeat experiment shown in other papers?
  • best practices to conduct experiments
  • overcoming difficulties in experimental setups, e.g., overheads introduced by tracing/profiling tools
  • controlling system noise on parallel machines
• Experimental testbeds
  • reproducible deployments of virtual machines
  • languages to define experimental workflows
• Analysis of experimental data
  • rigorous statistical analysis of experiments
  • data mining techniques to trim solution space
  • parallel data analysis
  • automated document/protocol generation
• Tools for reproducible research
  • versioning of source code
  • archiving experimental data
  • tools for automation / re-execution of experiments
• Visualization of experimental data
  • visualization techniques for large experimental data sets
  • tools for interactive data analysis of experimental results