experimentation资料.pptVIP

lbsh: Breadcrumbs as You Work Eric Osterweil Problem Measurement studies, simulations, and many other investigations require a lot of data work Data processing (or experimentation) can be ad-hoc Given some raw data (measurements, observations, etc.) we often “try a number of things” Simulations are often tweaked and re-run numerous times In this mode, experiments can recursively lead subsequent experiments How can a researcher always remember the exact provenance of their results? What is Data Provenance? The concept of data provenance is a lot like “chain of custody” More specifically, we borrow a definition from [1]: “… We defined data provenance as information that helps determine the derivation history of a data product, starting from its original sources.” Why Does Provenance Matter? Do we need to be able to remember exactly how we got results? Setting: Student: does a lot of processing, gets “compelling” results Advisor: wants to re-run with new data Student: panics (silently of course) Reviewers: hope that results are reproducible Sharing Work With Others? How many people have had to re-implement someone else’s algos for a paper? How about getting a tarball of crufty scripts from an author and trying to get them to work? What if you could get a tarball that was totally self-descriptive What if the tarball could totally describe the work that lead to that user’s results? What if it could allow you to re-run the whole thing? Example sort data2.out sorted-data2.out awk ‘{print $1 “\t” $5}’ sorted-data2.out dook sort data3.out sorted-data3.out join -1 1 -2 1 dook sorted-data3.out blah vi script.sh script.sh dook awk ‘{tot+=$1*$1}END{print tot}’ blah day1.out vi blah.pl blah.pl data1.out data2.out day2.out sort data1.out sorted-data1.out join -1 1 -2 1 dook sorted-data1.out blah vi blindluck.awk blindluck.awk blah day3.out Results? What if it turns out that “day3.out” has the results I wanted? Can anyone recall what the commands were for that? Were any