DStore An Easy-to-Manage Persistent State Store.ppt

Outline Project overview Consistency guarantees Failure detection Benchmarks Next steps and bigger picture Background: Scalable CHTs DStore: An easy-to-manage CHT Cheap recovery: Principles and costs Nothing new under the sun, but… Cheap recovery simplifies state management Outline Project overview Consistency guarantees Failure detection Benchmarks Next steps and bigger picture Consistency guarantees Usage model: Guarantee: For a key k, DStore enforces a global order of operations that is consistent with the order seen by individual clients. C1 issues w1(k, vnew) to replace current hash table entry (k, vold) w1 returns SUCCESS: subsequent reads return vnew w1 returns FAIL: subsequent reads return vold w1 return UNKNOWN (due to Dlib failure): two cases Case 1: Another user U2 performs a read Case 2: U1 performs a read Consistency guarantees C1 issues w1(k, vnew) to replace current hash table entry (k, vold) w1 returns SUCCESS: subsequent reads return vnew w1 returns FAIL: subsequent reads return vold w1 return UNKNOWN (due to Dlib failure): U1 reads – w1 is immediately committed or aborted U2 reads – if vold is returned, no user has read vnew if vnew is returned, no user will later read vold Versus sequential consistency Two-phase commit vs. single phase writes Recovery behavior Application-generic failure detection Failure detection and repartitioning behavior Bigger picture: What is “self-managing”? Bigger picture: What is “self-managing”? Bigger picture: What is “self-managing”? Nothing new under the sun, but… Cheap recovery simplifies state management Two-phase commit vs. single phase writes Bigger picture Big picture Use simple metrics to trigger scaling Brick load Cache hit rate Online data reconstruction Simple, aggressive failure detection Bricks send operating statistics CPU load, average queue delay, number of requests processed, etc. Statistical methods Median absolute deviation – compares one brick’s behavior with the current behavi