一个在Spark 和Cassandra之间的集成层.pdfVIP

StratioDeep: an integration layer between Spark and Cassandra Our customers #StratioBD StratioDeep An efficient data mining solution “Two and two are four? Sometimes… Sometimes they are five.” G. Orwell #StratioBD Why we use Cassandra Case A One User – Lots of data #StratioBD Case B Many Users – Few data #StratioBD Case C Many users – Lots of data #StratioBD Why we also need Spark • In Cassandra, you need to design the schema with the query in mind • Every other type of query is either very inefficient or impossible to resolve #StratioBD Challenge Accepted StratioDeep features (I) • Supports CQL3 features • Use of secondary Indexes • Small codebase (less bugs) #StratioBD StratioDeep features (II) Provides a Java friendly API: • Developers map Column Families to custom serializable POJOs • StratioDeep wraps the complexity of performing Spark calculations directly over the user provided POJOs. • SQL-Like Domain Specific Language #StratioBD Stratio DSL (I) SQL-Like domain specific language: • Built on-top of Spark’s API. • SQL + Linq abstractions. • Unique interface to all Stratio platform modules #StratioBD Stratio DSL (II) Stratio RT extension • Built on-top of Spark Streaming API. Stratio BUS extension • Registration of new channels/consumer/producers Cross-module integration with StratioMeta • Lets us create flows of data between StratioDeep StratioRT • Materialized views