一个在Spark 和Cassandra之间的集成层.docxVIP

StratioDeep: an integration layer between Spark and Cassandra Our customers #StratioBD StratioDeep An efficient data mining solution “Two and two are four? Sometimes… Sometimes they are five.” G. Orwell #StratioBD Why we use Cassandra Case A One User – Lots of data #StratioBD Case B Many Users – Few data #StratioBD Case C Many users – Lots of data #StratioBD Why we also need Spark In Cassandra, you need to design the schema with the query in mind Every other type of query is either very inefficient or impossible to resolve #StratioBD Challenge Accepted StratioDeep features (I) ? ? ?  Supports CQL3 features Use of secondary Indexes Small codebase (less bugs) #StratioBD StratioDeep features (II) Provides a Java friendly API: ? ? ?  Developers map Column Families to custom serializable POJOs StratioDeep wraps the complexity of performing Spark calculations directly over the user provided POJOs. SQL-Like Domain Specific Language #StratioBD Stratio DSL (I) SQL-Like domain specific language: ? ? ?  Built on-top of Spark’s API. SQL + Linq abstractions. Unique interface to all Stratio platform modules #StratioBD Stratio DSL (II) Stratio RT extension Built on-top of Spark Streaming API. Stratio BUS extension Registration of new channels/consumer/producers Cross-module integration with StratioMeta Lets us create flows of data between StratioDeep ? StratioRT Materialized views, live queries, alerts, etc… #StratioBD Conclusion Use case A Use case C #StratioBD THANKS Luca Rosellini  Alvaro Agea @luca_rosellini  @alvaroagea