Exadata在1号店数据平台的应用实践.pptx

Exadata在1号店数据平台的应用实践overview? as an Hardware with Intelligence– offloading– smart scan– storage index? why we used for BI OLAP– above all– ehcc1号店数据平台整体架构 Service EDW real computesystem EXADATA Hadoop cluster Replication layer?????Maximum AvailabilityArchitectureHigh performanceScalableExadata OLAPDistributed DatabaseSystems流技基术于实时应StormPlatform用Oracle clusterMysql clusterNosql cluster北京IDCHbase clusterreports数据分析挖掘Thousand ETL jobsbackend报表查询自助取数平台CMSPISWe need?个性精准化广 告业务SCMHow to do?订单交易处理财务、SCM报表流量数据挖掘日志数据分析????订单交易数据、财务数据、CRM数据流量明细基础 数据日志数据structured and unstructured Integration1号店DW系统框架Exadata Hadoop Mysql 构成: ? Exadata：非流量数据的数据镜像层、数据整合层，数据中间层，应用层 ? Hadoop：流量数据的数据镜像层、数据整合层 ? Mysql：依据应用做数据集市1号店Exadata migrateQuarter Rack X2-2 ? X2 active data guard ? X4 one database many services ? Why migrateHalf Rack X4-2Use Exadata Better——smart scan? The Bottleneck on Many (Most) Large Databases is between the Disk andthe DB Server(s)!? How to Speed Up?– Make the Pipe Bigger/Faster– Reduce the Volume? 触发smart scan的前提(三条都满足):– Full scan: Full Table Scan or Fast Full Index Scan– Direct Path Reads: 并发读取数据到program global area(PGA)– 数据保存在exadata storage上: smart scan offloading是storage server独有? Need Index or not?– When indexes are present, Exadata databases tend to over use them– Make them invisible first, and then remove themUse Exadata Better——EHCC? The hybrid columnar compression (HCC) Requirements:– Direct path loads– Impdp with DIRECT option or DIRECT path inserts– Alter table move? Data is stored in compression unit– not real columnar compression– advantage vs. disadvantage? If you update rows in HCC table, The entire compression unit is locked. Therows that are updated may be moved to a lower compression level such asNo compression/ OLTP compression? How to use betterUse Exadata Better——ESFC? 一种减少数据库访问IO cost的方式 – 不依赖smart scan，一般还会被忽略 – 最有效的单block离散读访问方式，降低小IO读请求响应时间 – Write-through缓存，写操作绕过缓存直接写磁盘? The OLTP acceleration(flash disk) – 以SSD的方式提供给ASM用于数据库存储 – 高并发oltp中，redo l