基于GraphX的社区发现-TAOBAO技术部数据挖掘与计算复杂算法.pptxVIP

基于GraphX的社区发现淘宝技术部数据挖掘与计算复杂算法引言复杂网络节点表?系统中的个体边表?个体之间的关系Spark技术讲座猎头公司云服务户外活动……打折机票酒店住宿旅游攻略社区发现 Community Detection相关工作FastUnfolding算法基于GraphX实现总结相关工作-算法调研GN AlgorithmGN算法[1]每次选择betweenness最大的边进行删除当节点数量较大时，计算全局的betweenness很复杂，可以考虑抽样的方法[2][1] /pdf/cond-mat/0112110.pdfcommunity structure in social and biological /pdf/cond-mat/0112110.pdfnetworks[2] /people/matteo/papers/RiondatoKornarop-BetweennessSampling.pdfFast Approximation of /people/matteo/papers/RiondatoKornarop-BetweennessSampling.pdfBetweenness/people/matteo/papers/RiondatoKornarop-BetweennessSampling.pdf Centrality through /people/matteo/papers/RiondatoKornarop-BetweennessSampling.pdfSampling相关工作-算法调研Label Propagation Algorithm每个点会把它的标签传播到它的邻居，达到稳定后属于同一个标签的点划分到一个社区[3]COPRA[4],每个节点记录多个label，删去概率低于阈值的labelSLPA[5]，不删去label，最后统计各label的频率，去除低频率的label[3] /pdf/0709.2938.pdfNear linear time algorithm to detect community structures in large-scale networks[4] /ftp/arxiv/papers/0910/0910.5516.pdfFinding overlapping communities in networks by label propagation[5] /pdf/1202.2465.pdfTowards Linear Time Overlapping Community Detection in Social Networks相关工作-算法调研其他算法KCore Subgraph[6]Local Expansion[7]Particle Competition[8]Game Theory[9]…[6] /pdf/1403.2226.pdfAccelerating Community Detection by Using K-core /pdf/1403.2226.pdfSubgraphs[7] /~inderjit/public_papers/overlapping_commumity_cikm13.pdfOverlapping Community Detection Using Seed Set /~inderjit/public_papers/overlapping_commumity_cikm13.pdfExpansion[8] /artigos/aici2009.pdfUncovering Overlap Community Structure in Complex Networks using /artigos/aici2009.pdfParticle Competition[9] http://www.cs.bgu.ac.il/~snean142/wiki.files/21-A%20game-theoretic-framework-to-identify-overlapping-communities.pdfA game-theoretic framework to identify overlapping communities in social http://www.cs.bgu.ac.il/~snean142/wiki.files/21-A%20game-theoretic-framework-to-identify-overlapping-communities.pdfnetworks相关工作-评估标准模块化 Modularity节点i到节点j存在连线可能性kikjAij实际i与j间边数为Aij与i相连共ki条边总边数为2m与j相连总边数为kjji