大数据的惩罚整合分析方法.pdfVIP

大数据的惩罚整合分析方法 【摘要】：大数据具有数据来源差异性、高维性及稀疏性等特点，如何挖掘数据集间的异质性和共同性 并降维去噪是大数据分析的目标与挑战之一。惩罚整合分析（Penalized Integrative Analysis ）同时分析多个 独立数据集，避免因地域、时间等因素造成的样本差异而引起模型不稳定，是研究大数据差异性的有效方 法。它的特点是将每个解释变量在所有数据集中的系数视为一组，通过惩罚函数对系数组进行压缩，研究 变量间的关联性并实现降维。本文从同构数据整合分析、异构数据整合分析以及考虑网络结构的整合分析 三方面梳理了惩罚整合分析方法的原理、算法和研究现状。统计模拟发现，在弱相关、一般相关和强相关 L L L 三种情形下， Group Bridge、 Group MCP 、Composite MCP 都表现良好，其中 Group Bridge 的假阳数 1 1 1 L 最低且最稳定。最后，将 Group Bridge 整合分析用于农村新农合的家庭医疗支出分析，发现不管在总体 1 还是各个地区，它都比单数据集分析的预测效果更好。 关键词：大数据；惩罚整合分析；变量选择；医疗支出 中图分类号：F222.3 文献标识码：A Penalized Integrative Analysis Approaches for Big Data Abstract:The difference of data source, high dimensionality and sparsity are the main characteristics of Big Data. How to mining the heterogeneity and association of different datasets and to achieving dimension reduction is one of the goals and challenges of Big data analysis.Integrative analysis provides an effective way of analyzing Big Data. It simultaneously analyzes multiple datasets, avoiding the model instability from individual variations caused by regional and time factor and so on. The coefficients of each covariate across all datasets are treated as a group and use penalty function to shrinkage these groups of coefficients to achieve variable selection. In this paper, we review the existingresearch of penalizedintegrative analysis from three aspects of homogeneity integrative analysis ， heterogeneity integrative analysis and network integrative analysis. Three simulations are conducted to verify the performance of integrative analysis, including weak, moderate and strong correlations. It shows that LGroup