毕业设计（论文）-基于Spark的语义数据行推理策略研究.doc

PAGE \* MERGEFORMAT PAGE \* MERGEFORMAT IV基于Spark的语义数据并行推理策略研究摘 要伴随着网络技术的快速发展，语义Web数据大规模出现。海量语义数据的隐含信息通常需要推理方能得到。语义数据的推理过程极具挑战性。其中的难点在于数据量大、计算耗时长。显然，运行于单机环境下的传统推理工具难以适用于如此大规模的语义数据。并且单节点推理引擎面对日益增长的数据存在计算性能和可扩展性不足等问题。由此，如何实现海量语义数据在云环境下的高效推理已成为当前语义领域的研究热点。论文基于开源并行框架Spark采用RDF/RDFS、OWL、HDFS等语义Web相关技术研究海量语义数据的推理策略，实现海量语义数据的高效推理。针对现有推理系统存在的相关问题，提出了基于RDFS推理规则的语义数据并行推理策略，详细阐述了并行推理过程中RDFS语义推理规则并行执行次序、数据压缩以及去除重复数据三个关键问题的实现策略。主要研究工作及成果如下：研究了RDFS推理规则以及传统的语义推理引擎，通过对比现有的云计算相关技术，引出了基于Spark的语义数据并行推理，设计实现了基于MD5的数据压缩算法以及基于Spark的语义数据并行化RDFS推理算法。该算法通过优化推理执行顺序及减少推理执行步骤，提高了推理执行速度；基于内存的迭代计算框架避免了中间结果的重复读取，减少了冗余计算；基于MD5的数据压缩，节约了数据导入和存储时间并降低了计算节点之间的数据通信开销。论文提出的推理策略解决了现有单机推理策略难以处理的并行问题以及可扩展性问题，并且减少了大量数据导入和存储时间以及大量不必要的数据传输。实现的语义Web推理引擎可以对海量语义数据实现高效的推理响应。关键词：Spark 语义数据 并行推理 RDFS Semantic Data Parallel Reasoning Research with SparkABSTRACT Along with the rapid development of network technology, large-scale semantic Web data have emerged. Without reasoning, huge amounts of semantic data cannot be available. While, the process is extremely challenging. The difficulty lies in the large amount of data and long time consuming in computing. Obviously, traditional reasoning tools used in the single machine do not apply to such large-scale semantic data. What’s more, in the face of surging data, the single node inference engine has many problems such as insufficient computing performance and scalability. Thus, how to realize that the massive semantic data in a cloud environment can be efficiently reasoned has become a current hot topic in the field of semantics. Based on open source parallel framework this thesis uses semantic Web technology such as RDF/RDFS, OWL and HDFS related to study large amounts of semantic data reasoning strategy, to realize high reasoning of massive semantic data. Based on relevant problems of the existing system, this thesis proposes a parallel semantic data reasoning strategy based on RDFS reasoning rule, elaborates RDFS semantic reasoning rules for parallel exe