基于网络爬虫的城市住房租金可视化分析-毕业论文.doc

基于网络爬虫的城市住房租金可视化分析 毕业论文中文摘要 基于网络爬虫的城市住房租金可视化分析 摘 要： 本文将选取北京这个城市，利用Python语言用网络爬虫的技术获取北京贝壳网上的租房区域的租赁信息，包括城区、地点、租赁方式、面积、户型、房屋朝向与价格，通过数据挖掘中的决策树模型分析出北京的住房租金受到哪些因素的影响，然后基于影响因子和房屋租金的关系对北京租房相关信息进行可视化分析。在可视化网站的总览页面中，对北京地图上的所有城区根据其均价使用深浅程度不同的红色进行区分，展示了不同城区的租房均价以及它们的排名和差异，在同一城区的页面上展示了不同小区租房均价及它们的排名和对比效果、整租与合租房均价的对比效果、各种户型的占比等等，为在北京需要短期或长期租房的人们提供价格参考，具有现实的租房指导意义。 关键词：北京市；城区；住房租金；网络爬虫；可视化 毕业论文外文摘要 Title Reptile?CityVisualization Analysis of Urban Housing Rent Based on Web Crawler Abstract This article select the city of Beijing as study object, use Python language and the technology of web crawlers to obtain information of different rental area in Beijing from shell network, including urban areas, location, leasing mode, area, household type, housing orientation and price, analyze the factors affecting housing rents in Beijing through the Decision Tree Algorithm of Data Mining, and then display the relationship between these impact factors and housing rents on the website. On the overview, all the urban areas are marked with varying degrees of red color according to their average price. The website also shows the average price of rented houses in different urban areas, as well as their rankings and comparisons. The pages in different urban areas show the average price of rented houses in different districts, their rankings and comparisons,. the proportion of various types of houses, etc.. Some guidance can be given to those who want to rent a house in Beijing for short or long term, which brings some practical significance. Keywords: Beijing; urban areas; housing rent; web crawler; visualization 目 录 TOC \o 1-3 \h \z \u 9931 1绪论 1 196 1.1 研究的背景及意义 1 2988 1.2 国内研究现状 1 20775 1.3 研究方法介绍 2 31759 2 网络爬虫的介绍 3 16255 2.1爬虫的工作原理 3 17958 2.2 网络爬虫常用的框架 3 9981 2.2.1 Nutch爬虫框架 3 12032 2.2.2 Crawler4j爬虫框架 3 29568 2.2.3 Scrapy爬虫框架 3 32377 2.3 BeautifulSoup库的介绍 3 8753 3 数据的获取 5 11601 3.1 爬虫的流程 5 7310 3.2多线程的设计 5 686 3.3数据的解析 5 17335 3.4爬取的数据存储结构 6 1