基于网络爬虫的实时交通数据自动采集.pdfVIP

Advances in Geosciences 地球科学前沿, 2016, 6(3), 169-179 Automatic Acquisition of Real-Time Traffic Date Based on Web Crawlers * Wenhao Yan, Yuqin Shu , Zhiqin Huang School of Geography, South China Normal University, Guangzhou Guangdong th th th Received: May 24 , 2016; accepted: Jun. 17 , 2016; published: Jun. 20 , 2016 Copyright © 2016 by authors and Hans Publishers Inc. This work is licensed under the Creative Commons Attribution International License (CC BY). Abstract Real-time traffic data is an essential data source to research fields such as intelligent city, low- carbon city, optimum of road net and so on. But the real-time traffic data is not available for free and public. In this paper, the Python programming language and the Tornado Net Framework are used to design a stable, efficient and timely web crawler program which could grab 1723 roads’ traffic data of Guangzhou city, including its basic attributes and real-time traffic data every 5 mi- nutes for each road, from four-dimensional traffic index page and save the data in the local MySQL database at the same time. The result shows that the web crawl technology is feasible and efficient in acquiring real-time traffic data. Keywords Web Crawler, Automatic Acquisition, Real-Time Traffic Data, Python Programming Language, Tornado Net Framework 基于网络爬虫的实时交通数据自动采集 * 闫文豪，舒娱琴，黄植钦 华南师范大学地理科学学院，广东 广州 收稿日期：2016年5月24 日；录用日期：2016年6月17 日；发布日期：2016年6月20 日 *通讯作者。 文章引用: 闫文豪, 舒娱琴, 黄植钦. 基于网络爬虫的实时交通数据自动采集[J]. 地球科学前沿, 2016, 6(3): 169-179. /10.12677/ag.2016.63020 闫文豪 等 摘 要 实时交通数据是低碳出行、智能交通、道路网络优化等研究领域必不可少的数据源。为了解决目前存在