基于微博数据的用户兴趣挖掘概要1.pptVIP

* * Given the size of the dataset, these estimates are reasonable reconstructions of the parameters used to generate the data. * Through analysis of the Microblog data, we find several problems/challenges that affect the accuracy of the result. The most significant ones are, short text, noise data, and the changes of concern. * Based on the consideration, we build the framework of the experiment. It contains 6 modules. Use Sina API to get … Use ensemble ML to train a model to identify the noise microblogs Use an Opensorce LDA tool and Wikipedia to get the topic model Use the topic model and Microblogs to build the user interest Transform the docs into topic distribution Compare and recommendation * The following is the details of each module. We use an ensemnble ML approach which combine the … together. The features used include… * In order to tackle the interest change issue over time, we design a representation model by introducing a weight companying the probability of each topic. 2. P (k) indicates the users level of interest in subject k, which is obtained from all the historical microBlogs the user published. Intuitively, more the content of the topic k a user published, the greater P (k) will be calculated. 3. If a year ago the user published a lot of microblogs about topic k, even if the user is no longer interested in Subjects k, P (k) would still be large. 4. Hence, wt(k) can be used to reflect the increase/decrease of concern on topic k, according to the time of the microblogs. 5. The longer the user discuss on topic k ago, the smaller wt (k) will be. * In our model, we first train a LDA model on external repository, Wikipedia, using standard LDA algorithm. Using external repository to train LDA model has several advantages. Firstly, external repository can overcome the shortcoming that each post is too short to catch its deep semantic information. Secondly, if we train LDA model on external repository, Wikipedia, we only need to determine the nu