Cost-effective online trending topic detection and popularity prediction in microblogging

Zhongchen Miao, Kai Chen*, Yi Fang, Jianhua He, Yi Zhou, Wenjun Zhang, Hongyuan Zha

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review


Identifying topic trends on microblogging services such as Twitter and estimating those topics’ future popularity have great academic and business value, especially when the operations can be done in real time. For any third party, however, capturing and processing such huge volumes of real-time data in microblogs are almost infeasible tasks, as there always exist API (Application Program Interface) request limits, monitoring and computing budgets, as well as timeliness requirements. To deal with these challenges, we propose a cost-effective system framework with algorithms that can automatically select a subset of representative users in microblogging networks in offline, under given cost constraints. Then the proposed system can online monitor and utilize only these selected users’ real-time microposts to detect the overall trending topics and predict their future popularity among the whole microblogging network. Therefore, our proposed system framework is practical for real-time usage as it avoids the high cost in capturing and processing full real-time data, while not compromising detection and prediction performance under given cost constraints. Experiments with real microblogs dataset show that by tracking only 500 users out of 0.6 million users and processing no more than 30,000 microposts daily, about 92% trending topics could be detected and predicted by the proposed system and, on average, more than 10 hours earlier than they appear in official trends lists.
Original languageEnglish
Article number18
JournalACM Transactions on Information Systems
Issue number3
Early online date4 Jan 2017
Publication statusPublished - 9 Jun 2017


  • cost
  • microblogging
  • prediction
  • topic detection


Dive into the research topics of 'Cost-effective online trending topic detection and popularity prediction in microblogging'. Together they form a unique fingerprint.

Cite this