Random walk with restart over dynamic graphs

Weiren Yu, Julie McCann

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Random Walk with Restart (RWR) is an appealing measure of proximity between nodes based on graph structures. Since real graphs are often large and subject to minor changes, it is prohibitively expensive to recompute proximities from scratch. Previous methods use LU decomposition and degree reordering heuristics, entailing O(|V|3) time and O(|V|2) memory to compute all (|V|2) pairs of node proximities in a static graph. In this paper, a dynamic scheme to assess RWR proximities is proposed: (1) For unit update, we characterize the changes to all-pairs proximities as the outer product of two vectors. We notice that the multiplication of an RWR matrix and its transition matrix, unlike traditional matrix multiplications, is commutative. This can greatly reduce the computation of all-pairs proximities from O(|V|3) to O(|Δ|) time for each update without loss of accuracy, where |Δ| (<<|V|2) is the number of affected proximities. (2) To avoid O(|V|2) memory for all pairs of outputs, we also devise efficient partitioning techniques for our dynamic model, which can compute all pairs of proximities segment-wisely within O(l|V|) is a user-controlled trade-off between  memory an I/O costs. (3) For bulk update, we also devise aggregation and hashing methods, which can discard many unneccessary updates further and handle chuncks of unit updates simultaneously. Our experimental results on various datasets demonstrate that our methods can be 1-2 orders of magnitude faster than other competitors while securing scalability and exactness...
Original languageEnglish
Title of host publicationProceedings: 16th IEEE International Conference on Data Mining, ICDM 2016
EditorsFrancesco Bonchi, Josep Domingo-Ferrer, Ricardo Baeza-Yates, et al
Place of PublicationPiscataway, NJ (US)
PublisherIEEE
Pages589-598
Number of pages10
ISBN (Electronic)978-1-5090-5473-2
ISBN (Print)978-1-5090-5474-9 , 978-1-5090-5472-5
DOIs
Publication statusPublished - 2017
Event16th IEEE International Conference on Data Mining and Workshops - World Trade Center, Bacelona, Spain
Duration: 12 Dec 201615 Dec 2016

Publication series

NameProceedings. IEEE International Conference on Data Mining
PublisherIEEE
ISSN (Print)2374-8486
ISSN (Electronic)2374-8486

Conference

Conference16th IEEE International Conference on Data Mining and Workshops
Abbreviated titleICDM &amp; ICDMW 2016
CountrySpain
CityBacelona
Period12/12/1615/12/16

Fingerprint

Data storage equipment
Scalability
Dynamic models
Decomposition
Costs

Bibliographical note

© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

Keywords

  • time measurement
  • loss measurement
  • Symmetric matrices
  • noise measurement
  • computational modeling
  • matrix decomposition
  • matrix converters

Cite this

Yu, W., & McCann, J. (2017). Random walk with restart over dynamic graphs. In F. Bonchi, J. Domingo-Ferrer, R. Baeza-Yates, & et al (Eds.), Proceedings: 16th IEEE International Conference on Data Mining, ICDM 2016 (pp. 589-598). (Proceedings. IEEE International Conference on Data Mining). Piscataway, NJ (US): IEEE. https://doi.org/10.1109/ICDM.2016.0070
Yu, Weiren ; McCann, Julie. / Random walk with restart over dynamic graphs. Proceedings: 16th IEEE International Conference on Data Mining, ICDM 2016. editor / Francesco Bonchi ; Josep Domingo-Ferrer ; Ricardo Baeza-Yates ; et al. Piscataway, NJ (US) : IEEE, 2017. pp. 589-598 (Proceedings. IEEE International Conference on Data Mining).
@inproceedings{d7d8106f92754556be4be0ea06810992,
title = "Random walk with restart over dynamic graphs",
abstract = "Random Walk with Restart (RWR) is an appealing measure of proximity between nodes based on graph structures. Since real graphs are often large and subject to minor changes, it is prohibitively expensive to recompute proximities from scratch. Previous methods use LU decomposition and degree reordering heuristics, entailing O(|V|3) time and O(|V|2) memory to compute all (|V|2) pairs of node proximities in a static graph. In this paper, a dynamic scheme to assess RWR proximities is proposed: (1) For unit update, we characterize the changes to all-pairs proximities as the outer product of two vectors. We notice that the multiplication of an RWR matrix and its transition matrix, unlike traditional matrix multiplications, is commutative. This can greatly reduce the computation of all-pairs proximities from O(|V|3) to O(|Δ|) time for each update without loss of accuracy, where |Δ| (<<|V|2) is the number of affected proximities. (2) To avoid O(|V|2) memory for all pairs of outputs, we also devise efficient partitioning techniques for our dynamic model, which can compute all pairs of proximities segment-wisely within O(l|V|) is a user-controlled trade-off between  memory an I/O costs. (3) For bulk update, we also devise aggregation and hashing methods, which can discard many unneccessary updates further and handle chuncks of unit updates simultaneously. Our experimental results on various datasets demonstrate that our methods can be 1-2 orders of magnitude faster than other competitors while securing scalability and exactness...",
keywords = "time measurement, loss measurement, Symmetric matrices, noise measurement, computational modeling , matrix decomposition, matrix converters",
author = "Weiren Yu and Julie McCann",
note = "{\circledC} 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.",
year = "2017",
doi = "10.1109/ICDM.2016.0070",
language = "English",
isbn = "978-1-5090-5474-9",
series = "Proceedings. IEEE International Conference on Data Mining",
publisher = "IEEE",
pages = "589--598",
editor = "Francesco Bonchi and Josep Domingo-Ferrer and Ricardo Baeza-Yates and {et al}",
booktitle = "Proceedings: 16th IEEE International Conference on Data Mining, ICDM 2016",
address = "United States",

}

Yu, W & McCann, J 2017, Random walk with restart over dynamic graphs. in F Bonchi, J Domingo-Ferrer, R Baeza-Yates & et al (eds), Proceedings: 16th IEEE International Conference on Data Mining, ICDM 2016. Proceedings. IEEE International Conference on Data Mining, IEEE, Piscataway, NJ (US), pp. 589-598, 16th IEEE International Conference on Data Mining and Workshops, Bacelona, Spain, 12/12/16. https://doi.org/10.1109/ICDM.2016.0070

Random walk with restart over dynamic graphs. / Yu, Weiren; McCann, Julie.

Proceedings: 16th IEEE International Conference on Data Mining, ICDM 2016. ed. / Francesco Bonchi; Josep Domingo-Ferrer; Ricardo Baeza-Yates; et al. Piscataway, NJ (US) : IEEE, 2017. p. 589-598 (Proceedings. IEEE International Conference on Data Mining).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - Random walk with restart over dynamic graphs

AU - Yu, Weiren

AU - McCann, Julie

N1 - © 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.

PY - 2017

Y1 - 2017

N2 - Random Walk with Restart (RWR) is an appealing measure of proximity between nodes based on graph structures. Since real graphs are often large and subject to minor changes, it is prohibitively expensive to recompute proximities from scratch. Previous methods use LU decomposition and degree reordering heuristics, entailing O(|V|3) time and O(|V|2) memory to compute all (|V|2) pairs of node proximities in a static graph. In this paper, a dynamic scheme to assess RWR proximities is proposed: (1) For unit update, we characterize the changes to all-pairs proximities as the outer product of two vectors. We notice that the multiplication of an RWR matrix and its transition matrix, unlike traditional matrix multiplications, is commutative. This can greatly reduce the computation of all-pairs proximities from O(|V|3) to O(|Δ|) time for each update without loss of accuracy, where |Δ| (<<|V|2) is the number of affected proximities. (2) To avoid O(|V|2) memory for all pairs of outputs, we also devise efficient partitioning techniques for our dynamic model, which can compute all pairs of proximities segment-wisely within O(l|V|) is a user-controlled trade-off between  memory an I/O costs. (3) For bulk update, we also devise aggregation and hashing methods, which can discard many unneccessary updates further and handle chuncks of unit updates simultaneously. Our experimental results on various datasets demonstrate that our methods can be 1-2 orders of magnitude faster than other competitors while securing scalability and exactness...

AB - Random Walk with Restart (RWR) is an appealing measure of proximity between nodes based on graph structures. Since real graphs are often large and subject to minor changes, it is prohibitively expensive to recompute proximities from scratch. Previous methods use LU decomposition and degree reordering heuristics, entailing O(|V|3) time and O(|V|2) memory to compute all (|V|2) pairs of node proximities in a static graph. In this paper, a dynamic scheme to assess RWR proximities is proposed: (1) For unit update, we characterize the changes to all-pairs proximities as the outer product of two vectors. We notice that the multiplication of an RWR matrix and its transition matrix, unlike traditional matrix multiplications, is commutative. This can greatly reduce the computation of all-pairs proximities from O(|V|3) to O(|Δ|) time for each update without loss of accuracy, where |Δ| (<<|V|2) is the number of affected proximities. (2) To avoid O(|V|2) memory for all pairs of outputs, we also devise efficient partitioning techniques for our dynamic model, which can compute all pairs of proximities segment-wisely within O(l|V|) is a user-controlled trade-off between  memory an I/O costs. (3) For bulk update, we also devise aggregation and hashing methods, which can discard many unneccessary updates further and handle chuncks of unit updates simultaneously. Our experimental results on various datasets demonstrate that our methods can be 1-2 orders of magnitude faster than other competitors while securing scalability and exactness...

KW - time measurement

KW - loss measurement

KW - Symmetric matrices

KW - noise measurement

KW - computational modeling

KW - matrix decomposition

KW - matrix converters

UR - http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=7837883

UR - http://www.scopus.com/inward/record.url?scp=85014580445&partnerID=8YFLogxK

U2 - 10.1109/ICDM.2016.0070

DO - 10.1109/ICDM.2016.0070

M3 - Conference contribution

AN - SCOPUS:85014580445

SN - 978-1-5090-5474-9

SN - 978-1-5090-5472-5

T3 - Proceedings. IEEE International Conference on Data Mining

SP - 589

EP - 598

BT - Proceedings: 16th IEEE International Conference on Data Mining, ICDM 2016

A2 - Bonchi, Francesco

A2 - Domingo-Ferrer, Josep

A2 - Baeza-Yates, Ricardo

A2 - et al,

PB - IEEE

CY - Piscataway, NJ (US)

ER -

Yu W, McCann J. Random walk with restart over dynamic graphs. In Bonchi F, Domingo-Ferrer J, Baeza-Yates R, et al, editors, Proceedings: 16th IEEE International Conference on Data Mining, ICDM 2016. Piscataway, NJ (US): IEEE. 2017. p. 589-598. (Proceedings. IEEE International Conference on Data Mining). https://doi.org/10.1109/ICDM.2016.0070