True real-time change data capture with web service database encapsulation

Mitchell J. Eccles, David J. Evans, Anthony J. Beaumont

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

This research is investigating the claim that Change Data Capture (CDC) technologies capture data changes in real-time. Based on theory, our hypothesis states that real-time CDC is not achievable with traditional approaches (log scanning, triggers and timestamps). Traditional approaches to CDC require a resource to be polled, which prevents true real-time CDC. We propose an approach to CDC that encapsulates the data source with a set of web services. These web services will propagate the changes to the targets and eliminate the need for polling. Additionally we propose a framework for CDC technologies that allow changes to flow from source to target. This paper discusses current CDC technologies and presents the theory about why they are unable to deliver changes in real-time. Following, we discuss our web service approach to CDC and accompanying framework, explaining how they can produce real-time CDC. The paper concludes with a discussion on the research required to investigate the real-time capabilities of CDC technologies.

Original languageEnglish
Title of host publicationProceedings : 2010 6th World Congress on Services, Services-1
Place of PublicationPiscataway, NJ (US)
PublisherIEEE
Pages128-131
Number of pages4
ISBN (Print)978-0-7695-4129-7
DOIs
Publication statusPublished - 2010
Event2010 6th World Congress on Services - Miami, FL, United States
Duration: 5 Jul 201010 Jul 2010

Congress

Congress2010 6th World Congress on Services
Abbreviated titleServices-1
CountryUnited States
CityMiami, FL
Period5/07/1010/07/10

Fingerprint

Encapsulation
Web services
Data acquisition
Data base
Scanning

Keywords

  • change data capture
  • log scanning
  • real-time
  • timestamps
  • triggers
  • web services

Cite this

Eccles, M. J., Evans, D. J., & Beaumont, A. J. (2010). True real-time change data capture with web service database encapsulation. In Proceedings : 2010 6th World Congress on Services, Services-1 (pp. 128-131). Piscataway, NJ (US): IEEE. https://doi.org/10.1109/SERVICES.2010.59
Eccles, Mitchell J. ; Evans, David J. ; Beaumont, Anthony J. / True real-time change data capture with web service database encapsulation. Proceedings : 2010 6th World Congress on Services, Services-1. Piscataway, NJ (US) : IEEE, 2010. pp. 128-131
@inproceedings{71356f6619d449968b2c3997be9508b0,
title = "True real-time change data capture with web service database encapsulation",
abstract = "This research is investigating the claim that Change Data Capture (CDC) technologies capture data changes in real-time. Based on theory, our hypothesis states that real-time CDC is not achievable with traditional approaches (log scanning, triggers and timestamps). Traditional approaches to CDC require a resource to be polled, which prevents true real-time CDC. We propose an approach to CDC that encapsulates the data source with a set of web services. These web services will propagate the changes to the targets and eliminate the need for polling. Additionally we propose a framework for CDC technologies that allow changes to flow from source to target. This paper discusses current CDC technologies and presents the theory about why they are unable to deliver changes in real-time. Following, we discuss our web service approach to CDC and accompanying framework, explaining how they can produce real-time CDC. The paper concludes with a discussion on the research required to investigate the real-time capabilities of CDC technologies.",
keywords = "change data capture, log scanning, real-time, timestamps, triggers, web services",
author = "Eccles, {Mitchell J.} and Evans, {David J.} and Beaumont, {Anthony J.}",
year = "2010",
doi = "10.1109/SERVICES.2010.59",
language = "English",
isbn = "978-0-7695-4129-7",
pages = "128--131",
booktitle = "Proceedings : 2010 6th World Congress on Services, Services-1",
publisher = "IEEE",
address = "United States",

}

Eccles, MJ, Evans, DJ & Beaumont, AJ 2010, True real-time change data capture with web service database encapsulation. in Proceedings : 2010 6th World Congress on Services, Services-1. IEEE, Piscataway, NJ (US), pp. 128-131, 2010 6th World Congress on Services, Miami, FL, United States, 5/07/10. https://doi.org/10.1109/SERVICES.2010.59

True real-time change data capture with web service database encapsulation. / Eccles, Mitchell J.; Evans, David J.; Beaumont, Anthony J.

Proceedings : 2010 6th World Congress on Services, Services-1. Piscataway, NJ (US) : IEEE, 2010. p. 128-131.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - True real-time change data capture with web service database encapsulation

AU - Eccles, Mitchell J.

AU - Evans, David J.

AU - Beaumont, Anthony J.

PY - 2010

Y1 - 2010

N2 - This research is investigating the claim that Change Data Capture (CDC) technologies capture data changes in real-time. Based on theory, our hypothesis states that real-time CDC is not achievable with traditional approaches (log scanning, triggers and timestamps). Traditional approaches to CDC require a resource to be polled, which prevents true real-time CDC. We propose an approach to CDC that encapsulates the data source with a set of web services. These web services will propagate the changes to the targets and eliminate the need for polling. Additionally we propose a framework for CDC technologies that allow changes to flow from source to target. This paper discusses current CDC technologies and presents the theory about why they are unable to deliver changes in real-time. Following, we discuss our web service approach to CDC and accompanying framework, explaining how they can produce real-time CDC. The paper concludes with a discussion on the research required to investigate the real-time capabilities of CDC technologies.

AB - This research is investigating the claim that Change Data Capture (CDC) technologies capture data changes in real-time. Based on theory, our hypothesis states that real-time CDC is not achievable with traditional approaches (log scanning, triggers and timestamps). Traditional approaches to CDC require a resource to be polled, which prevents true real-time CDC. We propose an approach to CDC that encapsulates the data source with a set of web services. These web services will propagate the changes to the targets and eliminate the need for polling. Additionally we propose a framework for CDC technologies that allow changes to flow from source to target. This paper discusses current CDC technologies and presents the theory about why they are unable to deliver changes in real-time. Following, we discuss our web service approach to CDC and accompanying framework, explaining how they can produce real-time CDC. The paper concludes with a discussion on the research required to investigate the real-time capabilities of CDC technologies.

KW - change data capture

KW - log scanning

KW - real-time

KW - timestamps

KW - triggers

KW - web services

UR - http://www.scopus.com/inward/record.url?scp=78049338304&partnerID=8YFLogxK

UR - http://ieeexplore.ieee.org/document/5575585/

U2 - 10.1109/SERVICES.2010.59

DO - 10.1109/SERVICES.2010.59

M3 - Conference contribution

AN - SCOPUS:78049338304

SN - 978-0-7695-4129-7

SP - 128

EP - 131

BT - Proceedings : 2010 6th World Congress on Services, Services-1

PB - IEEE

CY - Piscataway, NJ (US)

ER -

Eccles MJ, Evans DJ, Beaumont AJ. True real-time change data capture with web service database encapsulation. In Proceedings : 2010 6th World Congress on Services, Services-1. Piscataway, NJ (US): IEEE. 2010. p. 128-131 https://doi.org/10.1109/SERVICES.2010.59