An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation

Miguel Jimenez, Luis F. Rivera, Norha M. Villegas, Gabriel Tamura, Hausi A. Muller, Nelly Bencomo

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Continuous experimentation enables companies to reduce development risks and operational costs by continuously and directly assessing user response with respect to software updates. The increasing need for data-driven rapid decisions to face unpredictable context situations demands the automation of continuous experimentation practices. Furthermore, variable conditions and constraints associated with the experimentation process, such as changes in the experimentation goals and the cost of conducting experimental trials, demand from experiments to be adaptive. This paper presents our proposal towards what we call quality-driven adaptive continuous experimentation. Our contributions are as follows. First, we present a metamodel for experimental design to enable automatic planning and execution of experiments at run-time. Second, we propose a mesh of run-time models to allow autonomic managers conduct experiments while assisting in the continuous evolution of the subject system. Finally, we propose an architecture for quality-driven adaptive experimentation. Our architecture addresses separation of concerns in the experimentation process by dedicating three feedback loops to (1) control the satisfaction of high-level experimentation goals through experimental design; (2) conduct experimental trials for infrastructure configuration variants; and (3) conduct experimental trials for architectural design variants.
Original languageEnglish
Title of host publicationProceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019
PublisherIEEE
Pages20-23
Number of pages4
ISBN (Electronic)978-1-7281-2247-2
ISBN (Print)978-1-7281-2248-9
DOIs
Publication statusE-pub ahead of print - 29 Aug 2019
Event2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution (RCoSE/DDrEE) - Montreal, QC, Canada
Duration: 27 May 201927 May 2019

Conference

Conference2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution (RCoSE/DDrEE)
Period27/05/1927/05/19

Fingerprint

Design of experiments
Architectural design
Experiments
Costs
Managers
Automation
Feedback
Planning
Industry

Bibliographical note

Funding: National Sciences and Engineering Research Council (NSERC) of Canada, IBM Canada Ltd. and IBM Advanced Studies (CAS), the University of Victoria (Canada), and Universidad Icesi (Colombia).

Keywords

  • Autonomic Computing
  • Continuous Experimentation
  • Models at Run time
  • Software Evolution

Cite this

Jimenez, M., Rivera, L. F., Villegas, N. M., Tamura, G., Muller, H. A., & Bencomo, N. (2019). An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation. In Proceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019 (pp. 20-23). [8818213] IEEE. https://doi.org/10.1109/RCoSE/DDrEE.2019.00012
Jimenez, Miguel ; Rivera, Luis F. ; Villegas, Norha M. ; Tamura, Gabriel ; Muller, Hausi A. ; Bencomo, Nelly. / An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation. Proceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019. IEEE, 2019. pp. 20-23
@inproceedings{63dbb0dea6b64fa39ee62026b1fe09dc,
title = "An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation",
abstract = "Continuous experimentation enables companies to reduce development risks and operational costs by continuously and directly assessing user response with respect to software updates. The increasing need for data-driven rapid decisions to face unpredictable context situations demands the automation of continuous experimentation practices. Furthermore, variable conditions and constraints associated with the experimentation process, such as changes in the experimentation goals and the cost of conducting experimental trials, demand from experiments to be adaptive. This paper presents our proposal towards what we call quality-driven adaptive continuous experimentation. Our contributions are as follows. First, we present a metamodel for experimental design to enable automatic planning and execution of experiments at run-time. Second, we propose a mesh of run-time models to allow autonomic managers conduct experiments while assisting in the continuous evolution of the subject system. Finally, we propose an architecture for quality-driven adaptive experimentation. Our architecture addresses separation of concerns in the experimentation process by dedicating three feedback loops to (1) control the satisfaction of high-level experimentation goals through experimental design; (2) conduct experimental trials for infrastructure configuration variants; and (3) conduct experimental trials for architectural design variants.",
keywords = "Autonomic Computing, Continuous Experimentation, Models at Run time, Software Evolution",
author = "Miguel Jimenez and Rivera, {Luis F.} and Villegas, {Norha M.} and Gabriel Tamura and Muller, {Hausi A.} and Nelly Bencomo",
note = "Funding: National Sciences and Engineering Research Council (NSERC) of Canada, IBM Canada Ltd. and IBM Advanced Studies (CAS), the University of Victoria (Canada), and Universidad Icesi (Colombia).",
year = "2019",
month = "8",
day = "29",
doi = "10.1109/RCoSE/DDrEE.2019.00012",
language = "English",
isbn = "978-1-7281-2248-9",
pages = "20--23",
booktitle = "Proceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019",
publisher = "IEEE",
address = "United States",

}

Jimenez, M, Rivera, LF, Villegas, NM, Tamura, G, Muller, HA & Bencomo, N 2019, An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation. in Proceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019., 8818213, IEEE, pp. 20-23, 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution (RCoSE/DDrEE), 27/05/19. https://doi.org/10.1109/RCoSE/DDrEE.2019.00012

An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation. / Jimenez, Miguel; Rivera, Luis F.; Villegas, Norha M.; Tamura, Gabriel; Muller, Hausi A.; Bencomo, Nelly.

Proceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019. IEEE, 2019. p. 20-23 8818213.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

TY - GEN

T1 - An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation

AU - Jimenez, Miguel

AU - Rivera, Luis F.

AU - Villegas, Norha M.

AU - Tamura, Gabriel

AU - Muller, Hausi A.

AU - Bencomo, Nelly

N1 - Funding: National Sciences and Engineering Research Council (NSERC) of Canada, IBM Canada Ltd. and IBM Advanced Studies (CAS), the University of Victoria (Canada), and Universidad Icesi (Colombia).

PY - 2019/8/29

Y1 - 2019/8/29

N2 - Continuous experimentation enables companies to reduce development risks and operational costs by continuously and directly assessing user response with respect to software updates. The increasing need for data-driven rapid decisions to face unpredictable context situations demands the automation of continuous experimentation practices. Furthermore, variable conditions and constraints associated with the experimentation process, such as changes in the experimentation goals and the cost of conducting experimental trials, demand from experiments to be adaptive. This paper presents our proposal towards what we call quality-driven adaptive continuous experimentation. Our contributions are as follows. First, we present a metamodel for experimental design to enable automatic planning and execution of experiments at run-time. Second, we propose a mesh of run-time models to allow autonomic managers conduct experiments while assisting in the continuous evolution of the subject system. Finally, we propose an architecture for quality-driven adaptive experimentation. Our architecture addresses separation of concerns in the experimentation process by dedicating three feedback loops to (1) control the satisfaction of high-level experimentation goals through experimental design; (2) conduct experimental trials for infrastructure configuration variants; and (3) conduct experimental trials for architectural design variants.

AB - Continuous experimentation enables companies to reduce development risks and operational costs by continuously and directly assessing user response with respect to software updates. The increasing need for data-driven rapid decisions to face unpredictable context situations demands the automation of continuous experimentation practices. Furthermore, variable conditions and constraints associated with the experimentation process, such as changes in the experimentation goals and the cost of conducting experimental trials, demand from experiments to be adaptive. This paper presents our proposal towards what we call quality-driven adaptive continuous experimentation. Our contributions are as follows. First, we present a metamodel for experimental design to enable automatic planning and execution of experiments at run-time. Second, we propose a mesh of run-time models to allow autonomic managers conduct experiments while assisting in the continuous evolution of the subject system. Finally, we propose an architecture for quality-driven adaptive experimentation. Our architecture addresses separation of concerns in the experimentation process by dedicating three feedback loops to (1) control the satisfaction of high-level experimentation goals through experimental design; (2) conduct experimental trials for infrastructure configuration variants; and (3) conduct experimental trials for architectural design variants.

KW - Autonomic Computing

KW - Continuous Experimentation

KW - Models at Run time

KW - Software Evolution

UR - https://ieeexplore.ieee.org/document/8818213/

UR - http://www.scopus.com/inward/record.url?scp=85072777057&partnerID=8YFLogxK

U2 - 10.1109/RCoSE/DDrEE.2019.00012

DO - 10.1109/RCoSE/DDrEE.2019.00012

M3 - Conference contribution

SN - 978-1-7281-2248-9

SP - 20

EP - 23

BT - Proceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019

PB - IEEE

ER -

Jimenez M, Rivera LF, Villegas NM, Tamura G, Muller HA, Bencomo N. An Architectural Framework for Quality-Driven Adaptive Continuous Experimentation. In Proceedings - 2019 IEEE/ACM Joint 4th International Workshop on Rapid Continuous Software Engineering and 1st International Workshop on Data-Driven Decisions, Experimentation and Evolution, RCoSE/DDrEE 2019. IEEE. 2019. p. 20-23. 8818213 https://doi.org/10.1109/RCoSE/DDrEE.2019.00012