Random Articles
Reseach Article
Change Data Capture on OLTP Staging Area for Nearly Real Time Data Warehouse Base on Database Trigger
| International Journal of Computer Applications |
| Foundation of Computer Science (FCS), NY, USA |
| Volume 52 - Number 11 |
| Year of Publication: 2012 |
| Authors: I Made Sukarsa, Ni Wayan Wisswani, I K. Gd. Darma Putra, Linawati |
10.5120/8248-1762
|
I Made Sukarsa, Ni Wayan Wisswani, I K. Gd. Darma Putra, Linawati . Change Data Capture on OLTP Staging Area for Nearly Real Time Data Warehouse Base on Database Trigger. International Journal of Computer Applications. 52, 11 ( August 2012), 32-37. DOI=10.5120/8248-1762
Abstract
A conventional data warehouse use to produce summary from an organization information system in a long time period. This condition will make the management unable to get the most up to date data every time it needed. Therefore a nearly real time data warehouse which will manage the ETL process with a more compact data and a shorter period is needed. The design of nearly real time data warehouse in this research is implemented in two steps. The first step is done by data collection technique modeling to make a more compact ETL data managed. This step is done by putting the staging area on an Online Transactional Processing (OLTP). It can minimize the failure of data movement process from the OLTP to the staging area. Besides that, the CDC method is also had applied on the OLTP. This method will be implemented with a trigger active database. The trigger will capture of the data changing on the OLTP, transform it and then load it to the staging area in one time. The second step is the synchronization process of the data movement from the staging area to the nearly real time data warehouse. This process is done by mapping the movement which is ran by the SQL Yog. The mapping result will accomplished by the windows task scheduler
References
- Robert M. Bruckner, Beate List, and Josef Schiefer, Striving towards Near Real-Time Data Integration for Data Warehouses , Data Warehousing and Knowledge Discovery Lecture Notes in Computer Science, 2002, Volume 2454/2002, 173-182, DOI: 10. 1007/3-540-46145-0_31
- Javed, Dr. Muhammad Younus. , Nawaz, Asim. ,2010. Data Load Distribution by Semi Real Time Data Warehouse, In: Computer and Network Technology (ICCNT), 2010 Second International Conference On page(s): 556 - 560
- Inmon, W. H. 2005. Building The Data Warehouse Fourth Edition. Canada : Wiley Publishing. Inc.
- Simitsis, A. ; Vassiliadis, P. ; Sellis, T. ;, Optimizing ETL Processes in Data Warehouses. In Data Engineering, 2005. ICDE 2005. Proceedings. 21st International Conference on Digital Object, Page(s): 564 – 575
- Vandermay, John. , 2001. Considerations for Building a Real-time Data Warehouse
- Savitri, F. N. , Laksmiwati, H. ,Study of localized data cleansing process for ETL performance improvement in independent datamart, Electrical Engineering and Informatics (ICEEI), 2011 International Conference on, [diunduh : 13 Agustus 2011]
- Langseth ,Justin. , 2004, Real-Time Data Warehousing: Challenges and Solutions.
- Jie Song; Yubin Bao; Jingang Shi; 2010, A Triggering and Scheduling Approach for ETL . Computer and Information Technology (CIT), 2010 IEEE 10th International Conference on , Page(s): 91 – 98.
- R. Kimball and J. Caserta, The Data Warehouse ETL Toolkit: Practical Techniques for Extracting, Cleanin. John Wiley & Sons, 2004.
- Mitchell J Eccles, David J Evans and Anthony J Beaumont, True Real-Time Change Data Capture WithWeb Service Database Encapsulation, 2010, 2010 IEEE 6th World Congress on Services
- Attunity Ltd , 2009, Efficient and Real Time Data Integration With Change Data Capture, Tersedia di http://www. attunity. com/cdc_for_etl
- Jingang Shi, Yubin Bao, Fangling Leng, Ge Yu. 2008,Study on Log-Based Change Data Capture and Handling Mechanism in Real-Time Data Warehouse. In International Conference on Computer Science and Software Engineering, CSSE 2008, Volume 4: Embedded Programming / Database Technology / Neural Networks and Applications / Other Applications, December 12-14, 2008, Wuhan, China. pages 478-481, IEEE Computer Society, 2008.
- Liu Jun; Hu ChaoJu; Yuan HeJin. 2010. Application of Web Services on The Real-time Data Warehouse Technology, Advances in Energy Engineering (ICAEE), 2010 International Conference on , Page(s): 335 – 338
Index Terms
Keywords