Storing Processed/transformed data

Last Post 17 Oct 2012 11:41 AM by Lee. 2 Replies.
AddThis - Bookmarking and Sharing Button
Author Messages
Liam32
New Member
New Member

--
28 Sep 2012 07:25 AM
Hi all i have a database which contains raw unprocessed data. This data is processed by an etl type tool where the results will be acccessed by downstream systems that are meant access it. If anyone has had experience with this before have they stored the data back in the source database in custom tables or moved it to somthing like a data warehouse. Gven that a single raw piece of data can be transformed by several algorithms storing (therefore on raw result can have multiple processed results) It would also mean that if the raw data changes then the processed results would require recalculation.Any information would be appreciated

Thanks in advance
rm
New Member
New Member

--
29 Sep 2012 05:19 PM
You can create staging tables to hold those data, etl processes them and save results to target tables.
Lee
New Member
New Member

--
17 Oct 2012 11:41 AM
Agreed just apply the appropriate retry logic if necessary so that rerunning the process doesn't bomb out the stage process

Lee Everest


Acceptable Use Policy
---