Duplicate rows in the source

Last Post 01 Jun 2011 12:59 PM by PaulMcKibben. 1 Replies.
AddThis - Bookmarking and Sharing Button Printer Friendly
  •  
  •  
  •  
  •  
  •  
Sort:
PrevPrev NextNext
You are not authorized to post a reply.
Author Messages
PaulMcKibben
New Member
New Member

--
01 Jun 2011 09:05 AM
I have a situation where the source is a file dump from a mainframe, and I have duplicate rows in the source. 

I have loaded it into a work table that has no keys or RI.

When I go to load it into the final table, I only can load a portion of the rows in the work table.  The package is performing like the first time that it hits a duplicate row, it is sending the rest of the rows to my error table.

I ultimately want to load only unique records into the final table.  I have no idea how to though.

Any assistance would be greatly appreciated.

Thank you.
PaulMcKibben
New Member
New Member

--
01 Jun 2011 12:59 PM
I found the answer which is really simple.

After loading the work table, add an Exeute SQL Task with the following code:

INSERT INTO dbo.tbl1
SELECT DISTINCT user_id, access_control_cd, group_id, division_id, product_ind, network_ind
FROM work_db.dbo.work_tbl1
You are not authorized to post a reply.

Acceptable Use Policy