Showing posts with label step. Show all posts
Showing posts with label step. Show all posts

Thursday, March 8, 2012

Dataflow slows dramatically down after passing 1.6 million records ?

If i have a dataflow witch moves records from one OLE DB Source to a OLE DB Destination, the step slow down when getting above passing 1.5 - 1.6 million records.The Source is a SQL 2000 and the Destination is SQL Server 2005. the Package is run from a laptop with 1.5 GB ram

What can i try to help it ?Check the perf counters to see if the package starts swapping when the slowdown occurs. If this is the case then there is not much to be done. You could try adding more memory or making the destination run faster (i.e. drop indicies, faster spindle, etc). Is the destination on the same PC as the package is running? If so then they are competing for memory so you could run the package on a different PC. Also, the next CTP has a minor change that you may find helps with this.

Thanks,|||Start by checking in Task Manager how much memory the package (i.e. dtexec.exe) uses - does it grow all the time?
And when it slows down, are you out of memory + swapping starts + CPU usage goes down?
If this is the case - do you expect memory usage to grow like this, given your data flow? Which component could be gobbling up the memory? Can you change its settings to improve the situation? Or change the data flow to not use the troublesome component?
Or is there a memory leak that creates a problem?
If the bottleneck isn't memory at all, you obviously have to look in other places...

Cheers,
KDog|||

Matt David wrote:

Check the perf counters to see if the package starts swapping when the slowdown occurs.

How do you check the perf counters?
Regards,
Henk|||Use the Windows Performance Monitor (Administrative Tools-->Performance). In there, there is an IS object containing various counters that you can use to monitor performance.

There is a help file provided that explains how to use Windows Performance Monitor.

-Jamie

Wednesday, March 7, 2012

Data-Driven Subscription - Empty Field List on Step 4

I am attempting to create a data-driven subscription. I step through the
wizard, paste in the query, validate and click next. When I get to the screen
to map the fields from the database table to the fields in the subscription
(to, cc, reply-to, etc).
There are no fields in the drop-down. I have tried numerous different
queries, that I know work when I run them in a report, but don't seem to want
to populate the field list. The source database is Teradata, if that makes a
difference.
There is no error, and I can't seem to find any log entries, so I am not
sure where to turn. Any Ideas?
SteveUPDATED: I have confirmed that, in fact the data-driven subscription will
pull fields from SQL Server, so the issue seems to be with the Teradata
connection.
Any help?
"Steve Muise" wrote:
> I am attempting to create a data-driven subscription. I step through the
> wizard, paste in the query, validate and click next. When I get to the screen
> to map the fields from the database table to the fields in the subscription
> (to, cc, reply-to, etc).
> There are no fields in the drop-down. I have tried numerous different
> queries, that I know work when I run them in a report, but don't seem to want
> to populate the field list. The source database is Teradata, if that makes a
> difference.
> There is no error, and I can't seem to find any log entries, so I am not
> sure where to turn. Any Ideas?
> Steve