Episode 144
| August 29, 2018
| 44 minutes
Is SSIS still relevant? Part 1
Moving data with SSIS is a forgone conclusion for most of us — it is part of SQL Server, it has a interface we are somewhat familiar with, and connects to lots of common data sources like csv, excel, and of course SQL Server. Are our needs changing as our environments become more hybrid — as we introduce cloud services? Are you still building packages the same way you were 10 years ago? Just as our data sources have grown, the capabilities needed to move data around have increased. This episode, with Jason Horner and Cathrine Wilhelmsen, challenges some of the thoughts around data movement and gives some things to think about as new tools are available to move data around.
Listen to Learn
00:40 Intro
01:33 Compañero Shout-Outs
02:08 SQL Trail
03:31 SQL Server in the News
05:03 Intro to the guest and topic
06:33 The genesis of the new ELT approach
10:16 Choosing between ETL and ELT
13:00 Is Azure Data Factory replacing SSIS? When should you be using ADF instead of SSIS?
15:06 Real-world use case and Azure Data Factory V1 vs V2
18:59 What is Azure Databricks?
22:37 It’s not just the guy in the closet who has to know what tools are being used
24:02 The tool you choose impacts the way you need to deal with your data — Patterns
28:00 Issues that ADF makes easier to deal with than SSIS did
30:24 Someone has to pay for whatever you plan to do
32:24 DBAs or data engineers have to be either reducing costs or enhancing revenues for the business
34:49 Common patterns Cathrine has built in her SSIS frameworks
36:12 SQL Family Questions – Cathrine
42:47 Closing Thoughts
Credits
Music for SQL Server in the News by Mansardian
Our Guests
Cathrine Wilhelmsen
Cathrine loves teaching and sharing knowledge. She is based in Norway and works as a consultant, focusing on Business Intelligence and Data Warehouse projects. Her core skills are ETL, SSIS, Biml and T-SQL development, but she enjoys everything from programming to data visualization. Outside of work she’s active in the SQL Server community as a Microsoft Data Platform MVP, BimlHero Certified Expert, author, speaker, blogger, organizer and chronic volunteer.
Cathrine’s blog: https://www.cathrinewilhelmsen.net/
Jason Horner
Jason Horner is a Microsoft Certified Master of SQL Server and has worked with it for more than 10 years. He began work as a DBA/Application Developer in 1999. He made the transition to delivering BI solutions in 2002 and began working with spatial data in 2008. Jason’s latest focus has been architecting large-scale spatially enabled BI solutions.
Is SSIS still relevant? Absolutely yes, if you’re all on-premises, if you’re not in the cloud, if you don’t have a hybrid environment, then SSIS is still the way to go.
Meet the Hosts
Carlos Chacon
With more than 10 years of working with SQL Server, Carlos helps businesses ensure their SQL Server environments meet their users’ expectations. He can provide insights on performance, migrations, and disaster recovery. He is also active in the SQL Server community and regularly speaks at user group meetings and conferences. He helps support the free database monitoring tool found at databasehealth.com and provides training through SQL Trail events.
Eugene Meidinger
Eugene works as an independent BI consultant and Pluralsight author, specializing in Power BI and the Azure Data Platform. He has been working with data for over 8 years and speaks regularly at user groups and conferences. He also helps run the GroupBy online conference.
Kevin Feasel
Kevin is a Microsoft Data Platform MVP and proprietor of Catallaxy Services, LLC, where he specializes in T-SQL development, machine learning, and pulling rabbits out of hats on demand. He is the lead contributor to Curated SQL, president of the Triangle Area SQL Server Users Group, and author of the books PolyBase Revealed (Apress, 2020) and Finding Ghosts in Your Data: Anomaly Detection Techniques with Examples in Python (Apress, 2022). A resident of Durham, North Carolina, he can be found cycling the trails along the triangle whenever the weather's nice enough.
Want to Submit Some Feedback?
Did we miss something or not quite get it right? Want to be a guest or suggest a guest/topic for the podcast?