The Engineering Side of Data

Alex Debrie and Bob Haffner recap their favorite announcements from 2023 re:Invent #data #dataengineering #aws Connect with Alex Twitter: @alexbdebrie Blog: alexdebrie.com Book: dynamodbbook.com Podcast: youtube.com/@SoftwareHuddle Connect with Bob Twitter - @bobhaffner LinkedIn - linkedin.com/in/bobhaffner Alex’s talk https://www.youtube.com/watch?v=PVUofrFiS_A

Juan Pablo (JP) Urrutia and Bob Haffner discuss Data Quality #data #dataengineering #dataquality Connect with JP Twitter - @the_datachef Linkedin - https://www.linkedin.com/in/jpurrutia Substack - https://substack.com/@thedatachef Connect with Bob Twitter - @bobhaffner LinkedIn - https://www.linkedin.com/in/bobhaffner Follow the show on Twitter @EngSideOfData Data quality camp slack community:https://join.slack.com/t/dataqualitycamp/shared_invite/zt-2452i21wm-iZjqABfR7Hr8gcsshYuh2A Data Quality Fundamentals Book https://www.oreilly.com/library/view/data-quality-fundamentals/9781098112035/

Roy Hasson and Bob Haffner recap the 2023 Snowflake Summit. #data #dataengineering #snowflake  Connect with Roy  Linkedin - https://www.linkedin.com/in/royhasson/ Connect with Bob  Twitter - @bobhaffner  LinkedIn - https://www.linkedin.com/in/bobhaffner  Follow the show on Twitter  @EngSideOfData

Joseph Machado and Bob Haffner discuss the many aspects of developer ergonomics and how they relate to data engineering.   #data #dataengineering  Connect with Joseph  Blog - https://www.startdataengineering.com/ Linkedin - https://www.linkedin.com/in/josephmachado1991/  Connect with Bob Twitter - @bobhaffner LinkedIn - https://www.linkedin.com/in/bobhaffner

Alex Debrie and Bob Haffner recap their favorite announcements from 2022 re:Invent   #data #dataengineering #aws    Connect with Alex Twitter: @alexbdebrie  Blog: alexdebrie.com  Book: dynamodbbook.com   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner    Top Announcements  https://aws.amazon.com/blogs/aws/top-announcements-of-aws-reinvent-2022/   Yan Cui’s Choreography vs Orchestration blog post  https://theburningmonk.com/2020/08/choreography-vs-orchestration-in-the-land-of-serverless/

Join Eric Tolotti and Bob Haffner for a discussion about Data Lakes, Data Lakehouses and Data Warehouses #data #dataengineering #datalake #datalakehouse #datawarehouses Check out Eric's YT channel https://www.youtube.com/c/nullQueries Connect with Bob Twitter - @bobhaffner LinkedIn - linkedin.com/in/bobhaffner Show notes nullQueries’ Intro to the Data LakeHouse https://www.youtube.com/watch?v=yDXgvsnmUCs nullQueries’ Do you Need a Data Warehouse https://www.youtube.com/watch?v=K12lMYE3k0s nullQueries’  Dealing With Bad Analytics Requirements https://www.youtube.com/watch?v=8Td0A50cWzw

Join Alex Merced and Bob Haffner for a discussion about the Open Data Lakehouse concept   #data #dataengineering #datalake #datalakehouse    Connect with Alex  Twitter - @amdatalakehouse   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner    Show notes   The DataNation Podcast   Available on iTunes/Spotify/Stitcher   The Subsurface Data Lakehouse Community  dremio.com/subsurface   Dremio  dremio.com   Follow the podcast on Twitter @EngSideOfData

Sven Balnojan and Bob Haffner discuss the goals and principles of Functional Data Engineering   #data #dataengineering #functionaldataengineering   Connect with Sven  Twitter - @sbalnojan  LinkedIn - linkedin.com/in/dr-sven-balnojan-a55b4072  Blog - https://medium.com/three-data-point-thursday   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner   Follow the podcast on Twitter @EngSideOfData   Show Notes   https://github.com/sbalnojan/easy-functional-data-engineering   http://svenbalnojan.com/   https://maximebeauchemin.medium.com/functional-data-engineering-a-modern-paradigm-for-batch-data-processing-2327ec32c42a   Infrastructure as Code by Keif Morris  https://infrastructure-as-code.com/book/

Michael Meyer and Bob Haffner chat about data catalogs   #data #dataengineering #datacatalogs  Connect with Mike  Twitter - @dataguyatheart  LinkedIn - linkedin.com/in/michael-meyer-6972286   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner   Follow the podcast on Twitter @EngSideOfData

Johnny and Bob discuss Glue the serverless ETL tool from AWS   #dataengineering #etl #aws    Connect with Johnny   Youtube Channel  https://www.youtube.com/c/JohnnyChivers   The QuestionBank - Completely free Bank of community Questions AWS Certifications developed on AWS Amplify  https://www.thequestionbank.io/   Personal Website for Contact, forum and free AWS learning resource  https://johnnychivers.co.uk/   Connect with Bob   Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner https://twitter.com/EngSideOfData

Mário Pereira and Bob Haffner chat about Data Engineering at the Edge   #data #dataengineering #iot #edge #edgecomputing    Connect with Mário LinkedIn - linkedin.com/in/xmariopereira/   Connect with Bob Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner   Vopak is hiring (https://www.werkenbijvopak.nl/vacatures?lastchecked=discipline&discipline=9)

Adi Polak and Bob Haffner host a Twitter Spaces conversation on Careers and Community in Data Engineering.   #data #dataengineering  Connect with Adi  Twitter - @AdiPolak   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner   Join the Community - https://lakefs.io/community/

Cindi, Mike and Bob are back with the sequel to their 2021 Data Vault conversation.   Check out the upcoming World Wide Data Vault Consortium!  https://wwdvc.com/  Part 1 of this discussion https://www.youtube.com/watch?v=4U9HcFS93tM Connect with Cindi  Twitter - @Data_Rebels   Connect with Mike Twitter - @dataguyatheart  LinkedIn - linkedin.com/in/michael-meyer-6972286   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner   Show notes   Learn More: https://www.datavaultalliance.com - THE Standards Board for DV2 and Certification Education   https://www.datarebels.com - Authorized Training Partner, CDVP2 Certified Instruction, and Consulting Services   https://www.danLinstedt.com - Extensive Blogs by Daniel Linstedt, Data Vault Inventor and Found   https://kentgraziano.com/ - Kent Graziano's Data Warrior Blogs   https://twitter.com/dlinstedt  Participation and Getting Involved:  https://www.wwdvc.com - Annual Data Vault Conference  https://www.meetup.com/data-vault-north-american-user-group - Data Vault North American User' Group  https://www.linkedin.com/groups/9050802/ - Data Vault North American Users' Group LinkedIn Profile   Certified DV2 Training and Education:  https://datarebels.com/get-schooled/  https://datavaultalliance.com/certification/authorized-trainers/  https://datavaultalliance.com/event-directory/  https://www.arihovi.com/en/courses/data-vault-2-0-bootcamp-certification/  Other Resources:  Building a Scalable Data Warehouse with Data Vault 2.0 Published by Elsevier Paperback ISBN: 9780128025109 eBook ISBN: 9780128026489  Amazon:  https://www.amazon.com/Building-Scalable-Data-Warehouse-Vault/dp/0128025107/ref=sr_1_1?crid=335M6BS7BRLQA&dchild=1&keywords=building+a+scalable+data+warehouse+with+data+vault+2.0&qid=1633796921&sprefix=building+a+scala%2Caps%2C148&sr=8-1 #data. #dataengineering #datavault

Brian McMillan and Bob Haffner talk about Data Engineering for Data Discovery   #data #dataengineering #elt #etl #analysticsengineering  Connect with Brian  Like to discuss this further? First 5 people to schedule a meeting with me and receive a free copy of the book (https://calendly.com/building-data-products).   Purchase the book: https://www.minimumviablearchitecture.com    Brian McMillan – Brian [at] minimumviablearchitecture.com  Twitter – @brianmcmillan01  LinkedIn – linkedin.com/in/brianmcmillan01  GitHub – https://github.com/brianmcmillan/intro_to_data_and_analytics_engineering Schedule a discussion – https://www.minimumviablearchitecture.com/contact.html  The Book: Building Data Products: Introduction to Data and Analytics Engineering for non-programmers – https://www.minimumviablearchitecture.com   Full Length Demo - https://www.youtube.com/watch?v=xLjQhz1KJoQ&t=4s  Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner    Links: 3X - Kent Beck – https://www.youtube.com/watch?v=FlJN6_4yI2A Why Use Make (Mike Bostock) – https://bost.ocks.org/mike/make/ SQLite – https://www.sqlite.org/index.htm, https://blog.wesleyac.com/posts/consider-sqlite Building Production Applications Using Go & SQLite – https://www.youtube.com/watch?v=XcAYkriuQ1o)  CSVkit – https://csvkit.readthedocs.io/en/latest/ SQLite-Utils – https://sqlite-utils.datasette.io/en/stable/ Datasette – https://datasette.io Vega Lite –https://vega.github.io/vega-lite/

Dan Silberman and Bob Haffner discuss the traditional approach of Extract, Transform and Load (ETL) vs Extract, Load and Transform (ELT).   What each one means, why has ELT become so popular and how ELT will evolve. #data #dataengineering #elt #etl #analysticsengineering  Check out Mozart Data  https://www.mozartdata.com/  Connect with Bob Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner

Alex DeBrie joins Bob Haffner to recap the 2021 re:Invent from the data perspective   #data #dataengineering  Connect with Alex  Twitter: @alexbdebrie  Blog - alexdebrie.com  Book - dynamodbbook.com    Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner

Arjun Narayan and Bob Haffner discuss the state of streaming.   Connect with Arjun Twitter - @narayanarjun  LinkedIn - linkedin.com/in/arjunravinarayan/   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner   Show Notes  Materialize website - materialize.com  Materialize Twitter - @MaterializeInc  Materialize LinkedIn - https://www.linkedin.com/company/materializeinc Kafka is not a database blog post - https://materialize.com/kafka-is-not-a-database/ Streaming SQL blog post - https://materialize.com/streaming-sql-intro/  #data #dataengineering #streaming

Cindi Meyersohn joins Michael Meyer and Bob Haffner for a discussion about Data Vault.     Connect with Cindi  Twitter - @Data_Rebels   Connect with Mike  Twitter - @dataguyatheart  LinkedIn - linkedin.com/in/michael-meyer-6972286   Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner    Show notes   Learn More:  https://www.datavaultalliance.com - THE Standards Board for DV2 and Certification Education   https://www.datarebels.com - Authorized Training Partner, CDVP2 Certified Instruction, and Consulting Services   https://www.danLinstedt.com - Extensive Blogs by Daniel Linstedt, Data Vault Inventor and Found   https://kentgraziano.com/ - Kent Graziano's Data Warrior Blogs   https://twitter.com/dlinstedt   Participation and Getting Involved:  https://www.wwdvc.com - Annual Data Vault Conference  https://www.meetup.com/data-vault-north-american-user-group - Data Vault North American User' Group  https://www.linkedin.com/groups/9050802/ - Data Vault North American Users' Group LinkedIn Profile   Certified DV2 Training and Education:  https://datarebels.com/get-schooled/ https://datavaultalliance.com/certification/authorized-trainers/  https://datavaultalliance.com/event-directory/  https://www.arihovi.com/en/courses/data-vault-2-0-bootcamp-certification/  Other Resources:  Building a Scalable Data Warehouse with Data Vault 2.0 Published by Elsevier Paperback ISBN: 9780128025109 eBook ISBN: 9780128026489  Amazon:   https://www.amazon.com/Building-Scalable-Data-Warehouse-Vault/dp/0128025107/ref=sr_1_1?crid=335M6BS7BRLQA&dchild=1&keywords=building+a+scalable+data+warehouse+with+data+vault+2.0&qid=1633796921&sprefix=building+a+scala%2Caps%2C148&sr=8-1

Caleb Welchans and Bob Haffner discuss GIS.  GIS is a topic that everyone in data should be familiar with.     Connect with Caleb  Twitter - @calebwelchans  LinkedIn - linkedin.com/in/welchans        Intro/Outro music by John Yasut   #data #dataengineering #gis  Connect with Bob  Twitter - @bobhaffner  LinkedIn - linkedin.com/in/bobhaffner   Please Like, Subscribe and Comment!

Johnny Chivers and Bob Haffner discuss Data Lakes on AWS. Connect with Johnny!   Youtube Channel  The QuestionBank - Completely free Bank of community Questions AWS Certifications developed on AWS Amplify    Personal Website - Contact, forum and free AWS learning resource    Please Like, Subscribe and Comment!