In episode 16 of Recsperts, we hear from Michael D. Ekstrand, Associate Professor at Boise State University, about fairness in recommender systems. We discuss why fairness matters and provide an overview of the multidimensional fairness-aware RecSys landscape. Furthermore, we talk about tradeoffs, methods and receive practical advice on how to get started with tackling unfairness.In our discussion, Michael outlines the difference and similarity between fairness and bias. We discuss several stages at which biases can enter the system as well as how bias can indeed support mitigating unfairness. We also cover the perspectives of different stakeholders with respect to fairness. We also learn that measuring fairness depends on the specific fairness concern one is interested in and that solving fairness universally is highly unlikely.Towards the end of the episode, we take a look at further challenges as well as how and where the upcoming RecSys 2023 provides a forum for those interested in fairness-aware recommender systems.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts. (00:00) - Episode Overview (02:57) - Introduction Michael Ekstrand (17:08) - Motivation for Fairness-Aware Recommender Systems (25:45) - Overview and Definition of Fairness in RecSys (46:51) - Distributional and Representational Harm (53:59) - Relationship between Fairness and Bias (01:04:43) - Tradeoffs (01:13:36) - Methods and Metrics for Fairness (01:28:06) - Practical Advice for Tackling Unfairness (01:32:24) - Further Challenges (01:35:24) - RecSys 2023 (01:38:29) - Closing Remarks Links from the Episode: Michael Ekstrand on LinkedIn Michael Ekstrand on Mastodon Michael's Website GroupLens Lab at University of Minnesota People and Information Research Team (PIReT) 6th FAccTRec Workshop: Responsible Recommendation NORMalize: The First Workshop on Normative Design and Evaluation of Recommender Systems ACM Conference on Fairness, Accountability, and Transparency (ACM FAccT) Coursera: Recommender Systems Specialization LensKit: Python Tools for Recommender Systems Chris Anderson - The Long Tail: Why the Future of Business Is Selling Less of More Fairness in Recommender Systems (in Recommender Systems Handbook) Ekstrand et al. (2022): Fairness in Information Access Systems Keynote at EvalRS (CIKM 2022): Do You Want To Hunt A Kraken? Mapping and Expanding Recommendation Fairness Friedler et al. (2021): The (Im)possibility of Fairness: Different Value Systems Require Different Mechanisms For Fair Decision Making Safiya Umoja Noble (2018): Algorithms of Oppression: How Search Engines Reinforce Racism Papers: Ekstrand et al. (2018): Exploring author gender in book rating and recommendation Ekstrand et al. (2014): User perception of differences in recommender algorithms Selbst et al. (2019): Fairness and Abstraction in Sociotechnical Systems Pinney et al. (2023): Much Ado About Gender: Current Practices and Future Recommendations for Appropriate Gender-Aware Information Access Diaz et al. (2020): Evaluating Stochastic Rankings with Expected Exposure Raj et al. (2022): Fire Dragon and Unicorn Princess; Gender Stereotypes and Children's Products in Search Engine Responses Mitchell et al. (2021): Algorithmic Fairness: Choices, Assumptions, and Definitions Mehrotra et al. (2018): Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommender Systems Raj et al. (2022): Measuring Fairness in Ranked Results: An Analytical and Empirical Comparison Beutel et al. (2019): Fairness in Recommendation Ranking through Pairwise Comparisons Beutel et al. (2017): Data Decisions and Theoretical Implications when Adversarially Learning Fair Representations Dwork et al. (2018): Fairness Under Composition Bower et al. (2022): Random Isn't Always Fair: Candidate Set Imbalance and Exposure Inequality in Recommender Systems Zehlike et al. (2022): Fairness in Ranking: A Survey Hoffmann (2019): Where fairness fails: data, algorithms, and the limits of antidiscrimination discourse Sweeney (2013): Discrimination in Online Ad Delivery: Google ads, black names and white names, racial discrimination, and click advertising Wang et al. (2021): User Fairness, Item Fairness, and Diversity for Rankings in Two-Sided Markets General Links: Follow me on Twitter: https://twitter.com/MarcelKurovski Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode 15 of Recsperts, we delve into podcast recommendations with senior data scientist, Mirza Klimenta. Mirza discusses his work on the ARD Audiothek, a public broadcaster of audio-on-demand content, where he is part of pub. Public Value Technologies, a subsidiary of the two regional public broadcasters BR and SWR.We explore the use and potency of simple algorithms and ways to mitigate popularity bias in data and recommendations. We also cover collaborative filtering and various approaches for content-based podcast recommendations, drawing on Mirza's expertise in multidimensional scaling for graph drawings. Additionally, Mirza sheds light on the responsibility of a public broadcaster in providing diversified content recommendations.Towards the end of the episode, Mirza shares personal insights on his side project of becoming a novelist. Tune in for an informative and engaging conversation.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts. (00:00) - Episode Overview (01:43) - Introduction Mirza Klimenta (08:06) - About ARD Audiothek (21:16) - Recommenders for the ARD Audiothek (30:03) - User Engagement and Feedback Signals (46:05) - Optimization beyond Accuracy (51:39) - Next RecSys Steps for the Audiothek (57:16) - Underserved User Groups (01:04:16) - Cold-Start Mitigation (01:05:06) - Diversity in Recommendations (01:07:50) - Further Challenges in RecSys (01:10:03) - Being a Novelist (01:16:07) - Closing Remarks Links from the Episode: Mirza Klimenta on LinkedIn ARD Audiothek pub. Public Value Technologies Implicit: Fast Collaborative Filtering for Implicit Datasets Fairness in Recommender Systems: How to Reduce the Popularity Bias Papers: Steck (2019): Embarrasingly Shallow Auoencoders for Sparse Data Hu et al. (2008): Collaborative Filtering for Implicit Feedback Datasets Cer et al. (2018): Universal Sentence Encoder General Links: Follow me on Twitter: https://twitter.com/MarcelKurovski Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode number 14 of Recsperts we talk to Daniel Svonava, CEO and Co-Founder of Superlinked, delivering user modeling infrastructure. In his former role he was a senior software engineer and tech lead at YouTube working on ad performance prediction and pricing.We discuss the crucial role of user modeling for recommendations and discovery. Daniel presents two examples from YouTube’s ad performance forecasting to demonstrate the bandwidth of use cases for user modeling. We also discuss sources of information that fuel user models and additional personlization tasks that benefit from it like user onboarding. We learn that the tight combination of user modeling with (near) real-time updates is key to a sound personalized user experience.Daniel also shares with us how Superlinked provides personalization as a service beyond ecommerce-centricity. Offering personalized recommendations of items and people across various industries and use cases is what sets Superlinked apart. In the end, we also touch on the major general challenge of the RecSys community which is rebranding in order to establish a more positive image of the field.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Chapters: (03:35) - Introduction Daniel Svonava (10:18) - Introduction to User Modeling (17:52) - User Modeling for YouTube Ads (35:43) - Real-Time Personalization (57:29) - ML Tooling for User Modeling and Real-Time Personalization (01:07:41) - Superlinked as a User Modeling Infrastructure (01:31:22) - Rebranding RecSys as Major Challenge (01:37:40) - Final Remarks Links from the Episode: Daniel Svonava on LinkedIn Daniel Svonava on Twitter Superlinked - User Modeling Infrastructure The 2023 MAD (Machine Learning, Artificial Intelligence, Data Science) Landscape Eric Ries: The Lean Startup Rob Fitzpatrick: The Mom Test Papers: Liu et al. (2022): Monolith: Real Time Recommendation System With Collisionless Embedding Table RSPapers Collection General Links: Follow me on Twitter: https://twitter.com/MarcelKurovski Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

This episode of Recsperts features Justin Basilico who is director of research and engineering at Netflix. Justin leads the team that is in charge of creating a personalized homepage. We learn more about the evolution of the Netflix recommender system from rating prediction to using deep learning, contextual multi-armed bandits and reinforcement learning to perform personalized page construction. Deep content understanding drives the creation of useful groupings of videos to be shown in a personalized homepage.Justin and I discuss the misalignment of metrics as just one out of many elements that is making personalization still “super hard”. We hear more about the journey of deep learning for recommender systems where real usefulness comes from taking advantage of the variety of data besides pure user-item interactions, i.e. histories, content, and context. We also briefly touch on RecSysOps for detecting, predicting, diagnosing and resolving issues in a large-scale recommender systems and how it helps to alleviate item cold-start.In the end of this episode, we talk about the company culture at Netflix. Key elements are freedom and responsibility as well as providing context instead of exerting control. We hear that being really comfortable with feedback is important for high-performance people and teams.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Chapters: (03:13) - Introduction Justin Basilico (07:37) - Evolution of the Netflix Recommender System (22:28) - Page Construction of the Personalized Netflix Homepage (32:12) - Misalignment of Metrics (37:36) - Experience with Deep Learning for Recommender Systens (48:10) - RecSysOps for Issue Detection, Diagnosis and Response (55:38) - Bandits Recommender Systems (01:03:22) - The Netflix Culture (01:13:33) - Further Challenges (01:15:48) - RecSys 2023 Industry Track (01:17:25) - Closing Remarks Links from the Episode: Justin Basilico on Linkedin Justin Basilico on Twitter Netflix Research Publications The Netflix Tech Blog CONSEQUENCES+REVEAL Workshop at RecSys 2022 Learning a Personalized Homepage (Alvino et al., 2015) Recent Trends in Personalization at Netflix (Basilico, 2021) RecSysOps: Best Practices for Operating a Large-Scale Recommender System (Saberian et al., 2022) Netflix Fourth Quarter 2022 Earnings Interview No Rules Rules - Netflix and the Culture of Reinvention (Hastings et al., 2020) Job Posting for Netflix' Recommendation Team Papers: Steck et al. (2021): Deep Learning for Recommender Systems: A Netflix Case Study Steck et al. (2021): Negative Interactions for Improved Collaborative Filtering: Don't go Deeper, go Higher More et al. (2019): Recap: Designing a more Efficient Estimator for Off-policy Evaluation in Bandits with Large Action Spaces Bhattacharya et al. (2022): Augmenting Netflix Search with In-Session Adapted Recommendations General Links: Follow me on Twitter: https://twitter.com/MarcelKurovski Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In this episode of Recsperts we talk to Rishabh Mehrotra, the Director of Machine Learning at ShareChat, about users and creators in multi-stakeholder recommender systems. We learn more about users intents and needs, which brings us to the important matter of user satisfaction (and dissatisfaction). To draw conclusions about user satisfaction we have to perceive real-time user interaction data conditioned on user intents. We learn that relevance does not imply satisfaction as well as that diversity and discovery are two very different concepts.Rishabh takes us even further on his industry research journey where we also touch on relevance, fairness and satisfaction and how to balance them towards a fair marketplace. He introduces us into the creator economy of ShareChat. We discuss the post lifecycle of items as well as the right mixture of content and behavioral signals for generating recommendations that strike a balance between revenue and retention.In the end, we also conclude our interview with the benefits of end-to-end ownership and accountability in industrial RecSys work and how it makes people independent and effective. We receive some advice for how to grow and strive in tough job market times.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Chapters: (03:44) - Introduction Rishabh Mehrotra (19:09) - Ubiquity of Recommender Systems (23:32) - Moving from UCL to Spotify Research (33:17) - Moving from Research to Engineering (36:33) - Recommendations in a Marketplace (46:24) - Discovery vs. Diversity and Specialists vs. Generalists (55:24) - User Intent, Satisfaction and Relevant Recommendations (01:09:48) - Estimation of Satisfaction vs. Dissatisfaction (01:19:10) - RecSys Challenges at ShareChat (01:27:58) - Post Lifecycle and Mixing Content with Behavioral Signals (01:39:28) - Detect Fatigue and Contextual MABs for Ad Placement (01:47:24) - Unblock Yourself and Upskill (02:00:59) - RecSys Challenge 2023 by ShareChat (02:02:36) - Farewell Remarks Links from the Episode: Rishabh Mehrotra on Linkedin Rishabh Mehrotra on Twitter Rishabh's Website Papers: Mehrotra et al. (2017): Auditing Search Engines for Differential Satisfaction Across Demographics Mehrotra et al. (2018): Towards a Fair Marketplace: Counterfactual Evaluation of the trade-off between Relevance, Fairness & Satisfaction in Recommender Systems Mehrotra et al. (2019): Jointly Leveraging Intent and Interaction Signals to Predict User Satisfaction with Slate Recommendations Anderson et al. (2020): Algorithmic Effects on the Diversity of Consumption on Spotify Mehrotra et al. (2020): Bandit based Optimization of Multiple Objectives on a Music Streaming Platform Hansen et al. (2021): Shifting Consumption towards Diverse Content on Music Streaming Platforms Mehrotra (2021): Algorithmic Balancing of Familiarity, Similarity & Discovery in Music Recommendations Jeunen et al. (2022): Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved Confounders General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In this episode of Recsperts we talk to Flavian Vasile about the work of his team at Criteo AI Lab on personalized advertising. We learn about the different stakeholders like advertisers, publishers, and users and the role of recommender systems in this marketplace environment. We learn more about the pros and cons of click versus conversion optimization and transition to econ(omic) reco(mmendations), a new approach to model the effect of a recommendations system on the users' decision making process. Economic theory plays an important role for this conceptual shift towards better recommender systems.In addition, we discuss generative recommenders as an approach to directly translate a user’s preference model into a textual and/or visual product recommendation. This can be used to spark product innovation and to potentially generate what users really want. Besides that, it also allows to provide recommendations from the existing item corpus.In the end, we catch up on additional real-world challenges like two-tower models and diversity in recommendations.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Chapters: (02:37) - Introduction Flavian Vasile (06:46) - Personalized Advertising at Criteo (18:29) - Moving from Click to Conversion optimization (23:04) - Econ(omic) Reco(mmendations) (41:56) - Generative Recommender Systems (01:04:03) - Additional Real-World Challenges in RecSys (01:08:00) - Final Remarks Links from the Episode: Flavian Vasile on LinkedIn Flavian Vasile on Twitter Modern Recommendation for Advanced Practitioners - Part I (2019) Modern Recommendation for Advanced Practitioners - Part II (2019) CONSEQUENCES+REVEAL Workshop at RecSys 2022: Causality, Counterfactuals, Sequential Decision-Making & Reinforcement Learning for Recommender Systems Papers: Heymann et al. (2022): Welfare-Optimized Recommender Systems Samaran et al. (2021): What Users Want? WARHOL: A Generative Model for Recommendation Bonner et al (2018): Causal Embeddings for Recommendation Vasile et al. (2016): Meta-Prod2Vec: Product Embeddings Using Side-Information for Recommendation General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode number ten of Recsperts I welcome David Graus who is the Data Science Chapter Lead at Randstad Groep Nederland, a global leader in providing Human Resource services. We talk about the role of recommender systems in the HR domain which includes vacancy recommendations for candidates, but also generating talent recommendations for recruiters at Randstad. We also learn which biases might have an influence when using recommenders for decision support in the recruiting process as well as how Randstad mitigates them.In this episode we learn more about another domain where recommender systems can serve humans by effective decision support: Human Resources. Here, everything is about job recommendations, matching candidates with vacancies, but also exploiting knowledge about career path to propose learning opportunities and assist with career development. David Graus leads those efforts at Randstad and has previously worked in the news recommendation domain after obtaining his PhD from the University of Amsterdam.We discuss the most recent contribution by Randstad on mitigating bias in candidate recommender systems by introducing fairness-oriented post- and preprocessing to a recommendation pipeline. We learn that one can maintain user satisfaction while improving fairness at the same time (demographic parity measuring gender balance in this case).David and I also touch on his engagement in co-organizing the RecSys in HR workshops since RecSys 2021.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode: David Graus on LinkedIn David Graus on Twitter David's Website RecSys in HR 2022: Workshop on Recommender Systems for Human Recources Randstad Annual Report 2021 Talk by David Graus at Anti-Discrimination Hackaton on "Algorithmic matching, bias, and bias mitigation" Papers: Arafan et al. (2022): End-to-End Bias Mitigation in Candidate Recommender Systems with Fairness Gates Geyik et al. (2019): Fairness-Aware Ranking in Search & Recommendation Systems with Application to LinkedIn Talent Search General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode number nine of Recsperts we talk with the creators of RecPack which is a new Python package for recommender systems. We discuss how Froomle provides modularized personalization for customers in the news and e-commerce sectors. I talk to Lien Michiels and Robin Verachtert who are both industrial PhD students at the University of Antwerp and who work for Froomle. We also hear about their research on filter bubbles as well as model drift along with their RecSys 2022 contributions.In this episode we introduce RecPack as a new recommender package that is easy to use and to extend and which allows for consistent experimentation. Lien and Robin share with us how RecPack evolved, its structure as well as the problems in research and practice they intend to solve with their open source contribution.My guests also share many insights from their work at Froomle where they focus on modularized personalization with more than 60 recommendation scenarios and how they integrate these with their customers. We touch on topics like model drift and the need for frequent retraining as well as on the tradeoffs between accuracy, cost, and timeliness in production recommender systems.In the end we also exchange about Lien's critical reception of using the term 'filter bubble', an operationalized definition of them as well as Robin's research on model degradation and training data selection.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode: Lien Michiels on LinkedIn Lien Michiels on Twitter Robin Verachtert on LinkedIn RecPack on GitLab RecPack Documentation FROOMLE PERSPECTIVES 2022: Perspectives on the Evaluation of Recommender Systems PERSPECTIVES 2022: Preview on "Towards a Broader Perspective in Recommender Evaluation" by Benedikt Loepp 5th FAccTRec Workshop: Responsible Recommendation Papers: Verachtert et al. (2022): Are We Forgetting Something? Correctly Evaluate a Recommender System With an Optimal Training Window Leysen and Michiels et al. (2022): What Are Filter Bubbles Really? A Review of the Conceptual and Empirical Work Michiels and Verachtert et al. (2022): RecPack: An(other) Experimentation Toolkit for Top-N Recommendation using Implicit Feedback Data Dahlgren (2021): A critical review of filter bubbles and a comparison with selective exposure General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode number eight of Recsperts we discuss music recommender systems, the meaning of artist fairness and perspectives on recommender evaluation. I talk to Christine Bauer, who is an assistant professor at the University of Utrecht and co-organizer of the PERSPECTIVES workshop. Her research deals with context-aware recommender systems as well as the role of fairness in the music domain. Christine published work at many conferences like CHI, CHIIR, ICIS, and WWW.In this episode we talk about the specifics of recommenders in the music streaming domain. In particular, we discuss the interests of different stakeholders, like users, the platform, or artists. Christine Bauer presents insights from her research on fairness with respect to the representation of artists and their interests. We talk about gender imbalance and how recommender systems could serve as a tool to counteract existing imbalances instead of reinforcing them, for example with simulations and reranking. In addition, we talk about the lack of multi-method evaluation and how open datasets incline researchers to focus too much on offline evaluation. In contrast, Christine argues for more user studies and online evaluation.We wrap up with some final remarks on context-aware recommender systems and the potential of sensor data for improving context-aware personalization.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode: Website of Christine Bauer Christine Bauer on LinkedIn Christine Bauer on Twitter PERSPECTIVES 2022: Perspectives on the Evaluation of Recommender Systems 5th FAccTRec Workshop: Responsible Recommendation Papers: Ferraro et al. (2021): What is fair? Exploring the artists' perspective on the fairness of music streaming platforms Ferraro et. al (2021): Break the Loop: Gender Imbalance in Music Recommenders Jannach et al. (2020): Escaping the McNamara Fallacy: Towards More Impactful Recommender Systems Research Bauer et al. (2015): Designing a Music-controlled Running Application: a Sports Science and Psychological Perspective Dey et al. (2000): Towards a Better Understanding of Context and Context-Awareness General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode number seven, we meet Jacopo Tagliabue and discuss behavioral testing for recommender systems and experiences from ecommerce. Before Jacopo became the director of artificial intelligence at Coveo, he had founded tooso, which was later acquired by Coveo. Jacopo holds a PhD in cognitive intelligence and made many contributions to conferences like SIGIR, WWW, or RecSys. In addition, he serves as adjunct professor at NYU.In this episode we introduce behavioral testing for recommender systems and the corresponding framework RecList that was created by Jacopo and his co-authors. Behavioral testing goes beyond pure retrieval accuracy metrics and tries to uncover unintended behavior of recommender models. RecList is an adaption of CheckList that applies behavioral testing to NLP and which was proposed by Microsoft some time ago. RecList comes with an open-source framework with ready set datasets for different recommender use-cases like similar, sequence-based and complementary item recommendations. Furthermore, it offers some sample tests to make it easier for newcomers to get started with behavioral testing. We also briefly touch on the upcoming CIKM data challenge that is going to focus on the evaluation of recommender systems.In the end of this episode Jacopo also shares his insights from years of building and using diverse ML Ops tools and talk about what he refers to as the "post-modern stack".Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode: Jacopo Tagliabue on LinkedIn GitHub: RecList CIKM RecEval Analyticup 2022 (sign up!) GitHub: You Don't Need a Bigger Boat - end-to-end (Metaflow-based) implementation of an intent prediction (and session recommendation) flow Coveo SIGIR eCOM 2021 Data Challenge Dataset Blogposts: The Post-Modern Stack - Joining the modern data stack with the modern ML stack TensorFlow Recommenders TorchRec NVIDIA Merlin Recommenders (by Microsoft) recbole Papers: Chia et al. (2022): Beyond NDCG: behavioral testing of recommender systems with RecList Ribeiro et al. (2020): Beyond Accuracy: Behavioral Testing of NLP models with CheckList Bianchi et al. (2020): Fantastic Embeddings and How to Align Them: Zero-Shot Inference in a Multi-Shop Scenario General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode number six, we welcome Manel Slokom to the show and talk about purpose-aware privacy-preserving data for recommender systems. Manel is a 4th year PhD student at Delft University of Technology. For three years in a row she served as student volunteer at RecSys - before becoming student volunteer co-chair herself in 2021. Besides working on privacy and fairness, she also dedicates herself to simulation and in particular synthetic data for recommender systems - also co-organizing the 1st SimuRec Workshop as part of RecSys 2021.This episode is definitely worth a longer run. Manel and I discussed fairness and privacy in recommender systems and how ratings can leak signals about sensitive personal information. For example, classifiers may exploit ratings in order to effectively determine one's gender. She explains "Personalized Blurring", which is the approach she developed to personalize gender obfuscation in user rating data, as well as how this can contribute to more diverse recommendations.In our discussion, we also touch "data-centric AI", a term recently formulated by Andrew Ng, and how adapting feedback data may yield underestimated effects on recommendations that can lead to "data-centric recommender systems". In addition, we dived into the differences between simulated and synthetic data which brought us to the SimuRec workshop that she co-organized as part of RecSys 2021.Finally, Manel provides some recommendations for young researcher to become active RecSys community members and benefit from exchange: talk to people and volunteer at RecSys.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from the Episode: Manel on Twitter Manel on LinkedIn Manel at TU Delft (find more papers referenced there) SimuRec Workshop at RecSys 2021 FAccTrec Workshop at RecSys 2021 Andrew Ng: Unbiggen AI (from IEEE Spectrum) Papers: Slokom et al. (2021): Towards user-oriented privacy for recommender system data: A personalization-based approach to gender obfuscation for user profiles Weinsberg et al. (2012): BlurMe: Inferring and Obfuscating User Gender Based on Ratings Ekstrand et al. (2018): All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Recommender Evaluation and Effectiveness Slokom et al. (2018): Comparing recommender systems using synthetic data Burke et al. (2018): Synthetic Attribute Data for Evaluating Consumer-side Fairness Burke et al. (2005): Identifying Attack Models for Secure Recommendation Narayanan et al. (2008): Robust De-anonymization of Large Sparse Datasets General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode five my guest is Zeno Gantner, who is a principal applied scientist at Zalando. Zeno obtained his PhD from the University of Hildesheim where he was investigating ML-based recommender systems. As a principal applied scientist he is responsible for strategy, mentoring and setting standards for different initiatives on fashion recommendations impacting over 48 million customers in Europe.We discuss the ramifications and limitations of positive-only implicit feedback, touch on how reinforcement learning and more rating-like feedback can help as well as how to treat multiple feedback levels. In the main part, we turn our focus towards fashion recommendations and the “usual suspects” of typical e-commerce recommender systems.  We also discuss the goal of creating more fashion-specific recommendations and making users come back for inspiration. This involves a lot of domain-specific modeling and design of experiences to cater the needs for various user segments: from fashionistas to pragmatic customers. This also involves putting users into the “driver seat” of recommenders as well as understanding how to achieve long-term customer satisfaction.Finally, we briefly touch on the topic of size and fit recommendations and finish with an outlook on the future developments leading to fashion recommendations becoming its own subfield within the recommender systems space.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from this Episode: Preferably reach out to Zeno Gantner via email (find his address mentioned by the end of the episode) Fashion DNA by Zalando Research (Paper) Fashion MNIST (image dataset) Workshop on Recommender Systems in Fashion 2021 RecSys Challenge 2022 on Session-based Fashion Item Recommendation by Dressipi H&M Personalized Fashion Recommendation Challenge on Kaggle Spotify: A Product Story - Episode 4: Human vs Machine Dataset for trivago RecSys Challenge 2019 RecSys 2020: Tutorial on Conversational Recommender Systems Papers: Rendle et al. (2009): Bayesian Personalized Ranking from Implicit Feedback (2009) Loni et al. (2016): Bayesian Personalized Ranking with Multi-Channel User Feedback Sheikh et al. (2019): A Deep Learning System for Predicting Size and Fit in Fashion E-Commerce Wilhelm et al. (2018): Practical Diversified Recommendations on YouTube with Determinantal Point Processes General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode four my guest is Felice Merra, who is an applied scientist at Amazon. Felice obtained his PhD from Politecnico di Bari where he was a researcher at the Information Systems Lab (SisInf Lab). There, he worked on Security and Adversarial Machine Learning in Recommender Systems.We talk about different ways to perturb interaction or content data, but also model parameters, and elaborated various defense strategies.In addition, we touch on the motivation of individuals or whole platforms to perform attacks and look at some examples that Felice has been working on throughout his research.The overall goals of research in Adversarial Machine Learning for Recommender Systems is to identify vulnerabilities of models and systems in order to derive proper defense strategies that make systems more robust against potential attacks.Finally, we also briefly discuss privacy-preserving learning and the challenges of further robustification of multimedia recommender systems.Felice has published multiple papers at KDD, ECIR, SIGIR, and RecSys. He also won the Best Paper Award at KDD's workshop on Adversarial Learning Methods.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from this Episode: Felice's Website Felice Merra on LinkedIn and Twitter Adversarial Machine Learning in Recommender Systems (PhD Thesis Final Presentation) Workshop on Adversarial Personalized Ranking Optimization at ACM KDD 2021 (awarded Best Paper) Adversarial Recommender Systems: Attack, Defense, and Advances (chapter in 3rd edition of Recommender Systems Handbook) Information Systems Lab (SisInf Lab) Thesis and Papers: Merra et al. (2020): How Dataset Characteristics Affect the Robustness of Collaborative Recommendation Models Merra et al. (2021): A survey on Adversarial Recommender Systems: from Attack/Defense strategies to Generative Adversarial Networks find all the papers on Felice's website General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode three I am joined by Olivier Jeunen, who is a postdoctoral scientist at Amazon. Olivier obtained his PhD from University of Antwerp with his work "Offline Approaches to Recommendation with Online Success". His work concentrates on Bandits, Reinforcement Learning and Causal Inference for Recommender Systems.We talk about methods for evaluating online performance of recommender systems in an offline fashion and based on rich logging data. These methods stem from fields like bandit theory and reinforcement learning. They heavily rely on simulators whose benefits, requirements and limitations we discuss in greater detail. We further discuss the differences between organic and bandit feedback as well as what sets recommenders apart from advertising. We also talk about the right target for optimization and receive some advice to continue livelong learning as a researcher, be it in academia or industry.Olivier has published multiple papers at RecSys, NeurIPS, WSDM, UMAP, and WWW. He also won the RecoGym challenge with his team from University of Antwerp. With research internships at Criteo, Facebook and Spotify Research he brings significant experience to the table. Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from this Episode: Olivier's Website Olivier Jeunen on LinkedIn and Twitter Simulators: RecoGym RecSim RecSimNG Open Bandit Pipeline Blogpost: Lessons Learned from Winning the RecoGym Challenge RecSys 2020 REVEAL Workshop on Bandit and Reinforcement Learning from User Interactions RecSys 2021 Tutorial on Counterfactual Learning and Evaluation for Recommender Systems NeurIPS 2021 Workshop on Causal Inference and Machine Learning Thesis and Papers: Dissertation: Offline Approaches to Recommendation with Online Success Chen et al. (2018): Top-K Off-Policy Correction for a REINFORCE Recommender System Jeunen et al. (2021): Disentangling Causal Effects from Sets of Interventions in the Presence of Unobserved Confounders Jeunen et al. (2021): Top-𝐾 Contextual Bandits with Equity of Exposure General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In episode two I am joined by Even Oldridge, Senior Manager at NVIDIA, who is leading the Merlin Team. These people are working on an open-source framework for building large-scale deep learning recommender systems and have already won numerous RecSys competitions.We talk about the relevance and impact of deep learning applied to recommender systems as well as the challenges and pitfalls of deep learning based recommender systems. We briefly touch on Even's early data science contributions at PlentyOfFish, a Canadian online-dating platform. Starting with personalized recommendations of people to people he transitioned to realtor, a real-estate marketplace. From the potentially biggest social decision in life to the probably biggest financial decision in life he has really been involved with recommender systems at the extremes. At NVIDIA - to which he refers as the one company that works with all the other AI companies - he pushes for Merlin as large-scale, accessible and efficient platform for developing and deploying recommender systems on GPUs.This brought him also closer to the community which he served as industry Co-Chair at RecSys in 2021 as well as to winning multiple RecSys competitions with his team in the recent years.Enjoy this enriching episode of RECSPERTS - Recommender Systems Experts.Links from this Episode: Even Oldridge on LinkedIn and Twitter NVIDIA Merlin NVIDIA Merlin at GitHub Even's upcoming Talk at GTC 2021: Building and Deploying Recommender Systems Quickly and Easily with NVIDIA Merlin PlentyOfFish, realtor fast.ai Twitter RecSys Challenge 2021 Recommending music on Spotify with Deep Learning Papers Dacrema et al. (2019): Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches (best paper award at RecSys 2019) Jannach et al. (2020): Why Are Deep Learning Models Not Consistently Winning Recommender Systems Competitions Yet?: A Position Paper Moreira et al. (2021): Transformers4Rec: Bridging the Gap between NLP and Sequential / Session-Based Recommendation Deotte et al. (2021): GPU Accelerated Boosted Trees and Deep Neural Networks for Better Recommender Systems  General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/

In this first interview we talk to Kim Falk, Senior Data Scientist, multiple RecSys Industry Chair and author of the book "Practical Recommender Systems". We introduce into recommenders from a practical perspective discussing the fundamental difference between content-based and collaborative filtering as well as the cold-start problem - no mathematical deep-dive yet, but expect it to follow. In addition, we reason what constitutes good recommendations and briefly touch on a couple of ways of finding that out.Looking a bit into the history of the recommender systems community, we touch on the Netflix Prize that was running from 2006 to 2009 as well as on the RecSys - the leading conference in recommender systems, where we also met for the first time.In the end, we discuss a couple of challenges the field faces, in particular associated with approaches based on deep learning. Besides that, Spiderman will accompany our conversation at certain times. Plus many practical recommendations included on how to get started. Stay tuned!Links from this Episode: Kim Falk on LinkedIn and Twitter Book: Practical Recommender Systems (Manning) (get 37% discount with the code podrecsperts37 during checkout) GitHub Repository for PRS Book ACM Conference on Recommender Systems 2021 (Amsterdam) Recommender Systems Specialization at Coursera Amazon.com Recommendations: Item-to-Item Collaborative Filtering Netflix Prize Netflix Prize dataset on Kaggle New York Times: A $1 Million Research Bargain for Netflix, and Maybe a Model for Others Evaluation Measures for Information Retrieval Paper by Dacrema et al. (2019): Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches (best paper award at RecSys 2019) Recommending music on Spotify with Deep Learning MovieLens Recommenders General Links: Follow me on Twitter: https://twitter.com/LivesInAnalogia Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/ Twitter and LinkedIn posts for sharing: LinkedIn Twitter

Have you ever though about how Spotify is able to generate its fantastic Discover Weekly Playlist, how Amazon is generating a fortune by showing what other like you purchased in the past, or how Netflix achieves high user retention? The answer is personalization and in this show we focus on the most prominent way to achieve personalization: recommender systems.Whether you are a beginner and new to the field or you have already build recommenders, this show is to bring you the experts in recommender systems to share their knowledge and expertise with all of us. It is for making the topic more accessible and to provide a regular coverage of basics and advances in recommender systems research and application. I invite the experts to share their insights and to provide you with the right knowledge to get started and gain expertise yourself.In this introductory episode I am going to share some exemplary use cases from different industries (music streaming, e-commerce, travel, or social networks) along with challenges and problems in research and application. Plus, I am presenting the first guest for our upcoming episode.Links from the show: ACM Conference on Recommender Systems 2021 (Amsterdam): https://recsys.acm.org/recsys21/ Introductory Python RecSys Training: https://github.com/mkurovski/recsys_training Follow me on Twitter: https://twitter.com/LivesInAnalogia Read my RecSys Blogposts: https://medium.com/@marcel.kurovski Send me your comments, questions and suggestions to marcel@recsperts.com Podcast Website: https://www.recsperts.com/