The Art of Discoverability and Reverse Engineering User Happiness

TOCThe Problem with DiscoverabilityChallenges with present-day Metadata Platforms/TechnologiesThe Solution: Consolidation at a Global LevelWhat is a Global Metadata ModelImpact of this Model on Data TeamsImpact of this Model on BusinessHow is Data Disoverability Realised?What makes a Good Metadata ArchitectureWhere should the Metadata Engine sit in the Data StackWhat it means to build for the User: Discoverability-as-a-FeatureThe Philosophy of UXWhat does good UX imply in terms of DiscoverabilityDiscoverability SolutionsDiving right into a Discoverability requirement“As a data steward, I want to identify all the datasets where the past three quality runs have failed consecutively in a month. These datasets were derived from CRM and third-party data from a specific source system. Also, the datasets of interest should have impacted golden-tier downstream datasets in the past two weeks, eventually affecting production ML models and dashboards used for fraud detection.”In the current data ecosystem, the above problem is impossible to solve.The ProblemThe data ecosystem has flourished into a giant system with countless sources, pipelines, tools, and users.