The Good, The Bad and The Ugly
While integrating with Snowflake, we used their relatively new Internal Marketplace as the source for Data Products. We are very pleased with the Internal Marketplace and its potential, but for anyone considering its use, here is an overview of the Good, the Bad, and the Ugly as of early April.
The Good
- You can now promote high-quality data assets as Data Products available to consumers through the Internal Marketplace.
- You can define and add metadata and relevant information for each Snowflake Listing. Users can find Listings and request access.
- A Data Product is not merely a single table; it can consist of multiple data assets.
- The Marketplace provides an interface to search for Data Products suitable for analytical purposes or building BI reports and dashboards.
The Bad
- You cannot add assets from different schemas in Snowflake, meaning you must consolidate all your data within the same schema for each listing. Although feasible, this complicates processes, requiring additional analysis and development effort to ensure data availability in the correct schema.
- Search functionality is very basic, you cannot search using tags, column names, or comments. The search mainly works on Data Product names, limiting discoverability significantly.
The Ugly
Scaling Issues
- Access requests to Snowflake Listings are limited to either a basic URL or email. Using email is highly discouraged due to governance and scalability issues. It's only practical for organisations with very few Data Products and users.
- The basic URL approach increases the user's effort, as additional information such as purpose, start date, and end date must be provided manually. Ideally, all Data Product access requests should automatically route to the appropriate approver, clearly stating purpose and duration.
- Currently, you cannot request access to individual data assets, instead, you must request access to the entire Listing. This limitation forces users to justify access to all included data assets, potentially violating the principle of least privilege and purpose. While manual workarounds exists, such as listing assets individually via email (not recommended) or submitting multiple forms in ServiceNow, these solutions are cumbersome, especially when multiple different approvers are involved.
Governance Concerns
- Requesting access via email presents significant governance challenges and is strongly discouraged.
- The preview of actual data before explicit access approval poses serious GDPR and governance risks. Users should not see live data until they have obtained access with a clearly defined purpose.
- Again, the inability to request access at an individual data asset level is problematic. For large organizations, requiring access to entire Listings rather than individual assets complicates governance, security, and privacy compliance.
In summary, while Snowflake's Internal Marketplace offers promising capabilities, careful consideration of its current limitations is essential to avoid governance and operational challenges.
MeshFaro addresses all the issues mentioned above. That’s why we built the platform :-)
Request a demo for more information!