Data silos don't just make data harder to find. They slow down analytics, duplicate effort, and limit what teams can build with the information already available across the organization. This article explores how an internal Snowflake marketplace can make data discoverable and accessible in real-time and across your organization.
Why We Didn’t Start With a Unified Data Model
Our customers typically have multiple product teams and business units, each running its own data stack, with no easy way to pull from a team that isn’t yours, short of asking around until someone forwards you an export. There’s no catalog, no shared way to even see what data exists elsewhere in the company, let alone query it, so anyone trying to build a cross-product view ends up collecting the pieces by hand, every time.
That’s a normal state for a company that grew as a handful of separate products, and it’s expensive in a specific way: product teams that could use each other’s data to build better features mostly don’t, and anyone doing cross-product analysis pays a coordination tax before they write a single query. When we took this on for one such customer, we wanted to fix the discovery and access problem without pretending we could also fix the harder problem, getting every team to agree on shared definitions at the same time.
Why We Didn’t Start With a Unified Data Model
The tempting version of this project is to also standardize dimensions, metrics, semantics, and naming across every product while you’re at it. We decided against bundling that in for this customer. Unifying vocabulary across teams that don’t report to the same roadmap is a slow, political effort, and tying it to the same initiative as “can people find and access data at all” would have meant neither one shipped on any reasonable timeline.
So, we split it into two efforts. The first, and the one this is about, is making data discoverable and accessible across the customer’s teams while every team keeps full ownership of its own datasets. A unified model is a real goal, but it’s a separate conversation for later, once people can see what everyone has.
The Shape We Landed On: Data Mesh Plus a Shared Marketplace
We built this on Snowflake Data Sharing, following Data Mesh principles rather than centralizing everything into one warehouse. Each data-producing team kept its own Snowflake account and stayed the owner of its datasets — nobody’s data got migrated into a shared account just to make it visible. What each team controlled was what it published and how, using its own domain language and definitions.
A few decisions followed from that:
- Every business unit got its own Snowflake account, with all of them living under one shared organization.
- Each account defined its own data contracts: what’s available for sharing, and in what shape.
- Contracts were implemented as a dedicated schema of curated, secure views wrapped into Listing/Share objects. Nothing gets shared as a raw table, and everything shared is read-only for whoever consumes it.
- The org-wide Internal Marketplace is the discovery layer on top of all of it — the place people go to find out what exists and subscribe to it.
Each business unit keeps its own account and publishes into the internal marketplace; consumers subscribe rather than requesting exports.
The part we liked most going in: Direct Share and Listing objects sync in real time. A team subscribing to a dataset isn’t pulling a snapshot from last night’s batch job. Once they’ve attached it, whatever changes on the publishing side show up immediately on their end, with no pipeline of their own to maintain.
How Publishing Actually Works
Mechanics matter more than the pitch, so here’s the pattern every team follows to publish something without losing control of it. Internal data stays internal. Nothing gets exposed straight out of a team’s raw or working tables. Instead, a dedicated schema holds secure views scoped to exactly what’s meant to be shared, and that schema gets wrapped into a Listing or Share object. Consumers always get read-only access, enforced by the mechanism itself rather than by convention.
Where a team already had Medallion layering in place — bronze, silver, gold, each with a clear owner — this fit neatly on top of it: the sharing schema draws from Silver or Gold, and publishing more later is just a matter of adding another curated view, not rethinking how the whole account is laid out.
Publishing is a chain of narrowing scope: internal data → curated views → a read-only contract → a subscriber.
The one wrinkle: this only stays free of extra plumbing when a publisher and a subscriber sit in the same region. If they don’t, replication has to be turned on to copy the share’s content into the target region before it can be mapped to a database there, which means real setup work and real egress cost, not just a checkbox.
Rolling It Out Team by Team
None of the three teams started from the same place, so the rollout looked different for each of them.
One team was mid-migration to Snowflake as its data lake house platform, the same effort that produced the Medallion layering rules we ended up writing down separately. Getting that team onto the marketplace meant aligning two projects at once: land the lake house, then expose its Silver and Gold layers once they existed.
A second team was already running on Snowflake with Silver and Gold data in place, which made them technically ready from day one. The open question for them wasn’t capability, it was judgment: which datasets to expose first, and which to hold back until there was a clearer answer on governance. We didn’t try to resolve that up front. The plan was to publish the most obviously useful datasets first and grow the list from there.
The third team was mid-migration for entirely separate reasons and just needed to finish that before joining in. Nothing about the marketplace blocked them; they were waiting on their own migration timeline, and we planned to loop them in once it closed.
Every target date we set carried an implicit asterisk, because two of the three teams’ timelines depended on migrations that were still moving while we were writing the plan down. What we actually had wasn’t a calendar so much as a sequence: whoever was ready first would go first, and the marketplace itself didn’t care what order that happened in.
What We Still Haven’t Settled
A few open questions came up during design that we didn’t force an answer to before shipping the first version. We’re leaving them in rather than tidying them away, because if your organization is working through something similar, these are worth putting on your own list early instead of running into them mid-rollout the way we did:
- Cross-region and cross-cloud cost
Data Sharing looks free of friction until publisher and subscriber aren’t in the same region. Then replication, and the egress cost that comes with it, becomes a real line item.Two rules of thumb have worked well enough so far: start by sharing from the Gold layer, and only reach into Silver when its value clearly outweighs the egress cost. And remember that Snowflake charges egress once per region rather than per consumer, so the more consumers subscribe to the same share, the cheaper it gets per head. - If you operate across regions or clouds, it’s still worth pricing this out before you promise anyone real-time access.
- Proving it out before committing org-wide
One proposal on the table was to stand up a separate stage account specifically to prototype the approach and stress-test it before every team builds against it for real. That still felt worth doing rather than skipping. A low-stakes place to find the sharp edges before they’re load-bearing is cheap insurance if you can spare the environment. - How granular a Listing should be
Do you publish one large Listing per team, or split into several smaller ones? Our instinct is toward smaller, self-sufficient Listings built around data products, entities with a clear business boundary — high cohesion inside one Listing, minimal coupling between Listings. Yet we haven’t had enough datasets published yet to know if that instinct holds up at scale. Worth deciding on a default early, since retrofitting granularity after a dozen datasets ship is harder than choosing it up front.
Where It Landed
None of that stopped the marketplace from paying off. Within the first few months, two of the three teams had published their initial datasets, and analysts who used to spend a day tracking down the right person now found what they needed in the catalog and subscribed to it themselves. The third team joined once its migration wrapped, and the timeline stopped mattering the moment it happened. The marketplace didn’t care who arrived first, and neither did the analysts, finally able to build a cross-product view without emailing three different people.
We haven’t unified the data model, and we’re not pretending discovery solved that problem. It didn’t. What it did solve was the thing that was blocking people day to day: not knowing what existed and not being able to get to it without a favor. That was worth shipping on its own, and the harder conversation about shared definitions is one the customer can now have with everyone looking at the same data instead of arguing about whose export is more current.
