Google Search API for Shopping (GSA) is a useful tool for retailers to facilitate competitive price monitoring. It enables product discovery, search on UPCs, and the data is structured and clean. Many retailers built in-house competitive price monitoring solutions using GSA.
Similarly, many price intelligence vendors have used GSA to various degrees in building their own competitive monitoring solutions. 360pi used it at one point for product discovery, but we uncovered some issues with it. Namely, the data is reliant on retailers to provide it to Google. This means that it can be incomplete, delayed, and often included issues with the shipping prices.
But regardless of whether it is effective or not, Google announced that they will be doing a spring cleaning of their product line and, unfortunately, the Search API for Shopping ended up on the chopping block. You can view the official Google Post here, but here is an excerpt:
“We’re deprecating our Search API for Shopping, which has enabled developers to create shopping apps based on Google’s Product Search data. While we believe in the value this offering provided, we’re shifting our focus to concentrate on creating a better shopping experience for users through Google Shopping. We’ll shut the API down completely on September 16, 2013.”
Retailers still have about 5 months until the product is sunset. But then what?
I sat down with our Chief Technology Officer, Dominic Plouffe, to give us his take on what retailers can do to prepare for life after GSA:
PU: Dom, can you explain why GSA was such a valuable tool for retailers and price intelligence vendors alike?
DP: Google Shopping API was free and the quality of data was very helpful for discovering if a given retailer carries a given product. There are not many other solutions out there that are free, high quality and available to anyone. Most product APIs are meant to generate revenue. For example; Price Grabber has an API, but it’s only accessible with an agreement to display products from their feed on your website.
Currently, retailers can search GSA by UPC or model number and get a structured set of data. The data set contains all the UPCs, model number, prices, and titles from each retailer.
PU: For retailers who were relying on GSA for an in-house competitive price monitoring tool, what alternatives are available to them?
DP: Luckily, there are alternatives available. Some are better than others. For example, retailers can use a manual method. This is obviously not scalable and results in human-error.
Retailers’ second option is to build a solution in house, leveraging Comparison Shopping Engines (CSE’s). CSEs rely on retailers to voluntarily submit their data, and often, retailers will not upload their entire product assortments, resulting in an incomplete data set. Furthermore, the pricing and shipping data is often wrong, since retailers may, intentionally or unintentionally, upload incorrect data. Lastly, retailers may not send the changes to the CSE until sometime later, and then the CSE may be delayed posting new information, so as a result, there is often a lag in the pricing information. All together, dependence on a CSE will result in inaccurate pricing information.
The third option available to retailers is investment in a price intelligence solution. A best in class price intelligence solution will get you the most complete and highest accurate competitive price data directly from your competitors’ sites.
PU: What makes the alternatives to GSA challenging? What gotchas do retailers need to look out for if they choose to explore the alternatives?
DP: Product matching from different retailers is not trivial. Lack of access to product metadata and different product variations can make things complicated. In many cases, matching a silver doorknob to a gold doorknob works because they are sold at the same price, but your matching falls apart when they do not. Retailers also need to keep in mind that model numbers are not an accurate indicator on whether a product is a match or not.
PU: Can you give us an example of how product matching becomes a challenge?
DP: Sure, let’s say I search for “GE XYZ” on Best Buy (GE = Manufacturer – XYZ = Model Number). Bestbuy.com returns 5 search results, and all have the model number XYZ and no products have the manufacturer as “GE “but one of them has the manufacturer as “General Electric”. A good Price Intelligence solution can use artificial intelligence to determine that “GE” and “General Electric” are the same. Most in-house solutions are not sophisticated enough to do this.
PU: What about manual labor? Can a retailer rely on that to match products?
DP: A retailer can rely on manual labor to confirm the matches, but manual labor falls apart when you want to scale to a reasonable set of products. You can give someone 2 products to match on 5 different competitors and they’ll get it right 100% of the time. Give them 10 products and they’ll probably get one wrong. Give them 100 products, and they’ll probably get 15 to 20 wrong, if not more. Human nature is to try and find short cuts when the work becomes tedious and that causes errors.
The other factor that makes it difficult with manual labour, is that the volume and frequency of price changes is so high these days. As soon as someone has manually went through and captured the data, the prices could have changed.
PU: What could the impact [of the loss of GSA] be on third-party providers of price intelligence?
DP: For providers who are heavily reliant on GSA, the demise of GSA could be very problematic. Depending on how much they were using GSA this could mean serious gaps in their data.
PU: What kind of questions could retailers ask their price intelligence vendors to uncover the extent of which they were using GSA?
DP: Some questions I would suggest asking price intelligence vendors are:
- What are the other ways that they are planning on getting that data?
- What is the expected impact on quality and completeness?
- How are they currently doing product discovery?
- Where do they actually pull prices from?
- How are they calculating shipping costs?
PU: What can you tell retailers out there about the product that you and your team have developed?
DP: We’ve built technologies that can crawl websites at a large scale and that can accurately product match products from different retailers with a limited amount of metadata. This technology has been developed over the last 6 years and it is in production today. We do not rely on Comparison Shopping Engines or GSA to extract our pricing and product data, and therefore will not be as impacted when this service no longer exists.
We have also developed advanced spidering technologies that allow us to do advanced product discovery with very little amounts of data. We have sophisticated site-search agents that do direct product discovery and price extraction.
We are not going to say that we are completely immune from the loss of GSA, however we believe we will still be in a better position than anyone else to provide timely, accurate and complete competitive pricing information.
PU: Any last thoughts before we sign off, Dom?
DP: I would just say that we would be happy to have a conversation with any retailers out there who want to find out how 360pi will adapt to this change. Don’t wait till the last minute and get caught without price intelligence, explore your options now.
As Dom said, don’t wait until it’s too late, find out how you can mitigate the end of Google’s Search API for shopping. Contact us today.