How Computer Vision Can Lead to a More Personalized Holiday Shopping Season

Date Updated: Thursday, December 21, 2017

Tapping into new data sources to reveal key insights into customer preferences

As people around the world begin exchanging gifts this holiday season, it’s worthwhile to reflect on a commonly repeated refrain: when it comes to gifts, it’s the thought that counts. For e-commerce businesses, there is an important lesson to be learned in this phrase. Gifts need to carry meaning and resonate on a personal level with both the buyer and the recipient.

In the past, brands relied on customer surveys or analyst predictions of which products would be in demand by customers during the critical holiday sales period. Today, companies turn to the large social media platforms and other sources of data to perform deep analysis of users and make relevant predictions that businesses can act upon. While these analyses are based on real data, they often rely upon artificial or arbitrary decisions taken by users as they consume social media or browse the web. Critically, they fail to deliver personalized recommendations that can lead to a heart-felt gift.

While there is certainly valuable information to be gleaned by studying a user’s social media preferences, that doesn’t always yield valuable or accurate predictions for what that user is interested in purchasing. The simple fact that a user is a fan of a particular show, or interacts with specific types of content, doesn’t necessarily translate into a prediction of upcoming purchase behavior. This is especially true during a period such as the holiday shopping surge in which customers generally make purchases for friends and family, rather than themselves.

Now imagine that you’re the developer of an ecommerce app, and that I could tell you that during this current holiday season you could expect to see an increase in shoppers looking for new pets and an increase in customers who need furniture because they are preparing to move to a new home. In fact, that’s exactly what we at Pixoneye are predicting this season, and it’s based on a data source that provides a far more accurate, predictive, and personalized analysis of purchase intent and preferences.

That source is the photo library on users’ smartphones, one of the largest and fastest growing sources of data available. In 2017 alone, it was estimated that there will be 1.2 trillion digital images captured with the vast majority of those taken on smartphones.

Learning from photos

We have developed a technology that uses computer vision and machine learning to analyze a user’s photo gallery in order to provide insights about their behavior and preferences. We partner with app developers to integrate this technology into their apps, allowing them to gain a better understanding of what their customers are interested in. What might this look like? Imagine an e-commerce app the user has downloaded and granted permission to access their photos. Our solution can reveal preferences for specific brands that may never come to light via social media. After all, how often do users “like” their favorite brand of soap or paper towels? Beyond that, user photos have powerful predictive capabilities, such as the example about moving to a new apartment mentioned previously.

This analysis rests on the fact that user photo libraries represent a far more accurate and realistic picture of their lives than what they upload and interact with on social media. When it comes to platforms like Instagram for instance, users carefully tailor a specific image of what they want the world to see. However, when it comes to the photos on a user’s phone, this is a much more realistic representation that hasn’t undergone that social filtering. This level of accuracy enables much more personalized insights and recommendations. For brands, this kind of personalization is key and helps to strengthen their relationship with customers. For users, it means they need to spend far less time searching and filtering through results to find what they’re looking for.

Of course, photos are deeply personal windows into people’s lives, so it’s incredibly important to explore this technology in a way that protects users’ privacy. To do so, we developed our technology with two key principals. First, all analysis happens on device. This means that no images are ever uploaded to our servers or our partners. It also means that no one at Pixoneye or our partners can see the content of users’ photos. The insights are generalized into what we call “feature vectors,” specific sets of patterns that we share with our partners to help them provide a more personalized experience. Second, we practice what we call “data monogamy.” This means insights are shared only with the app that the user downloaded and granted access to. If a user deletes that app, the data is completely removed, and both us and our partner lose access to it.

This was an important step and while it could be construed as a potential limitation of our platform, we believe it is essential for this kind of analysis. Users are correctly concerned with their photos being misused and it is important for our industry to respect this even as we work to provide users with additional value based on the content of their libraries.

Our solution is just one application of this technology, and looks at one very specific set of data. However, as cameras become important elements of even more devices, including wearable devices, smart devices in the home, and connected cars, the potential for computer vision to tap into the power of this data to provide more personalized experiences becomes clear. That’s why the ability to scale this technology rapidly and massively is so vitally important.

In that regard, our collaboration with Microsoft has proved to be extremely helpful. After participating in the Microsoft Accelerator – Tel Aviv, we began the process of migrating to Azure, which provides a unique set of computer vision tools that are helping us deploy our solution on a large scale. Beyond that, Microsoft has provided considerable support in terms of access to customers and partners. Our technology utilizes an incredibly innovative solution that pushes the boundaries of current marketing and data norms - having Microsoft as our strategic partner lends us a great deal of credibility and helps open new pathways for us.

Computer vision holds the potential to unlock numerous insights across multiple markets and industries. Helping e-commerce platforms obtain valuable insights into their customers is a logical place to begin. However, we believe it is only a first step towards enhancing every aspect of enterprises’ sales and marketing strategies.

Subscribe to the Microsoft for Startups blog

Subscribe