Ideas2IT rewards key players with 1/3rd of the Company in New Initiative.  Read More >
Back to Blogs

Lipstick Shade Finder: Virtual Try-On with AR Technology

Virtual Makeup Tryon and Product Recommendation is cutting-edge use case of Augmented reality in the fashion e-commerce industry.Deciding which Lipstick shade suits best based on AI recommendations for individual skin tones can make women look beautiful and confident about their choice. It will also help e-commerce companies reduce costs associated with product returns.It can be used in multiple industries where virtual tryon can help people choose products.The frictionless shopping experience is very important in a cut-throat competition to stand out of the crowd and the feeling of ‘Try Before You Buy will dismiss any hesitation against e-commerce purchase

What is Augmented Reality & Virtual Assistants in Beauty Technology

Now, you should have a fair idea of what we are going to explore further in this blog post It's about technology that will revolutionize how fashion/cosmetics products are sold.The beauty industry is soaring to $530 billion and growing faster, the shopping experience is a prime factor that generates brand loyalty and drives sales.

Sephora's Virtual Assistant: Bridging the Gap Between Virtual Try-On and Real Beauty

Here we are going to briefly talk about Sephora (a French cosmetics major) that deployed a virtual assistant in a web application that lists products that Sephora manufactures.

For Example, Lipstick Try On Page list of all available shades of lipstick where the user can select a particular color and apply it live on a webcam stream of her face.

Augmented Reality Beautification

Solution design briefly includes:

  1. Augmented Reality for face-tracking>
  3. ML based face feature detection
  5. OpenCV / Image Processing to produce overlays after product application
  7. Rendering /Player for the user to see

Modiface - In-store MirrorsBased on the same solution stack but using standalone Hardware/Kiosk in a physical location to drive sales.

Augmented Reality Beautification

Like every product, there are some features that were spot on, while others were nowhere close to reality. After evaluating Sephora virtual assistant, here’s what we found:

Dazzling Features

  • Eyelashes - It tracked perfectly and worked exactly as expected in the real world
  • Eyebrows - Just like eyelashes, eyebrows simulation will also look great
  • Lipstick - Lip Color try on was good but for a few of the darker shades, the lighting wasn’t accurate

Dimmed Show

  1. Blush - skin color correction was not tracked accurately
  3. Foundation - lightening/difference was not noticeable enough

The Benefits of Virtual Try-On

The primary benefits of virtual try-on technology include:

  • Convenience: Customers can try out various products and shades without visiting a store or dealing with messy makeup trials.
  • Customization: Virtual try-on tools can suggest personalized product recommendations based on individual preferences and skin tone.
  • Cost savings: Brands can reduce costs associated with product samples and testers.
  • Increased customer satisfaction: Virtual try-on helps customers make more informed purchase decisions, leading to greater satisfaction and fewer returns.


  1. Image segmentation is still in the infancy stage. Hence, applying hair color doesn’t work with accuracy.
  3. Foundation/Blush depends on detecting skin surface accurately with the color application, which is nowhere close to reality.

The Future of Virtual Try-On Technology

  1. Enhanced Realism: As technology advances, virtual try-on experiences will become increasingly lifelike, providing users with hyper-realistic simulations that closely mimic physical makeup application.
  2. Expanded Product Range: Virtual try-on technology will allow users to explore a wider range of products virtually, not only limited to lipsticks but also including eyeshadows, blushes, foundations, and more.
  3. Augmented Reality Integration: Expect augmented reality (AR) to play a significant role in virtual try-on, introducing dynamic elements such as varying lighting conditions and simulated environments to provide users with a more immersive and realistic experience.
  4. Personalized Recommendations: Artificial intelligence (AI) algorithms will continue to evolve, enabling virtual try-on tools to offer personalized product recommendations tailored to individual skin tones, facial features, and preferences.
  5. Cross-Platform Accessibility: Virtual try-on features will be seamlessly integrated across multiple platforms, including dedicated mobile apps, e-commerce websites, and social media platforms, ensuring users can access them wherever they are.

Integration with Retail: In-store AR mirrors will become more prevalent, bridging the gap between online and offline shopping experiences by allowing customers to virtually try on makeup products before making a purchase in-store. This integration will create a cohesive and enhanced shopping journey for consumers.

With ongoing advancements in detection and tracking technology, augmented reality is on the brink of transformative breakthroughs. While still in its infancy, this burgeoning technology promises to revolutionize everyday life. Explore the possibilities with us. Let us know which of our augmented reality services for fashion technology development interests you, and our team will be in touch shortly!

Ideas2IT Team

Connect with Us

We'd love to brainstorm your priority tech initiatives and contribute to the best outcomes.