Reengineering Product Discovery With AI, Vector Search, And LLMs

Introduction

Every day, millions of digital shoppers type perfectly valid questions into search bars and are met with confusion. They ask for help with "dry skin" and get a list of arbitrary lotions. They want to compare two popular brands and are served a product grid with no context. Behind this failure is an outdated assumption: that keywords are the language of intent.

But customers don’t search in keywords. They search in problems. In goals. In language that reflects their real needs, not a product catalog’s taxonomy.

This mismatch is one of the most persistent and most costly gaps in e-commerce today. It affects how people find, evaluate, and buy products. It leads to frustration, abandonment, and untapped revenue.

At Axelerant, we believe this is a foundational problem worth solving. We've engineered a solution that doesn’t just search for matches, it understands what you’re asking and provides the information you need. It turns ambiguous queries into guided discovery. And it bridges the gap between user intent and product relevance with AI-native architecture designed for scale.

This isn’t a nice-to-have upgrade. It’s a shift toward search that actually serves the customer, and, by extension, the business.

The Business Case: A Modern Search Engine For The Intent Economy

Traditional search engines within e-commerce platforms are limited by rigid keyword matching. They struggle to handle real-world queries like, "What’s a good face cream for dry skin?" or "Compare Samsung and OnePlus battery performance." These limitations translate to significant business risks. Over 40% of search results can be irrelevant, leading to frustrated customers, lower conversions, and increased strain on support teams.

In an era where digital consumers expect personalized, conversational interactions, the lack of semantic understanding becomes a bottleneck to growth. This engine was designed not just as a technical showcase but as a strategic solution, giving users the ability to converse with the platform as they would with a knowledgeable sales assistant.

What We Built: A Context-Aware Semantic Product Discovery Engine

Our semantic search engine transforms how users explore and compare products. It allows them to engage with product data using natural, conversational language, while the system returns contextually relevant answers. Instead of treating queries as keyword bags, it interprets intent, enabling intelligent product discovery.

The LLM-powered semantic search supports question-and-answer style queries, contextual comparisons, and sentiment-based insights drawn from reviews. It can identify relevant products, summarize user opinions, compare features, and surface critical buying factors such as price bands, ratings, and sentiment, all within a single, intuitive interaction.

The Engineering Behind The Experience

To build a system capable of delivering this experience, we implemented a layered architecture with a mix of open-source and commercial AI tools.

We started with publicly available Amazon product and review datasets. These were cleaned and normalized using Python and Pandas, with dbt optionally used for data modeling. Reviews were tokenized and split into manageable chunks using LangChain’s RecursiveCharacterTextSplitter, ensuring that no important context was lost in embedding.

For vectorization, we used OpenAI’s text-embedding-3-small model for high-quality semantic vectors, and optionally supported local alternatives like all-MiniLM-L6-v2. These embeddings were stored in ChromaDB along with metadata such as brand, category, and price band to enable high-quality filtering and retrieval.

The core of the interaction is powered by a Retrieval-Augmented Generation (RAG) pipeline. When a user enters a query, it is embedded, and the system performs a similarity search against the vector database. The top-matching chunks are compiled into a custom prompt and passed to an LLM (like GPT-4 or GPT-4o-mini) via LangChain. The LLM then generates a grounded, context-aware answer that draws only from the retrieved data.

For visualizations, we incorporated HTML and PNG reports, and built optional dashboards using Superset to deliver sentiment heatmaps and price-performance charts, key tools for both user insight and executive reporting.

The entire experience is surfaced via a Streamlit-based front-end, making it simple to interact with. For production readiness, the architecture can be easily exposed via FastAPI for integration with larger platforms.

Walkthrough: Search That Understands The Shopper

Consider the query: "Best moisturizer for dry, sensitive skin."

In a traditional system, such a query would return products with exact keyword matches, often missing the context entirely. The user might see a list of moisturizers, some irrelevant, some too generic, with no regard for skin sensitivity or hydration focus.

Our Semantic Search engine, on the other hand, interprets the user’s intent, identifying the need for gentle, hydrating products. It then filters products that match the inferred attributes, summarizes review sentiment, and presents top-rated options complete with pros and cons. It doesn't just show what's available; it shows what matters.

Now imagine a second query: "What do people complain about in budget fitness trackers?" Our system scans reviews, summarizes the most common complaints (e.g., "inaccurate step tracking", "poor battery life"), and provides suggestions with better performance in those areas. The user sees more than a product; they see insights.

Business Impact: Why Executives Should Care

The Semantic Search Engine was created with business leaders in mind. It represents a shift in how e-commerce platforms can align with customer behavior and expectations. Instead of optimizing around keywords, brands can now optimize around meaning.

With semantic search, the business gains tangible benefits:

• Higher conversion rates through more relevant product discovery
• Reduced support overhead as users find what they need without assistance
• Increased product visibility, especially for niche or long-tail items
• A scalable and modern architecture that can grow with the catalog and user base

This isn’t just about improving search; it’s about reimagining the buying journey from the very first interaction.

AI That Understands, Engineers Who Deliver

This Semantic Search Engine is a statement of intent. It shows how deeply the Axelerant team cares about solving problems that matter to both customers and businesses. The product discovery experience is broken across countless platforms. We set out to show what’s possible when technology is designed to meet people where they are, not where the database thinks they should be.

If you’re exploring how AI can create clarity, boost engagement, and accelerate conversion, this is what building with intent looks like. And we’re just getting started.