You know that I love talking about vector search. I've had good success optimizing for vector search. Often though, that success does not last. I think I've realized why, and if this thought is true, it may turn out that doing a great job at optimizing for vector search could possibly do more harm than good.
Here is an excellent video to help explain vector search even further.
In the DOJ vs Google trial, we learned that RankBrain can be used to re-rank the top 20-30 results.
RankBrain has likely evolved tremendously since Google first told us about converting words to vectors. Still, there's no doubt that vectors and vector embeddings are important to Search. In fact, Google specifically tells us that vector search powers Google Search.
This is why we are seeing so much advice from SEOs to write in a way that looks good to vector search. Doing so will help our content look more relevant to machines. At least initially...
I did not believe him. (Sorry Gary! I had to test things for myself before coming to the realization that you were right.)
I developed a theory that I wrote about in my book. The concept involved expanding a query to include related micro-intents and then writing text that had well structured sentences that clearly and concisely answered each of those microintents. The idea was that this technique should cause the piece of content to be located in Google's vector space in a place that would be close to the query as well as the query expansions. We would therefore, look more relevant in the eyes of vector search.
I had good success doing this:
In some cases, we retained these rankings. In quite a few cases though, we won them for a short while and then lost all of our gains.
Before I explain why, it's worth mentioning that Google has systems that are far more sophisticated than my simple explanation of vector search. For example, in late 2024 they wrote a paper describing a technique where they gave an LLM a piece of content and asked it to create a query that was relevant to the content and another query that seemed relevant, but really was not. Then, they used those pairs to train a system to better understand the nuances of relevancy. I've written much more here with my thoughts on how Google determines relevancy and helpfulness.
But let's get back to why focusing mostly on optimizing for vector search may be the wrong path to walk down.
When we write for vector search, we look good to machines. But, what if users don't agree that we are the most relevant and helpful choice?
www.mariehaynes.com
What is vector search?
Vector search uses math to convert words and phrases into numbers in a way where the numbers represent multiple aspects or dimensions of the word. When those numbers are embedded into a multidimensional space, concepts that are similar to each other will appear close to each other in that space.Here is an excellent video to help explain vector search even further.
RankBrain and RankEmbed BERT use vector embedding
Google's AI systems like RankBrain and RankEmbed BERT use vector embeddings and vector search. When a query is searched, the query (and, likely expansions of the query) are embedded into a vector space and content that is embedded nearby in that space is seen as likely to be relevant. (Here's more on this from a 2016 Search Engine Land article written by Danny Sullivan which says Google tells us we can learn more about RankBrain by reading this 2013 Google blog post on converting words to vectors.)In the DOJ vs Google trial, we learned that RankBrain can be used to re-rank the top 20-30 results.
RankBrain has likely evolved tremendously since Google first told us about converting words to vectors. Still, there's no doubt that vectors and vector embeddings are important to Search. In fact, Google specifically tells us that vector search powers Google Search.
This is why we are seeing so much advice from SEOs to write in a way that looks good to vector search. Doing so will help our content look more relevant to machines. At least initially...
Google says writing for vector search won't help us rank better
A couple of years ago I had a fascinating conversation with Google's Gary Illyes about vector search. He told me it was something interesting to learn about, but knowing about vector search would not give me an advantage in Search.
I did not believe him. (Sorry Gary! I had to test things for myself before coming to the realization that you were right.)
I developed a theory that I wrote about in my book. The concept involved expanding a query to include related micro-intents and then writing text that had well structured sentences that clearly and concisely answered each of those microintents. The idea was that this technique should cause the piece of content to be located in Google's vector space in a place that would be close to the query as well as the query expansions. We would therefore, look more relevant in the eyes of vector search.
I had good success doing this:
In some cases, we retained these rankings. In quite a few cases though, we won them for a short while and then lost all of our gains.
How Google determines relevancy is evolving
I now understand why Gary encouraged me not to focus too much on vector search. I do think we can learn a lot by understanding Google's various methods for predicting which content a user will find relevant. But, I think that trying extensively to look good to Google's vector search systems can potentially do more harm than good.Before I explain why, it's worth mentioning that Google has systems that are far more sophisticated than my simple explanation of vector search. For example, in late 2024 they wrote a paper describing a technique where they gave an LLM a piece of content and asked it to create a query that was relevant to the content and another query that seemed relevant, but really was not. Then, they used those pairs to train a system to better understand the nuances of relevancy. I've written much more here with my thoughts on how Google determines relevancy and helpfulness.
But let's get back to why focusing mostly on optimizing for vector search may be the wrong path to walk down.
When we write for vector search, we look good to machines. But, what if users don't agree that we are the most relevant and helpful choice?
Could optimizing for vector search do more harm than good? - Marie Haynes
You know that I love talking about vector search. I've had good success optimizing for vector search. Often though, that success does not last. I think I've realized why, and if this thought is true, it may turn out that doing a great job at optimizing for vector search could possibly do more...
www.mariehaynes.com
