Google explains its recent indexing problems

Indexing problems

Since last April, the Google search engine has experienced many problems with de-indexing or miss-indexing content. In a new post on his blog for webmasters, some explanations are provided on these recent concerns, with the promise of better communication in the future …

As we know, Google has recently experienced ( especially in April and May this year ) several indexing problems, which have created many issues updating the index of the search engine. Some observers also identified new difficulties a few days ago.

In short, these worries became frequent, and Google felt it was time to provide some information about it. This is what he just did on his blog for webmasters via a post signed by Vincent Courson.

Some explanations of the different indexing incidents

The article explains that the Mountain View company uses many data centers around the world and that these data centers are synchronized continuously to return, as often as possible, the same information and have the most similar index on each one of them, depending on the variation and changes web pages crawled at a time T. However, a problem occurred on April 5 (a Friday more), during synchronization of data centers which has badly done, which generated a bug causing the loss of part of the search engine index. A backup (backup) index could be reinstalled in a few days until a system update on April 11th.

On top of that, because of the bug on the synchronization of the index on the data centers, it is the Search Console which started to send data sometimes incoherent, so finally logical: the primary data is erroneous, the reports of the Search Console followed the same path. Backtracking on an index saved in advance did not facilitate data management. The Search Console bugs were processed once indexing bugs were fixed, days after the start of the phenomenon. The corrections started on April 15, ended on the 28th of the same month.

To make good weight, and a priori unrelated to the previous concerns, it is the Google News tool which then experienced problems of indexing and unwanted redirects.

Following these repeated worries, Google tried to communicate to explain what had happened on Twitter in Search Console and its online help.

Decisions made to better communicate in the future

In short, all these incidents led Google to put in place actions due to their ” post-mortem culture ” to prevent this type of phenomenon from happening again. Decisions have been made for better communication on this subject with webmasters, namely:

  • Explore ways to more quickly share information in the Search Console itself about detected bugs, and ensure that this information serves as the primary point of reference for webmasters to check for problems, in case they suspect failures.
  • Promptly display warnings on the Search Console Data Anomalies page (if the disruption is visible in long-term Search Console data).
  • Keep tweeting as quickly as possible to promptly reassure webmasters that engineers working on the engine are aware and that the problem is on their side.

Recently, a new problem has allowed Google to experiment with these new communication initiatives and strategies. On May 22, during the processing of some URLs, the ” duplicate management system ran out of memory after a planned upgrade of the infrastructure, which resulted in stopping the processing of all URLs. Incoming “this is what exactly happened then as Vincent Courson explains on the blog:

  • We noticed the problem (around 5:30 am, California time, May 22);
  • We tweeted about the current worry (around 6:40 am, California time, May 22);
  • We tweeted about solving the problem (around 10 pm, California time, May 22nd);
  • We evaluated updating the “Data Anomalies” page in the Search Console Help Center but decided not to do so because we had not detected a long-term impact. A term for the majority of Search Console data;
  • The confusion that this problem has created for many of us has confirmed our earlier findings that we need a way to more clearly indicate in the Search Console itself that there may be a disruption of one of our systems impacting webmasters. Such a solution might take longer to implement. We will communicate on this subject in the future, as we have more news;
  • Last week we also had another indexing problem. Like last May 22, we tweeted to let people know that there was a problem that we were working on it and when it was resolved.

The conclusion of the story and the moral of the song? Stay connected on Google’s various communication channels for everything from SEO and technical (indexing, etc.):

With all this, you should be more aware in the future of various problems occurring on the Google search engine in the future …

But do not forget: the best way to stay abreast of SEO and search engine news is to read News every week! 🙂

/by