Google Updates Bots and Indexing API Testing

Welcome to Crawl or No Crawl - this is Carolyn Holzman and today is Sept 10 - lots to cover - let’s get a move on

On Friday Google announced - we’re done - but here’s what I think led to all the last minute volatility -

Within the last 7 days - we’ve had the introduction of 4 new standard google crawlers and two updates on the InspectionTool crawlers - that is a lot of change that could lead to scattered and quick testing - have not seen this much update of chrome within 7 days -

I had to run a quick test on a different site to use the search console method to request indexation since those InspectionTool crawlers come on that method and NOT the API method. And sure enough their chrome builds were updated to the not the newest chrome but the one before that -

Here - let me show the bot log - that’s not on the site - you can only see it if I show it here.

And here’s the one before the newest - this is MY numerical designation - which you should know it means this is the 43rd mobile bot chrome seen since I started looking at server logs which I didn’t do from the very beginning of the research.

For anyone new the chrome crawlers - the picture bots - the ones that draw our pages get updated - and the nature of those updates historically has provided Google engineers with opportunities for testing the new against the old chrome build and invoked incidents of what normally I’d term as rollbacks to a previous state of the primary index - the one that we can see and search in.

Plus you take a look at the search console page updates - three updates within 7 days spanning 5 days of data added into the search console - instead of it normally taking 2 updates to cover that much data the 5 days’ worth

You have to go back to the week when the core update was announced to see that -

So if you saw weirdness last week - please refrain from blurting out that you were hit with the update.

These are alternate explanations for what you’re seeing that frankly are probably more likely to be the source of the weirdness than the update was.

Here’s another thing I haven’t seen before - desktop seo weather volatility being higher than mobile. Probably indicates that they are doing something with mobile results -

Now what’s going on with the Google Indexing API testing - last week is when I switched over to testing the Indexing API on a desktop primary crawler site - and while the simple html keywords got found - the javascript ones - no. at least not yet -

I didn’t do another test on the API until yesterday - nothing so far

And

Today’s test I managed a little differently - the three other tests were set up to automatically submit via the Indexing API - today’s disconnected the automated notification to the API and published the new page and then submitted the new sitemap. THEN I went back and submitted the URL to the Indexing API and well - immediately the googlebot arrived -

Please note what I’m saying here - the Indexing API doesn’t seem to automatically send the bot
if the sitemap gets updated in search console BEFORE the request - the bot comes immediately

Let that sink in -

Timing people -

And it doesn’t look like any of three tests were render crawled on the day of publish - and so far no day since - that’s different that it used to be before December 5th 2022

So if you’re using the Indexing API and you have a heavy javascript website framework - you’re probably pulling your hair out.

I suggest for problematic pages - manually resubmit your sitemap and then manually submit the troublesome URL - yes, you are limited to 10 in a 24 hour timeframe and its probably not YOUR midnight to midnight timeframe - it’s likely GMT.

OK - stick around for more Indexing API tests - the forensic seo training class is closed.

Related Topics:
August Core Update 2023
Google Indexing API
SEO Testing