Microsoft’s Fabrice Canel confirmed this morning that the new Bingbot user-agent is now 100% live today. The new Bingbot will generally be used for crawling 100% of sites, unless something goes wrong and Bing has to fallback to the previous Bingbot user-agent.
Bing’s confirmation. Fabrice Canel posted on Twitter, “yes, 100% today. We fallback to previous user agents where appropriate.”
He linked to the updated Bing crawler user agents page that previously listed this as the user agent:
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www. bing.com/bingbot.htm) Chrome/ WXYZ Safari/537.36
Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/ WXYZ Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
The new user agent is documented as:
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) Chrome/
Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
W.X.Y.Z Safari/537.36Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
Some history. In 2019, Microsoft Bing announced new Bingbot user-agent names that fit better with its evergreen Bingbot crawling and rendering service. Then in April 2022, Microsoft announced a name change, a user agent name change, for BingBot. In December, Microsoft told us the new Bingbot user agent would be 100% live by January 2023.
Old user-agent. Microsoft said it would stop using its historical user agent by the Fall of 2022. That user-agent looks like this:
Mozilla/5.0 (compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm)
New user-agent. Bing will use a user-agent that identifies the specific version of Microsoft Edge is crawling your site. Here is the format for both desktop and mobile:
Desktop – Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm) Chrome/W.X.Y.Z Safari/537.36
Mobile – Mozilla/5.0 (Linux; Android 6.0.1; Nexus 5X Build/MMB29P) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/W.X.Y.Z Mobile Safari/537.36 (compatible; bingbot/2.0; +https://www.bing.com/bingbot.htm)
“W.X.Y.Z” will be substituted with the latest Microsoft Edge version Bing is using, for eg. “100.0.4896.127″.
Why we care. You probably should have been prepared for this change since it was announced back in 2019. But in any event, this change can impact your site if you have any user agent detection methods for BingBot. Make sure to test your site to see if it supports the new user agent. Most sites probably do not need to worry about this, but if you have done any advanced bot detection, you may need to take steps to update those scripts.
It seems that if you did not make the necessary changes, then Bing will use the old user agent as a fallback.