Here's the deal. My 404 log is filling up with requests for things like this:
/wp-content/cache/autoptimize/js/autoptimize_3262629d5d3427a82464844a0d29c75a.js
/wp-content/cache/autoptimize/js/autoptimize_0c00ec59d9b582cb71bcc0d9e2a1e600.js
/wp-content/cache/autoptimize/js/autoptimize_78f5f9ff7310c387db0c7ec7b18d057b.js
...
and so on, and so on. So numerous as to make the 404 logs quickly meaningless.
These requests come ONLY from googlebot. And none of them have a referrer.
Yes, the site does use WP Super Cache. But those links do not currently exist on my site. The site and the cache are in perfect sync. It is not possible to get these errors by looking at the live site.
Here is how this is happening. Google is crawling my site and saving the pages. Sometime after Google crawls the page, I update the site, and the cache and autoptimize cache refresh. The site is perfect.
However, some time later, Google comes back and tries to crawl all the links on the page it saved two days ago. These links (as above) no longer exist, and I get gobs of 404 errors - only from Googlebot, because only Googlebot saved the page.
Previously, I worked around this with robots.txt by preventing Googlebot from crawling wp-content/cache. This eliminates these errors.
However, with the new "mobile friendly" edict from Google, if I do not allow wp-content/cache to be crawled, the site appears mobile-un-friendly.
Do you have any suggestions for eliminating these 404 errors? Google only visits so many times a day, and now half of those seem to be 404s.
Thanks