SEO

How We Automated Google Search Console to Keep 95% of Client Pages Indexed

Publishing a page does not mean Google indexes it. Here is the daily Google Search Console automation we built on our own AI server to keep client pages indexed, at scale, without touching a keyboard.

R
Roy Perel
Perel Web Studio Team
How We Automated Google Search Console to Keep 95% of Client Pages Indexed

You write a page. You publish it. You move on.

And you assume Google will find it, read it, and show it to people searching.

Most of the time, it does not.

The page sits in a state Google calls “Crawled, currently not indexed.” Google saw it. Google decided not to add it to the index. So it ranks for nothing. It brings zero traffic. It might as well not exist.

Nobody tells you this. There is no error, no email, no red light. The page is just quietly invisible.

For one page, you would never notice. But we run SEO for a portfolio of client sites. Between them, that is close to 20,000 pages. You cannot watch that by hand: checking each one, one at a time, in Google Search Console, every day, is not a job a human should do.

So we gave it to a machine.

Meet Minimac

Minimac is our AI server.

It is a small computer that sits in our office and never sleeps. Always on, 24 hours a day. It runs every automation we have built for the agency and for our clients: content, lead generation, reporting, and the one I want to tell you about today.

The indexing job.

Every night, while everyone is asleep, Minimac opens a browser, logs into Google Search Console, and does the tedious work that keeps our clients’ pages alive in Google.

Here is exactly what it does.

Step 1: Check what Google actually indexed

At 1am, Minimac pulls each site’s sitemap.xml. Then, for every URL, it calls the Google Search Console URL Inspection API and asks one question: is this page indexed, yes or no?

It writes every answer into one Google Sheet. One tab per site we manage, color coded. Green for indexed. Red and yellow for the pages Google is ignoring, with the exact reason next to each one:

  • Crawled, currently not indexed
  • Discovered, currently not indexed
  • Page with redirect
  • Excluded by noindex tag
  • URL is unknown to Google

Each tab shows the percentage of indexed pages at the top, and a master tab sums it up across the whole portfolio. One glance and you see the index health of every client.

That reason column matters. “Excluded by noindex” is intentional. “Redirect error” is a bug to fix. “Crawled, currently not indexed” is a page begging for a push. The reason tells us what to do next.

Before this, finding a single unindexed page meant clicking through Search Console by hand. Now the whole portfolio lands in one sheet every morning.

Step 2: Speed test every page

At 3am, a second job runs every page through the Google PageSpeed Insights API and writes the mobile score into the same sheet.

Why here? Because slow pages and unindexed pages are often the same pages. A page that takes six seconds to load on a phone is a page Google is reluctant to index and reluctant to rank. Now we see indexing status and speed side by side, on one row, for every URL.

One glance and we know which pages have a technical problem and which pages just need a nudge.

Step 3: Push up to 10 pages a day

This is the part that is hard to do by hand.

Here is a truth most people do not know. There is only one way to tell Google “index this page now”: submit it by hand in Search Console’s URL Inspection tool. Paste the URL, click Request Indexing, done.

Yes, Google has an Indexing API. But officially it only covers job posting and live broadcast pages, not regular content. For a normal page, that button in Search Console is the only lever you have. There is no API that forces it. And Google caps the button at roughly ten URLs a day.

So every morning at 7am, Minimac does exactly what a human would do, just faster and without forgetting. It drives a real Chrome over CDP with Playwright, logs into Search Console, reads the “not indexed” list from the sheet, picks the best candidates, and clicks Request Indexing on up to ten of them. Priority sites first. It skips the pages that should never be indexed anyway (carts, checkouts, tag pages, filters) so it never wastes a request, and it spaces them to respect the rolling 24 hour quota.

Ten a day sounds like nothing. But it is 300 a month. Three hundred pages, pushed at exactly the URLs that need it, is the difference between a backlog that grows and one that shrinks. It is not a magic switch, because Google does not give anyone one. It is the most you are allowed to do, done every single day.

The result

On the sites that matter most, the ones in active SEO projects, we keep around 95% of pages indexed. And that number keeps climbing on its own, month after month.

For a client, that is the difference between a blog they paid for that sits invisible, and a blog that actually shows up when someone searches. Between a new service page that Google never notices, and one that starts pulling leads within days.

And it happens on its own. Nobody logs in. Nobody checks. The report just arrives in our inbox every morning, and the number goes up.

Why this is the boring work that wins

Indexing is not glamorous. It is not a redesign. It is not a clever campaign. It is plumbing.

But it is the plumbing that decides whether all your other SEO work is even visible. You can write the best article in your industry. If Google never indexes it, you wrote it for nobody.

Most agencies never look at this, because doing it by hand across a portfolio is genuinely painful. That is precisely why we automated it. We would rather have a machine do the boring, relentless, every-single-day work, so the pages our clients pay for actually earn their keep.

That is what Minimac is for. Quiet, patient, invisible work that never takes a day off.

Is a page indexed today that was invisible yesterday? For our clients, most mornings, the answer is yes.

Curious how many of your own pages Google is actually indexing? That is one of the first things we check in a site audit.

#seo #google search console #indexing #technical seo #automation

Ready to Transform Your Online Presence?

Let's discuss how we can help your business grow with a high-performing website

Roy Perelgut

Founder & Digital Strategist

With 22 years of IT experience, Roy founded Perel Web Studio with one core belief: passion is what separates a good web agency from a bad one.

Passionate about creating digital solutions that drive real results, he leads a team of 6 from Brussels, collaborating with talented developers in Sri Lanka, delivering projects that achieve #1 Google rankings and multiply leads.

His approach combines technical excellence, sharp SEO strategy, and an uncompromising commitment to every client's success.

Connect on LinkedIn

Client Testimonials

What our clients say about us

Gilles Van Doorne
Stéphane Pujol
Caroline Pujol
Tom Sellekaerts
Emanuel Cristea
Gregory Marlier
Raphael Galant
Iron Jordan
Maître Géraldine Balthazar
Lionel Majorovic
Stéphane Roche
Alexandre Gelfged
Michael Schipper
Pierre Gerondal

Gilles Van Doorne

Owner of Hercules Merchandise

Perelweb delivers on their promises, but more importantly, they think with you and are genuinely involved in your business. By staying up to date with the latest in technology and AI, they help us keep improving.