Every AI Tool on Google Cloud, and When to Use It

flagship · iphone app for riders

Record your rides, share them beautifully.

GPS track, speed, lean angle and weather, captured on the road and turned into a map poster. No accounts, no servers: everything stays on your iPhone.

Get Qroda ↗made for riders

I work at Xebia, mostly on Google Cloud, and more and more of that work touches AI. A while ago I wanted a proper overview of all the AI tools Google provides.

It turned out to be a moving target. In April 2026, Vertex AI became the Gemini Enterprise Agent Platform, and dozens of products under it were renamed with it. In May, Gemini CLI handed over to the Antigravity CLI. In October, Google stopped selling Gemini Code Assist, and Jules shuts down at the end of the year. Plenty of what's been written about Google Cloud AI from before all that is now out of date.

So I went through it product by product, against the current docs and release notes. These are my notes. The goal is the same as my Google Cloud Networking 101 post: one place that covers every major piece, says what it's for and when to reach for it, and leaves you with a mental map of how it all fits together.

The post has two halves. The first covers the AI tools you use to do your own work: coding agents, the console assistant, MCP servers and the data agents. The second covers what you build AI products with: models, agents, retrieval, AI inside the databases, the infrastructure underneath, and the guardrails around it. Where my AI Engineering for Developers post already goes deep, on ADK, agent deployment and Model Armor, I keep it short here and link there.

Everything below is as of early November 2026. I've marked the features Google still labels Preview, and some of the names will have moved again by the time you read this.

First, the new names

Most of the confusion goes away once you have the rename table in your head. On April 22, 2026, at Cloud Next, Google launched the Gemini Enterprise Agent Platform as "the evolution of Vertex AI", and all Vertex AI services and roadmap now ship under it. The release notes list the full mapping. These are the ones you'll run into most:

The API hostnames didn't change, which helps. The platform still answers on aiplatform.googleapis.com and Agent Search is still the Discovery Engine API, so existing code keeps working. The console, the docs and the SDKs are mid-migration, though, and you'll see old and new names side by side, sometimes on the same page. Also, don't mix up Gemini Enterprise, the app employees use, with Gemini Enterprise Agent Platform, which is what Vertex AI became. The app runs on the platform, and the two names share two words.

AI tools for your own engineering work

The 2026 changes hit this half hardest. Most of the AI coding tools Google offered a year ago are gone or on their way out, and nearly all of what replaced them lives in one product.

Antigravity and the agy CLI

Antigravity is where Google now puts its coding agents. It launched on November 18, 2025, alongside Gemini 3, as an agent-first IDE built on VS Code. At I/O in May 2026 it grew into a platform: Antigravity 2.0, a desktop app for running and supervising several agents at once; the Antigravity CLI, agy, a terminal agent on the same engine; and an SDK for building on that engine, still in preview. The original editor is now called the Antigravity IDE.

agy is the successor to Gemini CLI. It's a single Go binary and it can run agents in the background. It kept the parts of Gemini CLI people had built on: skills, hooks and subagents carry over, and Gemini CLI extensions come back as Antigravity plugins. Google said at launch there would be no one-to-one parity, so check that the extension you depend on made it across. The other difference is licensing. Gemini CLI is Apache-2.0, while agy is closed source, and its GitHub repository holds a README, a changelog and an issue tracker but no code.

Pricing follows the consumer and enterprise split Google uses everywhere. Individuals get Antigravity free with weekly rate limits, and Google AI Pro and Ultra subscribers get higher limits. Organizations pay through a Gemini Enterprise seat or pay-as-you-go on a Google Cloud project. Enterprise users don't get the Antigravity IDE itself; they get the desktop app, the CLI and IDE extensions, where VS Code and Visual Studio are GA and JetBrains, Zed, Xcode and Eclipse are in preview. Admins can also opt in to Claude models next to Gemini. A plugin marketplace and remote agents that keep working in the cloud were rolling out as I wrote this.

If you work on Google Cloud and want Google's agent in your terminal or editor, this is it now. The setup advice in my post on how I use Claude Code doesn't depend on the tool and applies here too: a project instructions file, tests and linters as gates, and a short list of tools.

Where Gemini CLI, Code Assist, Jules and Firebase Studio went

• Gemini CLI is still open source and still ships weekly releases. Since June 18, 2026, though, it has no free tier and doesn't work with Google AI subscriptions. It runs only on a Code Assist Standard or Enterprise license or a paid Gemini API or Agent Platform key, and its README says it isn't tuned for the Gemini 4 models and points you to agy. If you need an open-source agent you can fork, it's still the one to use. Otherwise, switch.
• Gemini Code Assist, the IDE assistant that started as Duet AI, is being wound down. The free tier for individuals stopped on June 18, 2026, and Google stopped selling Standard and Enterprise on October 9. Existing subscriptions renew through the end of 2026, and auto-renewal ends in 2027. The consumer GitHub app for pull request reviews shut down in July.
• Jules, the asynchronous coding agent, goes read-only after November 11, 2026 and shuts down on December 31, so export what you want to keep.
• Firebase Studio, the browser IDE that used to be Project IDX, stopped accepting new workspaces on June 22, 2026. It shuts down on March 22, 2027, and anything left is deleted. The migration guide sends code-first projects to Antigravity and prototypes to AI Studio.

Google AI Studio

Google AI Studio is the browser workbench for Gemini. You try a prompt against different models there, get an API key in a few seconds, and see new models before they show up anywhere else. I come back to the API side in the models section.

What changed this year is Build mode. Since March 2026 it builds full-stack apps, not just front-ends. It runs on the Antigravity agent, wires up Firestore and Firebase Auth for you, keeps secrets in a secrets manager, and generates React, Angular or Next.js. Deploying the result to Cloud Run went GA at Next '26, and when a project outgrows the browser you can export it to Antigravity.

AI Studio itself is free to use, and model calls bill against the Gemini API. It's a good fit for prototypes and small internal tools. If something it generates is headed for production, review it like any other pull request.

Gemini Cloud Assist

Gemini Cloud Assist is the assistant in the Cloud console and the Cloud mobile app, and it has your projects as context. Next to a chat panel for questions about resources, IAM and errors, it includes App Design Center, where you describe an architecture and get back a diagram plus gcloud, kubectl or Terraform, and Investigations, which does root-cause analysis across logs, metrics and configuration. It also helps with cost in Cloud Hub and FinOps Hub, with Flow Analyzer in Network Intelligence Center, and with observability, Database Center and Storage Insights.

The product as a whole is still in Preview, more than eighteen months after it was introduced. Since April 10, 2026, Investigations has been limited to Premium Support customers or accounts their Google team approves. MCP access and proactive background agents, which look into alerts and cost anomalies before you ask, are in private preview.

Where it earns a place is triage. You ask why a request is denied, why a service can't reach a database, or what changed before latency went up, and the answer takes your project into account. It works well next to Network Intelligence Center's Connectivity Tests, which I covered in the networking post.

MCP servers for Google Cloud

If you'd rather bring your own agent, Google now meets you there. Google-managed MCP servers are remote endpoints hosted on each service's own API domain, such as bigquery.googleapis.com/mcp. They authenticate with IAM, honor deny policies, write audit logs, can be screened by Model Armor, and show up in Agent Registry. There were more than fifty by Next '26. The ones for BigQuery, AlloyDB, Cloud SQL, Spanner, Firestore, Bigtable and Cloud Run are GA, and the ones for the gcloud CLI, Cloud Assist, Security Command Center and Billing are in preview.

Any MCP client can use them, whether that's agy, Claude Code or an agent you built with ADK. Give the agent a service account with read-only roles and let IAM decide what it can do, rather than the prompt. It's the same reason I run the Kubernetes MCP server in read-only mode.

Two open-source projects cover what the managed servers don't. MCP Toolbox for Databases, formerly Gen AI Toolbox, reached 1.0 in April 2026. You host it yourself, and it covers Google databases alongside Postgres, MySQL, MongoDB, Snowflake and Neo4j. gcloud-mcp runs local servers for gcloud, observability and Cloud Storage.

One more is worth knowing about if you use agents to work on Google Cloud. The Developer Knowledge API and MCP server, in preview since February 2026, serves Google's current documentation to agents. With names moving this fast, a model's training data is usually a few renames behind, and an agent that checks the docs before it answers won't recommend a product that stopped being sold last month.

Gemini in the data tools

Gemini in BigQuery does SQL and Python generation, data canvas and data prep at no extra cost on every BigQuery compute option. On top of that sit three agents. The Data Engineering Agent (GA in April 2026) builds Dataform pipelines from a description. The Data Science Agent (GA in May) plans and runs an analysis end to end in Colab Enterprise and BigQuery notebooks. Conversational analytics (GA in June) answers questions about your tables in plain language, and you can publish those data agents to Gemini Enterprise so the rest of the company can use them. The Data Agent Kit, still in preview, is an extension that makes VS Code, Antigravity and Cursor aware of your BigQuery datasets.

In Looker, Conversational Analytics is GA and grounded in your LookML semantic layer, so its answers use your metric definitions. There's an API for embedding it in your own product.

A few smaller ones

Stitch, a Google Labs experiment, designs UIs from a prompt, and since March 2026 it has an MCP server so a coding agent can pull a design straight in. Code Wiki generates and maintains wikis for public repositories (preview). Conductor started as a Gemini CLI extension for spec-driven development and is now a plugin for Antigravity and Claude Code.

Models: AI Studio, Agent Platform and the Gen AI SDK

Two doors to the same models

From here on the post is about building things, and that starts with calling a model. There are two ways into Gemini, and they suit different stages of a project.

The Gemini Developer API is what you get from AI Studio. It lives on generativelanguage.googleapis.com and authenticates with an API key and nothing else. It's the fastest way to start, and the free tier is generous. On the free tier, prompts may be used to improve Google's products; the paid tier doesn't do that.

Agent Platform serves the same models, and many more, on aiplatform.googleapis.com inside your Google Cloud project. You authenticate with IAM: Application Default Credentials on your laptop, the attached service account on Cloud Run or GKE, Workload Identity Federation from CI. In return you get what production needs: VPC Service Controls, CMEK and data residency, SLAs, Provisioned Throughput, third-party models, the MLOps tooling, and spend that counts toward your Google Cloud commitment.

Google also added an express mode (preview), where you sign in with a Gmail account and get an API key and a 90-day free tier on Agent Platform. Agent Platform accepts API keys now, but use IAM in production.

One SDK for both

For years, moving from one door to the other meant rewriting code, because there were two Python SDKs. Both are now deprecated. google-generativeai reached end of life on November 30, 2025, and its repository is archived. The vertexai.generative_models module was removed in google-cloud-aiplatform 2.0 in September 2026, so if something still imports it, pin below 2.0 while you migrate.

Everything goes through the Google Gen AI SDK now: google-genai for Python, @google/genai for JavaScript and TypeScript, plus Go, Java and C#. Switching doors is one constructor argument:

from google import genai
from google.genai import types
from pydantic import BaseModel
 
 
class IncidentSummary(BaseModel):
    root_cause: str
    affected_services: list[str]
    severity: str
 
 
# Prototyping with an AI Studio key:
# client = genai.Client(api_key="...")
 
# Production on Agent Platform, authenticated with IAM:
client = genai.Client(vertexai=True, project="my-prod-project", location="global")
 
response = client.models.generate_content(
    model="gemini-3.8-flash",
    contents="Summarize this incident timeline: ...",
    config=types.GenerateContentConfig(
        response_mime_type="application/json",
        response_schema=IncidentSummary,
    ),
)
 
summary: IncidentSummary = response.parsed

You can also leave the constructor empty and set GOOGLE_GENAI_USE_VERTEXAI, GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION in the environment. A developer can then prototype against an AI Studio key while staging and production run on Agent Platform through IAM, with the same application code.

What's in the model catalog

The lineup moves every few weeks, so treat this as a snapshot of early November 2026.

Gemini 4 Argon is the new flagship, GA since October 13, 2026. It has a 1M token context and can also output up to 1M tokens, but it isn't tunable yet. The workhorse tier is the Gemini 3.x Flash family: 3.8 Flash (GA in September), 3.5 Flash, 3.5 Flash-Lite and 3.1 Flash-Lite. Gemini 3.1 Pro is still in preview. Gemini 2.5 is past its listed retirement date, so if you're still on it, migrate.

Imagen was retired on June 30, 2026, and Gemini's image models replaced it. "Nano Banana 2" is gemini-3.1-flash-image and "Nano Banana Pro" is gemini-3-pro-image. For video there's Veo 3.1, and for music there's Lyria (Lyria 2 is GA, Lyria 3 in preview). Speech output goes through Gemini TTS and transcription through Chirp 3, and the Live API for real-time voice runs on Gemini 3.8 Live, GA since September.

For embeddings, gemini-embedding-2 went GA in April 2026 and is multimodal: it covers text, images, documents, audio and video. The older text-embedding-00x models retire on April 1, 2027, and if you still use them that means re-embedding your corpus. Gemma 4 is the open-weight family, from small on-device sizes up to 31B.

Model Garden and open models

Model Garden is the catalog of more than 200 models inside Agent Platform. It puts two very different things on the same grid of cards, and which one you pick decides how you pay.

The first is models as a service: serverless, per-token APIs that Google operates. Besides Gemini, that includes Anthropic's Claude models (the current Opus, Sonnet and Haiku 5.5 and Fable 5.1, plus older versions), xAI's Grok, Meta's Llama 4, Mistral, and a few others in preview. Claude on Agent Platform is a strong option if your company already has a Google Cloud commitment, because the models are billed against it and governed by IAM and VPC Service Controls, with no separate vendor contract. In October 2026 Google also retired the pay-per-token endpoints for many open models (DeepSeek V3, Qwen3, Kimi K2, gpt-oss and others), so those are self-deploy only now.

The second is self-deployed open weights. Click Deploy on Gemma, Llama, Qwen or a Hugging Face model, and Agent Platform puts it on a dedicated endpoint with vLLM, vLLM on TPU, Hex-LLM or a custom container. You get traffic splitting, private networking and monitoring without setting anything up. The catch has always been that an endpoint keeps its GPUs running whether anyone calls it or not. Scale-to-zero for endpoints is in preview now: you set the minimum replica count to 0. The first request after an idle period gets a 429 while a replica starts, so your client needs to retry. If you want scale-to-zero today without the preview label, Cloud Run with a GPU is the better home (more in the infrastructure section).

Paying for tokens

Pay-as-you-go now comes in tiers, and picking between them is the biggest cost decision most teams make:

• Standard PayGo is the default. Your quota grows in tiers with your 30-day spend.
• Priority PayGo (GA May 2026) costs more and gets served first when capacity is tight.
• Flex PayGo (preview) is 50% cheaper in exchange for flexible latency. It runs on the global endpoint only and doesn't support Argon.
• Batch inference is 50% cheaper for jobs that read from and write to Cloud Storage or BigQuery. On Gemini 3 and later, the discount stacks with caching.
• Provisioned Throughput reserves capacity in generative AI scale units (GSUs), on terms from one week to one year, and comes with a latency SLA. Spend above the reservation spills over to pay-as-you-go.

Context caching cuts input cost on repeated prefixes by 90%. Implicit caching is on by default and needs no code. Explicit caching lets you pin a large prefix, like a document set or a codebase, for a TTL you choose, and charges for storage.

Endpoints come in three flavors: global, multi-region (US or EU) and regional. The global endpoint has the most capacity and the fewest 429s. Use it when your data residency rules allow, since regional endpoints can cost more on newer models.

The order I'd pull those levers in:

1. Start on Standard, on the global endpoint.
2. Send anything without a user waiting (backfills, nightly classification, offline evals) to batch or Flex.
3. Structure prompts so the long, stable part comes first and gets cached.
4. Buy Provisioned Throughput only for a baseline you've actually measured, not your peak.

Tuning and the classic ML stack

Supervised fine-tuning went GA for Gemini 3 in October 2026, on 3.5 Flash, 3.5 Flash-Lite and 3.1 Flash-Lite. The bigger models (3.1 Pro, 3.8 Flash and Argon) aren't tunable. Reinforcement fine-tuning is in preview, and a Gemini distillation service is in early access. As I argued in the AI engineering post, tuning is the last lever, after prompting, structured output and retrieval. On Gemini its best use is distillation: take a prompt that works on a big, slow model, generate a few thousand checked examples, and train a Flash-Lite model to produce the same output faster and cheaper.

The classic ML stack is all still there, renamed with an "Agent Platform" prefix: Workbench and Colab Enterprise for notebooks, Pipelines (managed Kubeflow), a Feature Store that's now backed by BigQuery, Model Registry, Experiments, Model Monitoring, and Managed Training and endpoints for custom models. Reach for it when you train tabular models for things like fraud, churn or forecasting and need reproducible retraining. Don't wrap LLM glue in it. A chunking job or a prompt eval inside Kubeflow Pipelines adds container start-up time and YAML to work that belongs in a Cloud Run job or a CI step.

The pre-trained APIs

The single-purpose APIs are still around. Document AI now offers custom extractors built on Gemini (in preview), Speech-to-Text runs on Chirp 3, Translation includes a Translation LLM you can fine-tune, and Text-to-Speech and Vision are still there. A Gemini call can do most of their jobs now. The APIs still win when you want a fixed output format, predictable per-page or per-minute pricing, and no prompt to maintain.

Building agents

My AI Engineering for Developers post covers this area in depth, with code. Here I'll stick to the map and what has changed since I wrote it.

ADK and agents-cli

The Agent Development Kit is Google's open-source, code-first agent framework, and it now comes in Python, TypeScript, Go, Java and Kotlin. ADK 2.0 shipped for Python in May 2026, then for Go and TypeScript over the summer. It adds a graph-based workflow runtime, and the old Sequential, Parallel and Loop agents become template workflows on top of it. Otherwise it has what you'd expect: MCP tools in both directions (consume them, or expose your agent as one), A2A for talking to other agents, adk web for a local debugging UI, adk eval with simulated users, and deploy commands for Agent Runtime, Cloud Run and GKE. ADK is model-agnostic and runs outside Google Cloud too.

agents-cli replaced the archived Agent Starter Pack. It scaffolds a project, runs evals, deploys and publishes. It also ships skills for Claude Code, Codex and Antigravity, so whichever coding agent you use knows how to drive it.

Agent Runtime, Sessions, Memory Bank and Sandbox

Agent Runtime, formerly Agent Engine, is managed hosting for agents built with ADK, LangGraph or your own code, and it's GA. Since I last wrote about it, it has gained sub-second cold starts, custom containers, revisions with traffic splitting, bidirectional streaming, and runs of up to seven days. You pay per second for vCPU and memory, and idle time between turns isn't billed. At the time of writing that's $0.085 per vCPU-hour and $0.009 per GiB-hour, and the current pricing page shows no free tier.

Sessions stores conversation state turn by turn, and Memory Bank extracts long-term facts about a user across sessions, in the background. Both are GA. Their pricing changed on September 1, 2026: storage is now billed per GiB-month, and reads and writes as compute. Sandbox groups code execution and computer use. A sandbox starts in under a second, keeps state for up to 14 days and has no network access, and agents running anywhere can call it, not only agents on Agent Runtime.

Use Agent Runtime when you want a stateful agent in production without building a session store, a memory pipeline and a sandbox yourself. Use Cloud Run when you need system packages, sidecars or a GPU in the same container.

Identity, registry and gateway

A few agents are easy to manage. Fifty agents across teams need the plumbing microservices needed, and Google shipped it this year. Agent Identity (GA in April 2026) gives each agent its own identity for IAM. Agent Registry (GA in June) catalogs agents and their A2A cards. Agent Gateway (GA in June) sits in the path of MCP and A2A traffic, enforces policy and can run Model Armor on it. Agents on Agent Runtime also send OpenTelemetry traces to Cloud Trace by default, which I come back to below.

The protocol under all of it, A2A, reached 1.0 in March 2026. It's a Linux Foundation project and, since August, part of the Agentic AI Foundation.

Gemini Enterprise

Gemini Enterprise, formerly Agentspace, is the finished product for employees. It's a company-wide assistant and search grounded in your systems (Workspace, Microsoft 365, Confluence, Jira, SharePoint, Salesforce and more), with permissions synced from the source. It also includes Workflow Builder for no-code agents (GA September 2026), and it's where IT approves and publishes the agents engineering teams build. ADK agents hosted on Agent Runtime and A2A agents can be registered there. It comes in several editions: Business starts at $21 per seat per month for up to 500 seats, and custom code agents need Standard, Plus or pay-as-you-go.

The reason an engineer should know it is so they don't rebuild it. When someone asks for "an internal ChatGPT over our Drive and Confluence", the cheaper answer is usually seats, plus a custom agent or two registered into it.

CX Agent Studio

For customer-facing chat and voice, Google launched Gemini Enterprise for Customer Experience in January 2026. Its core is CX Agent Studio, which is built on ADK and which Google describes as the evolution of Dialogflow CX. Agent Assist and the analytics product sit next to it. Dialogflow CX is still documented, and existing flow-based agents migrate through sales.

Retrieval and grounding

Sooner or later every company wants an AI that answers from its own documents. Google has four layers for that, from fully managed to build-your-own.

Agent Search

Agent Search, formerly Vertex AI Search, is managed search and RAG over data you put in Cloud Storage or BigQuery. You point a data store at it, and it parses, chunks, embeds and indexes your data. You get back a hybrid search API with reranking and generated answers, and there's an MCP server. The big change this year is that third-party connectors were removed from Agent Search. Confluence, Jira, SharePoint and the rest, with ACL sync, now live in Gemini Enterprise, so if your source is a SaaS tool rather than your own bucket, look there.

Agent Search gets you to good retrieval quality fast. The trade-off shows when a domain-specific query keeps pulling the wrong chunk, because you have far fewer knobs than you would with your own chunking, metadata and query weights.

RAG Engine

RAG Engine sits one level down: a managed RAG pipeline that you call from an agent you wrote. You choose a parser (the Document AI layout parser or an LLM parser), a reranker and a storage mode. The default mode is Serverless, backed by Agent Retrieval, and there's also a Spanner mode with customer-managed keys. The core is GA, many features are in preview, and new projects in some US and EU regions need an allowlist.

Vector Search and Agent Retrieval

Below that are the indexes themselves. Vector Search is the long-standing approximate nearest neighbor service, and it's GA. Agent Retrieval, formerly Vector Search 2.0, stores collections of objects rather than bare vectors, generates embeddings for you, reranks results and exposes an MCP server.

Grounding and the ranking APIs

Grounding connects a model call to a source: Google Search, Google Maps, Agent Search, RAG Engine, Elasticsearch or your own search API. Two smaller APIs are useful wherever your retrieval lives. The ranking API is a fast, stateless reranker. The check grounding API scores how well an answer is supported by the sources you pass it and returns citations.

How I'd choose:

• Agent Search when you want search over your own data working this week.
• RAG Engine when you own the agent but not the pipeline.
• Vector search in your database (next section) when you need control over every step, joins with relational data, and one less system to run.

AI inside the databases

A lot of AI work in a company isn't conversational. It's classification, extraction, translation and matching over rows that are already in a database. Pulling those rows into a Python service, calling a model in a loop and writing the results back is glue code, and Google's databases now let you skip it.

BigQuery

BigQuery now has a full set of SQL functions that call models, whether that's Gemini, Claude or an open model registered as a remote model. Next to the general AI.GENERATE, there's AI.GENERATE_TABLE for structured output and typed variants like AI.GENERATE_BOOL. The managed functions AI.IF, AI.CLASSIFY, AI.SCORE and AI.AGG take care of the prompt and the model for you. AI.PARSE_DOCUMENT, AI.CHUNK_DOCUMENT and AI.TRANSLATE work on documents and text, and AI.EMBED, AI.SIMILARITY and VECTOR_SEARCH cover embeddings and search. AI.FORECAST and AI.DETECT_ANOMALIES run on Google's TimesFM model with no training step.

Most of these are GA. Here's classification and extraction over yesterday's support tickets, written straight into typed columns:

-- One-time setup: register Gemini as a remote model
CREATE OR REPLACE MODEL `analytics.gemini_flash`
  REMOTE WITH CONNECTION `us.vertex-conn`
  OPTIONS (endpoint = 'gemini-3.5-flash');
 
SELECT ticket_id, category, sentiment_score, requires_escalation
FROM AI.GENERATE_TABLE(
  MODEL `analytics.gemini_flash`,
  (
    SELECT
      ticket_id,
      CONCAT('Classify this support ticket: ', body) AS prompt
    FROM `analytics.raw_tickets`
    WHERE created_at >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 1 DAY)
  ),
  STRUCT(
    'category STRING, sentiment_score FLOAT64, requires_escalation BOOL' AS output_schema,
    0.0 AS temperature
  )
);

BigQuery handles batching, retries and parallelism against the model. The danger is how easy it makes spending money: one query without a WHERE clause can send millions of rows to a model. Develop with LIMIT 10, and run AI queries in a project with quota and budget caps. Prompts that live in SQL strings need the same versioning and review as any other prompt, too.

AlloyDB AI

AlloyDB is Google's PostgreSQL-compatible database for heavy transactional workloads, and AlloyDB AI is its vector and model layer. The headline is the ScaNN index, which uses the same algorithm Google uses internally. Against HNSW on standard PostgreSQL, Google's docs claim up to 6x faster vector queries, 10x faster filtered queries, 16x faster index builds and a quarter of the memory. Since April 2026, ScaNN indexes tune themselves by default:

CREATE EXTENSION IF NOT EXISTS alloydb_scann CASCADE;
 
CREATE INDEX products_embedding_idx ON products
  USING scann (embedding cosine)
  WITH (MODE = 'AUTO');
 
-- A relational filter and a vector search in one query
SELECT product_id, title, price
FROM products
WHERE category_id = 14
  AND in_stock
ORDER BY embedding <=> google_ml.embedding(
  model_id => 'gemini-embedding-001',
  content => 'waterproof touring boots'
)::vector
LIMIT 20;

You can also call models from a query. ai.if filters rows on a natural-language condition, ai.rank orders them by natural-language criteria, ai.generate does free-form generation, and automatic embeddings stay in sync with a column. Those are GA, while the summarization, extraction and classification functions are in preview. Natural language to SQL now goes through a preview feature called QueryData. AlloyDB doesn't charge for the AI calls themselves; the model bills you.

Use AlloyDB AI for a production RAG or semantic search system that needs transactions, joins and tens of millions of vectors without a separate vector database. Keeping application state and embeddings in one database removes a whole class of sync bugs. The catch is the cost floor, because AlloyDB doesn't scale to zero. For a side project or a small knowledge base, start on Cloud SQL with pgvector and move later.

Vectors in the other databases

The rest of the family supports vectors too. Cloud SQL for PostgreSQL has pgvector, and Cloud SQL for MySQL has its own vector search. Spanner has exact search on every edition and approximate indexes on Enterprise and Enterprise Plus. Firestore has exact nearest-neighbor search with vector indexes, and Bigtable has distance functions but no approximate index. All of them have a Google-managed MCP server now.

Infrastructure: Cloud Run, GKE, TPUs and GPUs

When a managed API doesn't fit, you run the model yourself. Maybe you have a fine-tuned Gemma, a speech model that isn't in Model Garden, or enough volume that per-token pricing hurts. The options sit on a gradient: Cloud Run first, GKE second, raw accelerators last.

Cloud Run

Cloud Run is the easiest place to run a GPU workload on Google Cloud. A service can attach one NVIDIA L4 (24 GB) or one RTX PRO 6000 Blackwell (96 GB, GA since April 2026). It scales to zero when idle and starts a GPU instance in seconds:

gcloud run deploy gemma-vllm \
  --image us-docker.pkg.dev/my-project/llm/vllm-gemma:latest \
  --region us-central1 \
  --gpu 1 \
  --gpu-type nvidia-l4 \
  --cpu 8 \
  --memory 32Gi \
  --no-cpu-throttling \
  --no-gpu-zonal-redundancy \
  --max-instances 3

--no-gpu-zonal-redundancy trades failover across zones for a lower price, which is usually the right call for inference you can retry.

The rest of Cloud Run caught up this year. Jobs with GPUs are GA, and GPU jobs of up to 12 hours are in preview. Worker pools reached GA in April 2026, giving you pull-based workers that consume from Pub/Sub or Kafka without a fake HTTP port; a GPU on a worker pool is in preview and doesn't autoscale. Sandboxes for running untrusted or agent-generated code are in preview too.

Use Cloud Run for agent backends that need their own container, and for serving open models with vLLM or Ollama where traffic is spiky. The limits are one GPU per instance and a 60-minute request timeout, so a model that needs several GPUs with a fast interconnect has outgrown it.

GKE

GKE is where self-hosted inference goes when it gets serious, and the feature that justifies the extra work is GKE Inference Gateway, now described as "powered by llm-d". Standard Kubernetes load balancing is a poor fit for LLMs. Two requests of the same size can differ a hundredfold in GPU time, and sending a request to a pod whose KV cache is full causes latency spikes. Inference Gateway routes on KV-cache utilization, queue length and prefix-cache hits instead, plus LoRA adapter affinity so one base model can serve many fine-tunes. It costs nothing extra. InferencePool is GA, the objective CRD (which replaced InferenceModel) is still alpha and moving to the llm-d project, and a multi-cluster version is in preview.

Several other pieces sit around it. Inference Quickstart (gcloud container ai profiles) gives you benchmarked serving profiles and generated manifests, so you start from measured numbers rather than guesses. The Dynamic Workload Scheduler gets you scarce accelerators: flex-start runs a job when capacity frees up, at a discount, and calendar mode reserves a block for one to 90 days. Agent Sandbox runs agent code under gVisor. At Next '26 Google also announced automatic KV-cache tiering across memory, local SSD and storage, and llm-d joined the CNCF as a sandbox project.

Use GKE for high-throughput serving where cache-aware routing cuts your GPU count, for models that need several GPUs or hosts, or when your organization already runs a mature GKE platform. My posts on platform engineering and GCP networking cover what running one involves. Don't start here if Agent Runtime or Cloud Run can do the job, or you'll spend your time on GPU stockouts, driver versions and out-of-memory crashes instead of your product.

TPUs and GPUs

Google's own accelerators are the Cloud TPUs. The current generation is TPU7x (Ironwood), GA since March 31, 2026, with 192 GiB of HBM per chip and pods of up to 9,216 chips, positioned for both training and inference. v6e (Trillium), v5p and v5e are still offered. TPU 8t (for training) and TPU 8i (for inference) were announced at Next '26 as "available soon", and as I write this they aren't in the docs yet.

TPUs make sense when your model fits them. vLLM's TPU backend now serves the standard open architectures well, JAX runs natively, and native PyTorch on TPU is in preview with selected customers. The catch is that TPUs compile your model through XLA, so dynamic shapes and custom CUDA kernels that haven't been ported lead to recompilation stalls or unsupported operations. If your team runs research code full of custom kernels, stay on GPUs.

The GPU VM families, newest first:

AI Hypercomputer is the umbrella name for all of this plus the software around it, and Cluster Director builds and runs Slurm or GKE training clusters on it.

Getting weights onto accelerators

Large models spend a surprising share of their start-up time just loading weights. Managed Lustre is the high-throughput file system for training and large serving fleets. Cloud Storage Rapid, with Rapid Cache (formerly Anywhere Cache), keeps hot objects next to your accelerators, and Google says the Run:ai Model Streamer on top of Rapid Cache loads models up to five times faster. If cold starts on Cloud Run or GKE hurt, look here before you buy bigger instances.

Guardrails, evals and observability

Around the model you need a way to stop it doing something harmful, a way to tell whether a change made it better or worse, and a way to see what it did.

Model Armor

Model Armor screens prompts and responses for prompt injection and jailbreaks, harmful content, sensitive data (through Sensitive Data Protection) and malicious URLs. Since June 2026 it can screen images too, in preview. It has two levels of control. Templates are per-application settings. Floor settings are minimums a security team sets at the organization, folder or project level, and they apply to Gemini calls on Agent Platform and to Google's MCP servers even when an application team forgets.

It now plugs in wherever traffic flows: Agent Platform and Gemini Enterprise, Agent Gateway, Apigee (through policies that use templates), and load balancers, GKE and Inference Gateway through Service Extensions. Violations show up as findings in Security Command Center, and pricing is per token.

Each check is a remote call, so put Model Armor at the perimeter, on user input and output and on the gateway, not on every internal hop of a multi-step agent. The setup commands are in the AI engineering post.

The rest of the security picture

Apigee works as an AI gateway in front of any model provider. It has prompt and response sanitization through Model Armor, semantic caching, token quotas and MCP support, so the central rules don't depend on each team's code.

Security Command Center AI Protection inventories the models, agents and datasets across your organization, shows a dashboard of AI risks, and scans agents on Agent Runtime for vulnerabilities and leaked secrets. If you're on SCC's Enterprise tier, note that it shuts down in May 2027.

Sensitive Data Protection discovers sensitive data in training and tuning datasets, and since August 2026 it also detects leaked AI provider API keys.

Gen AI evals

Everyone agrees you need evals, and plenty of teams still test a prompt change by trying three examples by hand. Gen AI evals, formerly the Gen AI evaluation service, went GA in July 2026. You give it a dataset and a set of metrics. Those can be deterministic checks such as exact match, JSON schema and custom code, model-graded rubrics (including adaptive rubrics, which write specific pass/fail criteria for each prompt), or agent metrics for tool use and multi-turn task success. It can also simulate users with personas to exercise an agent, and run online monitors on production traffic that alert on drift.

Code-based metrics are free, and model-graded ones bill at model rates, which adds up quickly. Keep the set that runs on every pull request small, a hundred or so high-signal cases, and run the large suite nightly. If you already use LangSmith, Braintrust or Phoenix across clouds, there's no reason to move. If you want traces, datasets and evals inside one Google Cloud project and IAM boundary, this does the job.

Tracing agents

ADK agents on Agent Runtime send OpenTelemetry traces to Cloud Trace by default, following the OTel conventions for generative AI. Every model call, tool call and sub-agent shows up as a span, and token and latency metrics go to Cloud Monitoring. Agents running elsewhere can send OTLP to the same Telemetry API. When an agent does something strange in production, the trace is the first place to look.

What to use when

With the names out of the way, most decisions come down to a short list of defaults:

• An agent in your terminal or editor: Antigravity or agy. Gemini CLI only if you need open source and have a paid key.
• Prototyping with Gemini: AI Studio and an API key, using the google-genai SDK so the same code moves to Agent Platform later.
• Calling models in production: Agent Platform on Standard PayGo and the global endpoint, batch or Flex for work without a user waiting, caching for shared prefixes, and Provisioned Throughput for a measured baseline.
• Claude or another partner model under your Google Cloud contract: Model Garden's managed APIs.
• An open-weight model: Cloud Run with a GPU for spiky traffic, GKE with Inference Gateway for steady high volume, or Agent Platform endpoints if you want it managed and can live with scale-to-zero in preview.
• Enriching data at rest: BigQuery's AI functions, or AlloyDB's for transactional data, rather than a Python service in a loop.
• "Chat with our company docs" for employees: Gemini Enterprise, not something you build.
• Search or RAG inside your product: Agent Search to start, RAG Engine if you own the agent, vectors in AlloyDB or Cloud SQL when you need control and joins.
• A custom agent: ADK on Agent Runtime, or Cloud Run when you need your own container or a GPU.
• Customer-facing chat or voice: CX Agent Studio.
• Questions about your own cloud: Cloud Assist in the console, or the managed MCP servers with a read-only service account from your own agent.
• Safety: Model Armor floor settings at the organization level, templates at the perimeter.
• Quality: Gen AI evals, with a small set on every pull request and the full suite nightly.

The end

Google Cloud's AI stack is much better engineered than it is named. Under the 2026 renames there's a clear progression. Start with managed models and the unified SDK, run AI where your data already lives in BigQuery and AlloyDB, and use ADK and Agent Runtime when you want managed agents. Drop down to Cloud Run GPUs, GKE or TPUs only when you need to own the containers and the accelerators. For your own day-to-day work, the coding tools have collapsed into Antigravity.

If you take nothing else away: learn the rename table, because half of what you'll read online uses the old names. Default to the global endpoint, batch and caching before you buy capacity. Put budget alerts on any project that runs AI from SQL. Use Model Armor floor settings so safety doesn't depend on every team remembering it. And when an agent helps you with Google Cloud, point it at the current docs first, because left to its training data it will still call everything Vertex AI.

I hope you found this post useful. If you have any questions or feedback, please let me know!