Vertex AI and Cloud Deployment

Managed ML without managing infra

Vertex AI is Google Cloud's managed ML platform. Instead of running your own Kubernetes cluster and GPU instances, you describe what you want and Vertex handles the infra — training jobs, serving endpoints, autoscaling, logging, version management.

The three main capabilities you'll use:

Custom training jobs — run your training script on managed GPU/TPU machines, with experiment tracking
Model registry — upload SavedModels, version them, link to lineage
Online endpoints — deploy a registered model behind an autoscaling REST endpoint with one call

The serving containers are prebuilt: us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-{version}. Match your training TF version exactly — version mismatches between training and serving are a common source of "incompatible SavedModel" errors.

Code

Vertex AI: train, register, deploy·python

from google.cloud import aiplatform

aiplatform.init(
    project='my-gcp-project',
    location='us-central1',
    staging_bucket='gs://my-bucket',
)

# 1. Custom training job
job = aiplatform.CustomJob.from_local_script(
    display_name='my-tf-training',
    script_path='trainer/task.py',
    container_uri='us-docker.pkg.dev/vertex-ai/training/tf-cpu.2-12:latest',
    requirements=['tensorflow-datasets'],
    args=['--epochs=20', '--batch-size=256'],
)
job.run(
    machine_type='n1-standard-8',
    accelerator_type='NVIDIA_TESLA_T4',
    accelerator_count=1,
)

# 2. Upload SavedModel to Model Registry
model = aiplatform.Model.upload(
    display_name='flower-classifier-v2',
    artifact_uri='gs://my-bucket/models/flower_classifier/saved_model/',
    serving_container_image_uri=(
        'us-docker.pkg.dev/vertex-ai/prediction/tf2-cpu.2-12:latest'
    ),
    sync=True,
)

# 3. Deploy to autoscaling endpoint
endpoint = aiplatform.Endpoint.create(display_name='flower-endpoint')
deployed = endpoint.deploy(
    model=model,
    machine_type='n1-standard-4',
    traffic_percentage=100,
)

# 4. Online prediction
instances = [{"dense_input": [0.1, 0.2, 0.3, 0.4]}]
prediction = endpoint.predict(instances=instances)
print(prediction.predictions)

Vertex AI and Cloud Deployment

Managed ML without managing infra

Code

External links

Progress

Comments 0