Docker Deployment

Two commands and you have a serving endpoint

The fastest path to running TF Serving is the official Docker image. Mount your SavedModel directory, set the model name, and you have a production endpoint exposing both REST (8501) and gRPC (8500).

The required directory structure: {model_name}/{version_number}/saved_model.pb. TF Serving auto-detects new versions by scanning for the highest integer subdirectory and hot-swaps without restart.

For multiple models, write a models.config file listing each model and its base path. One server, dozens of models, single config update.

Code

Single-model serving·bash

# Pull TF Serving image
docker pull tensorflow/serving             # CPU
docker pull tensorflow/serving:latest-gpu   # GPU

# Serve a model on REST port 8501
docker run -p 8501:8501 \
  --mount type=bind,source=/path/to/my_model/,target=/models/my_model \
  -e MODEL_NAME=my_model \
  -t tensorflow/serving

# Expose both gRPC (8500) and REST (8501)
docker run -p 8500:8500 -p 8501:8501 \
  --mount type=bind,source=/path/to/my_model/,target=/models/my_model \
  -e MODEL_NAME=my_model \
  -t tensorflow/serving

# GPU serving
docker run --gpus all -p 8501:8501 \
  --mount type=bind,source=/path/to/gpu_model/,target=/models/gpu_model \
  -e MODEL_NAME=gpu_model \
  -t tensorflow/serving:latest-gpu

Multi-model with config file·bash

# models.config
cat <<EOF > models.config
model_config_list {
  config {
    name: 'image_classifier'
    base_path: '/models/image_classifier/'
    model_platform: 'tensorflow'
  }
  config {
    name: 'text_classifier'
    base_path: '/models/text_classifier/'
    model_platform: 'tensorflow'
    model_version_policy { latest { num_versions: 2 } }
  }
}
EOF

# Serve with config
docker run -p 8500:8500 -p 8501:8501 \
  --mount type=bind,source=$(pwd)/models/,target=/models/ \
  --mount type=bind,source=$(pwd)/models.config,target=/models/models.config \
  -t tensorflow/serving \
  --model_config_file=/models/models.config

Required directory layout·bash

my_model/
├── 1/                        # version 1 (first deploy)
│   ├── saved_model.pb
│   └── variables/
│       ├── variables.index
│       └── variables.data-00000-of-00001
└── 2/                        # version 2 (updated)
    ├── saved_model.pb
    └── variables/
# TF Serving auto-serves version 2 when detected

Two commands and you have a serving endpoint

Code

Progress

Comments 0