TF Recommenders, TF Agents, TF Federated

세 가지 문제 모양에 세 가지 특화 스택

TF Recommenders (TFRS)는 추천 시스템 빌딩 블록 제공 — Netflix, YouTube, Spotify 굴리는 retrieval → ranking 2단계 pipeline. Retrieval 단계가 ANN (approximate nearest neighbor)으로 수백만 후보를 밀리초 안에 수천으로 좁혀. Ranking 단계가 비싼 model을 그 후보들에 적용해서 top-N 선택.

TF Agents는 production-grade 강화학습 라이브러리: DQN, PPO, SAC, TD3, DDPG. 이산 action space는 DQN; 연속 action space (로봇 토크, 차량 조향)는 SAC.

TF Federated (TFF)는 분산 데이터에서 ML 가능하게 해. Training 데이터가 클라이언트 device (폰, 병원)에 머물고; model 업데이트만 중앙 서버로. Raw 데이터가 출처 떠날 수 없을 때 프라이버시 보존 대안.

Code

TFRS two-tower retrieval·python

import tensorflow as tf
import tensorflow_recommenders as tfrs

# pip install tensorflow-recommenders

user_model = tf.keras.Sequential([
    tf.keras.layers.StringLookup(vocabulary=unique_user_ids, mask_token=None),
    tf.keras.layers.Embedding(len(unique_user_ids) + 1, 64),
])
movie_model = tf.keras.Sequential([
    tf.keras.layers.StringLookup(vocabulary=unique_movie_titles, mask_token=None),
    tf.keras.layers.Embedding(len(unique_movie_titles) + 1, 64),
])

task = tfrs.tasks.Retrieval(
    metrics=tfrs.metrics.FactorizedTopK(
        candidates=movies.batch(128).map(movie_model),
    )
)

class MovielensModel(tfrs.Model):
    def compute_loss(self, features, training=False):
        u_emb = self.user_model(features["user_id"])
        m_emb = self.movie_model(features["movie_title"])
        return self.task(u_emb, m_emb)

model = MovielensModel(user_model, movie_model, task)
model.compile(optimizer=tf.keras.optimizers.Adagrad(0.5))
model.fit(ratings.batch(4096), epochs=3)

TF-Agents — CartPole의 DQN·python

import tensorflow as tf
from tf_agents.environments import suite_gym, tf_py_environment
from tf_agents.networks import q_network
from tf_agents.agents.dqn import dqn_agent

# pip install tf-agents

env = suite_gym.load('CartPole-v1')
train_env = tf_py_environment.TFPyEnvironment(env)

q_net = q_network.QNetwork(
    train_env.observation_spec(),
    train_env.action_spec(),
    fc_layer_params=(128, 64),
)

agent = dqn_agent.DqnAgent(
    train_env.time_step_spec(),
    train_env.action_spec(),
    q_network=q_net,
    optimizer=tf.keras.optimizers.Adam(1e-3),
    train_step_counter=tf.Variable(0),
)
agent.initialize()

TFF — federated averaging·python

import tensorflow_federated as tff

# pip install tensorflow-federated
trainer = tff.learning.algorithms.build_weighted_fed_avg(
    tff_model,
    client_optimizer_fn=tff.learning.optimizers.build_sgdm(learning_rate=0.1),
)

state = trainer.initialize()
for round_num in range(10):
    result = trainer.next(state, train_data)
    state  = result.state
    metrics = result.metrics
    print(f"Round {round_num}: "
          f"acc={metrics['client_work']['train']['accuracy']:.4f}")

TF Recommenders, TF Agents, TF Federated

세 가지 문제 모양에 세 가지 특화 스택

Code

Progress

댓글 0