Hugging Face Transformers — AutoModel 과 Pipeline

한 패키지, 모든 주요 Transformer architecture

transformers 라이브러리 (Hugging Face) 가 모든 주요 NLP, vision, audio, multimodal Transformer 의 uniform interface. 위에서 아래로 세 abstraction 층:

Pipeline — 한 줄 task-shaped API (pipeline("sentiment-analysis"), pipeline("summarization")). prototype 과 일회성 script 에 best.
Auto* class — AutoTokenizer, AutoModel, AutoModelForSequenceClassification 등. checkpoint 이름에서 architecture 감지하고 옳은 클래스 instantiate. 'model 과 loop 통제 원함' 의 옳은 level.
Specific model class — BertModel, GPT2LMHeadModel 등. Auto layer 가 숨기는 architecture-specific feature 필요할 때만.

Tokenizer + Model — 표준 쌍

모든 Transformers model 이 일치 tokenizer 와 함께. model 이 token ID (와 attention mask) 기대. tokenizer 가 text 를 ID 로. 일치해야 — 같은 checkpoint 이름에서 둘 다 로드.

'head' pattern

각 base model architecture 위 다른 task 위 여러 'head' variant:

AutoModel — base model, hidden state 반환.
AutoModelForSequenceClassification — classification head (logit 와).
AutoModelForTokenClassification — per-token label (NER).
AutoModelForCausalLM — language modeling (GPT-style).
AutoModelForSeq2SeqLM — encoder-decoder (T5-style).
AutoModelForQuestionAnswering — extractive QA (start/end position).

너 task 에 맞는 거 pick; 라이브러리가 옳은 output head 자동 hook.

Code

Pipeline — 작동 코드 가장 빠른 path·python

# pip install transformers
from transformers import pipeline

# Sentiment classification
clf = pipeline("sentiment-analysis")
print(clf("PyTorch makes deep learning feel native."))
# [{'label': 'POSITIVE', 'score': 0.9998}]

# Other tasks — same one-line idiom
summarizer = pipeline("summarization")
qa = pipeline("question-answering")
generator = pipeline("text-generation", model="gpt2")
ner = pipeline("ner", aggregation_strategy="simple")

Auto* — fine-tuning 의 옳은 level·python

import torch
from transformers import AutoTokenizer, AutoModelForSequenceClassification

model_name = "distilbert-base-uncased-finetuned-sst-2-english"
tok = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForSequenceClassification.from_pretrained(model_name)
model.eval()

# Tokenize — the inputs the model expects
inputs = tok(
    ["This tutorial is great!", "I'm bored."],
    padding=True, truncation=True, return_tensors='pt',
)
print(inputs.keys())   # dict_keys(['input_ids', 'attention_mask'])

# Forward
with torch.inference_mode():
    out = model(**inputs)
print(out.logits)
# tensor([[-1.9234, 2.0456], [1.7234, -1.8123]])

# Convert to labels
labels = ['NEGATIVE', 'POSITIVE']
preds = out.logits.argmax(-1)
for text, p in zip(["good", "bad"], preds):
    print(text, '→', labels[p])

base model + 너 head 추가·python

import torch
import torch.nn as nn
from transformers import AutoTokenizer, AutoModel

# Sometimes you want the encoder + a custom head (e.g. multi-task, custom loss)
tok = AutoTokenizer.from_pretrained("bert-base-uncased")
encoder = AutoModel.from_pretrained("bert-base-uncased")

class CustomBertClassifier(nn.Module):
    def __init__(self, encoder, num_classes, dropout=0.1):
        super().__init__()
        self.encoder = encoder
        self.dropout = nn.Dropout(dropout)
        hidden = encoder.config.hidden_size
        self.classifier = nn.Linear(hidden, num_classes)

    def forward(self, input_ids, attention_mask=None):
        out = self.encoder(input_ids=input_ids, attention_mask=attention_mask)
        cls_token = out.last_hidden_state[:, 0]      # [CLS] representation
        cls_token = self.dropout(cls_token)
        return self.classifier(cls_token)

model = CustomBertClassifier(encoder, num_classes=5)

Hugging Face Transformers — AutoModel 과 Pipeline

한 패키지, 모든 주요 Transformer architecture

Tokenizer + Model — 표준 쌍

'head' pattern

Code

External links

Exercise

Progress

댓글 0