Updating, Deleting, and Re-embedding

~20 min · chroma, lifecycle

Level 0Scout

0 XP0/41 lessons0/10 achievements

0/120 XP to next level120 XP to go0% complete

Upsert by id

Use collection.upsert() when re-ingesting. If the id exists, the row is updated; if not, it is inserted. This is the difference between an idempotent ingest pipeline and one that explodes the second time it runs.

Soft-delete by tombstone, hard-delete by id

For audit-friendly systems, mark deleted chunks with metadata {'deleted': True} and exclude them at query time. For production cleanup, collection.delete(ids=[...]) physically removes them.

Re-embedding strategy

Switching embedding models requires rebuilding every vector. Two patterns:

Side-by-side collection. Build docs-bge-m3-v2 in parallel with docs-bge-m3-v1. Switch the read pointer when ready. Delete v1 a week later.
In-place rebuild. Iterate ids, re-embed, upsert. Acceptable on small collections; risky on large because you lose query consistency mid-rebuild.

Always favor side-by-side. The rollback path is just "point back to v1".

Code

Idempotent ingest with upsert·python

docs.upsert(
    ids       =[c['id']       for c in chunks],
    documents =[c['text']     for c in chunks],
    metadatas =[c['metadata'] for c in chunks],
    embeddings=vecs,
)

Side-by-side re-embedding migration·python

src = client.get_collection('docs-bge-m3-v1')
dst = client.get_or_create_collection('docs-bge-m3-v2',
                                      metadata={'hnsw:space': 'cosine'})

batch = src.get(include=['documents', 'metadatas'])
new_vecs = new_model.encode(batch['documents'], normalize_embeddings=True).tolist()

dst.upsert(
    ids       =batch['ids'],
    documents =batch['documents'],
    metadatas =batch['metadatas'],
    embeddings=new_vecs,
)
# Flip the read pointer in your app config when dst.count() == src.count()

External links

Exercise

Build a mini side-by-side migration: 200 chunks in collection v1 with one model, copy them to v2 with a different model, run the same 5 queries against both, and report top-3 overlap. The number you get is your migration risk budget.

Progress

Progress is local-only — sign in to sync across devices.

← PreviousMetadata Filtering with Where Clauses Next →Backup, Restore, and Disaster Drills

Spotted a bug or have feedback on this page?Report an Issue

Comments 0

🔔 Reply notifications (sign in)

No comments yet — be the first.