Vectors and embeddings
Distinguish IDs, one-hot vectors, and learned embeddings.
- Distinguish IDs, one-hot vectors, and learned embeddings.
An ID is a label. A vector is a list of numbers. A one-hot vector marks just one vocabulary position with 1 and the others with 0; it does not express semantic similarity. An embedding is a learned vector. Training can place related uses near one another, depending on the data and learning objective.
A closer look
Modern Transformers use attention to combine information from positions in the available context, so a token's representation can change with its sentence. Similarity is not truth or identical meaning. The famous king − man + woman analogy is an illustrative result from some word embeddings, not a universal equation. Search compares query and document vectors in a compatible embedding space.
Explain it yourself
Do nearby embeddings prove that two statements say the same true thing?
Compare with an explanation
No. They may share a topic while disagreeing. Read and verify the actual claims.
You mark this yourself. Nothing is graded here.