word2vec uses the logistic or sigmoid function to predict if a context word \(c\) is a real context word for a target word \(t\). How can we compute \(P(+| t,c)\)? - word2vec & GloVe Review Assignment - Use this assignment in your class!

word2vec & GloVe Review Assignment

1.What differences would you expect for word embeddings that use a short context window vs. a large context window?

2.What are some problems with count-based methods for representing words?

3.Explain the intuition behind word2vec.

4.Consider the task of learning skip-gram embeddings. Provide 4 positive and 8 negative examples for the target word 'shovel' in the following excerpt: "... I purchased a shovel to rake the leaves in my lawn ..."

5.Consider the weighted unigram frequency formula for negative sampling in word2vec: \(P_\alpha(w) = {{count(w)^\alpha}\over{\sum_{w'} count(w')^\alpha}}\). Why is \(\alpha={3\over4}\) preferred over \(\alpha=1\)?

6.word2vec uses the logistic or sigmoid function to predict if a context word \(c\) is a real context word for a target word \(t\). How can we compute \(P(+| t,c)\)?

7.Compare and contrast CBOW and skip-gram. What are the advantages of each?

8.What's the intuition behind GloVe?

9.How does GloVe handle words that never co-occur together in a training corpus?

10.What are the advantages and disadvantages of GloVe compared to word2vec?

You answeredJUST NOW

Did you like this question?

(Voting helps us personalize your learning experience!)

Instructor solution

Alec KretchJAN 5, 2021, 11:26:08 AM

Was this helpful?

(Voting helps us personalize your learning experience!)

Vector Semantics Slide Deck (2) (pages 13-15)

Jurafsky, D., Martin, J.H. (2019). Vector Semantics, Part II [PowerPoint slides]. Retrieved from https://web.stanford.edu/~jurafsky/slp3/ (Apr 11, 2023)

Page

View full resource

Was this helpful?

(Voting helps us personalize your learning experience!)

Speech and Language Processing: Vector Semantics and Embeddings (pages 18-20)

Jurafsky, D., Martin, J.H.: Speech and Language Processing (3rd ed.). Retrieved from https://web.stanford.edu/~jurafsky/slp3/6.pdf (Apr 22, 2023)

Page

View full resource

Was this helpful?

(Voting helps us personalize your learning experience!)

Think you've got it?

Select one of the following options:

Submit answer

Was this helpful?

(Voting helps us personalize your learning experience!)

Continue review

You may exit out of this review and return later without penalty.