Instructor solution
Intro to tokenization clip (0:05 - 1:28)
You may exit out of this review and return later without penalty.
Tokenization Review Assignment
1.Tokenize the following sentence:
> Charles ate the french fries knowing they wouldn't be his last meal.
2.Tokenize the following sentence:
> It turns out you don't need all that stuff you insisted you did.
3.Tokenize the following sentence:
> There aren't enough towels in the world to stop the sewage flowing from his mouth.
4.Tokenize the following sentence:
> I am counting my calories, yet I really want dessert.
5.Tokenize the following sentence:
> They got there early, and they got really good seats.
6.Tokenize the following sentence:
> It doesn't sound like that will ever be on my travel list.
7.Tokenize the following sentence:
> As time wore on, simple dog commands turned into full paragraphs explaining why the dog couldn't do something.
8.Tokenize the following sentence:
> Had he known what was going to happen, he would have never stepped into the shower.
9.Tokenize the following sentence:
> After fighting off the alligator, Brian still had to face the anaconda.
10.Tokenize the following sentence:
> I am happy to take your donation; any amount will be greatly appreciated.
11.Tokenize the following sentence:
> I'm a living furnace.
12.Tokenize the following sentence:
> There were white out conditions in the town; subsequently, the roads were impassable.
13.Tokenize the following sentence:
> He spiked his hair green to support his iguana.
14.Tokenize the following sentence:
> She hadn't had her cup of coffee, and that made things all the worse.
15.Tokenize the following sentence:
> There was no ice cream in the freezer, nor did they have money to go to the store.
16.Tokenize the following sentence:
> He wondered why at 18 he was old enough to go to war, but not old enough to buy cigarettes.
17.Tokenize the following sentence:
> With a single flip of the coin, his life changed forever.
18.Tokenize the following sentence:
> I was very proud of my nickname throughout high school but today- I couldn't be any different to what my nickname was.
19.Tokenize the following sentence:
> I currently have 4 windows open up... and I don't know why.
20.Tokenize the following sentence:
> I often see the time 11:11 or 12:34 on clocks.
21.Tokenize the following sentence:
> Honestly, I didn't care much for the first season, so I didn't bother with the second.
22.Tokenize the following sentence:
> You're good at English when you know the difference between a man eating chicken and a man-eating chicken.
23.Tokenize the following sentence:
> The elephant didn't want to talk about the person in the room.
24.Tokenize the following sentence:
> I would have gotten the promotion, but my attendance wasn't good enough.
25.Tokenize the following sentence:
> They looked up at the sky and saw a million stars.
26.Tokenize the following sentence:
> Red is greener than purple, for sure.
27.Tokenize the following sentence:
> Improve your goldfish's physical fitness by getting him a bicycle.
28.Tokenize the following sentence:
> Don't step on the broken glass.
29.Tokenize the following sentence:
> I'd rather be a bird than a fish.
30.Tokenize the following sentence:
> The fog was so dense even a laser decided it wasn't worth the effort.
31.Tokenize the following sentence:
> You had a big fight with Tom last night, didn't you?
32.Tokenize the following sentence:
> My mother looks a lot like my grandmother, and I look very much like them.
33.Tokenize the following sentence:
> She's there.
34.Tokenize the following sentence:
> You ever make to-do lists and put a few things on there you've already done.
35.Tokenize the following sentence:
> It's much bigger than I thought it would be.
36.Tokenize the following sentence:
> We can offer these new products at 20% below list price.
37.Tokenize the following sentence:
> I think I've made a big mistake.
38.Tokenize the following sentence:
> The 3rd time I visited New York was on 1/15/2008.
39.Tokenize the following sentence:
> I'd like to talk about the benefits of having a pet pig.
40.Tokenize the following sentence:
> Everything you've ever wanted is right here.
41.Tokenize the following sentence:
> She promised she would reserve a room for me.
42.Tokenize the following sentence:
> If you had gone there, you would have found him.
43.Tokenize the following sentence:
> I can't find the page you're talking about.
44.Tokenize the following sentence:
> They're a guerilla movement.
45.Tokenize the following sentence:
> I think I'm in big trouble.
46.Tokenize the following sentence:
> I'll get right on it.
47.Tokenize the following sentence:
> Shoes have sizes; I am wearing your size.
48.Tokenize the following sentence:
> We've got a big job ahead of us.
49.Tokenize the following sentence:
> I shouldn't have used the word "password" as my password.
50.Tokenize the following sentence:
> You're on the list.
51.Tokenize the following sentence:
> There's a big pile of mail on your desk.
You may exit out of this review and return later without penalty.