abcabdabcabd

Sequitur is a method for inferring compositional hierarchies from strings. It detects repetition and factors it out of the string by forming rules in a grammar. The rules can be composed of non-terminals, giving rise to a hierarchy. It is useful for recognizing lexical structure in strings, and excels at very long sequences.

Craig Nevill-Manning, Google
Ian Witten, University of Waikato, New Zealand

pease porridge hot, pease porridge cold, pease porridge in the pot, nine days old. some like it hot, some like it cold, some like it in the pot, nine days old.

In the beginning, God created the heavens and the earth. And the earth was without form, and void; and darkness was upon the face of the deep. And the Spirit of God moved upon the face of the waters. And God said, Let there be light: and there was light. And God saw the light, that it was good: and God divided the light from the darkness. And God called the light Day, and the darkness he called Night. And the evening and the morning were the first day. And God said, Let there be a firmament in the midst of the waters, and let it divide the waters from the waters. And God made the firmament, and divided the waters which were under the firmament from the waters which were above the firmament: and it was so. And God called the firmament Heaven. And the evening and the morning were the second day. And God said, Let the waters under the heaven be gathered together unto one place, and let the dry land appear: and it was so. And God called the dry land Earth; and the gathering together of the waters called he Seas: and God saw that it was good. And God said, Let the earth bring forth grass, the herb yielding seed, and the fruit tree yielding fruit after his kind, whose seed is in itself, upon the earth: and it was so. And the earth brought forth grass, and herb yielding seed after his kind, and the tree yielding fruit, whose seed was in itself, after his kind: and God saw that it was good. And the evening and the morning were the third day. And God said, Let there be lights in the firmament of the heaven to divide the day from the night; and let them be for signs, and for seasons, and for days, and years: And let them be for lights in the firmament of the heaven to give light upon the earth: and it was so. And God made two great lights; the greater light to rule the day, and the lesser light to rule the night: he made the stars also. And God set them in the firmament of the heaven to give light upon the earth, And to rule over the day and over the night, and to divide the light from the darkness: and God saw that it was good. And the evening and the morning were the fourth day. And God said, Let the waters bring forth abundantly the moving creature that hath life, and fowl that may fly above the earth in the open firmament of heaven. And God created great whales, and every living creature that moveth, which the waters brought forth abundantly, after their kind, and every winged fowl after his kind: and God saw that it was good.

I am Sam I am Sam Sam I am That Sam-I-am That Sam-I-am! I do not like that Sam-I-am Do you like green eggs and ham I do not like them, Sam-I-am. I do not like green eggs and ham. Would you like them Here or there? I would not like them here or there. I would not like them anywhere. I do not like green eggs and ham. I do not like them, Sam-I-am Would you like them in a house? Would you like them with a mouse? I do not like them in a house. I do not like them with a mouse. I do not like them here or there. I do not like them anywhere. I do not like green eggs and ham. I do not like them, Sam-I-am. Would you eat them in a box? Would you eat them with a fox? Not in a box. Not with a fox. Not in a house. Not with a mouse. I would not eat them here or there. I would not eat them anywhere. I would not eat green eggs and ham. I do not like them, Sam-I-am. Would you? Could you? in a car? Eat them! Eat them! Here they are. I woould not , could not, in a car You may like them. You will see. You may like them in a tree? d not in a tree. I would not, could not in a tree. Not in a car! You let me be. I do not like them in a box. I do not like them with a fox I do not like them in a house I do mot like them with a mouse I do not like them here or there. I do not like them anywhere. I do not like green eggs and ham. I do not like them, Sam-I-am. A train! A train! A train! A train! Could you, would you on a train? Not on a train! Not in a tree! Not in a car! Sam! Let me be! I would not, could not, in a box. I could not, would not, with a fox. I will not eat them with a mouse I will not eat them in a house. I will not eat them here or there. I will not eat them anywhere. I do not like them, Sam-I-am. Say! In the dark? Here in the dark! Would you, could you, in the dark? I would not, could not, in the dark. Would you, could you, in the rain? I would not, could not, in the rain. Not in the dark. Not on a train, Not in a car, Not in a tree. I do not like them, Sam, you see. Not in a house. Not in a box. Not with a mouse. Not with a fox. I will not eat them here or there. I do not like them anywhere! You do not like green eggs and ham? I do not like them, Sam-I-am. Could you, would you, with a goat? I would not, could not. with a goat! Would you, could you, on a boat? I could not, would not, on a boat. I will not, will not, with a goat. I will not eat them in the rain. I will not eat them on a train. Not in the dark! Not in a tree! Not in a car! You let me be! I do not like them in a box. I do not like them with a fox. I will not eat them in a house. I do not like them with a mouse. I do not like them here or there. I do not like them ANYWHERE! I do not like green egss and ham! I do not like them, Sam-I-am. You do not like them. SO you say. Try them! Try them! ANd you may. Try them and you may I say. Sam! If you will let me be, I will try them. You will see. Say! I like green eggs and ham! I do!! I like them, Sam-I-am! And I would eat them in a boat! And I would eat them with a goat... And I will eat them in the rain. And in the dark. And on a train. And in a car. And in a tree. They are so goodm so goodm you see! So I will eat them in a box. And I will eat them with a fox. And I will eat them in a house. And I will eat them with a mouse. And I will eat them here and there. Say! I will eat them ANHYWHERE! I do so like green eggs and ham! Thank you! Thank you, Sam-I-am