iPedia Free Information Center

Information about Natural Language Processing

Natural language processing (NLP) is a subfield of artificial intelligence and computational linguistics. It studies the problems of automated generation and understanding of natural human languages. Natural language generation systems convert information from computer databases into normal-sounding human language, and natural language understanding systems convert samples of human language into more formal representations that are easier for computer programs to manipulate.

Tasks and limitations

In theory, natural language processing is a very attractive method of human-computer interaction. Early systems such as SHRDLU, working in restricted "blocks worlds" with restricted vocabularies, worked extremely well, leading researchers to excessive optimism which was soon lost when the systems were extended to more realistic situations with real-world ambiguity and complexity.

Natural language understanding is sometimes referred to as an AI-complete problem, because natural language recognition seems to require extensive knowledge about the outside world and the ability to manipulate it. The definition of "understanding" is one of the major problems in natural language processing.

Concrete problems

Some examples of the problems faced by natural language understanding systems:
  • The sentences We gave the monkeys the bananas because they were hungry and We gave the monkeys the bananas because they were over-ripe have the same surface grammatical structure. However, in one of them the word they refers to the monkeys, in the other it refers to the bananas: the sentence cannot be understood properly without knowledge of the properties and behavior of monkeys and bananas.
  • A string of words may be interpreted in myriad ways. For example, the string Time flies like an arrow may be interpreted in a variety of ways:
  • The common simile time moves quickly just like an arrow does;
  • measure the speed of flying insects like you would measure that of an arrow (thus interpreted as an imperative) - i.e. (You should) time flies as you would (time) an arrow.;
  • measure the speed of flying insects like an arrow would - i.e. Time flies in the same way that an arrow would (time them).;
  • measure the speed of flying insects that are like arrows - i.e. Time those flies that are like arrows;
  • all of a type of flying insect, "time-flies," collectively enjoys a single arrow (compare Fruit flies like a banana);
  • each of a type of flying insect, "time-flies," individually enjoys a different arrow (similar comparison applies);
  • the magazine, Time, travels straight when thrown
English is particularly challenging in this regard because it has little inflectional morphology to distinguish between parts of speech.
  • English and several other languages don't specify which word an adjective applies to. For example, in the string "pretty little girls' school".
  • Does the school look little?
  • Do the girls look little?
  • Do the girls look pretty?
  • Does the school look pretty?
  • We will often resolve ambiguities in language by the way we place stress on words. The sentence "I never said she stole my money" demonstrates the importance stress can play in a sentence, and thus the inherent difficulty a natural language processor can have in parsing it. Depending on which word the speaker places the stress, this sentence could have several distinct meanings:
  • "I never said she stole my money" - Someone else said it, but I didn't.
  • "I never said she stole my money" - I simply didn't ever say it.
  • "I never said she stole my money" - I might have implied it in some way or other, but I never explicitly said it.
  • "I never said she stole my money" - I said someone else took it, not her.
  • "I never said she stole my money" - I just said she probably borrowed it.
  • "I never said she stole my money" - I said she stole someone else's money.
  • "I never said she stole my money" - I accused her of stealing my elephant, but not my money.

Subproblems

Speech segmentation: In most spoken languages, the sounds representing successive letters blend into each other, so the conversion of the analog signal to discrete characters can be a very difficult process. Also, in natural speech there are hardly any pauses between successive words; the location of those boundaries usually must take into account grammatical and semantical constraints, as well as the context.


Text segmentation: Some written languages like Chinese, Japanese and Thai do not have single word boundaries either, so any significant text parsing usually requires the identification of word boundaries, which is often a non-trivial task.


Word sense disambiguation: Many words have more than one meaning; we have to select the meaning which makes the most sense in context.


Syntactic ambiguity: The grammar for natural languages is ambiguous, i.e. there are often multiple possible parse trees for a given sentence. Choosing the most appropriate one usually requires semantic and contextual information. Specific problem components of syntactic ambiguity include sentence boundary disambiguation.
Imperfect or irregular input
Foreign or regional accents and vocal impediments in speech; typing or grammatical errors, OCR errors in texts.


Speech acts and plans: Sentences often don't mean what they literally say; for instance a good answer to "Can you pass the salt" is to pass the salt; in most contexts "Yes" is not a good answer, although "No" is better and "I'm afraid that I can't see it" is better yet. And for the question "How many students failed the class last year?", "The class was not offered last year" is a better answer than "None".

Statistical NLP

Main article: statistical natural language processing
Statistical natural language processing uses stochastic, probabilistic and statistical methods to resolve some of the difficulties discussed above, especially those which arise because longer sentences are highly ambiguous when processed with realistic grammars, yielding thousands or millions of possible analyses. Methods for disambiguation often involve the use of corpora and Markov models. The technology for statistical NLP comes mainly from machine learning and data mining, both of which are fields of artificial intelligence that involve learning from data.

Major tasks in NLP

Evaluation of natural language processing

The goal of NLP evaluation is to measure one or more qualities of an algorithm or a system, in order to determine if (or to what extent) the system answers the goals of its designers, or the needs of its users. Research in NLP evaluation has received considerable attention, because the definition of proper evaluation criteria is one way to specify precisely an NLP problem, going thus beyond the vagueness of tasks defined only as language understanding or language generation. A precise set of evaluation criteria, which includes mainly evaluation data and evaluation metrics, enables several teams to compare their solutions to a given NLP problem.
  • History of evaluation in NLP
...

Depending on the evaluation procedures, a number of distinctions are traditionally made in NLP evaluation.
  • Intrinsic vs. extrinsic evaluation
Intrinsic evaluation considers an isolated NLP system and characterizes its performance mainly with respect to a gold standard result, pre-defined by the evaluators. Extrinsic evaluation, also called evaluation in use considers the NLP system in a more complex setting, either as an embedded system or serving a precise function for a human user. The extrinsic performance of the system is then characterized in terms of its utility with respect to the overall task of the complex system or the human user.
  • Black-box vs. glass-box evaluation
Black-box evaluation requires one to run an NLP system on a given data set and to measure a number of parameters related to the quality of the process (speed, reliability, resource consumption) and, most importantly, to the quality of the result (e.g. the accuracy of data annotation or the fidelity of a translation). Glass-box evaluation looks at the design of the system, the algorithms that are implemented, the linguistic resources it uses (e.g. vocabulary size), etc. Given the complexity of NLP problems, it is often difficult to predict performance only on the basis of glass-box evaluation, but this type of evaluation is more informative with respect to error analysis or future developments of a system.
  • Automatic vs. manual evaluation
In many cases, automatic procedures can be defined to evaluate an NLP system by comparing its output with the gold standard (or desired) one. Although the cost of producing the gold standard can be quite high, automatic evaluation can be repeated as often as needed without much additional costs (on the same input data). However, for many NLP problems, the definition of a gold standard is a complex task, and can prove impossible when inter-annotator agreement is insufficient. Manual evaluation is performed by human judges, which are instructed to estimate the quality of a system, or most often of a sample of its output, based on a number of criteria. Although, thanks to their linguistic competence, human judges can be considered as the reference for a number of language processing tasks, there is also considerable variation across their ratings. This is why automatic evaluation is sometimes referred to as objective evaluation, while the human kind appears to be more subjective.

Organizations and conferences

Software tools

See also

External links

Resources

Implementations

artificial intelligence (or AI) is "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions which maximizes its chances of success.
..... Click the link for more information.
Computational linguistics is an interdisciplinary field dealing with the statistical and/or rule-based modeling of natural language from a computational perspective. This modeling is not limited to any particular field of linguistics.
..... Click the link for more information.
In the philosophy of language, a natural language (or ordinary language) is a language that is spoken, written, or signed (visually or tactilely) by humans for general-purpose communication, as distinguished from formal languages (such as computer-programming
..... Click the link for more information.
Human–computer interaction (HCI), alternatively man–machine interaction (MMI) or computer–human interaction (CHI) is the study of interaction between people (users) and computers.
..... Click the link for more information.
SHRDLU was an early natural language understanding computer program, developed by Terry Winograd at MIT from 1968-1970. It was written in the Micro Planner and Lisp programming language on the DEC PDP-6 computer and a DEC graphics terminal.
..... Click the link for more information.
The blocks world is one of the most famous planning domains in artificial intelligence. The program was created by Terry Winograd and is a limited domain natural language system that can understand typed commands and move blocks around on a surface.
..... Click the link for more information.
In the field of artificial intelligence, the most difficult problems are informally known as AI-complete or AI-hard, implying that the difficulty of these computational problems is equivalent to solving the central artificial intelligence problem, in other words, making
..... Click the link for more information.
A simile is a comparison of two unlike things, typically marked by use of "like", "as", "than", or "resembles". Common examples are "the fog was thick like pea soup", "she was as quick as a whip", "madder than a bull", etc.
..... Click the link for more information.
time.

One view is that time is part of the fundamental structure of the universe, a dimension in which events occur in sequence, and time itself is something that can be measured.
..... Click the link for more information.
Time (whose trademark is capitalized TIME) is a weekly American newsmagazine, similar to Newsweek and U.S. News & World Report. A European edition (Time Europe, formerly known as Time Atlantic) is published from London.
..... Click the link for more information.
Inflection morphology is a part of the study of linguistics.

To apply an inflection is to change the form of a word so as to give it extra meaning. This extra meaning could be:
  • Number
  • Person
  • Case
  • Gender
  • Tense
  • Mood
  • Aspect

..... Click the link for more information.
Speech segmentation is the process of identifying the boundaries between words, syllables, or phonemes in spoken natural languages. The term applies both to the mental processes used by humans, and to artificial processes of natural language processing.
..... Click the link for more information.
Text segmentation is the process of dividing written text into words or other similar meaningful units. The term applies to mental processes used by humans when reading text, and to artificial processes implemented in computers, which are the subject of natural language processing.
..... Click the link for more information.
Chinese or the Sinitic language(s) (汉语/漢語, Pinyin: Hànyǔ; 华语/華語, Huáyǔ; or 中文, Zhōngwén) can be considered a language or language family.
..... Click the link for more information.
This article contains Japanese text.
Without proper ,
you may see question marks, boxes, or other symbols instead of kanji or kana.

Japanese
日本語
..... Click the link for more information.
Thai}}} 
Official status
Official language of: Thailand
Regulated by: The Royal Institute
Language codes
ISO 639-1: th
ISO 639-2: tha
ISO 639-3: tha

Thai (
..... Click the link for more information.
word sense disambiguation (WSD) is the problem of determining in which sense a word having a number of distinct senses is used in a given sentence. For example, consider the word bass, two distinct senses of which are:
  1. a type of fish
  2. tones of low frequency

..... Click the link for more information.
Syntactic ambiguity is a property of sentences which may be reasonably interpreted in more than one way, or reasonably interpreted to mean more than one thing. Ambiguity may or may not involve one word having two parts of speech or homonyms.
..... Click the link for more information.
Grammar is the study of the rules governing the use of a given natural language, and as such a field of linguistics. Traditionally, grammar included morphology and syntax, in modern linguistics commonly expanded by the subfields of phonetics, phonology, orthography, semantics, and
..... Click the link for more information.
In the philosophy of language, a natural language (or ordinary language) is a language that is spoken, written, or signed (visually or tactilely) by humans for general-purpose communication, as distinguished from formal languages (such as computer-programming
..... Click the link for more information.
Ambiguity is the property of words, terms, notations and concepts (within a particular context) as being undefined, undefinable, or without an obvious definition and thus having an unclear meaning.
..... Click the link for more information.
A parse tree or concrete syntax tree is a tree that represents the syntactic structure of a string according to some formal grammar. A program that produces such trees is called a parser.
..... Click the link for more information.
The introduction to this article provides insufficient context for those unfamiliar with the subject matter.
Please help [ improve the introduction] to meet Wikipedia's layout standards.

..... Click the link for more information.
Sentence boundary disambiguation (SBD) is the problem in natural language processing of deciding where the beginning and ends of sentences are.

External links

  • Search for 'sentence boundary disambiguation' , Google Scholar.

..... Click the link for more information.


Optical character recognition, usually abbreviated to OCR, is the mechanical or electronic translation of images of handwritten or typewritten text (usually captured by a scanner) into
..... Click the link for more information.
The notion speech act is a technical term in linguistics and the philosophy of language. There are several different conceptions of what exactly "speech acts" are.

Speech act as an illocutionary act

Following the usage of, for example, P. F. Strawson and John R.
..... Click the link for more information.
Stochastic, from the Greek "stochos" or "aim, guess", means of, relating to, or characterized by conjecture and randomness. A stochastic process is one whose behavior is non-deterministic in that a state does not fully determine its next state.
..... Click the link for more information.
Probability is the likelihood that something is the case or will happen. Probability theory is used extensively in areas such as statistics, mathematics, science and philosophy to draw conclusions about the likelihood of potential events and the underlying mechanics of
..... Click the link for more information.
Statistics is a mathematical science pertaining to the collection, analysis, interpretation or explanation, and presentation of data. It is applicable to a wide variety of academic disciplines, from the physical and social sciences to the humanities.
..... Click the link for more information.
This article or section may be confusing or unclear for some readers.
Please [improve the article] or discuss this issue on the talk page. This article has been tagged since September 2007.
..... Click the link for more information.


This article is copied from an article on Wikipedia.org - the free encyclopedia created and edited by online user community. The text was not checked or edited by anyone on our staff. Although the vast majority of the wikipedia encyclopedia articles provide accurate and timely information please do not assume the accuracy of any particular article. This article is distributed under the terms of GNU Free Documentation License.
Herod_Archelaus


page counter