Tokens Overview Index
Go Up to Lexical Elements Index
This section contains C++ Token topics. Tokens are word-like units recognized by a language. The compiler recognizes six classes of tokens.
Here is the formal definition of a token:
- punctuator (also known as separator)
As the source code is scanned, tokens are extracted in such a way that the longest possible token from the character sequence is selected. For example,
external would be parsed as a single identifier, rather than as the keyword extern followed by the identifier
See Token Pasting with ## Operator for a description of token pasting.