Tokens in C

Token is the smallest individual unit of a C program. This is the building block of a C Program. There are 6 categories under Token.

• Keyword
• Identifiers
• Constants
• Strings
• Special Symbols
• Operators

Keyword
Keywords are the reserved words for doing specific task. These words are predefined and always written in lower case. There are 32 keywords in C programming. Meaning of these keywords are already known to the compiler.

Identifiers
Identifier is the user defined words given to a variable, function, arrays, structures etc. There are certain rules to follow for naming identifiers.
1) Name should only consists of alphabets (both upper and lower case), digits
and underscore (_) sign.
2) First characters should be alphabet or underscore
3) Name should not be a keyword
4) Since C is a case sensitive, the upper case and lower case considered differently
5) Identifier should be up to 31 characters long.

Constants
Constant is any value that cannot be changed during program execution. In C any number, single character, or character string is known as a constant. For example, the number 10 represents a constant integer value.
const int PERSON = 5;
In this case, I have declared a constant integer variable.
Constants are also called as literals. It is best practice to define constants using upper case characters.

Strings
In C, string is referred as an array of characters having null character ‘\0’ at the end of the string. This null character indicates the end of the string. String in c are enclosed with double quotes, where as characters are enclosed with single quotes.
char result[] = “Hello World”;
char name[6]={‘w’,’o’,’r’,’l’,’d’,’\0′};

Special Symbols
There are few special symbols are used while writing C program, which has specific meaning and it cannot be used for some other purpose.

Square Brackets []
This is used to refer the array subscripts. This box is generally used to represent the amount of space that needs to be allocated in the memory.
example: char name[6]={‘w’,’o’,’r’,’l’,’d’,’\0′};

Round Brackets ()
It is used in function declaration, calling the function. While calling the function, we can pass values inside the brackets.
example: printf(“Hello World”);

Curly Brackets {}
This makes the start and end of a block of code.
example:
{
statement 1
statement 2
…..
}
Function, loop statement, if condition uses the curly braces to write block of code.

Comma ,
It is used for separating more than 1 statement. Example: While passing multiple parameter while calling the function, we can use comma symbol.

Hash/Pre-processor: #
It is used for pre processor directive. It is used to distinguish lines which need preprocessing before actual compilation begins.
Any line which begins with # is replaced/omitted with other piece of code and then that code is compiled.
For example when preprocessor encounters a line # include it replaces that line with all the text inside ‘stdio.h’.

Asterisk *
It is used to create a pointer variable. This symbol is also used as an operator for multiplication.
example: int a = 5*4;

Tilde ~
It is used as a destructor to free memory.

Operators
This is a symbol used to perform some operation on variables, operands or with the constant. The data items on which the operators are applied are known as operands. Operators are applied between the operands.
There are several operators.
Arithmetic Operators
Relational Operators
Logical Operators
Assignment Operators
Increment and Decrement
Conditional Operators
Bitwise Operators
Shift Operators

#include <stdio.h>
 
int main()
{
    printf("Hello World");
    int number1 = 10;
    return 0;
}

In the above program, can you identify different types of tokens ?
# : It is a special symbol.
#include is a preprocessor directive. Note: include is not a keyword. so you can use the name include for a variable.
<> is a special symbol.
stdio.h is a file, which comes along with any C compiler.
int is a keyword
main : It is a special function. It is a point at which execution of program starts.
{ } : Special Symbol. This makes the start and end of a block of code
printf is a inuilt function, which is defined in stdio.h file.
Hello World: Is a constant string
number1 is an Identifier.
= It is an assignment operator. int number1 =10; Here, code creates the variable number1 and assigns the value 10 to this variable.
return is a keyword
return 0; This method returns zero. This is the end statement of the function.