Leveraging Linguistics and Computer Science Notes – Class 11 AI (843)
Prepare smarter with pointwise, well-organized, and CBSE syllabus-aligned Leveraging Linguistics and Computer Science Notes for Class 11 AI (843), carefully desinged to simplify complex concepts, and support effective exam preparation.
Introduction to Natural Language Processing
- Natural Language Processing (NLP) helps machines understand human language.
- NLP breaks sentences into smaller units called tokens (usually words).
- A token is the basic unit used for analyzing text.
- Human language follows grammar but is still messy and ambiguous.
- To handle this complexity, NLP uses steps like:
- Computers handle unstructured data by processing one sentence at a time. (Sentence Segmentation)
- After that, sentences are broken into small units called tokens. (Tokenization)
- Each token (word) is then individually analysed and classified.
- Tokens are organized into a meaningful structure.
- Once structured, NLP can easily understand and process the text.
Emotion Detection and Sentiment Analysis
| Aspect | Emotion Detection | Sentiment Analysis |
| Definition | Identifies distinct human emotion types | Measures the overall sentiment (feeling) |
| Examples | Detecting emotions like anger, happiness, sadness, etc. | Classifying text as positive, negative, or neutral |
| Use Cases | Reading social media posts, customer service chats | Analyzing user ratings, feedback, surveys |
| AI Training | Trained to classify specific emotions | Uses a scale from positive to negative |
| Purpose | Identifying emotional tokens in text | Understanding overall tone of the text |
Classification problem
- A classification problem in NLP occurs when a system must correctly interpret words or sentences that have multiple meanings or ambiguity in language.
- Human language often confuses machines because words can have different meanings depending on the context, which humans understand easily but AI may misinterpret.
- To solve this, AI uses machine learning (supervised learning) and is trained on large datasets with labelled examples.
- However, AI cannot achieve perfect accuracy and may still make errors.
- Therefore, AI systems also provide a confidence value, which shows how certain the system is about its answer.
Chatbots
- Chatbots are software programs designed to simulate conversation with humans using text or voice-based interactions.
- They use Artificial Intelligence (AI), Natural Language Processing (NLP), and Machine Learning to understand user inputs and generate appropriate responses.
- Chatbots can be used on websites, messaging apps, and voice assistants, and they help in areas like customer support, information retrieval, task automation, and entertainment.
- They provide 24/7 availability, answering repetitive questions and helping businesses reduce the need for human effort in routine tasks.
- Chatbots can work with small or large datasets, depending on their purpose, such as handling specific queries (like movie bookings) or analyzing broader information (like social media trends).
| Aspect | Rule-based Chatbots | AI-powered Chatbots |
| Description | Operate on predefined rules and decision trees; respond based on fixed programmed rules | Use NLP and machine learning; also called chat agents or virtual assistants |
| Advantages | Easy to develop and maintain; provide consistent and accurate answers for specific questions | 24/7 availability; personalized interactions; improve efficiency and reduce costs |
| Limitations | Struggle with complex language; cannot go beyond programmed rules | High development cost and resource needs; may show bias; lack transparency; ethical concerns |
| Use Cases | Customer service (FAQs, order updates); guiding users through simple processes | Entertainment and gaming (interactive experiences); finance and banking (queries about accounts and transactions, simple requests) |
Structure of a Chatbot
- A chatbot has two main parts: frontend and backend.
- Frontend – The frontend is the user-facing interface where users send and receive messages. It is simple and user-friendly, but it may not fully understand the context of what the user means beyond the given input.
- Backend – The backend is the main processing system where the chatbot works internally. It runs the application logic and can remember earlier parts of the conversation to maintain context during interaction.
Intent
- An intent is the purpose or goal behind a user’s message to a chatbot.
- It can be thought of as an action (like a verb) that the user wants to perform.
- Examples of intents include: asking for information, filing a complaint, getting directions, or contacting support.
- A chatbot system is designed to handle multiple intents, depending on the needs of the organization.
| Intent | Possible user inputs |
| Open | When do you open? |
| Open | What are your hours? |
| Open | You open now? |
| Open | How late are you open? |
| Open | Can I walk in at 7 pm? |
Entity
- An entity is a noun that represents a person, place, or thing.
- After identifying intents, the chatbot also needs to identify entities to understand the exact context of the user’s request.
- For example, in the question “What are the hours for the Bangalore office?”, the intent is asking for opening hours, and the entity is Bangalore (a place).
- Entities help the chatbot give specific and accurate responses.
| Intent | Possible user inputs | Entities |
| Open | When do you open? | Bangalore |
| Open | What are your hours? | Schedule |
| Open | You open now? | Time |
| Open | How late are you open? | Time |
| Open | And so forth; there are many inputs mapping to this intent. | And so forth; there are many entities implied by this intent. |
Dialog
- A dialog in a chatbot is like a flowchart (IF/THEN structure) that shows how the chatbot responds to user inputs.
- It defines the conversation flow between the user and the chatbot.
- When a user sends a message (even with poor grammar or casual language), the chatbot uses NLP to understand the intent and generate a response.
- A dialog includes:
- Possible user inputs
- Matching chatbot responses
- Next possible replies from the user
- Because conversations can have many possibilities, chatbots simplify this using nodes.
- Each node represents one chatbot response.
- It also contains a list of possible user replies that lead to the next step.
- In chatbot design, multiple different user questions can map to the same response, such as all questions about restaurant timing leading to one answer.
Natural Language Processing – Converting Speech to Text & analysing its intent
- Natural Language Processing (NLP) involves five phases that help machines analyze, categorize, and understand spoken and written language.
- These steps utilize deep neural network-style machine learning techniques to mimic the brain’s ability to process data accurately
Lexical Analysis
- Breaks text into paragraphs, phrases, and words.
- Focuses on structure of words in a language.
- Uses normalization techniques like:
- Stemming (removes suffixes like -ing, -ly, -es, -s)
- Lemmatization (converts words to dictionary form using POS context)
Syntactical Analysis
- Checks grammar, word order, and word relationships.
- Performs sentence structure validation.
- Uses Part of Speech (POS) tagging and Dependency Grammar.
- Invalid sentences are rejected if grammar does not make sense.
Semantic Analysis
- Focuses on meaning of sentences in context.
- Extracts correct and relevant meaning from text.
- Ensures the sentence is understood in a clear and meaningful way.
Discourse Integration
- Understands meaning based on previous sentences or context.
- Resolves references like pronouns and proper nouns.
- Helps interpret words like “it” using earlier context.
Pragmatic Analysis
- Studies meaning in real-life context and communication.
- Answers questions like “who said what to whom?”
- Understands intent, speaker context, and situation of conversation.
Example:
Let us see how each stage works with the help of an example sentence.
“The cat sat on the mat.”
Now, let’s go through the five stages of NLP using this sentence:
| Analysis Step | Description | Example / Outcome |
| 1. Lexical Analysis (Tokenization & Morphological Analysis) | Breaks down the sentence into individual words (tokens) and each word is analysed for its root form using lemmatization or stemming. | Tokens: [“The”, “cat”, “sat”, “on”, “the”, “mat”] |
| 2. Syntactic Analysis (Parsing & Grammar check) | Checks the grammatical structure of the sentence according to formal syntax rules (Subject-Verb-Object structure). | Subject: “The cat”Verb: “sat”Phrase: “on the mat” |
| 3. Semantic Analysis (Meaning Extraction) | Analyses the meaning of individual words and ensures the sentence makes logical sense. | “cat” identified as an animal.”sat” as an action.”mat” as an object. |
| 4. Discourse Integration (Context Understanding) | Considers the context of the sentence within a conversation or paragraph to extract additional meaning. | If previous sentence was “It was raining outside”, the model infers the cat sat inside to avoid rain. |
| 5. Pragmatic Analysis (Real-World Interpretation) | Looks at real-world meaning and intent based on context, speaker, and situation. | In casual conversation: describing a cat’s action.In chatbot: may suggest a comfortable pet mat. |
Applications of NLP
- General Use in Business
- NLP tools help businesses analyze data and discover useful insights.
- They automate time-consuming tasks.
- They help gain a competitive advantage.
- Sentiment Analysis
- Analyzes text like customer reviews, social media posts, and feedback.
- Identifies sentiment as positive, negative, or neutral.
- Helps businesses understand customer opinions and emotions about products or services.
- Voice Assistants
- Examples: Siri, Alexa, Google Assistant.
- Used for tasks like calling, reminders, meetings, alarms, and internet search.
- Works using voice recognition + natural language understanding + NLP.
- Helps connect humans with the internet using speech.
- Still improving in understanding human speech accurately.
- Email Filtering
- Organizes emails into categories like important or spam.
- Separates work-related or useful emails from promotional or unwanted emails.
- Helps manage daily email overload efficiently.
- Document Analysis
- Used in organizations to handle large amounts of data.
- Helps in organizing, storing, and searching documents.
- Goes beyond keyword search by categorizing information based on instructions.
- Supports decision-making in areas like claims and risk management.
- Automatic Summarization
- Summarizes large amounts of data into short, meaningful content.
- Reduces time and effort required for manual reading.
- Can also detect emotional meaning in text.
- Improves efficiency and accuracy of data processing.
Python program for a simple rule-based chatbot
import random
def get_response(user_input):
user_input = user_input.lower()
if “hello” in user_input:
return “Hi there! How can I assist you?”
elif “how are you” in user_input:
return “I’m just a bot, but thanks for asking!”
elif “bye” in user_input:
return “Goodbye! Have a great day!”
else:
return “I’m sorry, I didn’t understand that.”
def main():
print(“Welcome to the Simple Chatbot!”)
print(“Type ‘bye’ to exit.”)
while True:
user_input = input(“You: “)
if user_input.lower() == “bye”:
print(“Chatbot: Goodbye! Have a great day!”)
break
else:
response = get_response(user_input)
print(“Chatbot:”, response
if __name__ == “__main__”:
main()
output
Welcome to the Simple Chatbot!
Type ‘bye’ to exit.
You: hello good afternoon
Chatbot: Hi there! How can I assist you?
You: how are you
Chatbot: I’m just a bot, but thanks for asking!
You: bye
Chatbot: Goodbye! Have a great day!
