|
Matnni qayta ishlash va nutq qismlarini tahlil qilish misoli
|
| bet | 15/182 | | Sana | 19.05.2024 | | Hajmi | 5,69 Mb. | | #244351 |
Bog'liq Python sun\'iy intellekt texnologiyasi Dasrlik 2024Matnni qayta ishlash va nutq qismlarini tahlil qilish misoli:
import nltk
from nltk.tokenize import word_tokenize
from nltk import pos_tag
nltk.download('punkt’)
nltk.download('averaged_perceptron_tagger')
text = "Natural Language Processing is fascinating!"
tokens = word_tokenize(text)
pos_tags = pos_tag(tokens)
print("Tokens:", tokens)
print("POS Tags:", pos_tags)
SpaCy kutubxonasini o‘rnatish:
pip install spacy
Matnni qayta ishlash va ma’lumot olish misoli:
import spacy
nlp = spacy.load("en_core_web_sm")
text = "SpaCy is an amazing NLP library."
doc = nlp(text)
print("Tokens:", [token.text for token in doc])
print("Named Entities:", [(ent.text, ent.label_) for ent in doc.ents])
Hugging Face tomonidan Transformerlar:
Transformers kutubxonasini o‘rnatish:
pip install transformers
Matnni tasniflash vazifasini bajarish uchun oldindan o‘qitilgan BERT modelidan foydalanish misoli:
from transformers import BertTokenizer, BertForSequenceClassification
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertForSequenceClassification.from_pretrained('bert-base-uncased')
text = "Hugging Face transformers make NLP easy!"
inputs = tokenizer(text, return_tensors="pt")
outputs = model(**inputs)
predictions = torch.argmax(outputs.logits, dim=1).item()
print("Predicted class:", predictions)
Ushbu misollar Hugging Face-ning NLTK, spaCy va transformers kutubxonalarining asosiy funktsiyalarini namoyish etadi. Transformers-dan foydalanish uchun siz tanlagan modelga qarab PyTorch yoki Thensorflow-ni o‘rnatishingiz kerak.
Rasa kutubxonasi va Dialogflow platformasidan foydalangan holda chatbotlarni birlashtirish uchun namunaviy kodlarni ko‘rib chiqamiz.
|
| |
