c2b_default

05_Genre_Tagging

abc

abc

9
0.0 (0)
R. P.
Study
R. P.
This flashcard set delves into the university-level topic of genre tagging in computer science, focusing on sequence tagging, classifiers, and machine learning algorithms. It covers key concepts like Markov models, support vector machines, and conditional random fields, along with their applications in text classification and web genre identification. The flashcards are particularly useful for students and researchers in natural language processing and machine learning, providing insights into how to train classifiers, handle different types of features, and apply these techniques to real-world problems like spam detection and content filtering.
Flashcards
9
Students
2
Language
English
Category
Computer Science
Level
University
Created / Updated
07.02.2018 / 07.02.2018
  • 1  / 9

Flashcards

Motivation for Genres

  • Serve specific community
  • Enhance memorability
  • User satisfaction
  • Lack of genre conventions is source of user difficulty in digital world

There is no standard inventory of genres in the web context.

  • typology of genres varies widely in the few corpora that include it

Many paper based conventions have been adopted

  • Advantage:        familiarity to genre
  • Disadvantage:   may fail to support adequate design of new information types
    • E.g. Supermarket brochure as pdf
      • No search function
      • No interactivity
      • No personalization

The characteristic properties of a discourse genre

  • Content                              topic
  • Form                                  layout, design, text structure etc.
  • Function                             communicative purpose etc.

Characteristics of Web Genres

  • Higher complexity in comparison with traditional genres:
    • Hypertext links
    • Interactive features
    • Multimedia
  • Examples of web genres:
    • Personal homepage
    • Blog
    • Search engine
    • Web shop

Applications for Genres in NLP

  • Information Retrieval
    • Filter out irrelevant documents returned by keywords
      • Keywords mostly express the topic of a document
        • politics, sports, football, finance, etc.
      • Genre expresses the type of the text
        • e.g. newspaper article, technical report, PhD thesis, weather report, etc.
  • Information Extraction
    • Identify & extract useful relevant content from web pages
      • Use genre and layout characteristics
  • Information Science
    • Automatic extraction of metadata
    • filter in field-based search masks?
  • Accuracy of NLP tools can be increased if targeted for genres
    • Words and word meanings
    • Certain types of entities
  • Accuracy of Genre Identification can be increased with NLP-based features

Automatic Web Genre Identification

  • Frequent Classification methods
    • Naïve Bayes
      • Training of Naïve Bayes classifiers is very fast
      • Binary, numeric and nominal features can be mixed
      • fails if the independence assumption is violated too much
    • Support Vector Machines
  • Classification features employed (e.g.)
    • function words (or simply the most frequent words)
    • POS trigrams

Supervised Machine Learning 

  • Required
    • A set of web genre labels
    • A corpus annotated with these labels
  • Train the classifier on a training set
  • Test the classifier on a test set
  • Value ranges:
    • binary
    • numeric
    • nominal
  • now the classifier can be used on unseen items to assign the (predefined) classes to them

 

More features is not always better!

  • every feature introduces a parameter that has to be learned
    • the more features the more training data is required
  • some ML algorithms assume features to be independent
    • dependent features lead to deficient models
  • complex features take long to compute
  • some ML algorithms only deal with binary (numeric, nominal) features

Support Vector Machines

Log in to view images.

  • Advantages
    • deals well with high-dimensional, sparse vectors
      •  can convert nominal features into e.g. Boolean representation
    • very flexible
      • different kernel functions, variation in number of support vectors
    • robust classifiers, noise tolerant
    • efficient implementations available
  • Limitations
    • only two classes: must use “1 vs. all” or other schemes for multiclass problems
    • high computational complexity
    • choice of kernel function and its parameters is a trial-and-error enterprise
    • black box

 

Applications of Text Classification

  • Content vs. boilerplates for zoning
  • Spam detection
  • SafeSearch – content filtering
  • Email sorting

Sequence Tagging

  • Words often have more than one POS: back
    • The back door = JJ
    • On my back = NN
  • The sequence tagging problem is to determine the label for each word of a sequence
  • Sequence Classification classifiers
    • Markov Models
      • Conditional Markov Models (CMM) /Maximum-entropy Markov model
        • Combines features from Hidden Markov Models and MaxEnt Models
    • Conditional Random Field
      • Linear Chain Conditional Random Field is simplest form of CRF
        • Every hidden state has 2 neighbors
      • Prof. comment:
        • “CRF pretty much the best even if something else is used CRF layer on top has the best results”
      • Properties
        • Idea: Allow some transitions to vote more strongly than others, depending on the observations
          • CRF solves the label bias problem by normalizing over the whole observation sequence, unlike the CMM
          • like CMM, it is a discriminative exponential model
    • CRF subsumes the advantages of HMM and CMM:
      • per sequence normalization: no label bias problem
      • discriminative: arbitrary features possible 
      • downside: slow training
      • especially for small amounts of training data, CRFs are the framework of choice for modern sequence taggers

Study