04_UIMA

• Collection Reader
  • Reader
    • Textual information comes in various formats
      • Plain text, XML, .doc, .pdf, HTML
      • Usually format-specific requirements if not plain text
        • XML/HTML – detagger, pdf/doc – converter
      • Hide format-specific details from downstream components
        • Always represent as CAS!
      • Some formats (e.g. XML) may contain useful meta-data
        • Write annotations of a certain type
• Consumer
  • Writer/ Consumer
    • Write accumulated structure in the desired format (e.g. in DB or text file)
    • While annotators only process single documents, consumers may see the results of the whole document collection

1. Input Data Analysis
2. Result of input data analysis
3. Problem analysis: 
4. Output Analysis

• What is the data like and is converter or detagger required?
• What is the quality of the data?
• Which form of pre-processing is required? Why?
• Is there any internal structure that needs to be preserved?
  • E.g. Question Title split from text by newline, Best Answer part in the beginning?

Result of Input Data Analysis

• Specification of reader component
• Required pre-processing annotators
• Annotation-Types with attributes for preserved structure (if any)

• Pre-Processing
  • Which form of pre-processing is required? Why?
    • Spelling correction
      • Errors can be modeled as
        • “error annotation” with attribute Correction
          • Can represent ranked list of alternatives
        • Corrected view (corrected document in one chunk)
    • Emoticon detection
      • Use list-based lookup Annotator
• Result of Pre-Processing Analysis
  • Specification of pre-processing annotators
  • Annotation-Types with attributes for pre-processing results
• Processing
  • POS Tagging
    • Identify part-of-speech for each word in document)
    • Write your own? Rather not ...
    • Modeling of POS annotations is straightforward
  • Chunking
    • chunk refers to several words that are customarily used together in a fixed expression, such as
      • "in my opinion,"
      • "to make a long story short,"
      • "How are you?"
  • Sentiment Analysis
    • Detect pos./neg. utterances in text
    • Most simple form: List-based lookup of known opinion words
    • More complex form: Also detect targets of opinions.

• Which part of the result do I want?
• In which format?
• Result of output analysis:
  • Specification of output content and its format
  • Specification of consumer

• Productivity
  • UIMA enables easy re-use of components
  • A lot of already existing components
• Flexibility
  • Adapt an application to different input data
    • Only reader needs to be changed
  • Large documents can be split into smaller parts; results are merged later
• Efficiency
  • Shared Resources
    • Annotators sometimes use large resources
      • Parser model, NER model, n-gram statistics, etc…
    • Run one annotator instance per CPU core
    • Use only a single shared instance of the resource
• Scalability
  • Collection Processing Engine (CPE)
    • Reader - single threaded
    • Processing - may run multi-threaded
    • Consumer - single threaded
  • UIMA Asynchronous Scale-Out
    • Local and remote delegates
    • Error handling and recovery
    • Services register with central broker
    • Clients communicate with broker
• Multilinguality
  • View always knows the language of its content
  • Pipelines may select the correct language-specific annotators
    • E.g. different/language-dependent TreeTagger for POS Tagging