Class Evaluator

java.lang.Object
  extended by Evaluator

public class Evaluator
extends java.lang.Object

This class performs evaluations of the events affilated with a sentence. It is normally instantiated from EvEx. The justification for this class is documented in the main report, chapter "Implementation". Definition: Precision = TP/(FP+TP) = TP/(All guesses) Definition: Recall = TP/(TP+FN) = TP/(All actual events) Definition: F-score = 2*P*R/(P+R)

See Also:
Anto, Syno, GSentence, TQL, XMLWriter, EvEx

Field Summary
 Anto anto
           
 boolean debug_eval
           
 GSentence g
           
 java.util.ArrayList g_events
           
 int g_total
           
 EvEx parent
           
 Syno syno
           
 TQL t
           
 int t_all_correct
           
 int t_cause_correct
           
 int t_cause_total
           
 java.util.ArrayList t_events
           
 int t_theme_correct
           
 int t_theme_total
           
 int t_total
           
 int t_type_correct
           
 int t_type_total
           
 XMLWriter xw
           
 
Constructor Summary
Evaluator(GSentence g, TQL t, EvEx parent, Syno syno, Anto anto, XMLWriter xw)
          Create a new evaluation for the sentence defined by GSentence and TQL.
 
Method Summary
 void echoEvaluation(GEvent ge, Event te)
          Simple method to echo the evaulation to stdout
 void eval()
          Wrapper method to start evaluation.
 void evalAuto()
          Method to evaluate the events in a given sentence.
 int getTypeAhead(java.lang.String t_type, int g)
          Assisting method to determine how far ahead one have to skip to get to a given type in the golden array.
 void hello()
          Simple method for echoing an evaluation.
 boolean intervention(java.lang.String ge, java.lang.String te)
          Method to determine if two strings are "equal".
 boolean isAhead(java.lang.String t_type, int g)
          Assisting method to determine is an event type is ahead in the golden array of events.
 boolean isHit(java.lang.String ge, java.lang.String te)
          Wrapper method to ensure backward compatibility with evalManual(); Invokes intervention();
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

g

public GSentence g

t

public TQL t

g_events

public java.util.ArrayList g_events

t_events

public java.util.ArrayList t_events

parent

public EvEx parent

syno

public Syno syno

anto

public Anto anto

xw

public XMLWriter xw

debug_eval

public boolean debug_eval

g_total

public int g_total

t_total

public int t_total

t_type_correct

public int t_type_correct

t_theme_correct

public int t_theme_correct

t_cause_correct

public int t_cause_correct

t_all_correct

public int t_all_correct

t_type_total

public int t_type_total

t_theme_total

public int t_theme_total

t_cause_total

public int t_cause_total
Constructor Detail

Evaluator

public Evaluator(GSentence g,
                 TQL t,
                 EvEx parent,
                 Syno syno,
                 Anto anto,
                 XMLWriter xw)
Create a new evaluation for the sentence defined by GSentence and TQL.

Parameters:
g - the sentence from GOLD
t - the sentende from TEST
syno - the synonyme class
anto - the antonyme class
xw - the writer for outputting distilled events
Method Detail

eval

public void eval()
Wrapper method to start evaluation.


evalAuto

public void evalAuto()
Method to evaluate the events in a given sentence. It also takes care of writing distilled events to defined xml-files. An (test) event is evaluated only if it matches the (gold) event. If no equality between types are found (lexical equality or syno/anto), the user is queried when --manual is given at command line. When no other possibility for equality is found, the event are positively registeded to the recall score, but does not affect precision. Evaluation results is stored as member variables to be accessed from EvEx later. This method also wraps the now obsolete evalManual()


isHit

public boolean isHit(java.lang.String ge,
                     java.lang.String te)
Wrapper method to ensure backward compatibility with evalManual(); Invokes intervention();

See Also:
intervention(String, String)

echoEvaluation

public void echoEvaluation(GEvent ge,
                           Event te)
Simple method to echo the evaulation to stdout


intervention

public boolean intervention(java.lang.String ge,
                            java.lang.String te)
Method to determine if two strings are "equal". 1: if both string are "null", they are "equal" 2: if one of them are "null", they are not "equal" 3: if they are lexically equal, they are "equal" 4: if they are defined synonymes, they are "equal" 5: if they are defined antonymes, they are not "equal" 6: if --manual is supplied, the user gets to decide if they are "equal" 7: else they are not "equal" If user inputs a decision, it is stored to Anto or Syno for later usage.

Parameters:
ge - the GOLD word
te - the TEST word
See Also:
Syno, Anto

hello

public void hello()
Simple method for echoing an evaluation.


isAhead

public boolean isAhead(java.lang.String t_type,
                       int g)
Assisting method to determine is an event type is ahead in the golden array of events.


getTypeAhead

public int getTypeAhead(java.lang.String t_type,
                        int g)
Assisting method to determine how far ahead one have to skip to get to a given type in the golden array.