helpers.data_processing Namespace Reference

Functions
def	convert_doc_tokens (paragraph_text)
def	_check_is_max_context (doc_spans, cur_span_index, position)
def	convert_example_to_features (doc_tokens, question_text, tokenizer, max_seq_length, doc_stride, max_query_length)
def	read_squad_json (input_file)
def	_get_best_indexes (logits, n_best_size)
def	get_final_text (pred_text, orig_text, do_lower_case)
def	_compute_softmax (scores)
def	get_predictions (doc_tokens, features, results, n_best_size, max_answer_length)

Function Documentation

convert_doc_tokens()

def helpers.data_processing.convert_doc_tokens

(

paragraph_text

)

Return the list of tokens from the doc text 

Here is the caller graph for this function:

_check_is_max_context()

def helpers.data_processing._check_is_max_context (   doc_spans,   cur_span_index,   position  )

private

Check if this is the 'max context' doc span for the token.

Here is the caller graph for this function:

convert_example_to_features()

def helpers.data_processing.convert_example_to_features	(		doc_tokens,
			question_text,
			tokenizer,
			max_seq_length,
			doc_stride,
			max_query_length
	)

Loads a data file into a list of `InputBatch`s.

Here is the call graph for this function:

read_squad_json()

def helpers.data_processing.read_squad_json

(

input_file

)

read from squad json into a list of examples

Here is the call graph for this function:

_get_best_indexes()

def helpers.data_processing._get_best_indexes (   logits,   n_best_size  )

private

Get the n-best logits from a list.

Here is the caller graph for this function:

get_final_text()

def helpers.data_processing.get_final_text	(		pred_text,
			orig_text,
			do_lower_case
	)

Project the tokenized prediction back to the original text.

Here is the caller graph for this function:

_compute_softmax()

def helpers.data_processing._compute_softmax (   scores )

private

Compute softmax probability over raw logits.

Here is the caller graph for this function:

get_predictions()

def helpers.data_processing.get_predictions	(		doc_tokens,
			features,
			results,
			n_best_size,
			max_answer_length
	)

Here is the call graph for this function: