3.1. Task definition

Different from the general seq2seq models to take a sequence as input, SAN-T2T takes a table as input, which consists of an infobox of various records. We model the table-to-text generation as a language model based on the seq2seq structure. The given table T consists of $n$ field-value pairs and their corresponding descriptions. In the training phase, the input and output of the model are the records $r_{1:n}$ in T and the corresponding reference text $w_{1:t}$ . In the inference phase, only the input $r_{1:n}$ is given, and the output would be determined by the model. Among them, $w_{1:t}$ contains t words $\{{w_1,w_2,w_3, \dots,w_t}\}$ , and the purpose of inference is to generate $\widehat{w}_{1:t}$ that maximizes $P (w_{1:t} \mid r_{1:n} )$ .

(1) \begin{equation} \widehat{w}_{1: t}=\text{argmax}_{w_{1:t}} \prod _{s=1}^{t} P\!\left (w_{s} \mid w_{0: s-1}, r_{1: n}\right ) \end{equation}

We test the SAN-T2T model on three public datasets: WikiBio, RotoWire, and WeatherGov.

WikiBio contains 728,321 Wikipedia biographies. Each sample contains an infobox and the first paragraph of the corresponding biographical text, and each text contains 26.1 words on average.

RotoWire contains 4853 samples, which consist of human-written NBA basketball game summaries aligned with their corresponding box- and line-scores, and each summary text contains 337.1 words on average.

WeatherGov contains 29,528 samples of weather forecast records, and each sample contains 36 records of fixed length, such as temperature and wind speed, and is paired with a description text, which contains 28.7 words on average.

3.2. SAN-T2T

The neural language model SAN-T2T proposed in this paper is based on the selective attention network, which combines gate, attention, and copy mechanism for table-to-text generation. The architecture of SAN-T2T is shown in Figure 1.

Figure 1. Architecture of SAN-T2T.

SAN-T2T could be introduced through three aspects:

1. 1. Position encoding. In the preprocessing phase, we model the fields’ value and location jointly to learn semantic and structural information from the table, as shown in Section 3.3.

2. 2. Field-content selective encoder. We utilize the self-gate mechanism in the encoder to determine the importance of mutual decision-making between different fields. Its architecture is shown in Figure 4. Then attention vector can obtain more information from the source sequence and refine the alignment between the records and text.

3. 3. In the inference phase, a formally simple but effective copy mechanism is applied in SAN-T2T that uses attention vectors exclusively rather than training an additional layer of neural networks. This can significantly alleviate the problem of rare words while reducing the resource consumption of additional training.

3.3. Position encoding

Data records in structured tables usually contain vast field-value pairs, where values consist of sequences of words of unfixed length that correspond to the content of this field. The field values are encoded with word embedding, taking the field embedding, which is determined by the embedding vector of the field and its position in the table, as the key information to model the value. Take the WikiBio dataset as an example, Lebret et al. (Reference Lebret, Grangier and Auli2016) represent field embedding in triples: $ (f_w, p_w^+, p_w^- )$ , where $f_w$ represents word embedding of the field and $p_w^+, p_w^-$ represent its forward and backward positions, respectively. Inspired by this idea, we take the fields and their appearing positions as auxiliary information for input. As shown in Figure 2, (a) represents the content of Penny Ramsey’s Wikipedia infobox, involving her birth and death information and occupation, and (b) is the corresponding preprocessed representation. $r_w=\text{concat}\{f_w, v_w\}$ represents the input, where $v_w$ are the values corresponding to the field. $p_w=\text{concat}\{p_w^+,p_w^-\}$ is the auxiliary position information. $r_w$ and $p_w$ are taken as the joint input of a certain timestep of LSTM unit.

Figure 2. The wiki infobox of Penny Ramsey and the preprocessed representation.

3.4. Field-content selective encoder

The field-content selective encoder contains the field encoder and content selector. The field encoder is designed to encode the records with LSTM network and to get the semantic and structural features of the records. Then the content selector controls the amount of information to be flowed to the decoder, through the gated content-selection algorithm.

3.4.1. Field encoder

The purpose of the field encoder is to encode each input record $r_i$ . LSTM is suitable for processing and predicting related events with long intervals and lags in time series. But unidirectional LSTM can only predict the next output from the previous states. Therefore, we use an improved bi-directional LSTM to enable it to learn the complex structure and long dependencies of the field-value pairs. Following the design in Graves et al. (Reference Graves, Mohamed and Hinton2013), the structure used in SAN-T2T at each timestep is shown in Figure 3 and defined as:

(2) \begin{equation} h_t, c_t=\text{biLSTM}\left (r_t, h_{t-1}\right ) \end{equation}

where $r_t=\text{concat}\{f_t,v_t\}$ is the input at timestep $t$ , and $c_{t}$ and $h_{t}$ are the cell state and hidden state at $t$ , respectively. LSTM can reserve essential information through the cell states and hidden states.

Figure 3. The structure of LSTM unit.

As we mentioned above, in order to learn more about the structure of the table, we take $p_t=\text{concat}\{p_t^+, p_t^-\}$ as the joint input of LSTM. The new cell states and hidden states are calculated as:

(3) \begin{equation} \left (\begin{array}{c} \varphi _{t} \\ \omega _{t} \end{array}\right )=\left (\begin{array}{c} \text{sigmoid} \\ \text{tanh} \end{array}\right )\left (W_{p} p_{t}+b_{p}\right ) \end{equation}

(4) \begin{equation} \acute{c}_{t}=f_{t} \odot c_{t-1}+i_{t} \odot \widetilde{c}_{t}+\varphi _{t} \odot \omega _{t} \end{equation}

where $W_p \in \mathbb{R}^{2n \times 2d_p}$ , $b_p \in \mathbb{R}^{2n}$ are the weight matrix and bias. $\varphi _{t} \in (0, 1 )^n$ determines the amount of structural information to be stored in $c_t$ , and $\omega _{t} \in (\!-\!1, 1 )^n$ retains all the structural information at the current timestep. $f_t$ , $i_t$ , and $\tilde{c}_{t}$ are obtained from the forget gate and input gate. We believe that the improved LSTM can not only learn the hidden semantic information of the long sequence but also preserve its structural information for the field-value pairs, while the structure information keeps a very important role in the decision-making of the attention mechanism in the decoder.

Algorithm 1. Gated Content-Selection Algorithm

Figure 4. Architecture of the gated content selector.

3.4.2. Content selector with gate mechanism

For the different records in the table, we believe that the context of each record is important in determining the others. For example, if a person is a football player, then the other relevant records, such as his playing position and the team he is playing for, etc., should also appear in the description. Puduppully et al. take these into consideration and apply the CS gating mechanism to obtain the new representation in the encoder. We borrow this idea to better capture the latent dependency as Figure 1, replacing the output of the encoder with that of the content selector so that more accurate semantic information can be obtained for a specific record in the decoder. The details are as Algorithm 1, and Figure 4 illustrates the architecture of the content selector. We apply the output of the encoder’s LSTM layer to do a self-gate adaption and control the amount of information that flowed to the following decoder component through the sigmoid function. So the input to the decoder will be covered by the content selector’s result. Similar to the attention mechanism, we first calculate the importance score of each record $\alpha _{i,j}$ , which will be used later to get the dependency vector $d_i$ of this record, and then obtain the new encoder output $\acute{h}_i$ through the content-selection gate.

(5) \begin{equation} \alpha _{i, j}=\text{softmax}\!\left (h_i W_r h_j\right ) \end{equation}

(6) \begin{equation} d_i=\sum _j\alpha _{i, j} h_j \end{equation}

(7) \begin{equation} \tilde{d}_i=\text{sigmoid}\!\left (\!W_d\text{concat}\{h_i, d_i\} \ + \ b_d\right ) \end{equation}

(8) \begin{equation} \acute{h}_i=\tilde{d}_i \odot h_i \end{equation}

where $W_r \in \mathbb{R}^{n \times n}$ , $W_d \in \mathbb{R}^{n \times 2n}$ , and $b_d \in \mathbb{R}^n$ are the learnable weight matrix and bias. The $\acute{d}_i \in (0, 1 )^n$ controls the amount of information that can be obtained from the field-content selective encoder at each timestep in the decoder.

3.5. Descriptive decoder

We mainly use LSTM network in the decoder to learn the semantic information of the context. In the training phase, the encoded word embedding $x_t$ at each timestep, the context vector $z_t$ obtained from the selective encoder, and the previous hidden state $s_{t-1}$ are used as input and then output the conditional probability distribution $P_t$ of the next word $P (w_t \mid w_{0:t-1}, r_{1:n} )$ .

A standard unidirectional and single-layer LSTM network is used in the decoder, and the context vector $z_t$ of each timestep is calculated through the attention mechanism as Bahdanau et al. (Reference Bahdanau, Cho and Bengio2014):

(9) \begin{equation} e_{t, j}=\frac{s_{t-1} \odot \acute{h}_{j}}{\sqrt{d_{n}}} \end{equation}

(10) \begin{equation} \alpha _{t, j}=\frac{\text{exp} \!\left (e_{t, j}\right )}{\sum _{j}\text{exp} \!\left (e_{t, j}\right )} \end{equation}

(11) \begin{equation} z_t=\sum _j\alpha _{t, j}\acute{h}_j \end{equation}

where $s_t$ , $c_t$ are the hidden state and cell state output by the LSTM network.

We integrate the embedding vector of the target sequence word and context vector as input to the LSTM unit and then do the softmax function to calculate each word’s generating probability:

(12) \begin{equation} \acute{x}_t=\text{concat}\{x_t, z_t\} \end{equation}

(13) \begin{equation} s_t, c_t=\text{LSTM}\left (\acute{x}_t, s_{t-1}\right ) \end{equation}

(14) \begin{equation} P\!\left (w_t \mid w_{0:t-1}, r_{1:n}\right )=\text{softmax}\left (W_s s_t + b_s\right ) \end{equation}

(15) \begin{equation} \widehat{w}_{t}=\text{argmax}_{w_{t}}P\left (w_{t} \mid w_{0: t-1}, r_{1: n}\right ) \end{equation}

where $W_s \in \mathbb{R}^{d_{out} \times n}$ and $b_s \in \mathbb{R}^{d_{out}}$ are the weight matrix and bias. $\widehat{w}_{t}$ is the word to be generated at timestep $t$ .

Since the records in some specific datasets (e.g., WeatherGov) are fixed-length rather than variable-length key-value pairs, and there exists no reciprocal location between different records, it does not make much sense to encode each record with the field encoder, respectively. To ensure that the model can still learn the dependency between the output text and records through the content selector without implementing the field encoding, we apply the method proposed by Mei et al. (Reference Mei, Bansal and Walter2016), using the regularization-term-added cross entropy as the loss function:

(16) \begin{equation} \text{Loss}=-\sum _{j=1}^t \log \frac{\text{exp} \!\left (y\left [w_{j}\right ]\right )}{\sum _{j}\text{exp} (y[j])}+\left (\sum _{j=1}^{n} cs_{j}-\gamma \right )^{2}+\left (1-\max \!\left (c s_{j}\right )\right ) \end{equation}

where $cs_j \in (0, 1 )$ is the context vector obtained by content selector, which is calculated by the $\it \textit{sigmoid}$ function, and represents the probability of each record being selected; the prunable parameter $\gamma$ can force the model to depend on the specific number of records. $ (\sum _{j=1}^{n} cs_{j}-\gamma )^{2}$ enables the model to correct its weight based on the content selector and the context vector of the selected record through gradient descent, while $ (1-\max\!(c s_{j} ) )$ can ensure that the content selector has selected at least one record.

3.6. Inference model with copy mechanism

In the training phase, the records $r_{1:n}$ and reference text $w_{1:t}$ are both taken as inputs into the encoder and decoder, respectively, been trained to maximize the likelihood of generating the real text $\widehat{w}_{1:t}$ . However, in the inference phase, text is generated by finding the words with the largest posterior probability predicted by the trained model. In other words, the subsequent words are generated sequentially from the first word.

We also apply beam search to improve the model performance. The strategy is that the model will find a string of generated words that approximately maximizes the conditional probability given the previous states, so it will generate the text from begin-to-end and keep a fixed number (i.e., beam width) of candidates with the highest log-probability at each timestep.

Meanwhile, to alleviate the problem of rare words, a flexible copy mechanism is used to copy OOV words to the output text from the source sequence. This is done by replacing the <unk > character in the output with the most relevant word in the table, which is the field pointed to by the attention vector with $\it \textit{argmax}$ function. Experiments have proved that the copy mechanism we used can greatly improve the performance of the model without increasing additional consumption. As shown in Fig. 1, the model actually outputs <unk> when generating “actress,” but by pointing to the most relevant field “occupation” in the table via the attention vector $\alpha$ , the <unk> is replaced with its value “actress” accordingly.

The specific algorithms of beam search and copy mechanism are as follows:

(17) \begin{equation} w_{s}^{\beta }=k\text{argmax}_{w_{s},{\beta }} P\left (w_{s} \mid r_{1: n}, w_{1}^{\beta }, \ldots, w_{s-1}^{\beta }\right ) \cdot \prod _{i=1}^{s-1} P\left (w_{i}^{\beta } \mid r_{1: n}, w_{1}^{\beta }, \ldots, w_{i-1}^{\beta }\right ) \end{equation}

(18) \begin{equation} \widehat{w}_s=\text{argmax}\left (z_s\right )\text{if} \, w_s = \lt unk\gt \end{equation}

where $w_{s}^{\beta }$ indicates the words in beam $\beta$ at timestep $s$ , and $k\text{argmax}$ is the $\text{argmax}$ function extended to topk scale. When inferencing text, these $k$ words with the highest probability are inputs to the next timestep in turn, until the length of the text exceeds the pre-defined max_length. $z_s$ is the context vector given above. Replace $w_s$ with $\widehat{w}_s$ when the model generates <unk> characters at a certain timestep.

4.1. Datasets and evaluation metrics

In this section, we introduce the datasets we used and the evaluation metrics of the experiment.

4.1.1. Datasets

We used WikiBio (Lebret et al., Reference Lebret, Grangier and Auli2016) as the benchmark dataset, and RotoWire (Wiseman et al., Reference Wiseman, Shieber and Rush2017) and WeatherGov (Liang, Jordan, and Klein, Reference Liang, Jordan and Klein2009) as supplementary datasets to verify the generalization of SAN-T2T. WikiBio contains 728,321 samples from Wikipedia, using the first paragraph of each article as the description of the corresponding infobox, as shown in Table 1. RotoWire contains 4853 samples, which are NBA basketball game summaries aligned with their corresponding box- and line-scores. WeatherGov contains 29,528 samples, each of which has 36 records of fixed length and is paired with a description text. We split WeatherGov dataset according to the proportion of training (80%), development (10%), and test (10%).

Table 1. The Wikipedia infobox of Frederick Parker–Rhodes

The introduction of his biography reads: “Arthur Frederick Parker–Rhodes (21 November 1914–2 March 1987) was an English linguist, plant pathologist, computer scientist, mathematician, mystic, and mycologist.”

4.2. Experiment setup

For WikiBio,Footnote ^a we select the most frequent 20,000 words in the training set to build the vocabulary, and the others will be encoded with <unk>. In the encoder, the value embedding and its corresponding field and position embedding are used as input. The largest position number is limited to 30, that is, fields with lengths longer than 30 will be truncated. <sos> and <eos> are added at the start and end of the target text, respectively. Records exceeding the pre-defined length 100 will be truncated and insufficient ones will be padded with <pad>. We also tried different beam widths to verify the influence of beam search on the model.

Set the dimension of field embedding to 50, word embedding to 400, and position embedding to 5. Hidden layer size is 500, and batch size is 32. We also restrict the generated sentence by the max_length of 60 to avoid redundant or time-consuming generation. For the Transformer model, the hidden units of the multi-head component and the feed-forward layer are both 512, with 8 heads and 6 encoder/decoder layers.

For RotoWire, we use similar settings as WikiBio without implementing position encoding. Set the dimension of field embedding to 30, word embedding to 300. max_length is 700, and max_field is 770.

For WeatherGov, in the encoder, we only need to set the embedding vector of words in the table as input. The word embedding dimension is 300, and hidden layer size is 500. For cross entropy loss, $\gamma =4$ is used to set the regularization term, and we also have experimented on the influence of different $\gamma$ on the model.

Adam optimizer is used to perform gradient descent, the initial learning rate is 0.0003, and the gradient clipping value is 5.0.

4.3. Results and analysis

As shown in Table 2, SAN-T2T is compared with several previous works, including KN, Template KN (Heafield et al., Reference Heafield, Pouzyrevsky, Clark and Koehn2013), NLM, Table NLM (Lebret et al., Reference Lebret, Grangier and Auli2016), Table2Seq-Single (Bao et al., Reference Bao, Tang, Duan, Yan, Zhou and Zhao2019), Structure-aware Seq2Seq (Liu et al., Reference Liu, Wang, Sha, Chang and Sui2018), and Two-level model (Cao, Gong, and Zhang, Reference Cao, Gong and Zhang2019). Meanwhile, we provide a vanilla Seq2Seq model (without position encoding and content selector compared to SAN-T2T) and a Transformer model (Vaswani et al., Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez, Kaiser and Polosukhin2017) to conduct an ablation study.

Table 2. Results on WikiBio (test set)

The boldface values show the highest values in the corresponding column.

According to Table 2, we observe that neural NLM always surpasses the statistical ones, even the vanilla Seq2Seq model can achieve better results than Table NLM, and content selector significantly improves the baselines. The Transformer model based solely on attention mechanisms achieves competitive performance, having the ability that generates buffered positional encoding using sine or cosine functions according to the length of the fields (i.e., no need for additional training). Then, according to the case studies on WikiBio, although SAN-T2T often generates short texts accurately and fluently, there is still large room for improvement for long text generation (e.g., occasionally missing information that counts), which causes the low performance. Transformer performs better through replacing the chain-like forms in LSTM structure in SAN-T2T, no need to worry about exploding gradient when generating longer texts. To learn more information on this impact, we also did experiments on samples with different reference-text-length (we split WikiBio’s test set into five subsets with different lengths, less than 10, between 10 and 20, between 20 and 30, between 30 and 40, and more than 40). As shown in Table 3, samples with text length between 10 and 30 fit the best, and samples with text length more than 40 the worst (samples with less than 10 tokens fit badly on BLEU score, for BLEU metric only calculates precision of the candidate texts without considering recall, and the high score on ROUGE verifies this).

Table 3. Effects of text length on WikiBio (test set)

The boldface values show the highest values in the corresponding column.

Table 4. Effects of different beam width on WikiBio

The boldface values show the highest values in the corresponding column.

Table 4 shows the impact of beam search. The BLEU score degrades as the beam width increases. The reason lies in the search strategy. Increasing the beam width makes the model generates texts that are disproportionately based on those early and non-greedy decisions, typically including words with relatively low probability followed by words with quite high conditional probability, leading to an overall higher probability sentence but lower precision. For example, when the $k$ words in a certain timestep contain the character <eos>, then its subsequent posterior words would likely remain <eos> (i.e., the sentence length will be shortened, for <eos> will be ignored), making the sentence holding a quite high probability but badly poor performance. Increasing the beam width will also significantly reduce the inference speed (about three times slower when setting beam width to 5 in our experiments).

Results also show that the application of copy mechanism can significantly improve the model (the group with copy exceed those without copy about 5 points), and it does not need additional training because there is no need for the redundant copy-network like Gu et al. (Reference Gu, Lu, Li and Li2016) to determine whether words need to be copied at current timestep, for the copied words only come from the attention vector.

After finishing works on WikiBio, we also did experiments on RotoWire and WeatherGov to verify SAN-T2T’s generalization. Table 5 compares SAN-T2T with TEMPL (template system), WS-2017 (Wiseman et al., Reference Wiseman, Shieber and Rush2017), NCP + CC (Puduppully et al., Reference Puduppully, Dong and Lapata2019), Hierarchical-k (Rebuffel et al., Reference Rebuffel, Soulier, Scoutheeten and Gallinari2019), and DATA-TRANS (Li et al., Reference Li, Crego and Senellart2019) on RotoWire. The model DATA-TRANS is the state of the art of RotoWire, which has a Transformer-based model that learns CS and summary generation jointly, and uses two data augmentation methods synthetic data generation and training data selection. The template system has the highest RG precision, for it’s faithful to the infoboxes by design. SAN-T2T is worse than the comparison models, and the reason lies in its weak ability to generate long text and reserve information, which has already been discussed in WikiBio’s results.

Table 5. Results on RotoWire (test set)

Columns indicate relation generation (RG) count (#) and precision (P%), content selection (CS), precision (P%) and recall (R%), count ordering (CO) in normalized Damerau–Levenshtein distance (DLD%), and BLEU. These metrics are described in Section 4.1. The boldface values show the highest values in the corresponding column.

Table 6 compares SAN-T2T with KL (Konstas and Lapata, Reference Konstas and Lapata2013a), MBW (Mei et al., Reference Mei, Bansal and Walter2016), and Two-level model (Cao et al., Reference Cao, Gong and Zhang2019) on WeatherGov. There is no need to apply copy mechanism because its vocabulary size is small and each sample has 36 records of fixed length, but it could also be concluded that beam search has little improvements, which is consistent with Mei et al. (Reference Mei, Bansal and Walter2016).

Table 6. Results on WeatherGov (test set)

The boldface values show the highest values in the corresponding column.

For the experiments on WeatherGov, results in Table 7 show the effect of different $\gamma$ on the model. BLEU score reaches the highest when $\gamma =4$ . After analyzing the original dataset, it can be seen that the most records number aligned by the reference text of all samples is 4, and the role of $\gamma$ is to constrain the number of records that the model aligns. Therefore, the best result at $\gamma =4$ is in line with expectations.

Table 7. Effects of different $\gamma$ on WeatherGov

The boldface values show the highest values in the corresponding column.

4.4. Case study

Three randomly selected samples from WikiBio, WeatherGov, and RotoWire and their descriptions are presented below. We will analyze the strengths as well as the remaining weaknesses of SAN-T2T for these cases.

Analysis of the texts generated for Alonzo Cushing in Tables 8 and 9 shows that SAN-T2T can learn more semantic information and even correct reasoning about that are not included in the input: “u.s. military’s highest decoration” does not appear in the input, but the knowledge learned from other samples (the medal of honor = u.s. military’s highest decoration) completes this information. When applying copy mechanism for the model, the copied word “battery” not only alleviates the problem of rare words but also makes up for some missing information: the ignored field commands appear as “an artillery officer” in the reference text, while SAN-T2T (without copying) outputs “actions at the battle of <unk>.”

Table 8. The Wikipedia infobox of Alonzo H. Cushing

Table 9. The generated descriptions for Alonzo H. Cushing

Table 10. Human Evaluation on WikiBio (test set)

The boldface values show the highest values in the corresponding column.

Human evaluation: For more intuitiveness, we perform human evaluation based on the sentences’ fluency (natural and grammatical) and semantic faithfulness (supported by the records). We defined three levels of fluency as: fluent, mostly fluent, and not fluent, and the same for semantic faithfulness. Three annotators are asked to evaluate on 100 randomly selected samples from the generated and reference sentences. Specifically, we asked participants questions about RG, syntax, coherence, and brevity to estimate the output of the model. Inter-rater agreement follows Fleiss’ kappa (Fleiss, Reference Fleiss1971). For fluency, kappa score $k$ = 0.251 (annotators assign decisions on the same 20 samples with the three categories), for semantic faithfulness, $k$ = 0.334. Results are shown in Table 10. The reference texts achieve the highest fluency and faithfulness, and there is a considerable gap between the SAN-T2T outputs and the human-written sentences, mainly because SAN-T2T’s weakness to generate long texts, often outputting repetitive sentences, and sometimes learning what should be wrong from other samples. Beam search has the ability to improve SAN-T2T’s performance on all of the four measures; the top- $k$ strategy alleviates the disadvantage that greedy search cannot get the global optimum (although with the side effect of requiring much more inference time). SAN-T2T performs almost as well as Transformer. It is also structurally lighter than the previous best model, Transformer, with fewer parameters and faster computing speed. SAN-T2T with Beam Search performs better than Transformer, and the use of Beam Search increases the variety generated by SAN-T2T. Compared to greedy strategy sampling, Beam Search prevents SAN-T2T from failing at one step and then failing at all, resulting in better performance than Transformer.

Tables 11 and 12 are texts generated from the weather data of Northfield on 2009-02-08-1 in WeatherGov.Footnote ^b It can be seen that the text output by SAN-T2T is already very consistent with the reference, but there are still some factual errors (mainly numerical): the approximate range of temperature and wind speed has been increased (but still very close). We believe that this is because the semantic discrimination between different words encoded with one-hot is diminished, so the model’s ability to recognize different numerical tokens weaken. However, due to the large number of numerical inputs in WeatherGov, it is necessary to encode with one-hot rather than word embedding. We encode the numerical tokens to binary-array representation, which is proposed by Mei et al.

Table 11. Weather data for Northfield, Minnesota on 2009-08-1

Table 12. The generated descriptions for Northfield, Minnesota on 2009-08-1

Table 13. The box score and line score for Clippers - Bucks in 2014-21-20

The definition of table headers could be found at https://github.com/harvardnlp/boxscore-data.

Table 14. The generated descriptions for Clippers - Bucks

Tables 13 and 14 are the source records and generated texts from the game between Clippers and Bucks in RotoWire. SAN-T2T performs not well, missing some important information of the source records compared with the reference text. Erroneous includes the city of Bucks (which should be Milwaukee), the percentage of the three-pointers made by Bucks (which should be 33), and wrong inferences (the Bucks lost three straight games, and need to fight against Clippers on Monday). As the text gets longer, the generation quality degrades, that is, generating repetitive sentences, which further impairs the model’s performance.

Figure 5. An example of visualization for the attention. The image above is the attention from Seq2Seq model, while the below one is from SAN-T2T. The vertical axis represents the text generated by the model, while the horizontal represents the fields’ value. -lrb- and -rrb- indicate brackets (). Deeper colors depict a higher attention score.

4.5. Attention visualization

Figure 5 shows an example of a heat-map of the attention vector based on the text generated for John Uzzell Edwards. The sub-figure above comes from the vanilla Seq2Seq model, while the below from SAN-T2T. Both have applied the copy mechanism. The reference text of this sample is “john uzzell edwards (10 October 1934–5 March 2014) was a welsh painter," which consists of the SAN-T2T model. Decisive information is: {nationality: welsh}, {field: painting}, {movement: pure painting}, {awards: granada arts fellowship}. It can be seen that Seq2Seq model generates some duplicate contents (i.e., “pure and pure” aligns to the movement field) and has lost some critical information (e.g., “welsh painter” has been missed, while generating the sentence “pure artist”). However, SAN-T2T generates descriptions that are consistent with the reference text. The fields name, birth_date, death_date, occupation and other information are all obtained from the correct fields. For example, “john uzzell edwards” and “welsh Painter” are from the name field and “welsh pure painting,” respectively. This suggests that the enhanced attention mechanism can model the relationship between records through the content selector and can accurately map the alignment between the generated text and the fields.