Adversarial learning

Implicit disentanglement has been instantiated by adversarial learning in several ways. To ensure that the representation found by the encoder is devoid of any style-related information, Fu et al. (Reference Fu, Tan, Peng, Zhao and Yan2018) trained a style classifier adversarially, making it unable to recognize the input attribute, while Lin et al. (Reference Lin, Liu, Sun and Kautz2020) applied adversarial techniques to decompose the latent representation into a style code and a content code, demonstrating the feasibility of a one-to-many framework (i.e., one input, many variants). John et al. (Reference John, Mou, Bahuleyan and Vechtomova2019) inferred embeddings for both content and style from the data, with the help of adversarial loss terms that deterred the content space and the style space from containing information about one another, and with a generator that reconstructed input sentences after the words carrying style were manually removed. Note that, since John et al. (Reference John, Mou, Bahuleyan and Vechtomova2019) approximated content with words that do not bear sentiment information, they could also fit under the group of Explicit Style-Content Disentanglement. We include them here because the authors themselves noted that ignoring sentiment words can boost the transfer, but is not essential.

Backtranslation

A whole wave of research banked on the observation that backtranslation washes out some stylistic traits of texts (Rabinovich et al. Reference Rabinovich, Patel, Mirkin, Specia and Wintner2017) and followed the work of Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Salakhutdinov and Black2018b). There, input sentences were translated into a pivot language and back as a way to manipulate their attributes: the target values were imposed in the backward direction, namely, when decoding the latent representation of the (pivot language) text, thus generating styled paraphrases of the input (in the source language).

Attribute controlled generation

Attribute control proved to be handy to produce style-less representations of the content while learning a code for the stylistic attribute. This emerges, for instance, in Hu et al. (Reference Hu, Yang, Liang, Salakhutdinov and Xing2017), who leveraged a variational auto-encoder and some style discriminators to isolate the latent representation and the style codes, which were then fed into a decoder. While the discriminators elicited the disentanglement, the constraint that the representation of source and target sentence should remain close to each other favored content preservation.

Other methods

An alternative path to disentanglement stems from information theory. Cheng et al. (Reference Cheng, Min, Shen, Malon, Zhang, Li and Carin2020a) defined an objective based on the concepts of mutual information and variation of information as ways to measure the dependency between two random variables (i.e., style and content). On the one hand, the authors minimized the mutual information upper bound between content and style to reduce their interdependency; on the other, they maximized the mutual information between latent embeddings and input sentences, ensuring that sufficient textual information was preserved.

Entangled latent representation editing

Some works edited the latent representations of the input texts learned by an auto-encoder. A common practice in this direction is to jointly train a style classifier and iteratively update the auto-encoder latent representation by maximizing the confidence on the classification of the target attribute (Mueller, Gifford, and Jaakkola Reference Mueller, Gifford and Jaakkola2017; Liu et al. Reference Liu, Fu, Zhang, Pal and Lv2020a). Another approach trained a multi-task learning model on a summarization and an auto-encoding task, and it employed layer normalization and a style-guided encoder attention using the transformer architecture (Wang, Hua, and Wan Reference Wang, Hua and Wan2019a).

Attribute controlled generation

Proven successful by disentanglement-based studies, methods for learning attribute codes were also applied without the content-vs.-style separation. Lample et al. (Reference Lample, Subramanian, Smith, Denoyer, Ranzato and Boureau2019), for instance, employed a denoising auto-encoder together with backtranslation and an averaged attribute embedding vector, which controlled for the presence of the target attribute during generation. Instead of averaging the one-hot encoding for individual attribute values, Smith et al. (Reference Smith, Gonzalez-Rico, Dinan and Boureau2019) used supervised distributed embeddings to leverage similarities between different attributes and perform zero-shot transfer.

Reinforcement learning

Multiple training loss terms have been defined in style transfer to endow the output texts with the three desiderata of content preservation, transfer accuracy, and text naturalness—often referred to as “fluency”. The dependency on differentiable objectives can be bypassed with reinforcement learning, which uses carefully designed training rewards (Luo et al. Reference Luo, Li, Yang, Zhou, Tan, Chang, Sui and Sun2019a, i.a.). Generally, rewards that cope with the presence of the target attribute are based on some style classifiers or discriminators, those pertaining to naturalness rely on language models; and those related to content preservation use Bleu or similar metrics that compare an output text against some reference.

Gong et al. (Reference Gong, Bhat, Wu, Xiong and Hwu2019) worked in a generator-evaluator setup. There, the generator’s output was probed by an evaluator module, whose feedback helped improve the output attribute, semantics, and fluency. Two building blocks can also be found in Luo et al. (Reference Luo, Li, Zhou, Yang, Chang, Sui and Sun2019b). They approached style transfer as a dual task (i.e., source-to-target and target-to-source mappings) in which, to warm-up the reinforcement learning training, a model was initially trained on a pseudo-parallel corpus. Wu et al. (Reference Wu, Ren, Luo and Sun2019a), instead, explored a sequence operation method called Point-Then-Operate, with a high-level agent dictating the text position where the operations should be done and a low-level agent performing them. Their policy-based training algorithm employed extrinsic and intrinsic rewards, as well as a self-supervised loss to model the three transfer desiderata. The model turned out relatively interpretable thanks to these explicitly defined operation steps. Tuning their number, in addition, allowed to control the trade-off between the presence of the initial content and of the target attribute.

An exception among reinforcement learning studies is the cycled reinforcement learning of Xu et al. (Reference Xu, Sun, Zeng, Zhang, Ren, Wang and Li2018), which falls within the disentangling picture.

Probabilistic modelling

Despite being a common practice in unsupervised learning, the definition of task-specific losses can lead to training instability. These objectives are empirically determined among a vast number of possible alternatives. To overcome the issue, He et al. (Reference He, Wang, Neubig and Berg-Kirkpatrick2020) formulated a probabilistic generative strategy that follows objectives defined by some principles of probabilistic inference, and which makes clear assumptions about the data. This approach allowed them to reason transparently about their system design and to outperform many works choosing ad-hoc training objectives.

Data

Works on gender style transfer typically follow the choice of data by Reddy and Knight (Reference Reddy and Knight2016), who used tweets posted in the US in 2013 and some reviews from the YelpFootnote ^e dataset, and inferred gender information from the users’ names.

For this style, there also exists Pastel,Footnote ^f a corpus annotated with attributes of both unintended and intended styles. That is the result of the crowdsourcing effort conducted by Kang, Gangal, and Hovy (Reference Kang, Gangal and Hovy2019), in which $\approx$ 41K parallel sentences were collected in a multimodal setting, and which were annotated with the gender, age, country, political view, education, ethnicity, and time of writing of their authors.

The need to collect attribute-specific rewrites further motivated Xu, Xu, and Qu (Reference Xu, Xu and Qu2019a) to create Alter. As a publicly available toolFootnote ^g , Alter was developed to overcome one major pitfall of crowdsourcing when it comes to generating gold standards: human annotators might fail to associate textual patterns to a gender label, at least when dealing with short pieces of text. Alter facilitates their rewriting tasks (specifically, to generate texts which are not associated with a particular gender) by providing them with immediate feedback.

Methods

Though not focused on transfer, Preoţiuc-Pietro, Xu, and Ungar (Reference Preoţiuc-Pietro, Xu and Ungar2016a) were the first to show that automatic paraphrases can exhibit the style of writers of different ages and genders, by manipulating the lexical choices made by a text generator. A phrase-based translation model learned that certain sequences of words are more typically used by certain age/gender groups and, together with a language model of the target demographics, it used such information to translate tweets from one group to the other. Their translations turned out to perform lexical substitution, a strategy that was more directly addressed by others. Reddy and Knight (Reference Reddy and Knight2016), for instance, performed substitution in order to defeat a gender classifier. They did so with the guidance of three metrics: one measured the association between words and the target gender label, thus indicating the words to replace to fool the classifier as well as possible substitutes; another quantified the semantic and syntactic similarity between the words to be changed and such substitutes; and the last measured the suitability of the latter in context.

A pitfall of such heuristics, noticed by the authors themselves, is that style and content-bearing words are equal candidates for the edit. Some neural methods bypassed the issue with a similar three-step procedure. That is the case of Sudhakar et al. (Reference Sudhakar, Upadhyay and Maheswaran2019), who proposed a variation of the pipeline in Li et al. (Reference Li, Jia, He and Liang2018). There, (1) only style-bearing words are deleted upon the decision of a Bert-based transformer, where an attention head encodes the stylistic importance of each token in a sentence. Next, (2) candidate substitutes are retrieved: sentences from a target-style corpus are extracted to minimize the distance between the content words of the input and theirs. Lastly, (3) the final output is generated with a decoder-only transformer based on Gpt, having learned a representation of both the content source words and the retrieved attribute words. It should be noted that this method was not designed to transfer genre-related attributes specifically (it achieves different results when dealing with other styles). Also, Madaan et al. (Reference Madaan, Setlur, Parekh, Poczos, Neubig, Yang, Salakhutdinov, Black and Prabhumoye2020) addressed gender as an ancillary task. They used a similar methodology (further discussed in Section 5.3 under Politeness) that first identifies style at the word level, and then changes such words in the output.

Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Salakhutdinov and Black2018b), instead, separated content and style at the level of the latent input representation, by employing backtranslation as both a paraphrasing and an implicit disentangling technique. Since machine translation systems are optimized for adequacy and fluency, using them in a backtranslation framework can produce paraphrases that are likely to satisfy at least two style transfer desiderata (content preservation and naturalness). To change the input attribute and comply with the third criterion, the authors hinged on the assumption that machine translation reduces the stylistic properties of the input sentence and produces an output in which they are less distinguishable. With this rationale, a sentence in the source language was translated into a pivot language; encoding the latter in the backtranslation step then served to produce a style-devoid representation, and the final decoding step conditioned towards a specific gender attribute returned a stylized paraphrase.

Modelling content and style-related personal attributes separately are in direct conflict with the finding by Kang et al. (Reference Kang, Gangal and Hovy2019), who pointed out that features used for classifying styles are of both types. As opposed to the studies mentioned above, this work transferred multiple persona styles in conjunction (e.g., education and gender) and did so with a sequence-to-sequence model trained on a parallel dataset. Similarly, the proposal of Liu et al. (Reference Liu, Fu, Zhang, Pal and Lv2020b) did not involve any content-to-style separation. With the aim of making style transfer controllable and interpretable, they devised a method based on a variational auto-encoder that performs the task in different steps. It revises the input texts in a continuous space using both gradient information and style predictors, finding an output with the target attribute in such a space.

Evaluation

While Reddy and Knight (Reference Reddy and Knight2016) carried out a small preliminary analysis, others assessed the quality of the outputs with (at least some of) the three criteria, both with automatic and human-based studies. For instance, Sudhakar et al. (Reference Sudhakar, Upadhyay and Maheswaran2019) evaluated the success of the transfer with a classifier and quantified fluency in terms of perplexity. For meaning preservation, other than Bleu (Kang et al. Reference Kang, Gangal and Hovy2019; Sudhakar et al. Reference Sudhakar, Upadhyay and Maheswaran2019), gender transfer was evaluated automatically with metrics based on n-gram overlaps (e.g., Meteor) and embedding-based similarities between output and reference sentences [e.g., Embedding Average similarity and Vector Extrema of Liu et al. (Reference Liu, Lowe, Serban, Noseworthy, Charlin and Pineau2016), as found in Kang et al. (Reference Kang, Gangal and Hovy2019)].

Sudhakar et al. (Reference Sudhakar, Upadhyay and Maheswaran2019) also explored Gleu as a metric that better correlates with human judgments. Initially a measure for error correction, Gleu fits the task of style transfer because it is capable of penalizing portions of texts changed inappropriately while rewarding those successfully changed or maintained. As for human evaluation, the authors asked their raters to judge the final output only with respect to fluency and meaning preservation, considering the transfer of gender a too challenging dimension to rate. Their judges also evaluated texts devoid of style-related attributes.

Data

So far, this task explored the collection of image captions crowdsourced by Shuster et al. (Reference Shuster, Humeau, Hu, Bordes and Weston2019), who asked annotators to produce a comment for a given image which would evoke a given personality trait. Their dataset Personality-CaptionsFootnote ^h contains 241,858 instances and spans across 215 personality types (e.g., sweet, arrogant, sentimental, argumentative, charming). Note that these variables do not exactly correspond to personality traits established in psychology. As an alternative, one could exploit the corpus made available by Oraby et al. (Reference Oraby, Reed, Tandon, S., Lukin and Walker2018), synthesized with a statistical generator. It spans 88k meaning representations of utterances in the restaurant domain and matched reference outputs which display the Big Five personality traits of extraversion, agreeableness, disagreeableness, conscientiousness, and unconsciousness.Footnote ⁱ

Methods

Cheng et al. (Reference Cheng, Min, Shen, Malon, Zhang, Li and Carin2020a) provided evidence that the disentanglement between the content of a text and the authors’ personality (where personalities are categorical variables) can take place. Observing that such a disentanglement is in fact arduous to obtain, they proposed a framework based on information theory. Specifically, they quantified the style-content dependence via mutual information, i.e., a metric indicating how dependent two random variables are, in this case measuring the degree to which the learned representations are entangled. Hence, they defined the objective of minimizing the mutual information upper bound (to represent style and content into two independent spaces) while maximizing their mutual information with respect to the input (to make the two types of embeddings maximally representative of the original text).

Without complying with any psychological models, Bujnowski et al. (Reference Bujnowski, Ryzhova, Choi, Witkowska, Piersa, Krumholc and Beksa2020) addressed a task that could belong to this node in our hierarchy. Neutral sentences were transferred into “cute” ones, i.e., excited, positive, and slangy. For that, they trained a multilingual transformer on two parallel datasets, one containing paired mono-style paraphrases and the other containing stylized rewritings, for it to simultaneously learn to paraphrase and apply the transfer.

Evaluation

Other than typical measures for style (i.e., style classifiers’ accuracy) and content (Bleu), Cheng et al. (Reference Cheng, Min, Shen, Malon, Zhang, Li and Carin2020a) considered generation quality, i.e., corpus-level Bleu between the generated sentence and the testing data, as well as the geometric mean of these three for an overall evaluation of their system.

Data

Sources that have been used for English varieties are the New York Times and the British National Corpus for English (Lee et al., Reference Lee, Xie, Wang, Drach, Jurafsky and Ng2019). Krishna et al. (Reference Krishna, Wieting and Iyyer2020) employed the corpus of Blodgett, Green, and O’Connor (Reference Blodgett, Green and O’Connor2016) containing African American Tweets, and included this dialectal information in their own datasetFootnote ^j ; as for the diachronic variations that they considered, texts came from the Corpus of Historical American English (Davies Reference Davies2012). Also the Pastel corpus compiled by Kang et al. (Reference Kang, Gangal and Hovy2019) contains ethnic information, which covers some fine-grained labels, like Hispanic/Latino, Middle Eastern, Caucasian, and Pacific Islander. Their resource includes data about the education of the annotators involved in the data creation process, from unschooled individuals to PhD holders.

Methods

Logeswaran, Lee, and Bengio (Reference Logeswaran, Lee and Bengio2018) followed the line of thought that addresses content preservation and attribute transfer with separate losses. They employed an adversarial term to discourage style preservation, and an auto-reconstruction and a backtranslation term to produce content-compatible outputs. Noticing that the auto-reconstruction and backtranslation losses supported the models in copying much of the input, they overcame the issue by interpolating the latent representations of the input and of the generated sentences.

Other methods used for this style are not based on disentanglement techniques (e.g., Kang et al. Reference Kang, Gangal and Hovy2019). Among those is the proposal of Lee et al. (Reference Lee, Xie, Wang, Drach, Jurafsky and Ng2019), who worked under the assumption that the source attribute is a noisy version of the target one, and in that sense, style transfer is a backtranslation task: their models translated from a “clean” input text to their noisy counterpart, and then denoised it towards the target. Krishna et al. (Reference Krishna, Wieting and Iyyer2020) fine-tuned pretrained language models on automatically generated paraphrases. They created a pseudo-parallel corpus of stylized-to-neutral pairs and trained different paraphrasing models in an “inverse” way, that is, each of them learns to recover a stylistic attribute by reconstructing the input from the artificially-created and style-devoid paraphrases. Hence, at testing time, different paraphrasers transferred different attributes (given a target attribute, the model trained to reconstruct it was applied).

Evaluation

Krishna et al. (Reference Krishna, Wieting and Iyyer2020) proposed some variations on the typical measures for evaluation, hinging on an extensive survey of evaluation practices. As for content preservation, they moved away from n-gram overlap measures like Bleu which both disfavors diversity in the output and does not highlight style-relevant words over the others. Instead, they automatically assessed content with the subword embedding-based model by Wieting and Gimpel (Reference Wieting and Gimpel2018). With respect to fluency, they noticed that perplexity might misrepresent the quality of texts because it can turn out low for sentences simply containing common words. To bypass this problem, they exploited the accuracy of a RoBerta classifier trained on a corpus that contains sentences judged for their grammatical acceptability. Moreover, they jointly optimized automatic metrics by combining accuracy, fluency and similarity at the sentence level, before averaging them at the corpus level.

Data

Pryzant et al. (Reference Pryzant, Martinez, Dass, Kurohashi, Jurafsky and Yang2020) released a corpusFootnote ^k of aligned sentences, where each pair consists of a biased version and its neutralized equivalent. The texts are Wikipedia revisions justified by a neutral point of view tag, comprising 180k pre and post revision pairs.

Table 4.

Examples of dynamic states, namely writing time [from Kang et al. (Reference Kang, Gangal and Hovy2019)] and subjective bias [taken from Pryzant et al. (Reference Pryzant, Martinez, Dass, Kurohashi, Jurafsky and Yang2020)]. Note that the former is transferred in combination with other styles (i.e., background)

Methods

With the goal of generating a text that is neutralized, but otherwise similar in meaning to an input, Pryzant et al. (Reference Pryzant, Martinez, Dass, Kurohashi, Jurafsky and Yang2020) introduced two algorithms. One, more open to being interpreted, has two components: a neural sequence tagger that estimates the probability that a word in a sentence is subjectively biased, and a machine translation-based step dedicated to editing while being informed by probabilities about subjectivity. The alternative approach directly performs the edit, with Bert as an encoder and with an attentional Lstm as a decoder leveraging a copy and coverage mechanisms.

Evaluation

The models’ accuracy was equated to the proportion of texts that reproduced the changes of editors. In the human-based evaluation, the success of models was measured with the help of English-speaking crowdworkers who passed preliminary tests proving their ability to identify subjective bias.

Data

There exists a comparably large set of emotion corpora from various domains (Bostan and Klinger Reference Bostan and Klinger2018), but only a small subset has interested style transfer. Among them are Tec, the corpus of Tweets from Mohammad (Reference Mohammad2012), Isear, a collection of descriptions of events that elicited emotional responses in their experiencers (Scherer and Wallbott Reference Scherer and Wallbott1994), and the EmpatheticDialogues datasetFootnote ^l from Rashkin et al. (Reference Rashkin, Smith, Li and Boureau2019), found in Smith et al. (Reference Smith, Gonzalez-Rico, Dinan and Boureau2019), which encompasses a wide range of mental states. A corpus that is not dedicated to emotions but contains them as personality-related labels is the Personality-Caption dataset (Shuster et al., Reference Shuster, Humeau, Hu, Bordes and Weston2019), leveraged by Li et al. (Reference Li, Li, Zhang, Li, Zheng, Carin and Gao2020b).

Concerning emotions and arguments, Chakrabarty et al. (Reference Chakrabarty, Hidey and Muresan2021) collected 301k textual instances from the subreddit Change My View, a forum for persuasive discussions. They created a parallel corpus with the help of a masked language model and a resource that labels nouns and adjectives with their connotations, including the label Emotion Association (Allaway and McKeown Reference Allaway and McKeown2021). The authors matched the words in the arguments they gathered to the entries in such an external dictionary. They masked those which are associated with fear, trust, anticipation and joy, and constrained the replacements proposed by the language model to have a different emotional association than the original one.

A number of other emotion-related datasets could be adopted in the future, which cover different textual domains and follow varied psychological theories. Examples are the 10k English sentences of Buechel and Hahn (Reference Buechel and Hahn2017) labeled with dimensional emotion information in the Valence-Arousal-Dominance schema, the emotion-bearing dialogs of Li et al. (Reference Li, Su, Shen, Li, Cao and Niu2017), and the literary texts made available by Kim, Padó, and Klinger (Reference Kim, Padó and Klinger2017) annotated both with discrete emotions and the communication channels that express them (e.g., description of facial expressions or body movements).

Methods

Being an under-explored task, emotion style transfer was tackled by Helbig et al. (Reference Helbig, Troiano and Klinger2020) with a pipeline transparent for investigation. Subsequent components (1) identify textual portions to be changed, (2) find appropriate new words to perform the lexical substitution, and (3) from the resulting alternatives, pick one depending on its fluency, content preservation and presence of a target attribute. Each step was instantiated with many strategies, like (1) a rule-based identification of words vs. a selection mechanism informed by the attention scores of an emotion classifier, (2) retrieving new words from WordNet vs. leveraging the similarity between input embeddings and those of possible substitutes, (3) re-ranking the outputs with different weights for the three transfer criteria. The approach of Dryjański et al. (Reference Dryjański, Bujnowski, Choi, Podlaska, Michalski, Beksa and Kubik2018) used a neural network to perform phrase insertion, but it is similar to that of Helbig et al. (Reference Helbig, Troiano and Klinger2020) in the idea that specific portions of texts should be targeted for the change.

A filtering step based on re-ranking was also explored in Troiano et al. (Reference Troiano, Klinger and Padó2020), where style transfer is defined as a backtranslation post-processing. The authors leveraged the idea that neural machine translation systems maximize both the output fluency and its faithfulness to the input (thus guaranteeing content preservation and naturalness), and focused on their ability to generate multiple and lexically diverse outputs as a way to promote emotion variability. Hence, with the help of an emotion classifier, they re-ranked backtranslations with respect to their association with the target emotion and to perform the transfer, they selected the text that best fulfilled such a requirement. Similarly, Chakrabarty et al. (Reference Chakrabarty, Hidey and Muresan2021) generated multiple styled rewritings, picking the one with the same meaning as the input—in their case, the one with the highest entailment relation to the original text. Their model was a fine-tuned Bart which learned to generate texts on their parallel data (with the artificially created text being the input and the original argument representing the target). Generation was further controlled by inserting a special separator token as a delimiter for the words that the model needed to edit during fine-tuning.

Though not directly formulated in emotion-related terms, an effort of emotion style transfer can be found in Nangi et al. (Reference Nangi, Chhaya, Khosla, Kaushik and Nyati2021). Therefore, the produced paraphrases display a different degree of excitement than the original texts, mirroring the notion of arousal in the continuous models of emotions. This paper aimed at gaining control over the strength of the transfer by integrating counterfactual logic in a generative model. With a series of losses to promote disentanglement, their variational auto-encoder was trained to find two separate embeddings for style and content. Counterfactuals came into play in the form of a generation loss which guided the model to find a new representation for the input attribute, specifically, a representation that can push the prediction made by a style classifier (given the style embeddings) towards the target attribute.

Evaluation

In a small-scale human evaluation, Helbig et al. (Reference Helbig, Troiano and Klinger2020) defined a best-worst scaling task: two annotators chose the best paraphrase for a given sentence, picking among four alternatives generated from different pipeline configurations.

Consistent with the idea of making arguments more trustworthy, Chakrabarty et al. (Reference Chakrabarty, Hidey and Muresan2021) conducted a human evaluation in which workers on Amazon Mechanical Turk rated arguments with respect to the presence of fear, while simultaneously taking into consideration the preservation of meaning (i.e., a trustworthy text would have been penalized if it altered the input meaning).

Data

A fair share of sentiment-polarized datasets consists of mono-style resources. Commonly used are Yelp reviewsFootnote ^m, Amazon reviewsFootnote ⁿ, and Imdb reviewsFootnote ^o. Arguing that superior performance is observed for any sequence-to-sequence task with parallel data, Cavalin et al. (Reference Cavalin, Vasconcelos, Grave, Pinhanez and Alves Ribeiro2020) employed a semantic similarity measure to derive parallel data from (non-parallel) Amazon and Yelp reviews. Also, Jin et al. (Reference Jin, Jin, Mueller, Matthews and Santus2019) and Kruengkrai (Reference Kruengkrai2019) derived a pseudo-parallel corpus from mono-style data by aligning semantically similar sentences from the sides of the source and target attributes. For a subset of the Yelp reviews, they collected human-generated styled variations.Footnote ^p

Methods

Many approaches that attempted to obtain a sentiment neutralized latent representation of the content (e.g., Hu et al., Reference Hu, Yang, Liang, Salakhutdinov and Xing2017) employed methods like adversarial training (Shen et al., Reference Shen, Lei, Barzilay and Jaakkola2017; Fu et al., Reference Fu, Tan, Peng, Zhao and Yan2018; Zhao et al. Reference Zhao, Kim, Zhang, Rush and LeCun2018b; Fang et al. Reference Fang, Li, Gao, Dong and Chen2019; Lin et al., Reference Lin, Liu, Sun and Kautz2020) and fed this latent representation into a decoder to generate content with the desired polarity. Reinforcement learning-based methods have been adopted for sentiment transfer as well, to bypass the dependency on differentiable learning objectives like loss terms (Gong et al., Reference Gong, Bhat, Wu, Xiong and Hwu2019; Luo et al. Reference Luo, Li, Yang, Zhou, Tan, Chang, Sui and Sun2019a,b). In the cycled reinforcement learning approach of Xu et al. (Reference Xu, Sun, Zeng, Zhang, Ren, Wang and Li2018), a “neutralization” module removed sentiment from the semantic content of a sentence, and an “emotionalization” module introduced the style with the desired attribute in the newly generated text. A policy gradient-based method rewarded the neutralization step using the quality of the generated text from the emotionalization phase.Footnote ^q

Explicit disentanglement by identifying and changing style markers has been claimed effective in sentiment style transfer (Guerini, Strapparava, and Stock Reference Guerini, Strapparava and Stock2008; Whitehead and Cavedon Reference Whitehead and Cavedon2010; Li et al., Reference Li, Jia, He and Liang2018; Xu et al., Reference Xu, Sun, Zeng, Zhang, Ren, Wang and Li2018; Sudhakar et al. Reference Sudhakar, Upadhyay and Maheswaran2019; Leeftink and Spanakis Reference Leeftink and Spanakis2019; Wu et al. Reference Wu, Zhang, Zang, Han and Hu2019b; Lee Reference Lee2020; Madaan et al., Reference Madaan, Setlur, Parekh, Poczos, Neubig, Yang, Salakhutdinov, Black and Prabhumoye2020; Malmi et al. Reference Malmi, Severyn and Rothe2020), because such markers are less subtle compared to those of other styles (e.g., personality traits). Strategies designed to this end use frequency statistics-based methods (Li et al., Reference Li, Jia, He and Liang2018; Madaan et al., Reference Madaan, Setlur, Parekh, Poczos, Neubig, Yang, Salakhutdinov, Black and Prabhumoye2020), sentiment lexica (Wen et al., Reference Wen, Cao, Yang and Wang2020), techniques based on the attention scores of a style classifier (Xu et al., Reference Xu, Sun, Zeng, Zhang, Ren, Wang and Li2018; Zhang et al. Reference Zhang, Xu, Yang and Sun2018b; Sudhakar et al. Reference Sudhakar, Upadhyay and Maheswaran2019; Yang et al. Reference Yang, Lin, Xu, Xie, Su and Sun2019; Reid and Zhong Reference Reid and Zhong2021) or a combination of them (Wu et al. Reference Wu, Zhang, Zang, Han and Hu2019b). The sentiment-devoid content is then used as a template to generate text with the target sentiment. Wu et al. (Reference Wu, Ren, Luo and Sun2019a) achieved this with the contribution of two agents: one that iteratively proposes where the re-wordings should occur in the text, and another that performs such local changes. In Reid and Zhong (Reference Reid and Zhong2021), concurrent edits across multiple spans were made possible by generating a template with the Levenshtein edit operations (e.g., insert, replace, delete) which guided the transformation of the input text towards the desired attribute.

As stated by Yamshchikov et al. (Reference Yamshchikov, Shibaev, Nagaev, Jost and Tikhonov2019), the fact that content and style are hard to separate at the lexical level does not undermine the possibility that they can be separated in their latent representations—with the quality of such disentanglement depending on the used architecture. The machine translation framework of Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Salakhutdinov and Black2018b), already described in relation to genre style transfer (see Section 4.1), aimed at producing a style-devoid representation in the encoding step of the backtranslation. Compared to them, John et al. (Reference John, Mou, Bahuleyan and Vechtomova2019) pushed the disentanglement even further, by dividing such representation in two separate components, that is, a space of sentiment and a space for the content of the sentence (where the content is defined with bag-of-words, style-neutral features). For a given input, an auto-encoder represented the content (but not the style), which was then fed to the decoder, concatenated with an embedding of the desired output attribute. This is similar to Liao et al. (Reference Liao, Bing, Li, Shi, Lam and Zhang2018), who used two encoders to model content and target attribute (a value of the rating of sentences/reviews representing polarity). Claiming that the conditioning structure is essential for the performance of a style transfer model, Lai et al. (Reference Lai, Hong, Chen, Lu and Lin2019) refrained from treating the target attribute simply as part of the initial vector fed to the decoder. Instead, they concatenated the style vector with the output of a Gated Recurrent Unit (Chung et al. Reference Chung, Gulcehre, Cho and Bengio2015) cell at each time step. Style information was implicitly obfuscated at the token level by Lee et al. (Reference Lee, Tian, Xue and Zhang2021) under the assumption that the alternative option of explicit removal of tokens would result in an information loss. They opted for an adversarial strategy, which reversed the attention scores of a style discriminator to obtain a style-devoid content representation, and they applied conditional layer normalization on this representation to adapt it to the target attribute distribution.

In opposition to typical disentanglement-based studies, Yang et al. (Reference Yang, Hu, Dyer, Xing and Berg-Kirkpatrick2018) noticed that classifiers that guide the decoding step towards the desired attribute can be insufficient (their error signal is sometimes too weak to train the generator), and that their presence in adversarial setups as discriminators can lead to unstable optimization. To solve this problem, the authors moved to language models as a different type of discriminator which overcomes the need for adversarial training: a language model trained on the target sentiment data would not only assign low probabilities to outputs that do not contain the desired sentiment, but it would also allow outcome introspection (which word is responsible for such low probability?). In a similar vein, Li et al. (Reference Li, Li, Zhang, Li, Zheng, Carin and Gao2020b) proposed to gradually incorporate the style-conditional supervision signals in the successive training iterations, as long as the output quality does not degenerate. While these studies focused on the semantics of the input and the generated sentences, Gong, Song, and Bhat (Reference Gong, Song and Bhat2020) advocated the need for including the representation of their syntactic information in the transfer process. They encoded a sentence by considering dependency trees (to capture word relations) and structured semantic information (i.e., semantic roles) with the help of a Graph Neural Network (Marcheggiani and Titov Reference Marcheggiani and Titov2017), providing evidence that they can help a model identify the core information to be preserved.

Many limitations of disentanglement were pointed out in other sentiment-based style transfer studies (e.g., using fix-sized vectors for the latent representations might fail to retain the rich semantic information characterizing long texts), with some of them casting doubt on the feasibility of the style-to-content separation (e.g., Jafaritazehjani et al. Reference Jafaritazehjani, Lecorvé, Lolive and Kelleher2020). As an alternative to the manipulation of latent representations, Dai et al. (Reference Dai, Liang, Qiu and Huang2019) added a style embedding as an input to their transformer encoder, while Li et al. (Reference Li, Chen, Lin and Li2020a) directly proposed a novel architecture composed of two generators and no discriminator. They performed style transfer with a sentence noisification approach: after introducing noise to an input text, they found a number of variations and used them to learn the transfer by having the model reconstruct the original input attribute. The novel method proposed by Li, Sun, and Wang (Reference Li, Sun and Wang2021), which did not resort to disentanglement, used a generative adversarial network and a style classifier to regularize the distribution of latent representations from an auto-encoder. Instead, in the generative framework that Guu et al. (Reference Guu, Hashimoto, Oren and Liang2018) presented, a sequence of revisions was produced for some prototype sentences. First, they extracted a prototype from a corpus, next, they sampled an edit vector encoding the edit to be performed: both were fed into the neural editor to produce 1k sequences, and the sequence with the highest likelihood to contain the target attribute was selected.

According to Li et al. (Reference Li, Zhang, Gan, Cheng, Brockett, Dolan and Sun2019), a further problem that researchers should consider is that leveraging data from various domains might result in poor transfer performances. A model learned on movie reviews might not be appropriate to transfer polarity on restaurant reviews. Hence, they presented a domain adaptive approach which modifies sentiment in a domain-aware manner. Others focused on how to leverage pretrained text-to-text models. For instance, Mai et al. (Reference Mai, Pappas, Montero, Smith and Henderson2020) formulated a “plug and play” approach that allows to employ pretrained auto-encoders, and in which the transfer is learned within the latent space of the auto-encoder itself (i.e., embedding-to-embedding). For few-shot style transfer, Riley et al. (Reference Riley, Constant, Guo, Kumar, Uthus and Parekh2021) leveraged the presumably strong textual representations inherent to T5 (Raffel et al. Reference Raffel, Shazeer, Roberts, Lee, Narang, Matena, Zhou, Li and Liu2020). Their encoder-decoder model was trained to reconstruct a corrupted input. Generation was conditioned on a fixed-width style vector (similar to Lample et al. (Reference Lample, Subramanian, Smith, Denoyer, Ranzato and Boureau2019)) extracted from the preceding sentence, assuming that style is a feature which spans over large context windows. At inference time, the stylistic vector was inferred from a set of style transfer exemplar pairs. Interestingly, they demonstrated that a single model trained on generic web data can transfer multiple styles, including dialect, emotiveness, formality, and politeness.

Evaluation

As reported in the analysis of evaluation practices in (sentiment) style transfer by Mir et al. (Reference Mir, Felbo, Obradovich and Rahwan2019), content preservation is typically evaluated in an automatic fashion with metrics devised for machine translation, like Bleu, language models’ perplexity over the generated texts serves as a score for fluency, and sentiment classifiers quantify the transfer strength (i.e., transfer accuracy would be the percentage of output sentences that are classified as belonging to the target attribute). To overcome the limitations of these metrics, they suggested some alternative approaches. In their view, transfer strength is quantified by the Earth Mover’s Distance: observing the cost of turning the style distribution of the input into that of the output (Rubner, Tomasi, and Guibas Reference Rubner, Tomasi and Guibas1998) would acknowledge the transfer even if the output did not properly display the target attribute, but leaned toward it more than the input. With respect to content preservation, the authors experimented with two different settings, i.e., one in which the style-related words coming from a style lexicon were removed and one in which they were masked. Hence, they computed the Word Mover Distance to quantify the distance between the input and output word embeddings (Kusner et al. Reference Kusner, Sun, Kolkin and Weinberger2015). Lastly, naturalness was assessed via adversarial evaluation, with classifiers having to distinguish the input texts written by humans from the output of the generation system.

Mir et al. (Reference Mir, Felbo, Obradovich and Rahwan2019) also proposed some best practices with respect to human evaluation, with the main idea that annotators should be asked to perform pairwise comparisons: by rating the stylistic difference between input and output, by comparing the two after masking their style markers, and by choosing which of them is the most natural.

Yamshchikov et al. (Reference Yamshchikov, Shibaev, Nagaev, Jost and Tikhonov2019) leveraged human productions to propose some measures for the decomposition of textual information into content and styles (they corroborated the idea that better decomposition leads to better Bleu scores between output and human paraphrases). Yet another strategy was put forward by Pang and Gimpel (Reference Pang and Gimpel2019). They quantified content preservation as the average of the cosine similarities over all input/output sentence pairs, and observed perplexity using a language model trained on concatenated source and target attribute datasets. Moreover, they introduced a strategy to adapt to the task at hand which summarizes different metrics into a single score.

Data

A parallel sarcasm corpus, arguably the first of its kind, was introduced by Peled and Reichart (Reference Peled and Reichart2017). These authors crawled tweets with the hashtag “#sarcasm” and used crowdsourcing to generate non-sarcastic alternatives. The resulting dataset includes 3k sarcastic tweets and five non-sarcastic variants for each of them.

Methods

Driven by the idea that sarcastic statements have strong polarized connotations, Peled and Reichart (Reference Peled and Reichart2017) presented a machine translation-based algorithm targeting textual sentiment to “interpret” sarcasm and turn a sarcastic expression into a literal one. Mishra et al. (Reference Mishra, Tater and Sankaranarayanan2019) also leveraged the relation between sarcasm and sentiment, and managed to introduce the figurative-to-literal incongruity using an unsupervised approach with four steps: the first neutralizes the input statement that expresses a negative opinion, by removing the sentiment information with a classifier and a self-attention based filtering—e.g., “Hate when the bus is late” $\rightarrow$ “the bus is late”; next, positive sentiment is injected into the neutralized sentence with a sequence-to-sequence model trained on the neutralized and positive sentence pairs—e.g., “the bus is late” $\rightarrow$ “love when the bus is late”; the third step retrieves a negative-situation phrase fitting the input from their own collection of facts (e.g., canceled at short notice, getting yelled at by people) using an information retrieval system, with the input acting as a query (e.g., “waiting for bus”); and as a last step, the sarcastic statement is synthesized from the positive keywords and negative situation phrases, with a reinforcement reward.

Chakrabarty et al. (Reference Chakrabarty, Ghosh, Muresan and Peng2020a) worked with similar assumptions. Their system first reversed the valence of the input sentence by lexical antonym replacement or negation removal—e.g., “zero visibility in fog makes driving difficult” $\rightarrow$ “zero visibility in fog makes driving easy”. Next, it generated common sense knowledge using Comet (Bosselut et al. Reference Bosselut, Rashkin, Sap, Malaviya, Celikyilmaz and Choi2019), a pretrained language model fine-tuned on the ConceptNet knowledge graph (Speer, Chin, and Havasi Reference Speer, Chin and Havasi2017), by supplying keywords from the input and leveraging the causes relation - e.g., (zero, visibility, fog, driving, difficult) $\rightarrow$ accident. Lastly, this knowledge served to retrieve candidate sentences, which were corrected for grammatical consistency and ranked on a contradiction score, similar to a natural language inference problem.

Evaluation

Standard measures useful to quantify the lexical closeness between a candidate and a reference (Bleu, Rouge, Pinc (Chen and Dolan Reference Chen and Dolan2011)) were reported for automatic evaluations (Peled and Reichart Reference Peled and Reichart2017; Chakrabarty et al. Reference Chakrabarty, Ghosh, Muresan and Peng2020a). In addition, Mishra et al. (Reference Mishra, Tater and Sankaranarayanan2019) presented a metric, the “percentage of length increment”, based on the assumption that sarcasm requires more context than its literal counterpart.

As for the human evaluations, Peled and Reichart (Reference Peled and Reichart2017) collected ratings on the fluency and the adequacy of an interpretation, Mishra et al. (Reference Mishra, Tater and Sankaranarayanan2019) on the fluency and the relatedness to an input, and Chakrabarty et al. (Reference Chakrabarty, Ghosh, Muresan and Peng2020a) on the creativity, level of sarcasm, humor, and grammaticality. Mishra et al. (Reference Mishra, Tater and Sankaranarayanan2019) also had annotators label the sentiment of the transfer outputs.

Data

Ideated to study responses to gender, the corpus of Voigt et al. (Reference Voigt, Jurgens, Prabhakaran, Jurafsky and Tsvetkov2018) has also supported research in political slant transfer. Rt-Gender is a rich multi-genre dataset, with one subset including Facebook posts from the members of the House and Senate in the United States and their top-level responses. The posts include a label indicating if the Congressperson is affiliated with the Republican or the Democratic party. Posts and responses are publicly available,Footnote ^s but all information that could identify the users was removed for privacy.

The RtGender creators claimed that the dataset is controlled for content by nature, because the members of the Congress discuss similar topics. This represents an advantage for style transfer. According to Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Salakhutdinov and Black2018b), what reveals political slant are both topic and sentiment, markedly different for the two affiliations, like in the examples “defund them all, especially when it comes to the illegal immigrants” and “we need more strong voices like yours fighting for gun control” uttered by a republican and a democratic, respectively. Researchers interested in deepening such observation could make use of the dataset released by Mohammad et al. (Reference Mohammad, Zhu, Kiritchenko and Martin2015), as it includes electoral tweets annotated for sentiment, emotion, purpose of the communication (e.g., to agree, disagree, support), and information related to some rhetorical traits (e.g., whether it is sarcastic, humorous, or exaggerated).

To address political opinions more specifically, Chen et al. (Reference Chen, Wachsmuth, Al Khatib and Stein2018) collected 2196 pairs of news article headlines found on the platform all-sides.com, each of which is either left-oriented or right-oriented, depending on the newspapers and portals where they were published.

Methods

As for the stance flipping task addressed by Chen et al. (Reference Chen, Wachsmuth, Al Khatib and Stein2018), the authors started from the observation that not all news headlines are biased enough for a model to learn the task. Hence, they trained a generative model on the body of their articles, whose sentences are not semantically paired. Hence, they reproduced the cross-alignment setting proposed by Shen et al. (Reference Shen, Lei, Barzilay and Jaakkola2017) to transfer sentiment in the absence of parallel data, training two encoders and two decoders (one for each transfer direction).

No other method has been implemented exclusively for this task. The ones that have been applied are the backtranslation frameworks of Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Salakhutdinov and Black2018b) and Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Black and Salakhutdinov2018a) used for sentiment and gender style transfer, which include a separate decoder for each attribute (republican vs. democratic), and the tag-and-generate pipeline proposed by Madaan et al. (Reference Madaan, Setlur, Parekh, Poczos, Neubig, Yang, Salakhutdinov, Black and Prabhumoye2020) in the context of politeness (discussed in the next section).

Evaluation

Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Salakhutdinov and Black2018b) set up a comparison task. Eleven annotators compared the models’ outputs with an input sentence. In line with the definition of the task, they had to choose the paraphrase that maintained the intent of the source sentence, while changing its slant. The annotators also had the option to express no preference for any output. Their results showed that most of the time people did not select any of the outputs, suggesting that state-of-the-art systems still have a long way to go.

Chen et al. (Reference Chen, Wachsmuth, Al Khatib and Stein2018) framed the human evaluation task as one in which annotators judged the degree to which two headlines have opposite bias. Prabhumoye et al. (Reference Prabhumoye, Tsvetkov, Black and Salakhutdinov2018a), instead, refrained from measuring the presence of the target attributes in their human evaluation setting because judgments on political slants can be biased by personal worldviews.

Data

Research on formality transfer has been largely supported by the Grammarly’s Yahoo Answers Formality Corpus (Gyafc).Footnote ^t Introduced by Rao and Tetreault (Reference Rao and Tetreault2018), Gyafc contains around 110K formal/informal sentence pairs, where the informal side was generated via crowdsourcing. Next, the corpus curated by Briakou et al. (Reference Briakou, Lu, Zhang and Tetreault2021c), Xformal,Footnote ^u extended formality data to multiple languages. Like Gyafc, Xformal was built by extracting texts in the topic “family & relationship” from an existing corpus of Yahoo answers. Such texts, which are in Brazilian Portuguese, Italian, and French, were characterized by an informal style. Crowdworkers on the platform Amazon Mechanical TurkFootnote ^v provided multiple formal rewrites for each of them.

Depending on a single dataset might hinder the generalization capability over unseen domains. Hence, by taking Gyafc as ground truth, a few works based on data augmentation methods have created and made available more style transfer instances. The formality classifier of Xu et al. (Reference Xu, Ge and Wei2019b) was trained on Gyafc and made predictions on unlabeled texts; such predictions were filtered for a threshold confidence score of 99.5%. Czeresnia Etinger and Black (Reference Czeresnia Etinger and Black2019) augmented data with the assumption that POS tags are representative of style-independent semantics. After training a classifier on Gyafc, they applied it on a style-unlabelled corpus and created formal–informal sentence pairs, by aligning sentences that become equal as soon as their respective style markers are replaced with the corresponding POS tags.

Zhang et al. (Reference Zhang, Ge and Sun2020b) augmented approximately 4.9M sentence pairs with three techniques: backtranslation, formality discrimination, and multi-task transfer. Backtranslation employed a sequence-to-sequence model trained on parallel data in the formal to informal direction. It was then used to generate 1.6M informal sentences, given formal ones coming from the “entertainment & music” and “family & relationships” domains on Yahoo Answers L6.Footnote ^w Also the formality discrimination method exploited the observation that machine-translated informal texts can be rendered more formal: a number of informal English sentences from Yahoo Answers L6 were translated to different pivot languages and then back, followed by a discriminator with a predefined threshold that further filtered the augmented data, giving a total of 1.5M pairs. While these two strategies used the newly generated texts to augment data, the multi-task transfer method relied on sentence pairs annotated from previous tasks. For that, style transfer was formulated as a problem of Grammatical Error Correction under the assumption that informal sentences are prone to containing grammatical errors, character repetitions, spelling mistakes, unexpected capitalization, and so on. Accordingly, to improve the transfer of formality, they used the training data points for the Grammatical Error Correction task as augmented texts, namely, the Gec data (Mizumoto et al. Reference Mizumoto, Komachi, Nagata and Matsumoto2011; Tajiri, Komachi, and Matsumoto Reference Tajiri, Komachi and Matsumoto2012) and the Nucle corpus (Dahlmeier, Ng, and Wu Reference Dahlmeier, Ng and Wu2013).

Different from such resources, the Enron-Context corpus released by Cheng et al. (Reference Cheng, Gan, Zhang, Elachqar, Li and Liu2020b) contains paragraph-level data. It includes emails randomly sampled from the Enron dataset (Klimt and Yang Reference Klimt and Yang2004), in which sentences identified as informal by human annotators were rewritten in a more formal manner.

Methods

The availability of a relatively large parallel dataset has made formality transfer a go-to task. Rao and Tetreault (Reference Rao and Tetreault2018) spurred extensive research, benchmarking the performance of phrase-based and neural machine translation for this style. Following their work, Ge et al. (Reference Ge, Zhang, Wei and Zhou2019) performed style transfer on the Gyafc corpus as a problem of grammatical error correction.

Others have moved the challenge of formality transfer into a multi-lingual setting: Niu et al. (Reference Niu, Rao and Carpuat2018) opted for a multi-task learning approach to jointly perform monolingual transfer and multilingual formality-sensitive machine translation; Briakou et al. (Reference Briakou, Lu, Zhang and Tetreault2021c) leveraged machine translation for inter-language style transfer, learned both in a supervised and unsupervised manner. The translation model of Yang and Klein (Reference Yang and Klein2021) conditioned the output translation towards formality with the help of future discriminators. These consisted in some style predictors operating on an incomplete text sequence, which inform as to whether the desired attribute will hold for the complete text sequence, and can thus help adjust the generators’ original probabilities.

Many solutions were motivated by the need for massive amounts of parallel data to prevent overfitting in machine translation models. Among them are data augmentation attempts, like those by Czeresnia Etinger and Black (Reference Czeresnia Etinger and Black2019) and Zhang et al. (Reference Zhang, Ge and Sun2020b). The latter employed augmented texts to pretrain models, but acknowledging that such texts are less than perfect, the models were subsequently fine-tuned on the original natural data. Xu et al. (Reference Xu, Ge and Wei2019b) augmented data with a formality classifier. They trained a transformer model on a parallel corpus with each instance prefixed with a token to indicate the direction of transfer, such that a single model could go from formal to informal and vice versa.

This was also achieved by Wang et al. (Reference Wang, Wu, Mou, Li and Chao2020), a work belonging to the line of research that leverages pretrained language models. A sequence-to-sequence model with a single encoder captured the style-independent semantic representations with auxiliary matching losses, and two decoders were dedicated to each target attribute, jointly trained for bi-directional transfer. In Chawla and Yang (Reference Chawla and Yang2020), a pretrained language model-based discriminator helped to maximize the likelihood of the target attribute being in the output, and a mutual information maximization loss between input and output supported diversity in generation. Lai, Toral, and Nissim (Reference Lai, Toral and Nissim2021) worked on the parallel texts from Gyafc to fine-tune large pretrained language models, Gpt-2 (Radford et al. Reference Radford, Wu, Child, Luan, Amodei and Sutskever2019) and Bart (Lewis et al. Reference Lewis, Liu, Goyal, Ghazvininejad, Mohamed, Levy, Stoyanov and Zettlemoyer2020) and augmented them with rewarding strategies based on style discriminators (targeting the transfer of the attributes) and Bleu (targeting content preservation). They argued that pretrained models contribute to better content preservation, even with limited training data.

Wang et al. (Reference Wang, Wu, Mou, Li and Chao2019b) transformed informal sentences into formal ones in a rule-based fashion, with some transfer rules incorporated in their language model. The encoder was presented with an input as a concatenation of the original informal sentence and its formal revision to mitigate the consequent problem of noisy parallel data. Yao and Yu (Reference Yao and Yu2021) explored a similar architecture. The encoder’s input was created by concatenating the original sentence and additional information, comprising a list of all matched rules and the corresponding text alternatives, arranged as tuples. Keeping all rules in the input allowed the model to identify which ones to use dynamically.

Other approaches in formality transfer that circumvented the use of parallel corpora were reinforcement learning (Xu et al. Reference Xu, Ge and Wei2019b) and probabilistic modelling (He et al., Reference He, Wang, Neubig and Berg-Kirkpatrick2020). The work by Cheng et al. (Reference Cheng, Gan, Zhang, Elachqar, Li and Liu2020b) stands out in this panorama, in that it alters the formality of sentences while simultaneously considering the topic coherence to the text surrounding them. The context-aware model they proposed employs one decoder that translates the joint features from two separate encoders (which represent the main sentence and its contextual paragraph, respectively).

Evaluation

Outside NLP, researchers have used measurements based on diagnostic linguistic features to quantify the formality of text. A popular measure is the F-score (formality score) which is sensitive to the frequencies of different word classes in text, ranging from articles and pronouns to adjectives and interjections (Heylighen and Dewaele Reference Heylighen and Dewaele1999). There also exists a composite score that measures formality: Defined by Graesser et al. (Reference Graesser, McNamara, Cai, Conley, Li and Pennebaker2014), it is based on five principal component dimensions of Coh-Metrix,Footnote ^x and it takes into account syntax, discourse, and goals of communication (e.g., syntactic simplicity, referential cohesion, word concreteness, narrativity).

Style transfer studies have never opted for these measures. Indeed, while Rao and Tetreault (Reference Rao and Tetreault2018) raised the issue that the evaluation of style transfer (both human and automatic) is in need for best practices, formality transfer has insisted on evaluating the transfer accuracy with a style classifier, in line with other styles.

Data

The transfer task in Madaan et al. (Reference Madaan, Setlur, Parekh, Poczos, Neubig, Yang, Salakhutdinov, Black and Prabhumoye2020) is restricted to action-derivatives (e.g., “Let’s stay in touch”) which are rewritten as polite requests (e.g., “Can you call me when you get back?”). As these constructs are frequent in official communication, the authors built a politeness dataset starting from a collection of emails exchanged within the Enron corporation, contained in the Enron corpus (Klimt and Yang Reference Klimt and Yang2004). With the application of some filtering heuristics, 1.39 million sentences were gathered, annotated, and filtered with a politeness score assigned by a classifier. This dataset is open sourceFootnote ^y and includes both the texts and the politeness scores.

Politeness labels are also present in the resource of Danescu-Niculescu-Mizil et al. (Reference Danescu-Niculescu-Mizil, Sudhof, Jurafsky, Leskovec and Potts2013). Included in the collection of styled corpora from Kang and Hovy (Reference Kang and Hovy2021), it encompasses 10k requests produced in the context of Wikipedia edits and other administrative functions, as well as Stack Exchange, where requests are related to a variety of topics. Their work focused on the politeness markers of requests, characterized by strategies that minimize imposition through indirect phrases (e.g., “Could you please $\ldots$ ”) or apologies (e.g., “I’m sorry, but $\ldots$ ”).

Method

The task was introduced by Madaan et al. (Reference Madaan, Setlur, Parekh, Poczos, Neubig, Yang, Salakhutdinov, Black and Prabhumoye2020). Observing the complex, socio-cultural nature of politeness, these authors limited their study to the use of formal language among North American English speakers. They defined impoliteness as a lack of politeness markers and adopted a tag-and-generate approach. The linguistic realizations of the potential marker positions were tagged in the source sentence and the target attribute markers were then generated in such positions.

Reid and Zhong (Reference Reid and Zhong2021), who tested their method on the same dataset, introduced an unsupervised explicit disentanglement procedure. First, it transformed input texts into style-agnostic templates thanks to the attention scores of a style classifier; then, it filled the tagged positions in the templates using fine-tuned pretrained language models. Unlike other infilling methods for style transfer (Wang et al. Reference Wang, Wu, Mou, Li and Chao2019b; Malmi et al. Reference Malmi, Severyn and Rothe2020), theirs allowed concurrent edits over multiple textual spans.

Evaluation

For the automatic evaluation of transfer accuracy, Madaan et al. (Reference Madaan, Setlur, Parekh, Poczos, Neubig, Yang, Salakhutdinov, Black and Prabhumoye2020) calculated the percentage of generated sentences on which a classifier recognized the target attribute. For human evaluation, their annotators judged the match with the target attribute on a 5-point scale.

Data

Weller et al. (Reference Weller, Fulda and Seppi2020) used the Humicroedit dataset (Hossain, Krumm, and Gamon Reference Hossain, Krumm and Gamon2019), a resource where crowdworkers made single word edits to render a regular news headline more humorous (e.g., “Meet the wealthy donors pouring millions into the 2018 elections” $\rightarrow$ “Meet the wealthy sadists pouring millions into the 2018 elections”). Humicroedit contains around 15k edited headlines. A similar corpus was presented by West and Horvitz (Reference West and Horvitz2019). It was curated using an online game by asking participants to edit a humorous headline and make it sound serious. Not evaluated to date, this dataset could be useful for future research.

Additional data can be found in the Captions corpus (Gan et al. Reference Gan, Gan, He, Gao and Deng2017), which provides humorous captions describing images. Romantic and factual labels are also present as attributes opposite to humorous. Instead, researchers who prefer to treat “non-humorous” as such opposite could make use of the Short Text Corpus for Humor DetectionFootnote ^aa and the Short Jokes DatasetFootnote ^ab indicated by Kang and Hovy (Reference Kang and Hovy2021). These authors also provided a small sample of texts (2k instances) which allow to consider personal romanticism as a style on its own, with the two attributes “romantic” and “non-romantic”.

Method

Weller et al. (Reference Weller, Fulda and Seppi2020) did an exploratory investigation of the usability of the Humicroedit humor-based corpus for style transfer purposes. A transformer-based sequence-to-sequence model was trained for humor generation and a random POS tag replacement was taken as a baseline.

As humor is not the main focus of the other works mentioned above, we refer the reader to their respective discussions, under formality and sentiment.

Evaluation

Weller et al. (Reference Weller, Fulda and Seppi2020) conducted a human-based evaluation regarding the fluency and the level of humor of texts, which were rated on a 5-point scale. The authors reported that the manually edited sentences were considered more humorous than the machine-generated ones, which in turn were better than random replacements. This positively asserted the potential for the humor generation task, highlighting at the same time the subjectivity of the phenomenon in question. A similar conclusion was drawn by Amin and Burghardt (Reference Amin and Burghardt2020). Focusing on the broader task of humor generation, they analyzed possible evaluation approaches: human ratings on a Likert scale for humorousness, human ratings on a Likert scale for the likeness that a humorous text was written by a human—the soft Turing test as in Yu, Tan, and Wan (Reference Yu, Tan and Wan2018)—and “humorous frequency” as the proportion of funny instances out of a set of generated texts. All of them failed to present a criterion to evaluate humor in text objectively.

Data

To overcome the lack of parallel data, dos Santos et al. (Reference dos Santos, Melnyk and Padhi2018) opted to create a non-parallel resource, and did so by employing the offensive language and hate speech classifier from Davidson et al. (Reference Davidson, Warmsley, Macy and Weber2017b). The final dataset contains approximately 2M and 7M sentences from Twitter and Reddit, respectively, with the majority of instances being non-offensive. Also Cheng et al. (Reference Cheng, Gan, Zhang, Elachqar, Li and Liu2020b) created a parallel dataset of offensive and non-offensive texts (the latter were crowdsourced by asking annotators to produce two non-offensive alternatives for a given offensive input).

As for dictionary-based approaches, several open-access sources are available. For instance, Tran et al. (Reference Tran, Zhang and Soleymani2020) compiled a vocabulary of offensive terms by crawling a list of more than 1k English expressions made available by Luis von Ahn’s research group,Footnote ^ad and an online platform that contains an ever-growing inventory of profanities.Footnote ^ae

Method

dos Santos et al. (Reference dos Santos, Melnyk and Padhi2018) employed an encoder–decoder model with an attention mechanism. They ensured output quality with a cycle consistency loss and the help of a collaborative classifier providing signal about the effectiveness of the transfer. Interestingly, they noted that their model was unable to handle implicit offensive content (e.g., ordinarily inoffensive words used offensively), indicating that offensiveness cannot always be addressed at a lexical level by changing a few words.

Still, other researchers focused on the editing of offensive lexical items. For paraphrasing profane texts in Chinese, Su et al. (Reference Su, Huang, Chang and Lin2017) manually devised a rule-based system, equipped with an extensive set of profanity detection and paraphrasing strategies (the rules were language-specific, hence the system is not extendable to other languages). Similarly, Tran et al. (Reference Tran, Zhang and Soleymani2020) developed a transparent modular pipeline around the idea that a text is offensive if it contains profanity. The pipeline had different modules. First comes the retrieval module: it extracts ten part-of-speech (POS) tag sequences from a dataset of non-offensive texts, which are similar to the POS sequence found in an offensive sentence. Next is the generation module, which creates non-offensive sentences by matching the words from the input into possible positions in the generated POS sequences, and then filling the unmatched positions with a pretrained language model. An edit step further corrects word order. The selected output was the one with the best fluency, meaning preservation and transfer—which in this case corresponds to the absence of profanities.

Evaluation

In addition to the regular metrics for content preservation and fluency, dos Santos et al. (Reference dos Santos, Melnyk and Padhi2018) reported the classification accuracy using the classifier from Davidson et al. (Reference Davidson, Warmsley, Macy and Weber2017b).

Data

A parallel dataset for similes with approximately 87k sentences was created by Chakrabarty et al. (Reference Chakrabarty, Muresan and Peng2020b). It was built in an automatic manner, crawling self-labelled simile expressions from Reddit via the comparative phrase like a (e.g., “The boy was like an ox”). The authors employed Comet (Bosselut et al., Reference Bosselut, Rashkin, Sap, Malaviya, Celikyilmaz and Choi2019), a pretrained language model fine-tuned on the ConceptNet (Speer et al. Reference Speer, Chin and Havasi2017) knowledge graph, to replace the logical object of the comparison (here, “an ox”) with its shared property (here, “being strong”) in order to generate the parallel sentence (e.g., “The boy was strong”).

Method

Chakrabarty et al. (Reference Chakrabarty, Muresan and Peng2020b) exploited a simplified lexical structure followed by a simile, with clearly defined roles for the lexical elements. In the example “Her smile is like sunshine”, the author intended to describe the topic, her smile, by comparing it to a logical object, sunshine, via a shared property, i.e., their brightness. The parallel dataset they curated with distant supervision served to fine-tune Bart (Lewis et al., Reference Lewis, Liu, Goyal, Ghazvininejad, Mohamed, Levy, Stoyanov and Zettlemoyer2020), a pretrained language model that is a combination of bidirectional and auto-regressive transformers. They also conducted experiments with baseline models based on conditional generation, metaphor masking, and retrieval using Comet (Bosselut et al., Reference Bosselut, Rashkin, Sap, Malaviya, Celikyilmaz and Choi2019). Hence, they demonstrated that incorporating structured common sense knowledge through Comet is effective and can be employed in related creative text generation tasks. The fine-tuned Bart model successfully generated novel sentences and generalized over unseen properties.

Evaluation

For automatic evaluation, Chakrabarty et al. (Reference Chakrabarty, Muresan and Peng2020b) reported Bleu after removing the common prefix in the generated and reference sentences. Moreover, they leveraged BertScore (Zhang et al. Reference Zhang, Kishore, Wu, Weinberger and Artzi2020a), a measure indicating the similarity between candidate and reference sentences that uses contextual embeddings, for the contextual vectors of the logical object of the comparison phrases. Human evaluation aimed at comparing the literal utterances against six generated outputs, rated on a scale of 1-to-5 with respect to creativity, overall quality, relevance of the comparison object in portraying the shared property, and relevance of the suggested comparison object in the given topic context.

Table 9.

Literature on intended, non-targeted styles corresponding to conventional genres, divided by method

Data

A useful newspaper dataset for style transfer was created by De Mattei et al. (Reference De Mattei, Cafagna, Dell’Orletta and Nissim2020), even though their work regarded style-aware generation rather than transfer. They collected news that are lexically similar from two newspapers, a subset of which are topic-aligned. Gatti et al. (Reference Gatti, Ozbal, Guerini, Stock and Strapparava2016) used the news of the day, extracted from the RSS feed of the New York Times and BBC News, and Lee et al. (Reference Lee, Xie, Wang, Drach, Jurafsky and Ng2019) resorted to articles from the New York Times and comments from Reddit.

Another dataset dedicated to news articles is the Gigaword corpusFootnote ^ah (Parker et al. Reference Parker, Graff, Kong, Chen and Maeda2011). This resource was acquired over several years by the Linguistic Data Consortium, and it spans seven international sources of English newswire (i.e., Agence France-Presse, Associated Press Worldstream, Central News Agency of Taiwan, Los Angeles Times/Washington Post Newswire Service, New York Times, Xinhua News Agency, and Washington Post/Bloomberg Newswire Service). Fu et al. (Reference Fu, Tan, Peng, Zhao and Yan2018) focused instead on news titles. They built a datasetFootnote ^ai of 108,503 titles belonging to the science and technology categories and which come from the UC Irvine Machine Learning Repository (Dua and Graff Reference Dua and Graff2017). As an attribute opposite to “news”, their corpus contains scientific-oriented language, specifically paper titles crawled from academic websites.

The reframing study of Chen et al. (Reference Chen, Al Khatib, Stein and Wachsmuth2021) made use of the corpus published by Card et al. (Reference Card, Boydstun, Gross, Resnik and Smith2015). Encompassing more than 35k news articles about death penalty, gun control, immigration, same-sex marriage and tobacco, the corpus is annotated with 15 framing dimensions (e.g., economics, morality, politics) developed by Boydstun et al. (Reference Boydstun, Card, Gross, Resnick and Smith2014).

Methods

Gatti et al. (Reference Gatti, Özbal, Guerini, Stock and Strapparava2015) performed lexical substitution by extracting keywords from the news and inserting them in well-known expressions coming from slogans, movie titles, song titles, and clichés: after pairing the two data based on a similarity measure, they used a dependency metrics to find the probability for the words in the slogan of being replaced with the same part-of-speech keywords from the news.

More recent neural attempts aimed at transferring news titles to scientific paper titles. This was done by Romanov et al. (Reference Romanov, Rumshisky, Rogers and Donahue2019), who fit in the picture of disentanglement based on adversarial methods. They had an encoder produce a continuous style vector and a meaning vector for a given input. Compared to other adversarial approaches, these authors employed two complementary forces. One was a discriminator that penalized the encoder if the meaning embeddings still carried information about style; the other was a motivator, and it pushed the encoder to produce representations that facilitate the correct attribute classification—encouraging, rather than penalizing, was proven to make the separation between the two types of embeddings bolder.

Moving on to news reframing, Chen et al. (Reference Chen, Al Khatib, Stein and Wachsmuth2021) characterized the problem in the following terms: given three consecutive sentences and a target frame, the middle sentence can be masked, and a new one generated to fill in such blank, which contains the target frame and links the preceding and follow up sentences coherently. The authors trained one generation model for each frame, and experimented with three strategies. Namely, fine-tuning a sequence-to-sequence model on a specific frame, including knowledge about named entities to promote topic coherence, and adding examples in the training data (the sentence to be generated has a different frame compared to the surrounding ones).

Evaluation

De Mattei et al. (Reference De Mattei, Cafagna, Dell’Orletta and Nissim2020) put forward the idea that news styles are more difficult to judge than others (e.g., sentiment), and that humans are not as reliable judges of said styles as machines. They proposed a framework for the automatic evaluation of style-aware generation that seems handy for style transfer as well. Their automatic classifier had to distinguish the newspaper style of lexically aligned headlines: such an alignment pushed the classifier to make decisions based on stylistic information rather than content-related one.

With respect to human evaluation, Gatti et al. (Reference Gatti, Özbal, Guerini, Stock and Strapparava2015) asked people if an output headline was grammatically correct and if it could work as a headline for a given article, while Chen et al. (Reference Chen, Al Khatib, Stein and Wachsmuth2021) conducted an extensive study in which they presented crowdworkers with multiple reframings for an input text, which had to be evaluated for their contextual coherence, topical congruence, and presence of a given frame.

Data

An obvious prerequisite for style transfer in a specialized genre is the availability of domain-specific data. Cao et al. (Reference Cao, Shui, Pan, Kan, Liu and Chua2020) introduced an expert-annotated parallel corpusFootnote ^aj in the medical domain. It was derived from human-written medical references tailored for consumers vs. healthcare professionals who, in their view, are set apart by two major knowledge gaps: one related to technical terminology (“dyspnea” $\rightarrow$ “shortness of breath”) and one related to the understanding of empirical evidence (e.g., “About 1/1,000” $\rightarrow$ “quite small”).

Methods

The major contribution of Cao et al. (Reference Cao, Shui, Pan, Kan, Liu and Chua2020) was the dataset itself, that they evaluated with five state-of-the-art models from prior style transfer (Hu et al., Reference Hu, Yang, Liang, Salakhutdinov and Xing2017; Li et al., Reference Li, Jia, He and Liang2018; Dai et al., Reference Dai, Liang, Qiu and Huang2019) and text simplification studies (Shardlow and Nawaz Reference Shardlow and Nawaz2019; Surya et al. Reference Surya, Mishra, Laha, Jain and Sankaranarayanan2019).

Evaluation

The adopted evaluation methods in Cao et al. (Reference Cao, Shui, Pan, Kan, Liu and Chua2020) were transfer accuracy based on a classifier’s performance, fluency based on the perplexity of a fine-tuned Bert model, and content preservation computed in terms of Bleu. In their human evaluation study, laypeople rated content preservation in the model-generated output on a 1-to-5 scale, given both the input and human-produced gold references. The metrics Sari (Xu et al. Reference Xu, Napoles, Pavlick, Chen and Callison-Burch2016) was also used to evaluate language simplicity, as it compares the n-grams in the generated output with the input and human references, taking into account the words that were added, deleted and retained by the model. The authors concluded that for transfers regarding this style, there exists a substantial difference between the quality of machine-produced and human-produced texts.

Data

Carlson et al. (Reference Carlson, Riddell and Rockmore2018) contributed to fixing the lack of parallel data for style transfer. They collected a high-quality parallel corpus without the involvement of any automatic alignment effort. Their resource contains 34 versions of the Bible produced by professionals and which are naturally aligned, given the structure of such texts, i.e., in chapters and verses. Each version corresponds to an English stylistic value (e.g., archaic, simple, American). They made the dataset available for the texts that were already public.

Pang and Gimpel (Reference Pang and Gimpel2019) limited themselves to two variants of English, with the old one taken from Dickens’ works in Project GutenbergFootnote ^an and the modern version from the Toronto Books Corpus. Focusing on Chinese, Shang et al. (Reference Shang, Li, Fu, Bing, Zhao, Shi and Yan2019) constructed a parallel corpus containing old and modern versions of poems. Xu et al. (Reference Xu, Ritter, Dolan, Grishman and Cherry2012) made a sentence-aligned corpus of Shakespearean plays and their modern translations freely available. Krishna et al. (Reference Krishna, Wieting and Iyyer2020) built a non-parallel English corpus containing 15 M sentences, which contain 11 styles, including the Bible, Shakespeare, James Joyce. Lastly, the philosophy texts, sci-fi and gothic novels of Gero et al. (Reference Gero, Kedzie, Reeve and Chilton2019) also come from mono-style sources. They were extracted from Project Gutenberg and the Pulp Magazine Archive,Footnote ^ao respectively.

Methods

The first attempt at dealing with literature styles explored statistical machine translation (Xu et al., Reference Xu, Ritter, Dolan, Grishman and Cherry2012); on top of that, Carlson et al. (Reference Carlson, Riddell and Rockmore2018) went for sequence-to-sequence translation models, trained for each target attribute. A sequence-to-sequence network was also leveraged by Jhamtani et al. (Reference Jhamtani, Gangal, Hovy and Nyberg2017). They added both a pointer that facilitates the copy of input words, and a dictionary of shakespearean-to-modern word pairs which allows to retrofit pretrained word embeddings, thus accounting for novel words or words that have changed in meaning.

On the unsupervised side, Pang and Gimpel (Reference Pang and Gimpel2019) experimented with models that include losses corresponding to the three criteria, and that could be used both for model tuning and selection. Among such losses, many of which had been already explored (Shen et al., Reference Shen, Lei, Barzilay and Jaakkola2017, i.a.), they tried to favor content preservation with a reconstruction loss, a cyclic consistency loss (similar to the former, but with the transfer happening twice, i.e., from source to target and back), and a paraphrase loss obtained with sentence–paraphrase pairs coming from a parallel dataset.

Author mimicking was addressed with the probabilistic approach of He et al. (Reference He, Wang, Neubig and Berg-Kirkpatrick2020); similarly aiming at minimizing the manually defined objectives (e.g., content-to-style separation), the semi-supervised method of Shang et al. (Reference Shang, Li, Fu, Bing, Zhao, Shi and Yan2019) employed an encoder-decoder that learns to represent a style within a specific latent space, and a projection function that maps the latent representations of one attribute onto the other. The two steps leveraged non-parallel and parallel data respectively. Instead, Krishna et al. (Reference Krishna, Wieting and Iyyer2020) adopted their inverse paraphrasing approach already introduced with the background styles.

Style and content were handled separately by Gero et al. (Reference Gero, Kedzie, Reeve and Chilton2019). In line with their POS-based characterization of style, they defined some low-level linguistic features (e.g., frequency of pronouns, prepositions) as the style of a text, and they performed style transfer by inputting an encoder-decoder with only the content words, which allowed the generation to maintain them while adjusting the features of the target attribute. By contrast, Mueller et al. (Reference Mueller, Gifford and Jaakkola2017) refrained from defining editing features or rules. Claiming that revisions of combinatorial structures are unlikely to be found by simple search procedures, they addressed the Shakespearization of language as a problem of finding improved rewrites of a text.

Evaluation

To measure the quality of paraphrases, Carlson et al. (Reference Carlson, Riddell and Rockmore2018), Jhamtani et al. (Reference Jhamtani, Gangal, Hovy and Nyberg2017) and Xu et al. (Reference Xu, Ritter, Dolan, Grishman and Cherry2012) accompanied Bleu, a measure that fundamentally favors textual similarity at the word level, with Pinc, which instead rewards the diversity of the output from the source text thanks to the number of n-grams in the candidate output that do not appear in the source. To measure the transfer strength criterion, Xu et al. (Reference Xu, Ritter, Dolan, Grishman and Cherry2012) used a language model to compute the posterior probability that a sentence was generated from a model of the target language.

Pang and Gimpel (Reference Pang and Gimpel2019) introduced a way to measure the success of transfer by aggregating the metrics: an adjusted geometric mean between the accuracy, content preservation and perplexity, which penalizes perplexity scores that are too low, often achieved with short phrases but not meaningful sentences. For human evaluation, their annotators decided which of two generated sentences they preferred with respect to the three transfer criteria. The sentences were taken from different model variants, to observe the correlation between human judgments and each system.

Data

This task leveraged non-parallel lyrics resources from MetroLyricsFootnote ^ap in which more than 500k songs are associated to specific music genres.

Methods

Lee et al. (Reference Lee, Xie, Wang, Drach, Jurafsky and Ng2019) treated the problem as a denoising one, with the same model used to transfer the background of persona described in Section 2.2.2. The non-parallel source data were noised with a model trained on clean-noisy sentence pairs extracted from a language learner forum; the newly synthesized texts were then re-ranked according to their proximity to the target attribute and to the meaning of the source inputs; lastly, a denoising model was trained to find the probability of a clean text (i.e., the target), given the noisy one (i.e., the source).

Evaluation

Unlike other studies, Lee et al. (Reference Lee, Xie, Wang, Drach, Jurafsky and Ng2019) defined the transfer strength criterion as the ratio between the probability of the output belonging to the target domain and the probability of observing it in the source domain.