2.1. Overview of NLP techniques

The main NLP methodologies include rule-based approaches, statistical methods, machine learning (ML), deep learning (DL), and transformer models. The choice of the algorithm depends on the application because of the varying strengths and weaknesses of these approaches. Rule-based approaches use predefined linguistic rules for tasks like tokenization and parsing. While they are interpretable and preferred for simple tasks, they may lack adaptability to natural language complexity. Statistical methods, such as hidden Markov models (HMMs) and conditional random fields, are utilized for tasks like part-of-speech (POS) tagging and named entity recognition (NER). They are robust and handle uncertainty. However, they require extensive feature engineering and may not capture long-range dependencies. Supervised ML models, e.g., support vector machines and naive Bayes classifiers, learn from labeled data for tasks like text classification. Unsupervised ML methods, like clustering and topic modeling, uncover hidden structures in text. While ML algorithms may overfit to training data and need large labeled datasets and feature engineering, they are versatile and generalize well to new data. DL techniques, such as RNNs, CNNs, and long short-term memory networks (LSTMs) learn hierarchical representations from raw text. They excel at sequence modeling, text classification, and machine translation but require large datasets. The ability to capture complex patterns and dependencies in text may become computationally expensive, possibly suffering from gradient issues. Transformer architectures like BERT and GPT use self-attention mechanisms to capture contextual relationships in text. They excel at language understanding, generation, translation, and summarization but need massive computational resources and extensive pre-training on large text corpora (Vaswani et al., Reference Vaswani, Shazeer, Parmar, Uszkoreit, Jones, Gomez and Polosukhin2017). While they achieve state-of-the-art performance across NLP tasks, computing and data needs may limit their use.

The emerging use of NLP emphasizes challenges and policy implications related to data privacy, fairness, bias mitigation, and ethical AI governance. First, the reliance of transformer-based models like BERT and GPT on large-scale datasets raises concerns regarding user data privacy, especially when training on sensitive information (Carlini et al., Reference Carlini, Tramer, Wallace, Jagielski, Herbert-Voss, Lee, Roberts, Brown, Song, Erlingsson, Oprea and Raffel2021). Regulatory frameworks such as the General Data Protection Regulation (GDPR) emphasize the need for data anonymization, consent-based collection, and user control over personal information, which may conflict with the vast, unregulated data sources often used in pretraining these models. Furthermore, bias in training data can propagate into model outputs, leading to unfair or discriminatory outcomes in applications such as automated hiring, content moderation, and sentiment analysis (Bender et al., Reference Bender, Gebru, McMillan-Major and Shmitchell2021). Additionally, the increasing carbon footprint of large-scale NLP models due to the computational needs raises sustainability concerns. This encourages the use of training techniques such as pruning, quantization, and knowledge distillation that could reduce energy consumption (Strubell et al., Reference Strubell, Ganesh and McCallum2020). Addressing these challenges requires a balanced regulatory approach that promotes innovation while ensuring ethical AI deployment in real-world applications. The use of NLP in adversarial environments amplifies some of these challenges, resulting in elevated policy implications (Schlarmann and Hein, Reference Schlarmann and Hein2023).

2.2. Bayesian methods for NLP

Bayesian approaches are versatile for NLP applications due to their natural ability to model uncertainty, embed prior knowledge, and decision making under incomplete information (Cohen, Reference Cohen2022). In NLP tasks, such as text classification, sentiment analysis, or machine translation, Bayesian inference can be used to estimate the posterior distribution of model parameters given observed data. This allows for the incorporation of prior knowledge, which can help in regularizing models to avoid overfitting and improve generalization to unseen data. For instance, Naive Bayes classifiers are foundational in text classification tasks. They are based on Bayes’ theorem and make the simplifying assumption that the features are conditionally independent given the class label. Despite this simplification, Naive Bayes often performs surprisingly well, especially in spam detection, sentiment analysis, and topic classification.

For probabilistic clustering of text data, a Bayesian hierarchical method, Latent Dirichlet Allocation (LDA) (Blei et al., Reference Blei, Ng and Jordan2003), and the subsequent development of topic models have become popular for document summarization and information retrieval. (Abdelrazek et al., Reference Abdelrazek, Eid, Gawish, Medhat and Hassan2023). Bayesian methods also can be used for sequence labeling tasks such as POS tagging. For example, HMMs, which are probabilistic models that assume a sequence of observed words is generated by a sequence of hidden states (POS tags), can be trained using Bayesian inference. Bayesian networks and HMMs can be used to model the probabilistic relationships that arise in machine translation, word sense disambiguation, information retrieval, parsing, and named entity recognition. Finally, LLMs where the goal is to predict the probability of a sequence of words benefit from Bayesian n-gram models and neural networks in capturing the probabilistic relationships between words (Chien, Reference Chien2019). They can be useful for speech recognition and text generation.

While Bayesian methods provide robust and interpretable solutions to a wide range of NLP tasks, their complexity and computational cost may limit the real-world adoption. Bayesian methods in NLP could also present policy challenges related to fairness and transparency. Bayesian inference inherently depends on prior probabilities, which can introduce bias if the training data are not representative (Evans and Guo, Reference Evans and Guo2021). This is particularly relevant in adversarial NLP, where robustness of the models against misinformation, spam, and adversarial attacks, is crucial.

3.1. Adversarial attacks in NLP

Adversarial attacks with varying complexity and knowledge levels have been developed to weaken the performance and reliability of NLP systems. These methods usually involve changing text data, at different levels such as characters (char), words, or sentences, to trick NLP models to produce incorrect outputs, as in misclassification. Character-level attacks involve altering individual characters in a text to trick NLP models, such as changing “cat” to “c at.” This minor modification can disrupt the model’s understanding and cause incorrect predictions (Huang et al., Reference Huang, Kajiwara and Arase2021)). In word-level attacks, individual words in a text are strategically altered to mislead NLP models without changing the overall meaning. An example involves changing “The product is excellent” to “The product is terrible” by substituting “excellent” with “terrible.” This subtle change can cause the model to misclassify the sentiment (Gao et al., Reference Gao, Lanchantin, Soffa and Qi2018)). Sentence-level (paraphrasing) attacks rephrase sentences to confuse NLP models while keeping the meaning the same. An example is rephrasing “The cat sat on the mat” to “The mat was where the cat sat” that can lead the model to misclassify this sentence (Zeng et al., Reference Zeng, Li, Song, Gao, Lyu and King2018)).

In terms of knowledge of the attacked model, adversarial attacks are at varying levels between white-box and black-box attacks. In white-box attacks, the attacker has complete information about the target model, including its architecture, parameters, and training data. A black-box attacker has little or no knowledge of the target model. They rely on querying the model and observing its outputs to create adversarial examples. Binary attacks correspond to attacks for binary classifications. Attacks can also be targeted with a particular objective or may be more general as non-targeted.

The attack methods broadly range from basic attacks with adding manually crafted inputs or altering words, to iterative gradient based refinements. Our work reflects the advances in NLP models where attackers used heuristic and gradient based methods to exploit model vulnerabilities. Heuristic methods are mostly specific to certain models lacking generalizability. Gradient-based techniques like the Fast Gradient Sign Method (FGSM) create perturbations that misled models while remaining undetectable to humans (Wang, Reference Wang2022)). These gradient-based adversarial perturbations could be iteratively refined, as in iterative FGSM and Projected Gradient Descent (PGD), resulting in higher success rates than one-shot methods (Chao et al., Reference Chao, Robey, Dobriban, Hassani, Pappas and Wong2023). Guo et al. (Reference Guo, Sablayrolles, Jégou and Kiela2021) presents an overview of adversarial attack techniques on deep learning models, while Hartl et al. (Reference Hartl, Bachl, Fabini and Zseby2020) and Cheng et al. (Reference Cheng, Jiang and Macherey2019) focus on RNNs and transformers based NMTs, respectively. Smaller variants of the transformer models (e.g., BERT, GPT-3) are more susceptible to adversarial attacks. Table 1 presents an overview of highlights of adversarial attacks, ranging from char-level to sentence-level attacks, against various NLP models.

Table 1. Review highlights of adversarial attacks in NLP

3.2 Defense mechanisms in NLP

The increasing effectiveness of adversarial attacks makes robust countermeasures necessary for NLP security. Defenses against adversarial attacks in NLP are either based on (reactive) detection or (proactive) model enhancement methods. Detection and filtering methods may have limited power against sophisticated and dynamic adversarial attacks. Therefore, model enhancement methods such as AdvT, functional improvement, and certification could be preferred. Among these, AdvT is based on proactive inclusion of adversarial examples in training data. For instance, SmoothLLM uses adversarial examples during training to improve the robustness of LLMs with remarkable results while requiring computational resources. Phrasing is a specific tailored method that trains models to recognize resilient phrases. Zero-Shot defender for adversarial sample detection and restoration (ZDDR) combines AdvT with zero-shot learning to detect and restore adversarial inputs. As shown in Table 2, several other methods enhance model robustness against specific attacks at the cost of additional fine-tuning. Input preprocessing and data transformation methods include “Synonym Encoding” which replaces words with synonyms to reduce sensitivity to specific words. Duplicate text filtering removes duplicate text to improve generalization. Despite its effectiveness, it may also discard useful data. Data sanitization is based on removing sensitive information from data. It protects privacy at the potential cost of altering semantics. Finally, knowledge expansion methods augment training data with external knowledge sources. They enhance understanding, but their performance depends on the quality and relevance of the knowledge added. While these defense techniques help improve NLP model robustness, increased computational complexity and vulnerabilities to specific attacks are among current limitations. Table 2 presents an overview of NLP defenses against adversarial attacks ranging from char to sentence levels.

Table 2. Review highlights of adversarial defenses in NLP

3.3. Practical guidance

Practitioners can use various techniques to design adversarial attacks that evaluate and stress-test model robustness. These include open source libraries like nlpaug Footnote ¹, TextAttack Footnote ², Foolbox Footnote ³, and CleverHans Footnote ⁴ that provide algorithms for crafting adversarial examples. One widely used option is to utilize genetic algorithmic frameworks that evolve adversarial examples through natural selection. Their standard steps include initialization (generation of initial adversarial examples), evaluation (assessing their effectiveness), selection (choosing high performing examples), and crossover and mutation (creating new examples).

In this study, we focused on gradient-based adversarial attacks by utilizing the grand framework of model selection, gradient calculation and perturbation generation, and evaluation, through the use of open-source Python library Nlpaug. This data augmentation library offers methods such as synonym replacement, contextual word substitution, and back translation, making it versatile for different NLP tasks, such as sentiment analysis and topic classification. Augmenting text data has been shown to enhance the performance of NLP models, particularly for classification tasks (Bayer et al., Reference Bayer, Kaufhold, Buchhold, Keller, Dallmeyer and Reuter2023). In particular, we generated adversarial attacks on text data by applying random attacks, e.g., adding spelling errors, word splitting, and random perturbationsFootnote ⁵. The relevant code can be accessed at GitHubFootnote ⁶. Figure 2 displays a practical example for a word substitution (synonym) attack.

Figure 2. Adversarial sentiment analysis example on IMDB dataset (Shaw et al., Reference Shaw, Ansari and Ekin2024).

Table 3 presents the impact of such adversarial attacks on different combinations of NLP methods against both IMDB and TwitterFootnote ⁷ datasets. The varying impacts of attacks on accuracy and F1 scores can be recognized. For instance, accuracy dropped by 30% under attack, illustrating the practical risk to deployed NLP systems.

Table 3. Select empirical results before and after attacks against ensembles of CNN and BiLSTM

In terms of defenses, Table 2 indicates the popularity of adversarial training (AdvT). In our practical implementations (Shaw and Ekin, Reference Shaw and Ekin2024), we also have recognized that AdvT not only enhances the robustness of the model but also improves generalization to unseen adversarial examples. This could offer practitioners a practical pathway to fortify NLP models against increasingly sophisticated attacks, ensuring more secure and reliable performance.

Datasets with adversarial examples, e.g., word substitutions, character-level perturbations, are crucial for training and testing NLP models’ robustness. Evaluating models on diverse datasets helps researchers understand their robustness and improve defense strategies and model architectures. Therefore, Tables 1 and 2 list the utilized datasets in adversarial NLP literature. Several key metrics are used to evaluate the impact of adversarial attacks and defenses in NLP. These metrics help assess how well different attack techniques, defense mechanisms, and models work against adversarial threats. While accuracy is among the measures used to quantify the proportion of correctly classified examples, attack success rate measures the effectiveness of adversarial attacks. Robustness measures a model’s ability to maintain performance when facing adversarial perturbations. Transferability assesses if adversarial examples for one model can deceive other models, indicating shared vulnerabilities. While text domain lacks universal benchmarks or data sets as introduced in image domain, there has been recent work such as Adversarial GLUE (Wang et al., Reference Wang, Xu, Wang, Gan, Cheng, Gao, Awadallah and Li2021a) and MITRE ATLAS MatrixFootnote ⁸ to address that.

4.1. Ethical challenges and governance

The applications of NLP systems may often result in ethical challenges related to fairness and privacy. One of the most pressing challenges in NLP is the presence of bias in training data. Models trained on biased data can inadvertently replicate and amplify discriminatory patterns. For instance, sentiment analysis systems may misclassify language from minority dialects, and automated hiring tools can exhibit gender or racial biases. Governments and organizations should possibly create standards for bias detection and mitigation. Policies could advocate for transparency in dataset composition and require regular audits of NLP models to evaluate fairness. Robust governance frameworks can help ensure that NLP technologies do not perpetuate or exacerbate societal inequities. Frameworks such as the European Union’s AI Act, which emphasizes accountability and fairness, serve as important precedents for global adoption.

Moreover, privacy is a critical consideration in NLP applications, especially in domains like healthcare or legal services, where sensitive information is processed. LLMs trained on vast amounts of data may inadvertently memorize and reproduce private or confidential information. Hence, policy interventions may be essential to safeguard user data and enforce compliance with privacy regulations like the GDPR. Adopting differential privacy techniques during model training and ensuring end-to-end encryption for NLP-powered systems can help address these concerns.

These ethical challenges become even more crucial in adversarial environments, and policy makers need to be aware of the implications. Accountability mechanisms that are crucial to prevent the misuse of NLP technologies should consider potential manipulation of data and models. Organizations can increase the transparency and document their decision-making processes of their models and provide avenues for recourse in cases of harm caused by algorithmic decisions.

4.2. Policy and regulatory needs for adversarial robustness

The threat posed by adversarial attacks to the reliability and security of the NLP systems creates the potential need for regulatory frameworks. Such policies that encourage secure model development are especially crucial for high-stakes applications. We can address some of these challenges by embedding safeguard regulations as part of the development lifecycle for NLP systems. Similarly to the required cybersecurity certifications (Alawida et al., Reference Alawida, Mejri, Mehmood, Chikhaoui and Isaac Abiodun2023), NLP models could undergo adversarial stress tests to confirm their resilience against attacks on data and models (Singh et al., Reference Singh, Grover and Kumar2022). Governments and public agencies can incentivize the development of more robust NLP technologies by funding research and offering tax incentives to companies that prioritize security. Public-private partnerships can also facilitate research into more effective techniques to secure NLP systems. For instance, frameworks could mandate transparency in how models handle adversarial examples, ensuring public trust and accountability (Al-Maliki et al., Reference Al-Maliki, Qayyum, Ali, Abdallah, Qadir, Hoang, Niyato and Al-Fuqaha2024).

Adversarial threats transcend borders, particularly when NLP models are deployed in multilingual or cross-cultural contexts (e.g., global customer service chatbots or translation systems). Such a global nature of NLP development and deployment may require international collaboration on policy frameworks and standardization. Differences in data privacy laws, ethical standards, and regulatory requirements between countries can hinder progress and create compliance challenges for organizations. Efforts to harmonize data privacy regulations, such as aligning GDPR with U.S. frameworks like the California Consumer Privacy Act, can streamline compliance for NLP applications operating across jurisdictions. In addition, policies promoting open data standards and interoperability can facilitate cross-border collaboration in NLP research. For example, shared datasets for adversarial training and multilingual NLP can help advance the field while adhering to ethical standards. As nations become more protective of their digital resources, policies should balance the need for digital sovereignty with the benefits of international collaboration. Agreements on data sharing and joint research initiatives can ensure mutual benefits while respecting national interests.

As LLMs integrate Bayesian techniques for probabilistic reasoning, regulatory guidelines should ensure that these models maintain ethical standards, transparency, and responsible AI governance. Encouraging open-source Bayesian NLP frameworks and federated learning approaches could further enhance privacy-preserving AI applications while maintaining model robustness and scalability. Future policies should focus on balancing innovation and regulation, ensuring that Bayesian NLP methods contribute to ethical, fair, and secure AI systems.

5.1. Bayesian methods for adversarial NLP

Bayesian methods are increasingly relevant in adversarial machine learning (AML) due to their robustness and ability to quantify uncertainty (Rios Insua et al., Reference Rios Insua, Naveiro, Gallego and Poulos2023). They can be used to generate attacks as well as detecting and defending against adversarial attacks. Bayesian sequence models can generate text while maintaining a measure of uncertainty, helping to avoid nonsensical or adversarially induced outputs. Similarly, Monte Carlo dropout could be used to approximate Bayesian inference in deep learning while providing uncertainty estimates. High uncertainty of Bayesian predictions may indicate possible adversarial manipulations (Zhao et al., Reference Zhao, Su and Ji2020). For instance, using Bayesian neural networks, where weights are treated as distributions rather than point estimates, can produce more reliable confidence estimates, making the system more robust to attacks that exploit overconfident but incorrect predictions. By continuously updating the model with new data, Bayesian approaches can also adapt to evolving threats, maintaining the security of the NLP system over time. Bayesian optimization can be used to tune hyperparameters or model architectures to find configurations that are less susceptible to adversarial attacks. In addition, Bayesian methods inherently provide regularization, which can make models more robust to perturbations. Bayesian generative models, such as variational autoencoders (Doersch, Reference Doersch2016), can generate adversarial text samples by sampling from the latent space, providing a range of AdvT examples. Lastly, Bayesian decision-theoretic models such as adversarial risk analysis (Banks et al., Reference Banks, Gallego, Naveiro and Ríos Insua2022) could be used for AML (Ekin et al., Reference Ekin, Naveiro, Insua and Torres-Barrán2023; Caballero et al., Reference Caballero, Camacho, Ekin and Naveiro2024). These could be beneficial to model the interactions among the decision makers within adversarial NLP contexts (Shaw, Reference Shaw2025).

Overall, Bayesian methods offer a principled framework for decision-making under uncertainty, an essential aspect of developing robust adversarial defenses. Their ability to incorporate prior knowledge, provide calibrated uncertainty estimates, and apply natural regularization makes them well-suited for identifying and mitigating adversarial inputs. These strengths can guide models toward safer predictions when faced with perturbed or ambiguous data. However, the high computational demands and the implementation complexity have limited their adoption relative to more conventional approaches. As interest in large-scale NLP systems, particularly LLMs, continues to grow, there is a renewed need to reassess these trade-offs and explore the broader application of Bayesian approaches in adversarial NLP settings.

5.2. Emerging policy challenges

The emergence of large-scale NLP models, particularly LLMs, has introduced significant new policy challenges and amplified existing concerns. These challenges span domains, such as national security, public trust, misinformation, access, sustainability, and regulatory oversight.

First, adversarial attacks can significantly undermine the reliability of government-run IT systems that use NLP models. For instance, LLMs deployed in public service portals, e.g., immigration, social benefits, and tax filing systems, could be manipulated into giving incorrect or misleading guidance. An adversarial input might alter eligibility decisions, misroute applications, or subtly redirect users to malicious third-party websites. In addition to affecting operational efficiency, these may also result in privacy breaches, denial of services, or legal liability for government agencies. These risks highlight the urgent need for adversarial robustness in publicly deployed AI systems.

Second, many autonomous systems (e.g., self-driving cars, automated drones) increasingly rely on NLP for tasks like interpreting commands, processing sensor data, or interacting with humans. Adversarial threats could disrupt these systems, leading to potential safety risks. This eventually could diminish public trust in NLP. Potential governmental intervention can help establish minimum robustness and transparency standards, incentivize industry practices like “adversarial audits” and robustness certification, and coordinate international policy to prevent cross-border adversarial attacks.

Third, training and deploying large NLP models consume substantial energy resources, contributing to environmental concerns. Policies can encourage the development of energy-efficient model architectures, sustainable deployment and environmental reporting standards in alignment with global sustainability goals.

Finally, the generative capabilities of large-scale models result in powerful tools for misinformation and propaganda. Governments may support transparency when it comes to detecting and mitigating the spread of (false) information generated by NLP systems. Public awareness campaigns can complement these efforts. This also has consequences on trust in NLP systems. As NLP systems become increasingly sophisticated, their accessibility to under-resourced communities and languages remains a concern. Policies can support initiatives to develop NLP tools for low-resource languages and to enhance equitable access to these technologies.

These implications emphasize the importance of detection mechanisms and their use for public benefit while keeping the systems accessible. Policymakers face the complex task of balancing technical defenses and accessibility with legislative oversight. Ensuring secure model development is essential, especially when models are used in sensitive domains like legal aid, medical triage, or education. Prevention and early detection require investment in both tooling and organizational workflows, including training for civil servants to identify anomalies. Legislation might also be necessary, especially in creating (international) standards for responsible deployment, redressal mechanisms for affected users, and penalties for malicious adversarial input crafting. International collaboration and proactive regulation may also help navigate the complexities of NLP systems. Establishing a globally shared knowledge hub that gathers and publishes real-time adversarial threat intelligence for NLP systems can be beneficial. This could help with standardization of adversarial robustness and help responding to adversarial incidents or failures (e.g., LLM misuse, model degradation under attack), in the spirit of cybersecurity breach notification laws. Ultimately, an integrated approach managing technical, procedural, and legal aspects is likely to be most effective for NLP governance.