Bálint Gyevnár

Scientific fraud is the instrument of doubt that malicious entities can use to establish controversy in science. Historically, it required the resources of a company: deep pockets, ghostwritten articles, and corrupt academics. Today, Artificial Intelligence (AI) is increasingly automating scientific research, so we ask: Can a remote adversary weaponize the honest use of AI in science to compromise scientific integrity? We envision and empirically evaluate a new attack, indirect data poisoning, in which an adversary corrupts an open dataset and uploads the poisoned variant to a public repository. Autonomous research agents may independently retrieve and process this data, turning honest scientists into the unpaid and unwitting distributors of fraud at scale. Across five socially-salient topics, from hiring discrimination to the safety of autonomous vehicles, three widely used frontier AI systems (Claude Code with Claude Opus 4.7, Codex with GPT-5.5, Gemini CLI with Gemini 3.1 Pro), and 450 ethically contained experimental runs, we find that poisoning succeeds in 49.56% of runs, while the rate of poisoning detection is only 6.0%. The attack requires no topic-specific trigger-words, agent access, indirect prompt injection, or fabricated papers, only the open data ecosystem and misleading metadata. To mitigate the attacks, we propose and evaluate two measures: a scientist persona and a data provenance audit with five checks (referencing papers, social markers, statistical anomalies, related datasets, poisoning caution). We find that the persona still leaves 16.67% of runs with a poisoned conclusion, but provenance auditing reduces attack success rate to zero. Our results suggest that indirect data poisoning may enable scientific fraud at unprecedented scale, but these attacks can be mitigated with suitable auditing by agents during data retrieval.

Autonomous vehicles, financial markets, and social media have at least one thing in common: we can describe them as multi-agent systems (MAS), in which several agents, artificial or otherwise, coordinate their actions to reach potentially conflicting goals. Unfortunately, a side effect of automating these complex systems is that increased machine autonomy leads to a decrease in human understanding. When autonomous MAS fail, as is the case with fatal self-driving car accidents, market flash crashes, and social media echo chambers, how can we then guarantee their accountability and transparency when we do not understand what the agents are doing? Motivated by this question, this dissertation explores how people explain the complex behaviour of interacting agents and how we can generate intelligible action explanations automatically with the goal of improving people’s understanding. We begin with first principles rooted in the philosophy of science and cognitive science, focusing particularly on the counterfactual effect size model (CESM) and the framework of explanatory modes, to map what "understanding" means in this context. We investigate naturalistic explanations from human participants using the domain of autonomous vehicles, curating more than 1,300 explanations and 5,200 annotations in the Human Explanations for Autonomous Driving Decisions (HEADD). Our findings indicate that people have different preferences for explanatory modes and ascribe mental states to autonomous agents such as beliefs, desires, and intentions, regardless of whether these agents were described as being operated by humans or machines. Using these insights about people, we formalize what is meant by action explanation in a MAS modelled as a partially observable stochastic game. We integrate the CESM and the framework of explanatory modes with multi-agent simulators to present a general algorithm template whose five subroutines - rollback, simulation, abstraction, inference, and communication - can be adapted to fit any MAS. We present two different instantiations of this algorithm template with different desirable properties and limitations depending on the choice of simulator, and show that each system can correctly and robustly identify the most important causes behind the actions of agents. Through user studies, we also demonstrate that our explanations are preferred by people and can improve their predictions for agents’ goals and next actions. We conclude by discussing the limitations of our approach and the major challenges for the field of explainable MAS, suggesting some ways to gain traction on these problems.

Tensions between AI Safety (AIS) and AI Ethics (AIE) have increasingly surfaced in AI governance and public debates about AI, leading to what we term the "responsible AI divides". We introduce a model that categorizes four modes of engagement with the tensions: radical confrontation, disengagement, compartmentalized coexistence, and critical bridging. We then investigate how critical bridging, with a particular focus on bridging problems, offers one of the most viable constructive paths for advancing responsible AI. Using computational tools to analyze a curated dataset of 3,550 papers, we map the research landscapes of AIE and AIS to identify both distinct and overlapping problems. Our findings point to both thematic divides and overlaps. For example, we find that AIE has long grappled with overcoming injustice and tangible AI harms, whereas AIS has primarily embodied an anticipatory approach focused on the mitigation of risks from AI capabilities. At the same time, we find significant overlap in core research concerns across both AIE and AIS around transparency, reproducibility, and inadequate governance mechanisms. As AIE and AIS continue to evolve, we recommend focusing on bridging problems as a constructive path forward for enhancing collaborative AI governance. We offer a series of recommendations to integrate shared considerations into a collaborative approach to responsible AI. Alongside our proposal, we highlight its limitations and explore open problems for future research. All data including the fully annotated dataset of papers with code to reproduce our figures can be found at: this https URL.

Autonomous multi-agent systems (MAS) are useful for automating complex tasks but raise trust concerns due to risks such as miscoordination or goal misalignment. Explainability is vital for users’ trust calibration, but explainable MAS face challenges due to complex environments, the human factor, and non-standardised evaluation. Leveraging the counterfactual effect size model and LLMs, we propose Agentic eXplanations via Interrogative Simulation (AXIS). AXIS generates human-centred action explanations for multi-agent policies by having an LLM interrogate an environment simulator using prompts like ’whatif’ and ’remove’ to observe and synthesise counterfactual information over multiple rounds. We evaluate AXIS on autonomous driving across ten scenarios for five LLMs with a comprehensive methodology combining robustness, subjective preference, correctness, and goal/action prediction with an external LLM as evaluator. Compared to baselines, AXIS improves perceived explanation correctness by at least 7.7% across all models and goal prediction accuracy by 23% for four models, with comparable action prediction accuracy, achieving the highest scores overall. Our code is open-sourced at this https URL.

Narratives about AI safety have recently ignited significant debate, with profound implications for policy decisions and resource allocation in AI development and regulation. Two distinct perspectives have emerged: one views AI safety primarily as a project for minimizing existential threats of advanced AI, while the other sees it as a natural extension of existing technological safety practices, focusing on immediate and concrete risks of current AI systems. This paper conducts a systematic literature review of primarily peer-reviewed research on AI safety to empirically investigate the diversity of studied safety risks associated with AI systems and their corresponding mitigation strategies. Our review shows a vast array of concrete research, including numerous AI system engineering studies, that address safety concerns irrespective of existential risk considerations. This research includes areas like adversarial robustness and interpretability, highlighting the immediate relevance and importance of AI safety for current AI systems. Based on these empirical findings, we argue that primarily tying AI safety research to efforts for minimizing the existential risks of advanced AI presents an overly narrow perspective on the breadth and scope of AI safety research. Instead, the perception of the scope and aims of AI safety research must be more epistemically inclusive, embracing the diverse concerns and aspirations that shape the ongoing discussion about safe AI. We recognize that the focus on existential risks remains a contested topic within the AI community and more broadly, with many researchers actively engaged in safe AI development and deployment holding neutral or dissenting views about existential risks from advanced AI. Consequently, given our findings, we advocate for a conception of AI safety that is pluralistic and incorporates a wider range of safety considerations, motivations, and perspectives.

It is often argued that effective human-centered explainable artificial intelligence (XAI) should resemble human reasoning. However, empirical investigations of how concepts from cognitive science can aid the design of XAI are lacking. Based on insights from cognitive science, we propose a framework of explanatory modes to analyze how people frame explanations, whether mechanistic, teleological, or counterfactual. Using autonomous driving, a complex safety-critical domain, we conduct an experiment consisting of two studies on (i) how people explain the behavior of a vehicle in 14 unique scenarios (N1=54), and (ii) how they perceive these explanations (N2=382). Our main finding is that participants deem teleological explanations significantly better quality than counterfactual ones, with perceived teleology being the best predictor of perceived quality. Based on our results, we argue that explanatory modes are an important axis of analysis when designing and evaluating XAI and highlight the need for a principled and empirically grounded understanding of the cognitive mechanisms of explanation.