AI |

Responsible AI in RSE - a living register to support practitioners, leaders and policymakers

Tue, 14 Jul 2026 00:00:00 +0000

Note

This collaborative post was originally published at the and was authored by some of the Responsible AI Study Group members: Joseph Shingleton, Carlos Cámara-Menoyo, Oscar Seip, Stephan Druskat, Sam Harrison, Arielle Bennett and B. Almarzouq.

The research software community is nothing if not diverse. It brings together practitioners loosely connected by their relationship with the broad and sometimes nebulous idea of ‘research software’. The day-to-day experience of, say, an engineer developing and maintaining computational fluid dynamics simulations is very different from that of a data architect supporting clinical data pipelines. It is unsurprising, then, that this diversity of experience gives rise to a wide range of hopes, concerns, and uncertainties - particularly within the context of Generative Artificial Intelligence (GenAI).

Research software’s rich diversity of experience is explicitly recognised in the Alliance for Data Science and AI’s (ADSA) and US-RSE’s recent community in the RSE workplace:

“The conversation around software engineering in the age of GenAI has […] left relatively little space for nuanced discussions of […] the complexities of developing research software in a future where GenAI is widely available.”

The nuance and complexity identified by ADSA make the development of GenAI guidance and policy particularly challenging. In the context of a highly diverse, reactive, and evolving discipline like research software engineering, there is no “one-size-fits-all” approach to GenAI governance. Rather, we must develop systems that capture the nuanced and shifting concerns of practitioners, leaders, funders and other stakeholders; systems that can also act as an evolving reference for best practice. The (RAIRR) is intended as one such system.

RAIRR is a living register designed to support open discussion of how GenAI adoption is affecting research software practice. It allows contributors to document risks identified in their day-to-day work, alongside possible mitigations, examples, and shared responsibility for responding to them. We expect the register to become a useful resource to inform the development of AI policies that are grounded in the actual experience of RSE practitioners, rather than hype, promises, expectations or unfounded fears. RAIRR is open for anyone to use and contribute to, being particularly relevant to practitioners, leaders and policymakers.

Pilot and technical implementation

We introduced RAIRR at the as an opportunity to test the idea in practice. Working in small groups, attendees were asked to identify the ways in which GenAI adoption might lead to harm, failure, or undesirable outcomes in their professional work. Crucially, they were also asked to consider valid approaches to mitigation of these risks, and suggest examples of best practices they have seen, either in institutional guidance or individual practice. This provides a pragmatic and actionable account of practitioners’ perception of AI risk in research software.

An example of one such risk is shown below ( ), which highlights the potential for the accidental reuse of licensed code in software with incompatible licenses (e.g. the use of copyleft-licensed code within more permissively licensed software). Given the provenance of AI-generated code is not visible to developers using GenAI (i.e. there is no way of knowing whether the generated code is mostly or entirely based on code with a restrictive license), there is a real risk that such code will be unintentionally reused in violation of licenses.

Risks can be added to and updated in the register through issue templates provided in the repository. As well as adding new risks or updating existing ones, the templates also allow users to add relevant or interesting resources. Since each risk is directly associated with a GitHub issue, users wishing to discuss a risk further can easily do so by adding a comment to the issue. This facilitates open discussion and allows users to add additional context or framing to existing risks. Of course, some users may wish to submit more sensitive risks – in such cases, an alternate route for anonymous contributions via an online form is also available.

What’s next?

The register is open to new contributions describing risks around AI adoption within research software practices. We also welcome discussion of existing risks via their associated GitHub Issues, and encourage people to add examples of how risks can be mitigated, and to point to examples of best practice. We will be running a further workshop at RSECon26 in Sheffield, so please keep an eye out for opportunities for collaborative contribution sessions.

Contributing to RAIRR

You can view the full register and contribute to it . Please make sure to read the and for more information about what makes a good contribution to the register.

The register is maintained by the SSI Responsible AI Study Group – a recently established group interested in understanding ways in which AI may be adopted safely, fairly, and constructively within research software practices. The group is open to new members from both within the SSI and externally – if you’re interested in joining, then please contact joseph.shingleton[at]glasgow.ac.uk or oscar.seip[at]manchester.ac.uk.

The UK must use the Break Clause and Terminate its Contract with Palantir

Sun, 14 Jun 2026 00:00:00 +0000

Note

This collaborative post was written by the Rights and Tech Group at the University of Warwick and was originally published at

As the digital transformation of health continues to reshape health systems globally, states face a stark choice: how to strengthen cash-strapped health systems and get command of health data? While some might think high-income countries are better at protecting their sensitive health data than low-income countries, in fact the UK’s current problems with security and defence firm Palantir show no one is immune to the pressures of the private sector. In this piece, we outline some of the key concerns that emerge through the outsourcing of critical infrastructure to the private sector in the case of the UK. We make an argument that given these concerns, the UK should exercise its break clause and terminate Palantir’s contract in 2027.

Photo by

In 2023, the NHS awarded U.S. data analytics firm Palantir a to provide software and manage the NHS Federated Data Platform to connect fragmented NHS databases and allow data from different systems to be shared and analysed. While the digital transformation of health has sped forward, it has led to a proliferation of tools and platforms, creating siloed pockets of health data that cannot easily be shared from clinic to hospital to decision-maker. This fragmentation creates multiple challenges, ranging from sharing lab results in time with doctors who need them, to spotting the next epidemic in time to prevent its spread. For cash-strapped health agencies, the appeal of Palantir is its promise to improve efficiency by integrating data and making it easier for diverse platforms to connect, communicate, and use the data for public health.

Health data, however, is extremely sensitive, and without building necessary governance safeguards, purpose limitations, and transparent data processing methodologies, the creation of large and centralised data infrastructures poses serious risks for everyone; in particular, for marginalised groups at greater risk of violence, discrimination and inequitable access to healthcare. These populations are the ones whose trust the NHS most needs to keep. The , is an illustration of how such incidents can undermine trust and drive those away from health services who need them most.

As a group of scholars working on digital rights at the University of Warwick, we are concerned and echo civil society calls for the UK government to exercise the break clause in the contract and terminate the agreement with Palantir for the reasons set out below:

Ideological incompatibility

Palantir’s appeal for those who manage public services is that it provides efficiencies at scale. However, as the recently published by the co-founder, Alex Karp, articulates a world view that is imperialist, US-centric, anti-democratic, anti-pluralist, and openly hostile to public institutions, including . Reflecting on the Palantir manifesto, philosopher Mark argues that ‘efficiency’ is framed as a political value, which takes precedence over values of pluralism, democratic deliberation and institutional independence.

This demonstrates that Palantir is not merely a tech company offering a technical solution, but an ideological project that is fundamentally at odds with the values of a democratic and multicultural UK. In addition, Palantir’s close ties with the US military and whose technology has been key to target citizens in warfare and border control in the US and internationally raises serious questions about its political independence and ethics. For the UK government, continuing the NHS contract, therefore, requires reflection on how it can justify working with a company whose stated principles undermine the very norms upon which public services depend.

Institutional dependency and loss of sovereignty

The continuation of Palantir’s role within the NHS must also be understood in the broader context of the company’s expanding deals with other wings of government. Palantir now provides or is in talks to provide services to the , the , , and . These institutions have distinct functions, mandates and forms of expertise. Yet the increasingly blur these boundaries and erode institutional independence.

The fact that Palantir is being used in other UK sectors means that it can require the NHS or local governments to extend their contracts to those areas to “increase functionality”. This participates in a race of tech companies for the monopolisation of public services, especially as this is where the “big money” is. Although it might provide faster access to records (for example, during an emergency surgery), the risks of hyper-centralised data infrastructures are substantial. These include increased vulnerability to hacking, physical fragility (where the failure of a single node can disrupt the entire system), and the potential for co-option by “unwanted” or “unelected” actors.

Further, through the outsourcing of such critical department functions to Palantir, the state cedes its sovereignty to private enterprises. Critical media scholars Grohman and Barbosa call this as . The successive adoption of Palantir across sectors renders the sovereignty of the UK state to be now purchased and offered on the terms of a platform as part of its proprietary offering.

Take the promise of interoperability. Although companies claim that they can deal with many different datasets from different sources, the proprietary condition of their software means that they are not interoperable with other platforms and datasets. This means in practice that the client is made dependant on their services across governmental areas and over time. This results in transnational big tech companies monopolising local and national services, eliminating competition and locking governments to pay for unsolicited updates and upgrades of their systems. It is, therefore, critical that the UK government does not become just a consumer of outsourced services. As a democratic state, the UK government derives legitimacy from the public and must retain the capacity to collaborate with multiple suppliers rather than surrendering control to a single platform provider whose CEO has publicly declared (and demonstrated) their allegiance to the US. The case of NHS and Palantir poses a further geopolitical question, which is whether critical infrastructures such as public health providers can be in the hands of powerful transnational companies.

Erosion of trust, especially among marginalised groups

Allowing Palantir access to data on patients, and in particular on patients from marginalised communities, also raises serious concerns. For instance, racialised communities and people with insecure legal status already worry about sharing data between the NHS and the Home Office. This has happened in the US, where Palantir’s systems facilitated data sharing and identification to ), which was then used in raids to unlawfully target and deport migrants. Palantir’s capacity to cross personal data with surveillance technologies or to share it with other bodies, like the police, increases fear and anxiety among vulnerable populations, damaging their trust in the public health system. As a result, they may avoid seeking healthcare altogether: for example, a migrant with an infectious disease might stay away from their GP out of fear of being interrogated or deported. This loss of trust undermines the right to healthcare for all.

Maldistribution of resources

Finally, the substantial sums allocated to Palantir to facilitate managerial decisions in the NHS are drawn from wider public budgets. This diverts resources way from welfare provision and eliminates services that are essential for addressing social inequality and supporting the most vulnerable members of society. The promises of combining large amounts of data and having speedy decision-making cannot replace the need for adequate healthcare infrastructure or the labour of care provided by healthcare professionals, from doctors and nurses to cleaners and other employees. These raise pressing questions. What are the broader implications of these cuts and repurposing of public resources? Are any of the savings that the use of Palantir is promising to be ploughed back into the healthcare system, for example by fixing hospitals, hiring more healthcare workers, or improving working conditions? There is a serious risk that this approach will amplify existing inequalities in access to, and quality of, healthcare across the UK (see or a survey from ). Palantir is monetising these inequalities and presenting technological solutions as quick fixes for deeply structural problems, which risk being exacerbated.

Palantir’s involvement in NHS databases has received significant reaction from civil society. Campaigns led by organisations, such as the and highlight the lack of transparency and human rights due diligence in the contractual process, as well as the dangers of infrastructural lock-in. Palantir’s purported is also called into question. These concerns provide compelling grounds for the UK government to exercise the break clause and terminate its agreement with Palantir, as system has. The decision should be based not only on technical considerations, but also on fundamental issues of ideological incompatibility, institutional dependency, digital sovereignty, public trust and unjust distribution of public resources.

Call to action

February 2027 marks a window of opportunity . For the reasons argued here (ideological incompatibility; loss of sovereignty; erosion of trust; and maldistribution of resources), we enjoin the UK government to exercise the break clause in the contract and terminate the agreement with Palantir as a first and immediate step towards ensuring that the rights and data of all UK citizens (and particularly, the most vulnerable) are protected. However, this should not be the end goal: other actors may come and pose similar threats. The case of Palantir has surfaced a series of structural issues when it comes to digital rights and technology providers. We have highlighted a few: how can digital sovereignty for critical infrastructures be preserved? How can vendor lock-in can be avoided? How can data from subjects be protected from unlawful or nefarious purposes? Who benefits the most with these contracts? How can any efficiency savings be redirected to underfunded and under-resourced critical services? We contend that technological decisions are also political and, therefore, call on the UK government to ensure these issues are addressed.

The case of the UK and Palantir is not unique, but it is paradigmatic of a global trend in turning data, and particularly, health data, into commodities that threaten the human subjects that lie behind those figures. As the World Health Organization drafts its new Global Strategy on Digital Health 2028-33, the challenges faced by the UK should spark broader discussions about health data governance, with development of new and clearer standards so that all countries establish clearer guardrails to ensure that the public sector continues to serve the public interest.

Author’s bios

Siddharth Peter de Souza is an Assistant Professor at the Centre for Interdisciplinary Methodologies, University of Warwick. His research and teaching focus on how data is governed globally in contested, and plural settings. He is the author of Designing Indicators for a Plural Legal World (Cambridge 2022)

Derya Ozkul is an Associate Professor and Co-Director of the Social Theory Centre at the Department of Sociology, University of Warwick. Her teaching and research focus on the politics of migration and (im)mobility in the digital age.

Serena Natile is an Associate Professor at Warwick Law School. Her work on digital technology is mainly concerned with questions of social reproduction and maldistribution, at the intersection of law, feminist political economy, labour, and finance. She is the author of (Routledge 2020) and is currently completing the ISRF project .

Carlos Cámara-Menoyo is a Senior Research Software Engineer at the Centre for Interdisciplinary Methodologies, University of Warwick. He is an interdisciplinary researcher whose work sits at the intersection of technology and society, with a particular focus on the ways in which worldviews embedded in technology shape the experiences of under-represented communities.

Nerea Calvillo Gonzalez is a Reader at the Centre for Interdisciplinary Methodologies, University of Warwick. They work on digital infrastructures, environmental pollution and queer ecologies, and they are author of Aeropolis: Queering air in toxipolluted worlds (Columbia, 2023).

Sara (Meg) Davis is a Professor at the Centre for Interdisciplinary Methodologies, University of Warwick; and Principal Investigator for the Digital Health and Rights Project, an international consortium of academics, health advocates, human rights lawyers, and communities of people living with and affected by HIV. She is author of The Uncounted: Politics of data in global health (Cambridge 2020).

No Vibes, Just Struggle: Notes From a Research Software Engineer on Vive Coding

Thu, 11 Jun 2026 00:00:00 +0000

Last 14th May, I participated in the workshop “Convivial Code: The Politics and Pedagogies of Vibe Coding”, organised by the and the at the University of Warwick. The workshop takes a cue from Illich’s notion of convivial tools: technics that expand autonomous, imaginative use without requiring prior certification or imposing compulsory dependence and moved beyond vibe-coding specifically to further ask what a politics of convivial code might look like today? (e.g., publicly-led, democratic governance of technology capable of attending to the challenges of the current conjuncture: pluralization versus monopolization, autonomy versus dependence, and situated invention versus the norm-seeking tendencies of generative output).

The workshop was a hybrid session combining methods workshop, studio and (see recording below) to chart this domain; where staff and student participants tested AI infrastructures, discussed their implications and develop a shared critical understanding of the potentials, limits and harms of AI-assisted programming. I had the opportunity to give a short talk in which I shared my positionality and concerns in relation to AI in general and vive coding in particular. The talk was controversial, insofar it raised some opposition, but also interesting questions and reactions, so I wanted to share it here, hopefully to continue discussions.

No Vibes, Just Struggle: Notes From a Research Software Engineer

I’m going to share a couple of struggles that I’m experiencing in my everyday work as a Senior Research Software Engineer at the Centre for Interdisciplinary Methodologies. But before doing so, I need to disclose my positionality in relation to AI and vive coding, particularly about its meaning and how I use it.

Positionality

Coding is a very polisemic word, and people may assign different meanings depending on their background or context. As a Research Software Engineer, when I think about coding I am mainly referring to the action of creating scripts or software that can be used in an academic setting. This setting is very important to me because it comes with a series of values and requirements that are different from other settings (e.g. hobbysts or industry), such as reproducibility, provenance, or correctly citing sources as an integral part of academic integrity.

I have never vive coded. I’ve seldom used vector-mediated coding: when I use it, I do it reluctantly or triggered by curiosity. Either case, I always feel it as a defeat. This is not because I’m so proud that I feel my ego threatened, or because I’m a good or fast coder that doesn’t need such aids. I’m actually neither. Probably because of that, through commons-based peer production, I’ve learnt a series of techniques, good practices and cultural norms that helped me to overcome my individual limitations. Two paradigmatic examples are to use and adapt code and frameworks that other, smarter people, have created and decided to share it with others through open licenses; and to rely on processes and automations¹ to reduce errors, to understand when something doesn’t work and to seek help when I can’t find the answer, and to do so, to be able to explain to others what I want to do and what I’ve done.

Why I’m struggling with AI

Now that my stance is clear, I’m in a better condition to share my struggles with AI in general and AI-assisted coding.

The first has to do with the fact that technology is never neutral, and AI has been built through extractive and exploitive practices that are assumed, validated and reproduced when using them. Examples of this can be seen in how LLMs have been trained by systematically stealing IP, or by employing humans to manually tag content in terrible working conditions and at the expense of their mental health.

And not to mention the large amount of resources this technology and the required infrastructure (i.e., datacentres) require for functioning, such as energy, water, oil but also hardware such as RAM or SSDs and Hard drives. Even though the tech and the companies behind ofuscate date to make it really difficult to quantify the environmental costs (and therefore, making it easier to be disregarded), we are already seeing their effects. This week alone news echoed how Maryland taxpayers would need to pay $2 billion dollars to upgrade the power grid to host data centers, or that important SSD and HDD manufacturers have sold all their production until 2029 exclusively to serve Data center needs, creating new forms of scarcity.

But probably, the clearer example about the worldviews and political agenda that are encapsulated in this technology is how AI companies are directly or indirectly promoting surveillance, and even genocides, and how their CEOs feel empowered enough to publish supremacist manifestos in the name of their companies ².

The combination of these factors makes me feel that every time I use AI-tools I’m forced to disregard my values and the political and moral beliefs that I stand for, and makes me feel closer to a world and a society I wouldn’t wannt anyone to live in. And this is not an easy pill to swallow.

The second one has to do with AI assisted coding, or vector coding. It is easy to be flashed by the ability to produce something that works in a reduced amount of time and with few or little knowledge on coding. It is really impressing to see that what would take a lot of time is accomplished (at least in appearance) in a matter of minutes, and with little effort. I can see how many people (particularly those who do not know how to code or are in early stages) may see it as a way to overcome their limitations and how this seems like it’s democratising coding. I really do, I would believe it myself it wasn’t because it is the opposite.

Vive coding produces code, but it does not make programmers, it makes prompters. This is not just a semantic difference. I contend that the former is empowering, whereas the latter is disempowering. Vive coding epitomises neoliberal worldviews by stating that only the outputs matter³, disregarding decades of disciplinary and interdisciplinary knowledge built over decades. Coding is not about just producing code, it is also a cultural and social practice, a method. Equally as important than the output are issues related to:

Correctly attributing everybody’s work, and complying with licences. Using AI can easily lead to copyright infringements and flags questions about Intellecutal Property and accountability: Who is the author of the code? Can we make sure that we are complying with the licence terms of the code that the AI is reproducing?Can we be sure that we are not leaking any personal or sensitive information (either in the code or the prompts)?.
Writing understandable and explainable code, not just to make it easier for us to maintain and improve it over time and for others to contribute to or scrutinise (again, this is important in an academic setting). If we cannot explain what the code does, how can our research rely on it? Can we use it in research if we cannot write a basic methods section?
Considering and addressing failure. Any relatively complex piece of code will, inevitabily, fail. Far from approaching failuire from a nihilistic standpoint, software engineer need to actively address them by putting measures to minimise the errors (e.g. automated testing), or to be able to debug them when they ocurr so they are easily fixable. Or, as Cory Doctorov : “Writing code” is about making code that runs well. “Software engineering” is about making code that fails well". How are we going to maintain the software in the medium and long term? What will we do when it fails? (because inevitabily, all code fails) Shall we start over again? Is then efficient?
Saving resources, both material and human. Is it fair to ask others to read, understand, test, and improve code that can be generated at faster pace than any human can review? Many projects⁴, such as drupal⁵ or some GNU/Linux distributions, are not accepting contributions if the authors cannot explain them, as it suposes a liability but also a waste of resources and time from a team that has already a lot on their plate.

But probably the most important flaw in this framing is how it shifts the paradigm away from processes. Because programing is a never-ending task, it requires constant iteration, improvement, learning… it is, above all, a process. And so it is the learning process that makes you be a (good) programmer. Admittedly, coding is challenging, it requires a series of skills that take time to learn, and even more to master. But once learnt, they are ours, nobody can take that from us. This is what is really empowering. On the other hand, being a promter means relying on something external (AI) to generate code. What will a prompter do if they lose access to those tools? Considering the track history of enshitification and lack of human consideration of these companies, this is not a hypothetical question.

Code linters, Continuous Integration, Automated testing… are examples of such automations that are not AI-powered and have been commonplace in software development. ↩︎
See excerpts from Alexander C. Karp & Nicholas W. Zamiska’s The Technological Republic: Hard Power, Soft Belief, and the Future of the West at Palantir’s X account: ↩︎
Open-AI cofounder Andrej Karpathy coined the concept of vive coding as this ( ):

There’s a new kind of coding I call “vibe coding”, where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. It’s possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like “decrease the padding on the sidebar by half” because I’m too lazy to find it. I “Accept All” always, I don’t read the diffs anymore. When I get error messages I just copy paste them in with no comment, usually that fixes it. The code grows beyond my usual comprehension, I’d have to really read through it for a while. Sometimes the LLMs can’t fix a bug so I just work around it or ask for random changes until it goes away. It’s not too bad for throwaway weekend projects, but still quite amusing. I’m building a project or webapp, but it’s not really coding - I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works.

↩︎
Refer to the for a complete list of software that are not accepting any form of AI, and an exhaustive rationale for not doing that. ↩︎
Drupal’s founder’s blog: “ ” ↩︎