Seeing the System: Assemblage Mapping and Accountability in AI-Supported Advice
- 32 minutes ago
- 5 min read
Kryss Macleod, Principal Lecturer, Manchester Law School, Manchester Metropolitan University
Bruce Austin, Manchester Metropolitan University/Citizens Advice SORT
Keywords: AI and legal advice; access to justice; lawtech; accountability infrastructure; human-in-the-loop; assemblage mapping
Legal advice is increasingly being reorganised at the level of infrastructure. Lawyers, advisers and advice organisations may use AI to draft, search or summarise; but the deeper shift is that legally consequential guidance is increasingly produced through configurations of people, documents, interfaces, retrieval systems, prompts, models, supervision routes and records. Access to justice depends not only on whether help is available, but also on whether it can be explained, checked, challenged and corrected.
Across England and Wales, large areas of legal need go without professional assistance. Early legal advice remains a gateway to rights, remedies and participation in the rule of law; the House of Lords Constitution Committee described accessible and affordable legal advice as a “key enabler of the rule of law”, while noting that the sector is “under pressure”.
AI-supported tools are often cast as an access-to-justice breakthrough, able to scale triage, drafting, search and advice work. The promise is real, but it is not self-executing: without adequate safeguards, they also risk creating thinner or two-tier forms of legal help through opacity, bias, weakened oversight, confidentiality breach and diminished care. The hard question is less if AI can produce useful legal text, but whether AI-supported advice can be made accountable.
Understanding AI in advice settings
We address that question through an assemblage mapping of Caddy, an AI-supported welfare advice tool developed at Citizens Advice Stockport, Oldham, Rochdale and Trafford (CASORT). Caddy is an AI copilot for front-line contact agents, drawing on verified Citizens Advice and GOV.UK sources.
Its significance lies in that awkward fit: Caddy is not solicitor-client advice in a regulated law firm, yet it operates in a setting where guidance may shape legally consequential decisions. The Legal Services Act 2007 reserves only certain activities for Authorised Persons, including rights of audience, conduct of litigation, reserved instrument activities, probate, notarial activities and administration of oaths. Much AI-supported guidance may fall outside these activities, while still shaping decisions about benefits, housing, debt, immigration, employment or family problems.
Assemblage mapping is useful precisely at this boundary. Drawing on assemblage and socio-material approaches, it treats Caddy not as a single tool but as a configuration of people, documents, interfaces, workflows, legal materials, prompts, retrieval systems, records and repair mechanisms. Assemblage approaches direct attention to how heterogeneous elements are brought into relation, and how those arrangements produce effects that cannot be explained by any one component alone (see Bennett and DeLanda).
Much legal AI debate focuses on the model, the lawyer using it, or the danger of hallucinated outputs. But AI in legal services is an infrastructure problem, not just a tool problem. Components of this infrastructure include corpus governance, retrieval ranking, prompt design, model selection, human review, supervision, audit logs, feedback, redress and repair.
The point is not confined to current LLM or retrieval-based tools: it will become more acute as agentic AI systems move from generating text to coordinating multi-step tasks. Where a system can draft, route, retrieve, recommend or escalate, accountability depends on understanding how those actions are authorised, constrained, recorded and reviewed.
Beyond Human in the Loop
This account challenges thin reliance on “human-in-the-loop”. In legal services, the human is often imagined as the solicitor, barrister, trainee, caseworker or supervisor who checks an AI-generated output. Yet review is meaningful only if the reviewer has enough visibility and authority: access to sources and source dates, understanding of system limits, power to reject or escalate uncertainty, and an auditable trace of what they saw and did. Without those conditions, the human may become a visible bearer of responsibility without the practical control needed to exercise it (see Elish and Nissenbaum).
Regulatory debate is already moving in this direction. The SRA has identified opportunities and risks for legal services using AI, while the Legal Services Board has framed AI regulation as pro-innovation but consumer-protective. In 2026, LSB Chief Executive Richard Orpin noted AI is blurring the boundary between regulated and unregulated legal advice, further strengthening the case for reviewing reserved activities.
Assemblage mapping turns those concerns into practical questions: what conditions does the organisation work within, what sources does the system rely on, who updates them, can the adviser inspect provenance, what prompts shape certainty and scope, what model was used, what is logged, and can errors be reconstructed?
Accountability and responsibility
In retrieval-based advice systems, the legal source corpus is not background material. It is an accountability object. Responsibility depends on which sources are included, who selected them, when they are updated, how retrieval and ranking work, whether stale sources are detected and whether advisers can inspect provenance. The corpus and retrieval layer therefore function as legal-authority infrastructure: they shape which legal materials become visible, usable and authoritative within the advice process.
Session logs and feedback make accountability reconstructable. If advice is challenged, it may matter what query was entered, which sources were retrieved, what prompt and model were used, what the adviser saw or edited, whether supervision was requested, and whether the issue led to repair. The question is whether the system leaves durable traces that make later scrutiny possible.
Responsibility in legal work, on this account, cannot be confined to the authorised professional or regulated entity. AI-supported advice cuts across the categories on which traditional legal-services regulation often depends. Developers configure prompts; product leads shape source selection; cloud providers host systems; retrieval tools rank materials; supervisors review exceptions; advisers authorise outputs. These actors do not all “give advice”, but they configure its production. Responsibility is increasingly assembled across roles, not merely attached to the professional visible at the end of the chain.
Confidentiality is also infrastructural. In AI-supported advice, client information may pass through prompts, logs, redaction tools, retrieval pipelines, cloud infrastructure, model providers, staff practices and organisational rules. Assemblage mapping makes visible that confidentiality depends on how these elements are brought into relation: who can access what, what is minimised or retained, which vendors process information, how staff are trained, what gets logged, and whether those arrangements can be audited when something goes wrong.
In welfare advice, an error does not remain abstract: a poor explanation or missed escalation may affect benefit entitlement, housing security, debt enforcement, immigration position, disability-related support or safeguarding. The accountability risk is that AI may widen access to help while weakening the conditions that make legal guidance robust and answerable: explanation, contestation, correction and remedy.
The regulatory question can no longer stop at whether a qualified professional is formally responsible for advice. It must also ask whether the organisation has assembled the conditions under which AI-supported advice can be seen, checked, justified, challenged and repaired. It shows that legally consequential help is already being assembled across advice-sector, technical and regulatory boundaries. In that setting, professional status, reserved activity, and human review do not tell us enough about whether users are protected.
Future governance must look beyond professional status and model risk to the infrastructure that makes advice accountable in practice: how it is produced, authorised, evidenced, challenged and repaired. Without that infrastructure, responsibility may remain formally attached while accountability becomes too thin to protect the people relying on the advice.