PolicyPad: Collaborative Prototyping of LLM Policies

WHO: Domain experts involved in high‑stakes areas such as mental health. WHAT: An interactive system called PolicyPad that enables collaborative prototyping of policies for large language models (LLMs). WHERE: The system was developed and piloted within a series of policymaking workshops held across multiple venues. WHEN: The researchers conducted 19 workshops with nine experts over a 15‑week observation period. WHY: The aim was to improve the speed and quality of policy design by supporting rapid experimentation, continuous feedback, and iterative refinement, thereby enhancing the safety and accountability of LLM deployments. PolicyPad was introduced as a response to gaps identified during the workshops, where participants highlighted difficulties in iterating on policy drafts, gathering timely feedback, and coordinating across stakeholders. The system leverages an interactive interface to allow experts to draft, test, and modify policy rules in real time, with visibility into how changes affect model outputs. Early pilots suggest that the tool shortens cycle times for policy iteration and increases transparency in the design process. The work, published as arXiv:2509.19680v2, proposes a framework for embedding human expertise directly into the lifecycles of LLM governance. Key to PolicyPad’s approach is the emphasis on structured collaboration: policy modules can be swapped, versioned, and annotated, while a shared repository tracks the evolution of rules and justifications. This infrastructure supports both sprint‑style rapid prototyping and more deliberative, long‑term policy refinement. The authors argue that as LLMs become integral in critical sectors, tools like PolicyPad will be essential for ensuring that domain experts can keep pace with technological advances and safeguard user well-being.