Abstract
Data-driven decision support in high-stakes domains like healthcare and finance faces significant barriers to data sharing due to regulatory, institutional, and privacy concerns. While recent generative AI models, such as large language models, have shown impressive performance in open-domain tasks, their adoption in sensitive environments remains limited by unpredictable behaviors and insufficient privacy-preserving datasets for benchmarking. Existing anonymization methods are often inadequate, especially for unstructured text, as redaction and masking can still allow re-identification. Differential Privacy (DP) offers a principled alternative, enabling the generation of synthetic data with formal privacy assurances. In this work, we address these challenges through three key contributions. First, we introduce a comprehensive evaluation framework with standardized utility and fidelity metrics, encompassing nine curated datasets that capture domain-specific complexities such as technical jargon, long-context dependencies, and specialized document structures. Second, we conduct a large-scale empirical study benchmarking state-of-the-art DP text generation methods and LLMs of varying sizes and different fine-tuning strategies, revealing that high-quality domain-specific synthetic data generation under DP constraints remains an unsolved challenge, with performance degrading as domain complexity increases. Third, we develop a membership inference attack (MIA) methodology tailored for synthetic text, providing first empirical evidence that the use of public datasets - potentially present in pre-training corpora - can invalidate claimed privacy guarantees. Our findings underscore the urgent need for rigorous privacy auditing and highlight persistent gaps between open-domain and specialist evaluations, informing responsible deployment of generative AI in privacy-sensitive, high-stakes settings.
Abstract (translated)
在高风险领域（如医疗和金融）中，基于数据的决策支持面临着由于监管、机构和隐私问题而导致的数据共享障碍。尽管最近的生成式人工智能模型（例如大规模语言模型）在开放域任务中表现出色，但它们在敏感环境中的应用仍受限于不可预测的行为以及缺乏用于基准测试的隐私保护数据集。现有的匿名化方法通常不足以处理非结构化的文本，因为即使删除和屏蔽也可能允许重新识别信息。差分隐私（DP）提供了一种有原则的替代方案，它能够生成具有正式隐私保证的合成数据。 在本研究中，我们通过三项关键贡献来应对这些挑战： 1. 我们引入了一个全面的评估框架，包括标准化的实用性和保真度指标，并涵盖了九个经过策划的数据集，捕捉特定领域的复杂性，如技术术语、长上下文依赖和专门化的文档结构。 2. 我们进行了一项大规模实证研究，对最先进的差分隐私文本生成方法以及不同规模和微调策略的语言模型进行了基准测试。结果表明，在差分隐私约束下生成高质量的领域特定合成数据仍然是一个未解决的问题，并且随着领域复杂性的增加，性能会下降。 3. 我们开发了一种专门针对合成文本的成员推断攻击（MIA）方法，提供了首次实验证据，证明使用公共数据集——可能存在于预训练语料库中——可以使声称的隐私保证无效。 我们的发现强调了对生成式人工智能在隐私敏感、高风险环境中的负责任部署进行严格隐私审核的紧迫需求，并且指出了开放领域与专业评估之间的持久差距。
URL
https://arxiv.org/abs/2509.14594
