Google says attackers used 100,000+ prompts to try to clone AI chatbot Gemini

Google’s Threat Analysis Group (TAG) revealed in a technical report published this week that its flagship AI chatbot, Gemini, has been targeted by commercially motivated actors using over 100,000 malicious prompts in an attempt to clone the large language model's proprietary data and logic. The cyberattacks, which occurred over several months through various access points, were designed to extract the underlying structures of Google’s artificial intelligence to create imitation software or competitive products without the associated research and development costs. This sophisticated campaign highlights a growing trend of 'model stealing' or 'distillation attacks' within the tech industry, where rival developers or malicious entities leverage a high-performing AI to train their own smaller, cheaper versions. Technically, these attackers utilize high volumes of queries to map out how Gemini responds to specific inputs, effectively reverse-engineering the model's decision-making processes. By capturing the outputs of more than 100,000 carefully crafted prompts, the attackers can create a synthetic dataset that mimics the behavior and quality of Google’s original system. Google noted that while Gemini is equipped with robust safety filters and rate limits, the sheer scale of the automated requests suggests a highly organized effort by entities looking to shortcut the expensive process of training advanced generative AI from scratch. This security revelation comes as the broader tech community grapples with the vulnerabilities of generative AI to specialized cyber threats. Beyond traditional data breaches, AI companies must now defend against information extraction and adversarial attacks that target the 'intelligence' of the software itself. Google emphasized that it has implemented enhanced monitoring and defense mechanisms to identify these patterns of mass-scale prompting, ensuring that its intellectual property remains protected from competitors who seek to bypass legitimate licensing and development phases.