Behind the popularity of generative AI, security, privacy and reliability issues are becoming increasingly prominent.MicrosoftIn order to prevent the occurrence of Supremacy AGI (AI that claims to control the human world),A series of solutions have been launched recently to prevent generative AI from getting out of control.
Microsoft said in an official announcement: "How generative AI can effectively prevent prompt word injection attacks has become a major challenge. In this attack, malicious actors try to manipulate artificial intelligence systems to do something beyond their intended purpose, such as creating harmful content or leaking confidential data."
Microsoft first restricted Copilot In addition, Microsoft introduced a "Groundedness Detection" feature designed to help users identify text-based hallucinations.
This feature will automatically detect “ungrounded material” in text to support the quality of LLM output, ultimately improving quality and confidence.
The relevant tools are introduced as follows:
-
Prompt Shields: for detecting and blocking prompt injection attacks. Includes new models for identifying indirect prompt attacks before they impact your models, coming soon and now available in preview in Azure AI Content Safety.
-
Groundedness detection: Mainly used to detect "hallucination" situations in model output, coming soon.
-
Safety system messages: Guide your model in a safe and responsible direction.
-
Safety evaluations: used to evaluate the vulnerability of applications to jailbreak attacks and generated content risks, now available in preview.
-
Risk and safety monitoring: Understand which model inputs, outputs, and end users trigger content filters to inform risk mitigation, coming soon and currently in preview on the Azure OpenAI service.