Nvidiaannounced an AI guardrail service called "NIM," which is now available as NVIDIA's NeMo Guardrails suite, enabling developers to add a set of guardrail rules to Large Language Models (LLMs) designed to address the issue of users "jailbreaking" the Large Model by prompting for words that do not meet expectations. The goal is to address the issue of users "jailbreaking" the LLM by prompting words, preventing the AI from generating content that doesn't meet expectations.
NVIDIA says the corresponding AI fencing service suiteTrained on NVIDIA's Aegis content security datasetThe dataset contains 35,000 labeled data samples and is publicly available on Hugging Face at the following address (Click here to visit).
NVIDIA pointed out that the corresponding AI fencing suite is characterized by small size and high efficiency, and can run smoothly in most occasions. Enterprises can directly embed the corresponding security suite in the development of AI models, which can improve AI security deployed in healthcare, automotive, manufacturing, and other fields.
In addition, NVIDIA has provided announced a vulnerability scanning tool called Garak to test the security of models against the possibility of them outputting hallucinatory content or leaking confidential information within the organization.
