Do Neutral Prompts Produce Insecure Code? FormAI-v2 Dataset: Labelling Vulnerabilities in Code Generated by Large Language Models

no code implementations • 29 Apr 2024 • Norbert Tihanyi, Tamas Bisztray, Mohamed Amine Ferrag, Ridhi Jain, Lucas C. Cordeiro

This study provides a comparative analysis of state-of-the-art large language models (LLMs), analyzing how likely they generate vulnerabilities when writing simple C programs using a neutral zero-shot prompt.

Code Generation

LLMs in Web Development: Evaluating LLM-Generated PHP Code Unveiling Vulnerabilities and Limitations

1 code implementation • 21 Apr 2024 • Rebeka Tóth, Tamas Bisztray, László Erdodi

This study evaluates the security of web application code generated by Large Language Models, analyzing 2, 500 GPT-4 generated PHP websites.

The FormAI Dataset: Generative AI in Software Security Through the Lens of Formal Verification

no code implementations • 5 Jul 2023 • Norbert Tihanyi, Tamas Bisztray, Ridhi Jain, Mohamed Amine Ferrag, Lucas C. Cordeiro, Vasileios Mavroeidis

This paper presents the FormAI dataset, a large collection of 112, 000 AI-generated compilable and independent C programs with vulnerability classification.