Tech

New EU AI Act Regulation Exposes AI Gaps 

  • Reading time: 2 min
the EU introduced the EU AI Act regulation as a framework for evaluating software cybersecurity and addressing discriminatory outputs. 

Following the implementations of several regulations and expressing concerns about security and privacy in AI models, the EU introduced the EU AI Act regulation as a framework for evaluating software cybersecurity and addressing discriminatory outputs. 

After OpenAI launched ChatGPT in late 2023, the EU proposed AI regulation and started working on specific legislation for “general-purpose” AI.  

LatticeFlow, in collaboration with ETH Zurich and the Institute for Computer Science, Artificial Intelligence and Technology (INSAIT), designed a tool called the “Large Language Model Checker,” which tests models from tech majors like Meta and OpenAI in various categories for compliance. 

The latest leaderboard from LatticeFlow put models from Alibaba, Anthropic, OpenAI, Meta, and Mistral above an average score of 0.75. However, it noted the Large Language Model (LLM) Checker signaled several compliance gaps that may call for resource reallocations for these firms. 

EU regulation for AI Compliance Challenges Ahead 

Any EU AI Act regulation violation can be fined up to 35 million euros ($38 million) or 7% of a company’s global annual turnover.   

As the Union’s countries work to enforce its EU AI regulations, it is gathering experts to draft a code of practice for regulating generative AI tools, expected by spring 2025.  

Analyses by LatticeFlow show that discriminatory outputs remain a big issue, with AI models often learning human biases related to gender and race. For example, OpenAI’s “GPT-3.5 Turbo” scored just 0.46 in this category, while Alibaba Cloud’s “Qwen1.5 72B Chat” performed worse, scoring just 0.37. 

LatticeFlow also rated models on “prompt hijacking,” a type of cybersecurity threat in which malicious prompts disguise themselves as valid requests. The ratings included: Meta’s “Llama 2 13B Chat” with 0.42, Mistral’s “8x7B Instruct” with 0.38.  

Anthropic’s “Claude 3 Opus” was the best with the highest average of 0.89. 

Consequences of AI EU Regulations 

Due to EU AI Act regulation, the new online access to the LLM Checker will enable developers to check the compliance of their models and give early insights into where improvements are necessary.  

Petar Tsankov, CEO and Co-founder of LatticeFlow, gave a rosy look at the general results: “With a greater focus on optimizing for compliance, we believe model providers can be well-prepared to meet regulatory requirements.” 

While companies such as Meta still did not comment, the European Commission did mention that they were among those who played a role in developing the LLM Checker but called it a “first step” toward the implementation of the EU on AI regulation Act. 

Robust compliance measures that may be required in this fast-growing field of AI are necessary and tech companies must have a balance between innovation and security, privacy, and ethical concerns in order to attain a sustainable success especially in the presence of the new EU commission AI regulation. 

Inside Telecom provides you with an extensive list of content covering all aspects of the tech industry. Keep an eye on our Tech sections to stay informed and up-to-date with our daily articles.

Tags: