News

This is a good survey on prompt injection attacks on large language models (like ChatGPT).

Abstract: We are currently witnessing dramatic advances in the capabilities of Large Language Models (LLMs). They are already being adopted in practice and integrated into many systems, including integrated development environments (IDEs) and search engines. The functionalities of current LLMs can be modulated via natural language prompts, while their exact internal functionality remains implicit and unassessable. This property, which makes them adaptable to even unseen tasks, might also make them susceptible to targeted adversarial prompting. Recently, several ways to misalign LLMs using Prompt Injection (PI) attacks have been introduced. In such attacks, an adversary can prompt the LLM to produce malicious content or override the original instructions and the employed filtering schemes. Recent work showed that these attacks are hard to mitigate, as state-of-the-art LLMs are instruction-following. So far, these attacks assumed that the adversary is directly prompting the LLM.

In this work, we show that augmenting LLMs with retrieval and API calling capabilities (so-called Application-Integrated LLMs) induces a whole new set of attack vectors. These LLMs might process poisoned content retrieved from the Web that contains malicious prompts pre-injected and selected by adversaries. We demonstrate that an attacker can indirectly perform such PI attacks. Based on this key insight, we systematically analyze the resulting threat landscape of Application-Integrated LLMs and discuss a variety of new attack vectors. To demonstrate the practical viability of our attacks, we implemented specific demonstrations of the proposed attacks within synthetic applications. In summary, our work calls for an urgent evaluation of current mitigation techniques and an investigation of whether new techniques are needed to defend LLMs against these threats.

The domain name registrar Freenom, whose free domain names have long been a draw for spammers and phishers, has stopped allowing new domain name registrations. The move comes just days after the Dutch registrar was sued by Meta, which alleges the company ignores abuse complaints about phishing websites while monetizing traffic to those abusive domains.

Freenom is the domain name registry service provider for five so-called “country code top level domains” (ccTLDs), including .cf for the Central African Republic; .ga for Gabon; .gq for Equatorial Guinea; .ml for Mali; and .tk for Tokelau.

Freenom has always waived the registration fees for domains in these country-code domains, presumably as a way to encourage users to pay for related services, such as registering a .com or .net domain, for which Freenom does charge a fee.

On March 3, 2023, social media giant Meta sued Freenom in a Northern California court, alleging cybersquatting violations and trademark infringement. The lawsuit also seeks information about the identities of 20 different “John Does” — Freenom customers that Meta says have been particularly active in phishing attacks against Facebook, Instagram, and WhatsApp users.

The lawsuit points to a 2021 study (PDF) on the abuse of domains conducted for the European Commission, which discovered that those ccTLDs operated by Freenom made up five of the Top Ten TLDs most abused by phishers.

“The five ccTLDs to which Freenom provides its services are the TLDs of choice for cybercriminals because Freenom provides free domain name registration services and shields its customers’ identity, even after being presented with evidence that the domain names are being used for illegal purposes,” the complaint charges. “Even after receiving notices of infringement or phishing by its customers, Freenom continues to license new infringing domain names to those same customers.”

Meta further alleges that “Freenom has repeatedly failed to take appropriate steps to investigate and respond appropriately to reports of abuse,” and that it monetizes the traffic from infringing domains by reselling them and by adding “parking pages” that redirect visitors to other commercial websites, websites with pornographic content, and websites used for malicious activity like phishing.

Freenom has not yet responded to requests for comment. But attempts to register a domain through the company’s website as of publication time generated an error message that reads:

“Because of technical issues the Freenom application for new registrations is temporarily out-of-order. Please accept our apologies for the inconvenience. We are working on a solution and hope to resume operations shortly. Thank you for your understanding.”

Although Freenom is based in The Netherlands, some of its other sister companies named as defendants in the lawsuit are incorporated in the United States.

Meta initially filed this lawsuit in December 2022, but it asked the court to seal the case, which would have restricted public access to court documents in the dispute. That request was denied, and Meta amended and re-filed the lawsuit last week.

According to Meta, this isn’t just a case of another domain name registrar ignoring abuse complaints because it’s bad for business. The lawsuit alleges that the owners of Freenom “are part of a web of companies created to facilitate cybersquatting, all for the benefit of Freenom.”

“On information and belief, one or more of the ccTLD Service Providers, ID Shield, Yoursafe, Freedom Registry, Fintag, Cervesia, VTL, Joost Zuurbier Management Services B.V., and Doe Defendants were created to hide assets, ensure unlawful activity including cybersquatting and phishing goes undetected, and to further the goals of Freenom,” Meta charged.

It remains unclear why Freenom has stopped allowing domain registration, but it could be that the company was recently the subject of some kind of disciplinary action by the Internet Corporation for Assigned Names and Numbers (ICANN), the nonprofit entity which oversees the domain registrars.

In June 2015, ICANN suspended Freenom’s ability to create new domain names or initiate inbound transfers of domain names for 90 days. According to Meta, the suspension was premised on ICANN’s determination that Freenom “has engaged in a pattern and practice of trafficking in or use of domain names identical or confusingly similar to a trademark or service mark of a third party in which the Registered Name Holder has no rights or legitimate interest.”

ICANN has not yet responded to requests for comment.

A copy of the amended complaint against Freenom, et. al, is available here (PDF).

Kaspersky said the figures represented a 1.5 increase compared with the second half of 2021

The memorandum highlights the need for states to include cybersecurity in periodic audits

Police also seized electronic equipment and are currently performing forensic examinations

Information was stolen during recent ransomware attack

Focus will be on enhancing prevention and identification skills

Online counseling service shared health data

Devices seized, suspects interrogated and arrested, allegedly connected to devastating cyberattack on University Hospital in Düsseldorf.

Last week, the Biden Administration released a new National Cybersecurity Strategy (summary here). There is lots of good commentary out there. It’s basically a smart strategy, but the hard parts are always the implementation details. It’s one thing to say that we need to secure our cloud infrastructure, and another to detail what the means technically, who pays for it, and who verifies that it’s been done.

One of the provisions getting the most attention is a move to shift liability to software vendors, something I’ve been advocating for since at least 2003.

Slashdot thread.