Extracting Personal Information from Large Language Models Like GPT-2

Researchers have been able to find all sorts of personal information within GPT-2. This information was part of the training data, and can be extracted with the right sorts of queries.

Paper: “Extracting Training Data from Large Language Models.”

Abstract: It has become common to publish large (billion parameter) language models that have been trained on private datasets. This paper demonstrates that in such settings, an adversary can perform a training data extraction attack to recover individual training examples by querying the language model.

We demonstrate our attack on GPT-2, a language model trained on scrapes of the public Internet, and are able to extract hundreds of verbatim text sequences from the model’s training data. These extracted examples include (public) personally identifiable information (names, phone numbers, and email addresses), IRC conversations, code, and 128-bit UUIDs. Our attack is possible even though each of the above sequences are included in just one document in the training data.

We comprehensively evaluate our extraction attack to understand the factors that contribute to its success. For example, we find that larger models are more vulnerable than smaller models. We conclude by drawing lessons and discussing possible safeguards for training large language models.

From a blog post:

We generated a total of 600,000 samples by querying GPT-2 with three different sampling strategies. Each sample contains 256 tokens, or roughly 200 words on average. Among these samples, we selected 1,800 samples with abnormally high likelihood for manual inspection. Out of the 1,800 samples, we found 604 that contain text which is reproduced verbatim from the training set.

The rest of the blog post discusses the types of data they found.

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk

Sealed U.S. Court Records Exposed in SolarWinds Breach

The ongoing breach affecting thousands of organizations that relied on backdoored products by network software firm SolarWinds may have jeopardized the privacy of countless sealed court documents on file with the U.S. federal court system, according to a memo released Wednesday by the Administrative Office (AO) of the U.S. Courts.

The judicial branch agency said it will be deploying more stringent controls for receiving and storing sensitive documents filed with the federal courts, following a discovery that its own systems were compromised as part of the SolarWinds supply chain attack. That intrusion involved malicious code being surreptitiously inserted into updates shipped by SolarWinds for some 18,000 users of its Orion network management software as far back as March 2020.

“The AO is working with the Department of Homeland Security on a security audit relating to vulnerabilities in the Judiciary’s Case Management/Electronic Case Files system (CM/ECF) that greatly risk compromising highly sensitive non-public documents stored on CM/ECF, particularly sealed filings,” the agency said in a statement published Jan. 6.

“An apparent compromise of the confidentiality of the CM/ECF system due to these discovered vulnerabilities currently is under investigation,” the statement continues. “Due to the nature of the attacks, the review of this matter and its impact is ongoing.”

The AO declined to comment on specific questions about their breach disclosure. But a source close to the investigation told KrebsOnSecurity that the federal court document system was “hit hard,” by the SolarWinds attackers, which multiple U.S. intelligence and law enforcement agencies have attributed as “likely Russian in origin.”

The source said the intruders behind the SolarWinds compromise seeded the AO’s network with a second stage “Teardrop” malware that went beyond the “Sunburst” malicious software update that was opportunistically pushed out to all 18,000 customers using the compromised Orion software. This suggests the attackers were targeting the agency for deeper access to its networks and communications.

The AO’s court document system powers a publicly searchable database called PACER, and the vast majority of the files in PACER are not restricted and are available to anyone willing to pay for the records.

But experts say many other documents stored in the AO’s system are sealed — either temporarily or indefinitely by the courts or parties to a legal matter — and may contain highly sensitive information, including intellectual property and trade secrets, or even the identities of confidential informants.

Nicholas Weaver, a lecturer at the computer science department at University of California, Berkeley, said the court document system doesn’t hold documents that are classified for national security reasons. But he said the system is full of sensitive sealed filings — such as subpoenas for email records and so-called “trap and trace” requests that law enforcement officials use to determine with whom a suspect is communicating via phone, when and for how long.

“This would be a treasure trove for the Russians knowing about a lot of ongoing criminal investigations,” Weaver said. “If the FBI has indicted someone but hasn’t arrested them yet, that’s all under seal. A lot of the investigative tools that get protected under seal are filed very early on in the process, often with gag orders that prevent [the subpoenaed party] from disclosing the request.”

The acknowledgement from the AO comes hours after the U.S. Justice Department said it also was a victim of the SolarWinds intruders, who took control over the department’s Office 365 system and accessed email sent or received from about three percent of DOJ accounts (the department has more than 100,000 employees).

The SolarWinds hack also reportedly jeopardized email systems used by top Treasury Department officials, and granted the attackers access to networks inside the Energy, Commerce and Homeland Security departments.

The New York Times on Wednesday reported that investigators are examining whether a breach at another software provider — JetBrains — may have precipitated the attack on SolarWinds. The company, which was founded by three Russian engineers in the Czech Republic, makes a tool called TeamCity that helps developers test and manage software code. TeamCity is used by developers at 300,000 organizations, including SolarWinds and 79 of the Fortune 100 companies.

“Officials are investigating whether the company, founded by three Russian engineers in the Czech Republic with research labs in Russia, was breached and used as a pathway for hackers to insert back doors into the software of an untold number of technology companies,” The Times said. “Security experts warn that the monthslong intrusion could be the biggest breach of United States networks in history.”

Under the AO’s new procedures, highly sensitive court documents filed with federal courts will be accepted for filing in paper form or via a secure electronic device, such as a thumb drive, and stored in a secure stand-alone computer system. These sealed documents will not be uploaded to CM/ECF.

“This new practice will not change current policies regarding public access to court records, since sealed records are confidential and currently are not available to the public,” the AO said.

James Lewis, senior vice president at the Center for Strategic and International Studies, said it’s too soon to tell the true impact of the breach at the court system, but the fact that they were apparently targeted is a “a very big deal.”

“We don’t know what the Russians took, but the fact that they had access to this system means they had access to a lot of great stuff, because federal cases tend to involve fairly high profile targets,” he said.

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk

All Aboard the Pequod!

Like countless others, I frittered away the better part of Jan. 6 doomscrolling and watching television coverage of the horrifying events unfolding in our nation’s capital, where a mob of President Trump supporters and QAnon conspiracy theorists was incited to lay siege to the U.S. Capitol. For those trying to draw meaning from the experience, might I suggest consulting the literary classic Moby Dick, which simultaneously holds clues about QAnon’s origins and offers an apt allegory about a modern-day Captain Ahab and his ill-fated obsessions.

Many have speculated that Jim Watkins, the administrator of the online message board 8chan (a.k.a. 8kun), and/or his son Ron are in fact “Q,” the anonymous persona behind the QAnon conspiracy theory, which holds that President Trump is secretly working to save the world from a satanic cult of pedophiles and cannibals.

Last year, as I was scrutinizing the computer networks that kept QAnon online, researcher Ron Guilmette pointed out a tantalizing utterance from Watkins the younger which adds tenuous credence to the notion that one or both of them is Q.

We’ll get to how the Great White Whale (the Capitol?) fits into this tale in a moment. But first, a bit of background. A person identified only as “Q” has for years built an impressive following for the far-right conspiracy movement by leaving periodic “Q drops,” cryptic messages that QAnon adherents spend much time and effort trying to decipher and relate to current events.

Researchers who have studied more than 5,000 Q drops are convinced that there are two distinct authors of these coded utterances. The leading theory is that those identities corresponded to the aforementioned father-and-son team responsible for operating 8chan.

Jim Watkins, 56, is the current owner of 8chan, a community perhaps now best known as a forum for violent extremists and mass shooters. Watkins is an American pig farmer based in the Philippines; Ron reportedly resides in Japan.

In the aftermath of back-to-back mass shootings on Aug. 3 and Aug. 4, 2019 in which a manifesto justifying one of the attacks was uploaded to 8chan, Cloudflare stopped providing their content delivery network to 8chan. Several other providers quickly followed suit, leaving 8chan offline for months before it found a haven at a notorious bulletproof hosting facility in Russia.

One reason Q watchers believe Ron and Jim Watkins may share authorship over the Q drops is that while 8chan was offline, the messages from Q ceased. The drops reappeared only months later when 8chan rebranded as 8kun.

CALL ME ISHMAEL

Here’s where the admittedly “Qonspiratorial” clue about the Watkins’ connection to Q comes in. On Aug. 5, 2019, Ron Watkins posted a Twitter message about 8chan’s ostracization which compared the community’s fate to that of the Pequod, the name of the doomed whaling ship in the Herman Melville classic “Moby Dick.”

“If we are still down in a few hours then maybe 8chan will just go clearnet and we can brave DDOS attacks like Ishmael on the Pequod,” Watkins the younger wrote.

Ishmael, the first-person narrator in the novel, is a somewhat disaffected American sailor who decides to try his hand at a whaling ship. Ishmael is a bit of a minor character in the book; very soon into the novel we are introduced to a much more interesting and enigmatic figure — a Polynesian harpooner by the name of Queequeg.

Apart from being a cannibal from the Pacific islands who has devoured many people, Queequeg is a pretty nice guy and shows Ismael the ropes of whaling life. Queequeg is covered head to toe in tattoos, which are described by the narrator as the work of a departed prophet and seer from the cannibal’s home island.

Like so many Q drops, Queequeg’s tattoos tell a mysterious tale, but we never quite learn what that full story is. Indeed, the artist who etched them into Queequeg’s body is long dead, and the cannibal himself can’t seem to explain what it all means.

Ishmael describes Queequeg’s mysterious markings in this passage:

“…a complete theory of the heavens and earth, and a mystical treatise on the art of attaining truth; so that Queequeg in his own proper person was a riddle to unfold; a wondrous work in one volume; but whose mysteries not even himself could read, though his own live heart beat against them; and these mysteries were therefore destined in the end to moulder away with the living parchment whereon they were inscribed, and so be unsolved to the last.”

THE GREAT WHITE WHALE

It’s perhaps fitting then that one of the most recognizable figures from the mob that stormed the U.S. Capitol on Wednesday was a heavily-tattooed, spear-wielding QAnon leader who goes by the name “Q Shaman” (a.k.a. Jake Angeli).

“Q Shaman,” a.k.a. Jake Angeli, at a Black Lives Matter event in Arizona (left) and Wednesday, confronted by U.S. Capitol Police. Image: Twitter, @KelemenCari.

“Angeli’s presence at the riot, along with others wearing QAnon paraphernalia, comes as the conspiracy-theory movement has been responsible for the popularization of Trump’s voter-fraud conspiracy theories,” writes Rachel E. Greenspan for Yahoo! News.

“As Q has become increasingly hands-off, giving fewer and fewer messages to his devotees, QAnon leaders like Angeli have gained fame and power in the movement,” Greenspan wrote.

If somehow Moby Dick was indeed the inspiration for the “Q” identity in QAnon, yesterday’s events at The Capitol were the inexorable denouement of a presidential term that increasingly came to be defined by conspiracy theories. In a somewhat prescient Hartford Courant op-ed published in 2018, author Steven Almond observed that Trump’s presidency could be best understood through the lens of the Pequod’s Captain Ahab. To wit:

“Melville is offering a mythic account of how one man’s virile bombast ensnares everyone and everything it encounters. The setting is nautical, the language epic. But the tale, stripped to its ribs, is about the seductive power of the wounded male ego, how naturally a ship steered by men might tack to its vengeful course.”

“Trump’s presidency has been, in its way, a retelling of this epic. Whether we cast him as agent or principal hardly matters. What matters is that Americans have joined the quest. In rapture or disgust, we’ve turned away from the compass of self-governance and toward the mesmerizing drama of aggression on display, the masculine id unchained and all that it unchains within us. With every vitriolic tweet storm and demeaning comment, Trump strikes through the mask.”

EPILOGUE

If all of the above theorizing reads like yet another crackpot QAnon conspiracy, that may be the inevitable consequence of my spending far too much time going down this particular rabbit hole (and re-reading Moby Dick in the process!).

In any case, none of this is likely to matter to the diehard QAnon conspiracy theorists themselves, says Mike Rothschild, a writer who specializes in researching and debunking conspiracy theories.

“Even if Jim Watkins was revealed as owning the board or making the posts, it wouldn’t matter,” Rothschild said. “Anything that happens that disconfirms Q being an official in the military industrial complex is going to help fuel their persecution complex.”

Rothschild has been working hard on finishing his next book, “The Storm is Upon Us: How QAnon Became a Movement, Cult, and Conspiracy Theory of Everything,” which is due to be published in October 2021. Who’s printing the book? Ten points if you guessed Melville House, an independent publisher named after Herman Melville.

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk

Poor Software Quality Costs US $2.08tn

Poor Software Quality Costs US $2.08tn

Poor-quality software cost America over $2tn last year, according to a new report by the Consortium for Information & Software Quality (CISQ).

The “Cost of Poor Software Quality in the US: A 2020 Report,” which was co-sponsored by American software company Synopsys, found that the cost of poor software quality (CPSQ) in the US in 2020 was approximately $2.08tn.

Researchers looked at poor software quality resulting from software failures, unsuccessful development projects, legacy system problems, technical debt, and cybercrime enabled by exploitable weaknesses and vulnerabilities in software.

Operational software failure was determined to be the leading driver of the total CPSQ. CISQ estimated the cost of operational software failure in the US in 2020 as $1.56tn, a figure that has increased 22% since 2018. 

The next largest growth area of the CPSQ, estimated at $260bn, was unsuccessful development projects, the cost of which has risen 46% since 2018.    

Unmitigated flaws in the software were reported as the primary underlying cause of operational software failure, while a lack of attention to quality was “a consistent theme” among the causes of unsuccessful development projects. 

“Software quality lags behind other objectives in most organizations,” wrote CISQ. “That lack of primary attention to quality comes at a steep cost, which is revealed in this report. 

“While organizations can monetize the business value of speed, they rarely measure the offsetting cost of poor quality.”

CISQ advised software shops to avoid unsuccessful projects by not creating arbitrary schedules. It further advised shops to “pay attention to defined quality objectives and measure against those objectives throughout a project’s lifecycle.”

Researchers put the CPSQ associated with operating and maintaining legacy software at $520bn, down from $635bn in 2018. 

“As poor software quality persists on an upward trajectory, the solution remains the same: prevention is still the best medicine,” said Joe Jarzombek, director for government and critical infrastructure programs at Synopsys.  

“It’s important to build secure, high-quality software that addresses weaknesses and vulnerabilities as close to the source as possible. This limits the potential damage and cost to resolve issues.”

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk

British Airways Plans £3bn Breach Settlement

British Airways Plans £3bn Breach Settlement

The UK’s flag-carrier airline is planning to begin settlement discussions that could see customers who became the victims of a data breach receive a compensation payout of up to £3bn.  

British Airways customers were impacted by two data breaches in 2018. Between April and July 2018, some 185,000 British Airways reward-booking customers were notified that their personal information and financial details had been compromised, while a further 380,000 users of the airline’s app and website had their information exposed between August and September 2018.

Data compromised in the breaches included customer names, billing addresses, and email addresses. Payment card information, including card numbers, expiry dates, and—in tens of thousands of cases—the CVV security code, was also exposed. No passport details were stolen.

In July 2019, the Information Commissioner’s Office (ICO) issued a notice of intention to fine the airline a record £183m over the breach. However, this penalty was reduced drastically to a £20m fine in October 2020.

According to a statement released today by consumer action law firm Your Lawyers, British Airways has voiced its intention to kick off settlement discussions in the first quarter of 2021.

In 2019, Your Lawyers was appointed to the Steering Committee responsible for the overall conduct of the BA data breach litigation. The firm described the airline’s plans to begin settlement discussions as an admission of culpability for the breach and an effort to avoid the burden of litigation.

“News that British Airways wants to settle compensation claims, with negotiations set to take place in the first quarter of 2021, is acknowledgement of its wrongdoing in failing to protect customer data,” said Aman Johal, director at Your Lawyers.

“This is incredibly positive news for the victims of the breach and for consumer rights in general, but people must act fast to avoid missing out.”

The deadline to join the Group Litigation Order (GLO) falls on March 19, 2021. Your Lawyers believes that affected customers could each potentially receive an average of £6,000 in compensation. Financial losses arising from the breach could also be claimed.

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk

ElectroRAT Drains Crypto Wallets

ElectroRAT Drains Crypto Wallets

Thousands of cryptocurrency users have fallen victim to a sophisticated threat campaign that uses trojanized apps to drain funds from digital wallets.

The recently discovered campaign is a wide-ranging operation that encompasses fake companies, a marketing campaign, custom-built cryptocurrency applications, and a new Remote Access Tool (RAT) written from scratch to avoid antivirus detection.

Researchers at Intezer who unearthed the operation in December believe it was initiated in January 2020.

“The campaign includes domain registrations, websites, trojanized applications, fake social media accounts and a new undetected RAT that we have named ElectroRAT,” wrote researchers. 

ElectroRAT is written in the open-source programming language Golang and is compiled to target Windows, Linux, and Mac operating systems.

“It is rather common to see various information stealers trying to collect private keys to access victims’ wallets,” wrote researchers. “However, it is rare to see tools written from scratch and used to target multiple operating systems for these purposes.”

The author of the malicious campaign entices cryptocurrency users to download trojanized applications by promoting the apps on social media and in dedicated online forums. 

“We estimate this campaign has already infected thousands of victims based on the number of unique visitors to the pastebin pages used to locate the command and control servers,” noted researchers.

Three different trojanized apps—Jamm, eTrade, and DaoPoker—have been created by the attacker, each with a Windows, Linux, and Mac version. The attacker then built websites specifically to host the binaries. 

The apps appear to offer easy-to-use tools that will help users trade and manage their cryptocurrency. 

“These applications were promoted in cryptocurrency and blockchain-related forums such as bitcointalk and SteemCoinPan,” wrote researchers. 

“The promotional posts, published by fake users, tempted readers to browse the applications’ web pages, where they could download the application without knowing they were actually installing malware.”

To make the DaoPoker app appear legitimate, the attacker created Twitter and Telegram accounts for it and paid a social media influencer with over 25,000 Twitter followers to advertise the app.

Among ElectroRAT’s extremely intrusive capabilities are keylogging, taking screenshots, uploading files from disk, downloading files, and executing commands on the victim’s console.

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk

Secure Chorus Transfers Ownership of Encrypted Messaging App Standards to ETSI

Secure Chorus Transfers Ownership of Encrypted Messaging App Standards to ETSI

Secure Chorus, a not-for-profit membership organization for the development of strategies, standards and capabilities in the field of information security, has announced the transfer of ownership of its interoperability standards for enterprise grade encrypted messaging apps to the European Telecommunication Standards Institute (ETSI).

ETSI produces globally applicable standards for ICT-enabled systems, applications and services deployed across all sectors of industry and society.

The standards – developed by Secure Chorus over a four-year project – provide solutions which offer state-of-the-art, end-to-end encryption powered by the open cryptographic standard MIKEY-SAKKE and alleviate vendor lock-in constraints, allowing enterprise users to choose their preferred service in terms of functionality

The transfer to ETSI was fully achieved in December 2020 and will allow for the wide adoption of the standards along with the development of new features in the future.

Stephen Brown, Secure Chorus’ chairman, said: “Secure Chorus is very pleased that ETSI has agreed to the transfer of ownership of our interoperability standards. ETSI is a center of excellence for the development of globally applicable standards for ICT, so we are confident they will continue the good work we started.”

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk

BlueVoyant Enters Strategic Partnership with Third Party Risk Management Consultancy

BlueVoyant Enters Strategic Partnership with Third Party Risk Management Consultancy

Cybersecurity company BlueVoyant has announced a strategic partnership with third-party risk management consultancy DVV Solutions. The collaboration will enable BlueVoyant’s Cyber Risk Management (CRx) services to be delivered to DVV Solutions’ global customer base.

The firm said it can now offer actionable intelligence to this cohort of customers, helping them identify and address supply chain vulnerabilities. This will be primarily via BlueVoyant’s Vendor Risk Management (CR3) solution, which offers visibility and expertise to meet vendor risk management requirements as well as improve organizations’ own cyber-risk strategies.

This will add to DVV Solutions’ currently dedicated third-party risk management portfolio, which encompasses on-site/virtual visits, risk assessments, questionnaires, security ratings, continuous monitoring, cyber-risk maturity consultancy and regulatory compliance services.

In addition, BlueVoyant’s Cyber Risk Management for Investors (CRi) solution will also be available within DVV Solutions’ mergers and acquisitions (M&A) consultancy. This service showcases and mitigates the cyber-risks associated with potential transactions of this kind.

Commenting on the agreement, Robert Hannigan, chairman at BlueVoyant International, said: “The knock-on effect of COVID-19 has led to squeezed financial margins and a reduction in the resources available to tackle third-party risk in the supply chain, just as the pandemic widens the attack surface. Our CR3 solution will provide the desired level of risk analysis and remediation for DVV Solutions’ customers and their vendor ecosystems, enabling them to effectively quantify, manage and remediate third-party security risks.”

Sean O’Brien, managing director of DVV Solutions, stated: “Partnering with BlueVoyant is a natural extension of our third-party risk management and security monitoring services. As businesses and their vendor ecosystems have changed throughout an unprecedented 2020, we have seen a sharp uptick in the requirement for managed services, as supply chains become increasingly complex. Organizations therefore require a managed service-based third-party risk management solution to cut through the noise, helping them to prioritize the most pertinent supply chain risks.”

Premium Domain Names – transcom.uk
Transcom ISP – The UK’s Best Business ISP
DoubleCheck any website at doublecheck.uk