Amazon investigates Perplexity AI for alleged content scraping

Amazon is investigating allegations that Perplexity AI, an artificial intelligence startup, removed content from major news sites without permission.

An Amazon spokesperson said Friday (June 28) that the company is investigating multiple reports from WIRED and Forbes claiming that Perplexity accessed content from websites that explicitly prohibit such scraping practices. Perplexity runs on servers provided by Amazon Web Services (AWS).

The representative also noted that all AWS customers must follow the instructions in the robots.txt file. These files are typically used on websites to instruct robots and crawlers to refrain from scraping their data, whether for generative AI tools or other uses.

“The AWS Terms of Service prohibit abusive and illegal activities and our customers are responsible for compliance with these terms. We regularly receive reports of alleged abuse from a variety of sources and urge our customers to understand these reports,” the representative said.

Forbes editor-in-chief and chief content officer Randall Lane accused Perplexity of committing “cynical theft,” accusing the company of creating “counterfeit stories” containing “eerily similar wording” and “completely copied fragments” of its articles.

He added: “More seriously, the post, which looked and read like a news article, made no mention of Forbes at all, aside from a line at the bottom of a few paragraphs that mentioned “sources” and a very small icon that looked like the “F” in the Forbes logo – if you squinted.

Did Perplexity AI plagiarize content?

San Francisco-based AI search startup Perplexity, once celebrated by major tech investors like Amazon's Jeff Bezos, has recently come under scrutiny following accusations of plagiarism.

Perplexity CEO Aravind Srinivas has denied allegations that his company “ignored bot exclusion protocol and then lied about it.” Srinivas acknowledged to Fast Company that Perplexity uses third-party crawlers in addition to its own, and confirmed that the crawler identified by WIRED was one of them.

However, he added, “Forbes has pointed out very clearly that they prefer more prominent source mentioning.” Srinivas also mentioned that sources are now more prominent.

ReadWrite has contacted Amazon and Perplexity for comment.

Featured Image: Canva / Perplexity AI