Unjustified Limits on Data Collection of Judicial Information in South Carolina
In a significant development, the American Civil Liberties Union (ACLU) and the South Carolina chapter of the National Association for the Advancement of Colored People (NAACP) filed a federal lawsuit on March 30, challenging restrictions on web scraping of public case records by the South Carolina Court Administration. However, as of August 2025, no confirmed updates exist on this case.
The lawsuit moves the debate about web scraping beyond the Computer Fraud and Abuse Act (CFAA), a federal law on cybercrime that has been used by some organizations to challenge web scraping activities. Recent court decisions have narrowed the interpretation of the CFAA, clarifying that breaching terms of service agreements or workplace policies does not necessarily violate the statute.
Facilitating access to government data is an important part of encouraging open data and government transparency. South Carolina's Public Index serves as the primary database for all legal documents in the state, containing information on court records and filings pertaining to individuals and organizations from all 45 counties in the state, covering all case categories from family law to housing issues to criminal matters.
However, the terms of service state that users should not use the database for unauthorized purposes, and manual downloading of each record from the Public Index database is a much more difficult task compared to streamlined access. The lawsuit argues that restrictions on data scraping inhibit access to public information.
Web scraping is an important tool for accessing publicly available information. The NAACP claims that the ban on data scraping on South Carolina's Public Index websites limits its ability to research racial disparities in eviction cases. Open access to large legal datasets can help researchers, journalists, and others understand legal trends, analyze policy outcomes, and identify critical areas where legal needs are unmet.
South Carolina should update its policies on access to its court records and work to ensure open access to these records. The state should always make available options for legitimate users when occasional technical restrictions on data scraping may be necessary, like for preventing denial-of-service attacks. South Carolina should allow users to more easily download its data through application programming interfaces (APIs) or bulk downloads.
The ACLU and NAACP's lawsuit is not the first instance where the issue of web scraping has been raised. The interpretation suggests that using web scraping to collect public data may not be a violation of the law because the information is already publicly accessible. No branch of government, at the federal or state level, should impose unnecessary technical or legal restrictions on web scraping activity.
As the case progresses, checking official federal court dockets for the District of South Carolina or statements directly from ACLU South Carolina and the South Carolina NAACP might provide updates beyond the available search results.
- The debate around web scraping has moved beyond the Computer Fraud and Abuse Act (CFAA), with the lawsuit filed by the ACLU and the South Carolina NAACP focusing on restrictions in South Carolina's Public Index.
- Facilitating access to government data is crucial for open data and transparency, but South Carolina's Public Index terms of service prohibit unauthorized use, making web scraping a necessary tool for accessing public information.
- The NAACP argues that the ban on data scraping on South Carolina's Public Index websites hinders its ability to research racial disparities in eviction cases, and access to large legal datasets can aid researchers, journalists, and others in understanding legal trends and identifying unmet legal needs.
- As the ACLU and NAACP's lawsuit progresses, it highlights the importance of maintaining open access to public data, and no government, at the federal or state level, should impede web scraping activity if the information is publicly accessible.
