Common Crawl - Blog - July 2026 Crawl Archive Now Available The Data OverviewCDXJ IndexURL IndexWeb GraphsLatest CrawlCrawl StatsGraph StatsErrata Resources Get StartedAI AgentBlogExamplesCCBotInfra StatusOpt-Out LedgerFAQ Community Research PapersMailing List ArchiveHugging FaceDiscordCollaborators About AboutTeamJobsPrivacy PolicyTerms of Use Search AI Agent Contact Us ←  Back to Blog July 28, 2026 July 2026 Crawl Archive Now Available The crawl archive for July 2026 is now available. The data was crawled between July 7th and July 25th, and contains 2.14 billion web pages (or 364.01 TiB of uncompressed content). We also announce some improvements and changes. Sebastian Nagel Sebastian is a Distinguished Engineer at the Common Crawl Foundation. The crawl archive for July 2026 is now available. The data was crawled between July 7th and July 25th, and contains 2.14 billion web pages (or 364.01 TiB of uncompressed content). Page captures are from 40.5 million hosts or 33.2 million registered domains and include 603 million new URLs, not visited in any of our prior crawls. File List #Files Total Size Compressed (TiB) Segments segment.paths.gz 100 WARC warc.paths.gz 100000 84.69 WAT wat.paths.gz 100000 14.09 WET wet.paths.gz 100000 5.89 Robots.txt robotstxt.paths.gz 100000 0.15 Non-200 responses non200responses.paths.gz 100000 2.71 URL index cc-index.paths.gz 302 0.22 Columnar URL index cc-index-table.paths.gz 900 0.20 Archive Location & Download The July 2026 crawl archive is located in the commoncrawl bucket at crawl-data/CC-MAIN-2026-30/. To assist with exploring and using the dataset, we provide gzipped files which list all segments, WARC, WAT and WET files. By simply adding either s3://commoncrawl/ or https://data.commoncrawl.org/ to each line, you end up with the S3 and HTTP paths respectively, please see Get Started for detailed instructions. Improvements and Changes This crawl comes with a series of improvements and changes. WARC-Record-ID Now Uses Type 7 UUID We switch from using Type 4 to Type 7 UUIDs for WARC record IDs. Type 7 UUIDs are defined by RFC 9562 (published in 2024 and updating RFC 4122) and combine a Unix timestamp (epoch seconds) with a random component. One example of a Type 7 WARC record ID: WARC/1.0 WARC-Type: response WARC-Date: 2026-07-10T22:25:47Z WARC-Record-ID: <urn:uuid:019f4e23-0ae1-7128-99c9-625a3048e3b1> Content-Length: 27888 Content-Type: application/http; msgtype=response WARC-Warcinfo-ID: <urn:uuid:019f4e20-cbc0-7088-8251-ecc9a008540a> WARC-Concurrent-To: <urn:uuid:019f4e23-0ae1-71ae-a89b-b0992fcec9b8> WARC-IP-Address: 2620:cb:2000::1 WARC-Target-URI: https://commoncrawl.org/ WARC-Protocol: h2 WARC-Protocol: tls/1.3 WARC-Cipher-Suite: TLS_AES_128_GCM_SHA256 WARC-Payload-Digest: sha1:BYSGGYPUHJIF4ZIOWKMPPLYUJ5U5YJQO WARC-Block-Digest: sha1:5OM6GSLZYKMJJ5MPXWM5BUVWW43GGZZU WARC-Identified-Payload-Type: text/html The following Python snippets demonstrates how the UUID can be parsed to extract the timestamp: import datetime as dt import uuid record_id = uuid.UUID('urn:uuid:019f4e23-0ae1-7128-99c9-625a3048e3b1') dt.datetime.fromtimestamp(record_id.time / 1000).isoformat() # returns '2026-07-11T00:25:47.233000' As you can see, the timestamp in the in the UUID mirrors the WARC-Date header. Of course, the UUIDs in the WARC response records follow the same pattern. For further details about this change, please see the corresponding issue on GitHub. WARC-Record-ID Included in URL Indexes The record ID of the WARC response records is now contained in our URL indexes without the decorating <urn:uuid:>: The CDXJ index stores the UUID in the field recordid . The URL index has a new column warc_record_id. In order to use the new column in the URL index with Amazon Athena, you need to use the updated table schema defined by the cc-index-create-table-flat.sql statement. The column warc_record_id is a binary column. You can cast from and to the data type UUID: select cast(warc_record_id as uuid) from ccindex where url = 'https://commoncrawl.org/' and url_host_name_reversed = 'org.commoncrawl' and crawl = 'CC-MAIN-2026-30'; To look-up a record ID: select url, fetch_time from ccindex where cast(warc_record_id as uuid) = uuid '019f4e23-0ae1-7128-99c9-625a3048e3b1' and url_host_name_reversed = 'org.commoncrawl' and crawl = 'CC-MAIN-2026-30'; Also DuckDb provides a UUID data type. You can use the new column the exact same way as in Amazon Athena. More details about the new column can be found in the pull request cc-index-table#42 on GitHub. The URL Index Switches to ZStandard Parquet Compression The Parquet files of the URL index, starting with the current crawl's partition (crawl=CC-MAIN-2026-30) are compressed using ZStandard compression. Given that most Parquet readers and query engines support ZStandard nowadays, no changes are required to query the recent partitions of the URL index. Additional information can be found in this issue report on GitHub. Statistics of IP Address and HTTP Protocol Versions We started publishing statistics of IP address and HTTP protocol versions observed by our crawler CCBot. The new metrics are found on the Crawl Stats site along with other crawler metrics. This release was authored by: Sebastian Nagel Sebastian is a Distinguished Engineer at the Common Crawl Foundation. Luca Foppiano Luca Foppiano is a Senior Engineer at the Common Crawl Foundation. Michael Paris Michael is a Senior Research Engineer at the Common Crawl Foundation. Thom Vaughan Thom is Principal Engineer at the Common Crawl Foundation. Erratum:  Content is truncated Originally reported by:  More details Some archived content is truncated due to fetch size limits imposed during crawling. This is necessary to handle infinite or exceptionally large data streams (e.g., radio streams). Prior to March 2025 (CC-MAIN-2025-13), the truncation threshold was 1 MiB. From the March 2025 crawl onwards, this limit has been increased to 5 MiB. The Data Overview CDXJ Index URL Index Web Graphs Latest Crawl Crawl Stats Graph Stats Errata Resources Get Started AI Agent Blog Examples CCBot Infra Status Opt-Out Ledger FAQ Community Research Papers Mailing List Archive Hugging Face Discord Collaborators About About Team Jobs Privacy Policy Terms of Use &copy; 2026 Common Crawl