czds

- ICANN Centralized Zone Data Service Tool
git clone git://git.acid.vegas/czds.git
Log | Files | Refs | Archive | README | LICENSE

README.md (5453B)

      1 # ICANN Centralized Zone Data Service API
      2 
      3 The [ICANN Centralized Zone Data Service](https://czds.icann.org) *(CZDS)* allows *approved* users to request and download DNS zone files in bulk, provided they represent a legitimate company or academic institution and their intended use is legal and ethical. Once ICANN approves the request, this tool streamlines the retrieval of extensive domain name system data, facilitating research and security analysis in the realm of internet infrastructure.
      4 
      5 ## Features
      6 * Asynchronous downloads with configurable concurrency
      7 * Support for both CSV and JSON report formats
      8 * Optional gzip decompression of zone files
      9 * Environment variable support for credentials
     10 * Comprehensive error handling and logging
     11 
     12 ## Zone Information
     13 Zone files are updated once every 24 hours, specifically from 00:00 UTC to 06:00 UTC. Access to these zones is granted in increments, and the total time for approval across all zones may extend to a month or longer. It is typical for more than 90% of requested zones to receive approval. Access to certain zone files may require additional application forms with the TLD organization. Please be aware that access to certain zones is time-bound, expiring at the beginning of the following year, or up to a decade after the initial approval has been confirmed.
     14 
     15 At the time of writing this repository, the CZDS offers access to 1,151 zones in total.
     16 
     17 1,079 have been approved, 55 are still pending *(after 3 months)*, 10 have been revoked because the TLDs are longer active, and 6 have been denied. Zones that have expired automatically had the expiration extended for me without doing anything, aside from 13 zones that remained expired. I have included a recent [stats file](./extras/stats.csv) directly from my ICANN account.
     18 
     19 ## Installation
     20 ```bash
     21 pip install czds-api
     22 ```
     23 
     24 ## Usage
     25 ### Command Line Interface
     26 ```bash
     27 czds [-h] [-u USERNAME] [-p PASSWORD] [-z] [-c CONCURRENCY] [-d] [-k] [-r] [-s] [-f {csv,json}] [-o OUTPUT]
     28 ```
     29 
     30 #### Arguments
     31 ###### Basic Options
     32 | Argument              | Description                                  | Default           |
     33 |-----------------------|----------------------------------------------|-------------------|
     34 | `-h`, `--help`        | Show help message and exit                   |                   |
     35 | `-u`, `--username`    | ICANN Username                               | `$CZDS_USER`      |
     36 | `-p`, `--password`    | ICANN Password                               | `$CZDS_PASS`      |
     37 | `-o`, `--output`      | Output directory                             | Current directory |
     38 
     39 ###### Zone Options
     40 | `-z`, `--zones`       | Download zone files                          |                   |
     41 | `-c`, `--concurrency` | Number of concurrent downloads               | `3`               |
     42 
     43 ###### Report Options
     44 | `-r`, `--report`      | Download the zone stats report               |                   |
     45 | `-s`, `--scrub`       | Scrub username from the report               |                   |
     46 | `-f`, `--format`      | Report output format (csv/json)              | `csv`             |
     47 
     48 ### Environment Variables
     49 ```bash
     50 export CZDS_USER='your_username'
     51 export CZDS_PASS='your_password'
     52 ```
     53 
     54 ### Python Module
     55 ```python
     56 import os
     57 from czds import CZDS
     58 
     59 async with CZDS(username, password) as client:
     60     # Download zone stats report
     61     await client.get_report('report.csv', scrub=True, format='json')
     62     
     63     # Download zone files
     64     zone_links = await client.fetch_zone_links()
     65     await client.download_zones(zone_links, 'zones', concurrency=3, decompress=True)
     66 ```
     67 
     68 ## Zone Information
     69 Zone files are updated once every 24 hours, specifically from 00:00 UTC to 06:00 UTC. Access to these zones is granted in increments, and the total time for approval across all zones may extend to a month or longer. It is typical for more than 90% of requested zones to receive approval. Access to certain zone files may require additional application forms with the TLD organization. Please be aware that access to certain zones is time-bound, expiring at the beginning of the following year, or up to a decade after the initial approval has been confirmed.
     70 
     71 At the time of writing this repository, the CZDS offers access to 1,151 zones in total.
     72 
     73 1,079 have been approved, 55 are still pending *(after 3 months)*, 10 have been revoked because the TLDs are longer active, and 6 have been denied. Zones that have expired automatically had the expiration extended for me without doing anything, aside from 13 zones that remained expired. I have included a recent [stats file](./extras/stats.csv) directly from my ICANN account.
     74 
     75 ## Respects & extras
     76 While ICANN does have an official [czds-api-client-python](https://github.com/icann/czds-api-client-python) repository, I rewrote it from scratch to be more streamline & included a [POSIX version](./extras/czds) for portability. There is some [official documentation](https://raw.githubusercontent.com/icann/czds-api-client-java/master/docs/ICANN_CZDS_api.pdf) that was referenced in the creation of the POSIX version. Either way, big props to ICANN for allowing me to use the CZDS for research purposes!
     77 
     78 ___
     79 
     80 ###### Mirrors for this repository: [acid.vegas](https://git.acid.vegas/czds) • [SuperNETs](https://git.supernets.org/acidvegas/czds) • [GitHub](https://github.com/acidvegas/czds) • [GitLab](https://gitlab.com/acidvegas/czds) • [Codeberg](https://codeberg.org/acidvegas/czds)