httpz

- Hyper-fast HTTP Scraping Tool
git clone git://git.acid.vegas/httpz.git
README.md (1724B)
      1 # HTTP-Z
      2 ###### This is still a work in progress...stay tuned for updates!
      3 
      4 ## Information
      5 This script is developed as a robust alternative to HTTPX, addressing the limitations in customizing JSON outputs and other functionalities that HTTPX lacks. It is specifically designed for asynchronous lookups on a list of domains, efficiently gathering DNS information and web content details such as page titles and body previews.
      6 
      7 ## Usage
      8 | Argument               | Description                                                 |
      9 | ---------------------- | ----------------------------------------------------------- |
     10 | `<input_file>`         | File containing list of domains                             |
     11 | `-c`, `--concurrency`  | Number of concurrent requests                               |
     12 | `-m`, `--memory_limit` | Number of results to store in memory before syncing to file |
     13 | `-o`, `--output`       | Output file                                                 |
     14 | `-t`, `--timeout`      | Timeout for HTTP requests                                   |
     15 | `-u`, `--user_agent`   | User agent to use for HTTP requests                         |
     16 | `-x`, `--proxy`        | Proxy to use for HTTP requests                              |
     17 | `-r`, `--retry`        | Number of times to retry failed requests                    |
     18 | `-v`, `--verbose`      | Increase output verbosity                                   |
     19 | `-p`, `--preview`      | Preview size in bytes for body & title *(default: 500)*     |
     20 
     21 ___
     22 
     23 ###### Mirrors
     24 [acid.vegas](https://git.acid.vegas/httpz) • [GitHub](https://github.com/acidvegas/httpz) • [GitLab](https://gitlab.com/acidvegas/httpz) • [SuperNETs](https://git.supernets.org/acidvegas/httpz)