golcg

- Unnamed repository; edit this file 'description' to name the repository.
git clone git://git.acid.vegas/-c.git
Log | Files | Refs | Archive | README | LICENSE

README.md (6995B)

      1 <h1 align="center">GoLCG</h1>
      2 
      3 <p align="center">
      4     <img src="./.screens/gopher.svg" width="30%" />
      5 </p>
      6 
      7 <h3 align="center">Linear Congruential Generator for scanning IP ranges in Go</h3>
      8 
      9 
     10 ###### A Python version of this project is also available [here](https://github.com/acidvegas/pylcg)
     11 
     12 
     13 ## Features
     14 * Memory efficient IP range scanning
     15 * Deterministic random IP generation
     16 * Sharding support for distributed scanning
     17 * State saving for resuming scans
     18 * Pure Go implementation
     19 
     20 ## Installation 
     21 ```bash
     22 go install github.com/acidvegas/golcg/cmd/golcg@latest
     23 ```
     24 
     25 ## Usage
     26 ```bash
     27 golcg -cidr CIDR [-shard INDEX/TOTAL] [-seed N] [-state STATE]
     28 ```
     29 
     30 ## Arguments
     31 | Argument  | Description                                           |
     32 | --------- | ----------------------------------------------------- |
     33 | `-cidr`   | The CIDR range to scan                                |
     34 | `-shard`  | Shard specification in INDEX/TOTAL format (e.g., 1/4) |
     35 | `-seed`   | The seed for the LCG                                  |
     36 | `-state`  | The state file to resume from                         |
     37 
     38 ## Examples
     39 ```bash
     40 # Scan entire IPv4 space
     41 golcg -cidr 0.0.0.0/0
     42 
     43 # Scan private ranges
     44 golcg -cidr 10.0.0.0/8
     45 golcg -cidr 172.16.0.0/12
     46 golcg -cidr 192.168.0.0/16
     47 
     48 # Distributed scanning (2 shards)
     49 golcg -cidr 0.0.0.0/0 -shard 1/2  # One machine
     50 golcg -cidr 0.0.0.0/0 -shard 2/2  # Second machine
     51 ```
     52 
     53 ## State Management & Resume Capability
     54 
     55 golcg automatically saves its state every 1000 IPs processed to enable resume functionality in case of interruption. The state is saved to a temporary file in your system's temp directory (usually `/tmp` on Unix systems or `%TEMP%` on Windows).
     56 
     57 The state file follows the naming pattern:
     58 ```
     59 golcg_[seed]_[cidr]_[shard]_[total].state
     60 ```
     61 
     62 For example:
     63 ```
     64 golcg_12345_192.168.0.0_16_1_4.state
     65 ```
     66 
     67 The state is saved in memory-mapped temporary storage to minimize disk I/O and improve performance. To resume from a previous state:
     68 
     69 1. Locate your state file in the temp directory
     70 2. Read the state value from the file
     71 3. Use the same parameters (CIDR, seed, shard settings) with the `--state` parameter
     72 
     73 Example of resuming:
     74 ```bash
     75 # Read the last state
     76 state=$(cat /tmp/golcg_12345_192.168.0.0_16_1_4.state)
     77 
     78 # Resume processing
     79 golcg -cidr 192.168.0.0/16 -shard 1/4 -seed 12345 -state $state
     80 ```
     81 
     82 Note: When using the `--state` parameter, you must provide the same `--seed` that was used in the original run.
     83 
     84 ## How It Works
     85 
     86 ### IP Address Integer Representation
     87 
     88 Every IPv4 address is fundamentally a 32-bit number. For example, the IP address "192.168.1.1" can be broken down into its octets (192, 168, 1, 1) and converted to a single integer:
     89 ```
     90 192.168.1.1 = (192 × 256³) + (168 × 256²) + (1 × 256¹) + (1 × 256⁰)
     91              = 3232235777
     92 ```
     93 This integer representation allows us to treat IP ranges as simple number sequences. A CIDR block like "192.168.0.0/16" becomes a continuous range of integers:
     94 - Start: 192.168.0.0   → 3232235520
     95 - End:   192.168.255.255 → 3232301055
     96 
     97 By working with these integer representations, we can perform efficient mathematical operations on IP addresses without the overhead of string manipulation or complex data structures. This is where the Linear Congruential Generator comes into play.
     98 
     99 ### Linear Congruential Generator
    100 
    101 golcg uses an optimized LCG implementation with three carefully chosen parameters that work together to generate high-quality pseudo-random sequences:
    102 
    103 | Name       | Variable | Value        |
    104 |------------|----------|--------------|
    105 | Multiplier | `a`      | `1664525`    |
    106 | Increment  | `c`      | `1013904223` |
    107 | Modulus    | `m`      | `2^32`       |
    108 
    109 ###### Modulus
    110 The modulus value of `2^32` serves as both a mathematical and performance optimization choice. It perfectly matches the CPU's word size, allowing for extremely efficient modulo operations through simple bitwise AND operations. This choice means that all calculations stay within the natural bounds of CPU arithmetic while still providing a large enough period for even the biggest IP ranges we might encounter.
    111 
    112 ###### Multiplier
    113 The multiplier value of `1664525` was originally discovered through extensive mathematical analysis for the Numerical Recipes library. It satisfies the Hull-Dobell theorem's strict requirements for maximum period length in power-of-2 modulus LCGs, being both relatively prime to the modulus and one more than a multiple of 4. This specific value also performs exceptionally well in spectral tests, ensuring good distribution properties across the entire range while being small enough to avoid intermediate overflow in 32-bit arithmetic.
    114 
    115 ###### Increment
    116 The increment value of `1013904223` is a carefully selected prime number that completes our parameter trio. When combined with our chosen multiplier and modulus, it ensures optimal bit mixing throughout the sequence and helps eliminate common LCG issues like short cycles or poor distribution. This specific value was selected after extensive testing showed it produced excellent statistical properties and passed rigorous spectral tests for dimensional distribution.
    117 
    118 ### Applying LCG to IP Addresses
    119 
    120 Once we have our IP addresses as integers, the LCG is used to generate a pseudo-random sequence that permutes through all possible values in our IP range:
    121 
    122 1. For a given IP range *(start_ip, end_ip)*, we calculate the range size: `range_size = end_ip - start_ip + 1`
    123 
    124 2. The LCG generates a sequence using the formula: `X_{n+1} = (a * X_n + c) mod m`
    125 
    126 3. To map this sequence back to valid IPs in our range:
    127    - Generate the next LCG value
    128    - Take modulo of the value with range_size to get an offset: `offset = lcg_value % range_size`
    129    - Add this offset to start_ip: `ip = start_ip + offset`
    130 
    131 This process ensures that:
    132 - Every IP in the range is visited exactly once
    133 - The sequence appears random but is deterministic
    134 - We maintain constant memory usage regardless of range size
    135 - The same seed always produces the same sequence
    136 
    137 ### Sharding Algorithm
    138 
    139 The sharding system employs an interleaved approach that ensures even distribution of work across multiple machines while maintaining randomness. Each shard operates independently using a deterministic sequence derived from the base seed plus the shard index. The system distributes IPs across shards using modulo arithmetic, ensuring that each IP is assigned to exactly one shard. This approach prevents sequential scanning patterns while guaranteeing complete coverage of the IP range. The result is a system that can efficiently parallelize work across any number of machines while maintaining the pseudo-random ordering that's crucial for network scanning applications.
    140 
    141 ---
    142 
    143 ###### Mirrors: [acid.vegas](https://git.acid.vegas/golcg) • [SuperNETs](https://git.supernets.org/acidvegas/golcg) • [GitHub](https://github.com/acidvegas/golcg) • [GitLab](https://gitlab.com/acidvegas/golcg) • [Codeberg](https://codeberg.org/acidvegas/golcg)