Improvement introducing batched, precise re-evaluation of candidates in the CELF algorithm By using a profiler and implementing optimizations during the initial candidate-counting stage of the CELF+heap process, we have managed to accelerate this stage of rulest by ~38%. Selection (CELF greedy) 13:13 33.6% (620.5 candidates/s) score calls=533,785 (batched=64,128, singleton=469,657), candidates=2,759,654; wall time=471.294 s; host allocation=2.361 s; host enqueueing=81.524 s; GPU kernel=293.786 s; H2D=0.516 s; filling=7.462 s; readback=2.024 s; wait for synchronization=315.712 s; average time per call=0.88 ms Selection (CELF greedy) 08:11 24.0% (1,006.0 candidates/s) score calls=525,423 (batched=88,982, singleton=436,441), candidates=2,833,397; wall time=409.614 s; host allocation=1.458 s; host enqueueing=74.758 s; GPU kernel=303.370 s; H2D=0.502 s; filling=6.669 s; readback=1.039s; synchronization wait=311.601s; avg wall/call=0.78ms Time saved: 5.02 min Reduction in stage duration: 38.23% What has changed The CELF algorithm continues to use simplified upper bounds (based on raw hits) to discard candidates. Candidates whose refined upper bounds still allow for a result better than the current best are now evaluated precisely and in batches - using a dedicated bitmap row of target IDs for each candidate. For the winning candidate, an additional registration scan is performed to retrieve the exact target IDs. The gain obtained this way is verified against the precise gain calculated during the batch process before the coverage state is committed. The OpenCL compute kernel supports three modes: fast upper-bound estimation, batch calculation of precise gains, and precise evaluation of a single candidate, including hit registration for the winner. The ProcessEngineProxy component issues the command to use this new precise-counting mode, while maintaining the CELF algorithm's execution on the previously selected single GPU. The method for allocating the coverage matrix remains unchanged: the full "candidate–target" coverage matrix is ​​not created. Ability to roll back changes (rollback) Batch-based, precise re-evaluation of candidates is enabled now by default. To use the legacy CELF-style re-evaluation (rescore) path, you must set this environment variable before running the program. RULEST_CELF_EXACT_BATCH=0 Please be aware, however, that for a period of 2–5 minutes, the rule processing rate will start/drop to 0 rules per second (depending on the heap load - specifically, how many "dead" candidates with inflated upper bounds caused by duplicates - reside at the top of the heap, ahead of the winning rule).

Added: Cost-weighted CELF (--cost-alpha) for postprocess Priority = gain / max(1, token_count)**cost_alpha Prefers shorter rules when coverage is similar Default 0.0 = same behavior as before Batched exact GPU scoring New kernel score_exact_batch_kernel scores up to 64 rules in one dispatch Controlled by RANKER_EXACT_BATCH env var (default: 64) Fewer GPU round-trips during CELF resolve Timing/profiling helpers in celf_select_recompute_gpu (upper-bound, seed-exact, exact-batch phases) Changed: Round 1 is now exact-scored (no longer trusts the raw upper-bound alone) Heap entries store (-priority, rule_index, raw_gain) — reported gains stay as raw coverage counts Cached OpenCL sub-regions in _score_single_recording (less overhead per chunk)

Using concentrator on debugged Hashcat rules

https://hcrt.pages.dev/email-rule-extractor

#!/bin/bash # simple script to save your time on file operations while cracking vBulletin < 3.8.5 hashes without salts using hashcat # popular types of vBulletin salt - ?a?a?a, ?h?h?h?h?h?h, ?d?d?d?d?d # download hashgen - https://github.com/cyclone-github/hashgen (used to convert $[HEX] hashes with colon-containing salts and generating hashed wordlist) # Function to read file paths with validation read_file_path() { local prompt="$1" local result_var="$2" local path while true; do read -r -p "$prompt" path if [ -f "$path" ]; then # Use eval to set the variable passed by name eval "$result_var=\"$path\"" break else echo "ERROR: File not found: $path. Please try again." fi done } echo "--- Interactive vBulletin Salt Cracker Configuration ---" # --- Input Acquisition --- # 1. Hash list path read_file_path "Enter the path to the hash list (e.g., hashlists/main.txt): " HASHLIST_PATH # 2. Wordlist path read_file_path "Enter the path to the wordlist (e.g., wordlists/top1000.txt): " WORDLIST_PATH # 3. Mask (optional, with default value) read -r -p "Enter the mask for salt cracking (e.g., ?h?h?h?h?h?h) [Default: ?a?a?a]: " CRACKING_MASK if [ -z "$CRACKING_MASK" ]; then CRACKING_MASK="?a?a?a" fi # Generate dynamic paths WORDLIST_BASENAME=$(basename "$WORDLIST_PATH") HASHED_WORDLIST_PATH="md5_wordlists/${WORDLIST_BASENAME}.md5" echo "--- Settings Confirmed ---" echo "Hash List: $HASHLIST_PATH" echo "Word List: $WORDLIST_PATH" echo "Mask: $CRACKING_MASK" echo "Hashed Word List Output: $HASHED_WORDLIST_PATH" echo "------------------------------" sleep 2s # --- Tool Verification --- if ! command -v hashcat &> /dev/null; then echo "ERROR: hashcat not found. Ensure it is in your PATH." exit 1 fi if [ ! -f ./hashgen ]; then echo "ERROR: The ./hashgen file was not found. Ensure it is in the current directory and is executable." exit 1 fi # --- STEP 1: Potfile Cleanup --- echo "--- STEP 1: Cleaning potfile ---" if [ -f "hashed.pot" ]; then echo "Creating backup: hashed.pot -> hashed.pot.bak" cat hashed.pot >> hashed.pot.bak 2>/dev/null else echo "hashed.pot not found, skipping backup." fi echo '' > hashed.pot sleep 3s # --- STEP 2: Generate Hashed Wordlist --- echo "--- STEP 2: Generating hashed wordlist ---" mkdir -p md5_wordlists ./hashgen -m 0 -w "$WORDLIST_PATH" -o "$HASHED_WORDLIST_PATH" sleep 3s # --- STEP 3: Crack Salts --- echo "--- STEP 3: Cracking salts (Mode -m0 -a6) ---" hashcat -d1 -m0 -a6 -w4 --hwmon-temp-abort=95 --potfile-path=hashed.pot "$HASHLIST_PATH" "$HASHED_WORDLIST_PATH" "$CRACKING_MASK" --remove sleep 3s # --- STEP 4: Prepare Hashes for Wordlist Cracking --- echo "--- STEP 4: Preparing hashes for cracking (m2611 format) ---" sleep 3s # 1. Process NON-HEX lines (already cracked passwords) # Corrected format: HASH:PLAINTEXT (Single colon) grep -v HEX hashed.pot | sed 's/^\(.\{65\}\)/\1:/' | cut -d: -f1,3 | sed -E 's/^(.{32}):(.*)$/\1:\2/' > 2611_to_crack.txt sleep 3s echo "Converting HEX strings" # 2. Convert HEX strings grep HEX hashed.pot | cut -d: -f2 > hex.tmp grep HEX hashed.pot | cut -d: -f1 > hash.txt ./hashgen -m plaintext -w hex.tmp -o hex.txt sleep 3s # 3. Combine HEX hashes and append to 2611_to_crack.txt # Corrected format: HASH:HEX_SALT (Single colon) paste -d : hash.txt hex.txt | sed -E 's/^(.{32}):(.*)$/\1:\2/' >> 2611_to_crack.txt sleep 3s echo "Deleting temporary files" # 4. Remove temporary files rm -f ./{hash.txt,hex.txt,hex.tmp} sleep 3s # --- STEP 5: Crack Hashes with Wordlist (Mode -m2611) --- echo "--- STEP 5: Cracking hashes with wordlist -a0 -m2611 ---" hashcat -d1 -m2611 -a0 -w4 --potfile-path=hashcat.pot.salted 2611_to_crack.txt "$WORDLIST_PATH" sleep 3s # --- STEP 6: Export Cracked Hashes --- echo "--- STEP 6: Writing/Appending last cracked hashes into file ---" hashcat -d1 -m2611 -a0 -w4 -O --hwmon-temp-abort=95 --hwmon-disable --potfile-path=hashcat.pot.salted 2611_to_crack.txt --show > 2611_cracked.txt sleep 3s echo "Done." Preview