pantagrule: hashcat rules generated from over 840 million compromised passwords

by do son · Published August 14, 2020 · Updated October 11, 2021

Pantagrule

gargantuan hashcat rulesets generated from over 840 million passwords

Pantagrule is a series of rules for the hashcat password cracker generated from large amounts of real-world password compromise data. While Pantagrule rule files can be large, the rules are both tunable and perform better than many existing rule sets.

Pantagrule was generated using the PACK‘s Levenshtein Reverse Path algorithm for automated rule generation (Kacherginsky, 2013). PACK’s output was then sorted based upon the number of times PACK generated the rule to make the base ruleset. This process is similar to the rules generated by _NSAKEY for password cracking competitions in 2014 (_NSAKEY, 2014), however, Pantagrule has generated off a significantly larger set of passwords. Pantagrule’s underlying password corpus contains 842,643,513 unique passwords, which alone was able to crack roughly 70% of the Have I Been Pwned Pwned Passwords NTLM list in a straight dictionary attack. This corpus is roughly 9.2 gigabytes in size and PACK generates 161 gigabytes of unique rules from the corpus, leaving the full Pantagrule rule file at an obscenely impractical 6,348,454,502 unique rules.

However, since most of the rules generated appear only a handful of times, most of the useful rules are the ones that are most commonly generated by the algorithm. This repository contains a subset of rules generated by PACK whilst iterating through the existing corpus.

Optimised variants

In order to generate a second-pass optimisation of the rules against real-world data, the top one million generated rules was run against the Pwned Passwords NTLM list using the rockyou wordlist. Any rule that cracked a password was added to its own list and poorer-performing rules were discarded.

Four optimisation types were created:

pantagrule.popular.rule: pantagrule.1m run against the top 25,000,000 passwords of the HIBP set.
pantagrule.random.rule: pantagrule.1m run against 25,000,000 randomly selected passwords from the HIBP set.
pantagrule.hybrid.rule: A sorted list of a combination of the most successful popular and random rules, then cut in half, in an attempt to make a lighter, “balanced” ruleset that works across a larger sample set.
pantagrule.one.rule: A version of OneRuleToRuleThemAll in which the top performing hybrid rules are appended, and the list is truncated to the size of the dive rule set. Interestingly, there is only a couple-thousand-rule overlap with OneRuleToRuleThemAll and the Pantagrule rules, making the two strategies complementary. The HIBP target of one offers a marginal 1.76% improvement on the top 25M passwords, and an approximately 5% gain on the top 100M over OneRuleToRuleThemAll, making this list perform better than other existing rule sets of this size.

Note that these rulesets are also sorted by number of successful cracks on their datasets, so they may also be truncated for speed using head. It is recommended that you start with these rulesets. All four optimisations crack more passwords in testing than previous commonly-used rule sets. You can find these original variants in the rules/original folder.

The royce variants

As stated above, Pantagrule was originally refined against the HIBP Pwned Passwords NTLM set as its target. Upon request of hashcat contributor Royce Williams, optimisations of the top one million rules were also run with the hashes.org founds list. This is due to the HIBP corpus being relatively dirty, and the hashes.org founds list being likely to yield a more practical ruleset for real-world cracking. These have been added as the royce variants. The royce optimisations appear to consist of marginally fewer rules overall, and random.royce is substantially more effective on a long tail of passwords than the existingrandom. Performance did not increase over the existing rules on some variants, but given that the training and validation data of the original Pantagrule are both from the Pwned Passwords dataset, this does not seem surprising. Pantagrule roycevariants exist in the rules/royce folder.