Skip to content

Finding the right hashcat -m mode (and what to do when you get it wrong)

Hashcat won't autodetect anything. Here is how to pick the correct -m mode, disambiguate look-alike hashes, and read the errors that mean you chose wrong.

Published on 4 min read

Hashcat does not look at your hash and figure out what it is. You tell it, with -m, and if you tell it wrong it will happily run for hours and crack nothing. That is the single most common reason people think a password is "uncrackable" when it was a one-word dictionary hit the whole time.

So the real skill is not running hashcat. It is choosing the mode.

The mode number is the algorithm, not the hash

-m 0 is MD5. -m 1000 is NTLM. -m 1800 is sha512crypt. The number encodes the exact construction, including how the salt is mixed in, so md5($pass) and md5($salt.$pass) are different modes (0 and 20) even though both produce 32 hex characters. Get the construction wrong and every candidate you try hashes differently from the target. No match, ever.

This is why identifying the hash comes first. The format tells you the family. The source tells you the specific mode.

Use --identify, then ignore most of what it says

Recent hashcat ships with a shortlist tool:

hashcat --identify 5f4dcc3b5aa765d61d8327deb882cf99

That returns the modes whose structure matches the string. For a bare 32-hex value you will get a long list: MD5, NTLM, MD4, the double-MD5 variants, and so on. It is correct and not very helpful, because all of those are structurally identical. The tool can only see the string. It cannot see that the hash came out of secretsdump.py against a domain controller, which would make it NTLM and nothing else.

So treat --identify as a sanity check. The decision comes from context:

  • Pulled from NTDS.dit, the SAM hive, or a secretsdump line that looks like user:1001:aad3b435...:b4b9b02e...:::? That right half is NTLM, -m 1000. The left half is LM, and if it is aad3b435b51404eeaad3b435b51404ee it is the empty LM hash, skip it.
  • Sitting in a users table in a web app dump? Overwhelmingly raw MD5 (-m 0) or SHA-1 (-m 100), occasionally stored as hash:salt.
  • Starts with $2y$ or $2b$? bcrypt, -m 3200, and you should lower your expectations accordingly.
  • Starts with $6$? sha512crypt out of /etc/shadow, -m 1800.

Length narrows the family. Provenance picks the mode.

The errors that mean you guessed wrong

Two messages come up constantly and people misread both.

"Token length exception" means the hash does not parse under the mode you picked. The structure is wrong. Nine times out of ten it is a trailing newline from a bad copy-paste, a username column you forgot to strip, or simply the wrong -m. Run it through xxd if you are unsure what is actually in the file:

xxd hash.txt | tail -2

A 0a at the end is a newline. Hashcat tolerates one trailing newline per line, but two hashes glued together or a Windows 0d 0a will break parsing.

"Separator unmatched" shows up when you use --username or a salted mode and the colon count is off. Salted modes expect hash:salt. If the salt itself contains a colon, or you left the user: prefix on, the parser splits in the wrong place.

Neither error means the hash is exotic. It almost always means a formatting problem or the wrong mode.

The silent failure is the dangerous one

There is a worse case than an error: the wrong mode that still parses. md5($pass) and md5($pass.$salt) both accept a 32-hex token. Point -m 0 at a salted MD5 and hashcat runs cleanly, churns through your whole wordlist, and reports nothing cracked. No error. Just a wrong answer that looks like a hard hash.

The defense is a known-answer test. Before you trust a long run, hash a password you already know with the same construction and confirm hashcat cracks it in seconds:

echo -n "password" | md5sum
# 5f4dcc3b5aa765d61d8327deb882cf99
echo "5f4dcc3b5aa765d61d8327deb882cf99" > test.txt
hashcat -m 0 -a 0 test.txt <(echo password)

If that does not return password, your mode is wrong and there is no point launching the real attack.

When in doubt, let John triage

John the Ripper autodetects, and that makes it a good first pass when you genuinely do not know what you are holding. Throw the file at john with no format flag and read what it picked:

john hash.txt
john --show hash.txt

If John settles on Raw-MD5 you have your answer, and you can move the work to hashcat with -m 0 for the GPU throughput. I cover when to stay in John versus move to hashcat in the tooling comparison.

One more thing about optimized kernels

If you run with -O (optimized kernels) hashcat caps the candidate password length, often at 31 characters or less depending on the mode. For most dictionary work that is irrelevant. But if you are running a mask that generates long candidates and nothing cracks, the optimized kernel quietly skipped everything past the limit. Drop -O and rerun before you conclude the password is long and random. It might just be 33 characters of something you already had in your list.

Related articles

Found a mystery hash? Learn the signals that reveal its type — length, character set and prefixes like $2y$ or $6$ — and how to identify it privately in your browser.
A practical comparison of hashcat and John the Ripper — GPU vs CPU strengths, autodetection, -m modes, jumbo formats, wordlists and rules — with example commands.
Why bcrypt drops cracking throughput from billions to thousands per second: the cost factor, its GPU-hostile key schedule, and the 72-byte truncation gotcha.