Symbolism, Imagery, Allegory
The fear grows shadowy; and Imagination, the enemy of men, the father of all terrors, unstimulated, sinks to rest in the dullness of exhausted emotion. Jim saw nothing but the disorder of his tossed cabin. He lay there battened down in the midst of a small devastation, and felt secretly glad he had not to go on deck. (2.3)
The enemy of men? Imagination? Surely you must be joking, Marlow. Nope. The guy is deadly serious:
His confounded imagination had evoked for him all the horrors of panic, the trampling rush, the pitiful screams, boats swamped – all the appalling incidents of a disaster at sea he had ever heard of. (7.24)
From these two quotes, it's clear that Lord Jim is not talking about imagination of the Willy Wonka variety. In this novel, imagination is something much more sinister. It is a source of fear.
Jim imagines all sorts of horrors when he is aboard the Patna, which leads him to jump overboard. He lets his imagination run wild, and that is a very dangerous thing for a sailor.
But there is another side to the coin as well. While Jim shows us the dangers of an overactive imagination, Brierly shows us the dangers of having no imagination at all:
"What's the use of it? It is the stupidest set out you can imagine," he pursued hotly. I remarked that there was no option. He interrupted me with a sort of pent-up violence. "I feel like a fool all the time." (7.13)
Brierly wasn't able to conceive of anything like the Patna incident at all. It is simply beyond him. But when he is faced with the facts and must acknowledge that scandals like the Patna incident can and do occur, he is shocked into total despair.