According to Wikipedia:
In the 1914 translation of Richard von Krafft-Ebing's Psychopathia Sexualis, "psychology of abnormal sexuality" was rendered in Japanese as 『変態性慾心理』 (Hentai Seiyoku Shinri).
In Japanese, the word hentai is a kanji compound of 変 (hen meaning "change" or "strange") and 態 (tai meaning "attitude" or "appearance"). The term is used as a shortened form of the phrase 変態性欲 (hentai seiyoku), or "sexual perversion." In slang, 変態 (hentai) is used as an insult meaning roughly "pervert" or "weirdo". The term is not often applied to pornography in Japan. Instead, terms such as 18-kin (18禁, literally "18-prohibited") meaning "prohibited to those not yet 18 years old", and seijin manga (成人漫画 "adult manga") are used when referring to pornography.
The English use of "hentai" is more similar to the way the Japanese use the slang term エッチ (H, etchi, often spelled ecchi), which refers to any sexually explicit content or behavior. Etchi is simply the spelling-out of the Japanese pronunciation of the letter H. The origins of this term are uncertain, but it may be a shortened form of hentai used as a polite codeword in the 1960s.
"H" in Japan is now broadly used to refer to all sexual content or activity, so "H manga" are manga with sexual content. Also, the term "ero" (エロ), short for "erotic" but closer in meaning to "porn", is now used more often instead of "H".
Exactly how the term hentai came to refer to all sexually explicit content in American anime fandom is unknown. With the rise of the World Wide Web, however, the term was extensively promoted by pornographic sites selling access to (frequently bootlegged) erotic manga. Banner ads promoting these sites might, for instance advertise "live girls and hentai", with the latter meaning erotic manga as opposed to photographs. Compare otaku for another word altered somewhat in this transition.