" ... a Hill-Billie is a free and untrammeled white citizen of Alabama, who lives in the hills, has no means to speak of, dresses as he can, talks as he pleases, drinks whiskey when he gets it, and fires off his revolver as the fancy takes him."
New York Journal, 1900 (apparently the first ever mention in print of the expression)