Two possibilities here. My two favourite albums - both only available on vinyl and relatively hard to get hold of. So they're not just my favourite abstractly, they're actual favourite objects.
Above is Let's Get Small by Trouble Funk which I first came across on this compilation and which completely beguiled me in every way. The graphics, the language, the noise. My favourite ever gig was Trouble Funk at the Brixton Acadamy where they, basically, played the same groove for three hours. My second favourite ever gig was Trouble Funk at The Fridge where they played it again.
The second is Funny How Time Slips Away by Joe Hinton from a brilliant compilation called If It's Not A Hit I'll Eat My Hat - lots of rare, early bluesy, soul-type stuff. Completely exotic sounding. Heard about it on John Peel, found it in a discount record shop in Derby the next day. Bliss. Promise me you'll listen all the way to the note he hits at the end.