After casual relationship...?

Well this is tricky. He obviously used you as a "rebound girl" and hey that happens. So going back to his ex didn't work out and that sux for him. 5 years is a long time and he will have had time enough to get over the ex/love of his life, but do you really wanna go through this again? Maybe his feelings for you have changed but i doubt it. If you want to date him tread carefully and set some ground rules. He must be exclusive with you no "behind closed doors" crap, and i would even go so far as to withhold for awhile and just date so he has to get to know you as a person and not just a piece of meat as you put it. If he is annoyed with not getting any from you just mention that last time you were together it seemed you rushed into and didnt get to know one another, this phrase and his response to it will speak volumes. If he wants to date you and try to get serious he will be ok with this request, if he's just looking for a call he'll probably whine or roll his eyes and get p*ssed. But either way your in control and the in your court.