Where did he come form a shelter or the roadside.
Some cats are just shy of humans.  It takes time for an animal to except
people.

Animals don't hate people  they may mistrust people.  Which is not in your
case  he jumps up when you sit or sleeps with you in the same bed.  Here is
a question  does the furball turn his back to you and lay down to go to
sleep  that is a sigh of complete trust in the animal world.

He shys away when you walk up to him  some cats are like that.  My rumble
who I love dearly does this sometimes  cats are cats.   You need to have
patience.  If you want him to stay around you need to socialize a little
more with him.  Take some treats or catnip call to him.  When you get near
him  start calling his name so you don't scare him or make him nervous.  Let
him come to you also  get him used to the idea of humans as friends

Spend time softly talking to him, and play with interactive toys (like
a Cat Dancer). Let him come to you--if he'll sit on your knee or sleep
in your bed, he's already making progress. He just needs a little more
time to settle in.