I forgot to add that if I leave the machine long enough on (2-3 days) the RAM 
usage goes down to like the normal 5/6% BUT the swap is totaly used up with 
maybe like 5/6% left free.
Also, I forgot to attach my kernel config in case it helped,, so here it is