Why do I have low self-esteem?

I also have very low self esteem, but I know that I'm not ugly or stupid. Its really held me back in life and I'm sick of it. To make it worse, one of my biggest regrets thanks to my lack of confidence/self esteem had something to do with a girl :(