If I may reply: coming out is almost a lifelong process. First we come out to ourselves, then maybe close friends or family, then perhaps the larger world or @ our place of employment.
That you came out 4 years ago is fairly recent. I came out 30 years ago and still struggle at times with certain aspects of the process.
That said, if you are a fairly regular size, you can always buy clothing online. I almost never shop in stores any more and I am a femme! I shop online all the time. You can order anything at all and if it doesn't fit, ship it back and exchange it! All the online stores I shop at have great mens clothing: J.crew, LLBean, Nordstroms, Lands End, or even Amazon has shops or you can buy on EBay. I have found some fantastic clothes, like NotYourDaughtersJeans that cost a fortune @ Nordstroms for 1/2 price on EBay and they were brand-new!
Regardless, be kind to yourself. Self-acceptance is difficult but not impossible!
You can dress any damn way you please that makes you feel good and I bet you look awesome

__________________
~Anya~
Democracy Dies in Darkness
~Washington Post
"...I'm deeply concerned by recently adopted policies which punish children for their parents’ actions ... The thought that any State would seek to deter parents by inflicting such abuse on children is unconscionable."
UN Human Rights commissioner