I think my take would vary depending on the level of weird.
For slightly weird stuff, as discussed in Tessa's answer, it's not actually obvious this is a net cost – the gain in memorability and inferred bravery/independent-mindedness could easily outweigh any costs to credibility.
For very weird stuff, I think it probably is net bad for your expected impact / career prospects, so if you don't care overmuch I'd advise against it. But if conforming to conventional appearance norms is very difficult/alienating/demoralising to you, then you should probably do the weird thing. In that respect, I put it in the same category as having kids, or living in a non-standard location with worse network effects. To quote Julia Wise:
Some things I can do cheerfully. It turns out that giving up children was not one of them. Other people would have no problem giving up parenthood, but I suspect that everyone has something that would cause an inordinate amount of pain to sacrifice.
So test your boundaries, and see what changes you can make that will help others without costing you too dearly. But when you find something is making you bitter, stop. Effective altruism is not about driving yourself to a breakdown. We don't need people making sacrifices that leave them drained and miserable. We need people who can walk cheerfully over the world, or at least do their damnedest.
Now, all that said, what counts as "slightly weird" vs "very weird" will vary depending on the place you live and the sector you work in, and so needs to be evaluated on a case-by-case basis. Academia values conformity in appearance far less than, say, the US Government. And of course different kinds of "weird" will imply different things – kooky vs messy vs uneducated, et cetera.
It's also sadly probably true that if your appearance is unavoidably different from most of the people around you, that will impact how additional voluntary weirdness is perceived – though I'm very unsure how that last thing cashes out in practice, in terms of the impact of marginal weirdness on your reputation.
To me, reducing your weirdness is equivalent to defection in a prisoner's dilemma, where the least weird person gets the most reward but the total reward shrinks as the total weirdness shrinks.
Of course you can't just go all-out on weirdness, because the cost you'd incur would be too great. My recommendation is to be slightly more weird than average. Or: be as weird as you perceive you can afford, but not weirder. If everyone did that, we would gradually expand the range of acceptable things outward.