I don't think there's a really an EA reason to pick a certain college. Just pick based on the normal considerations (e.g. where you'll most enjoy living; where you think you fit with the culture / most fun; quality of tutorials & housing & funding; general reputation; academics https://en.wikipedia.org/wiki/Norrington_Table).

I did physics and philosophy and went to Balliol and was happy about that.

The main reason was that Balliol was clearly the phys phil college at that time, with ~5 people studying it each year, out of a total of ~15 in the university. It also had David Wallace, who was a great phys phil tutor (though he's now left). I'd guess it still the biggest phys phil, but haven't checked. It's also well-known for PPE.

I think this was a decent reason to choose it - I appreciated having other phys phils to work with, and they tend to be an interesting bunch (and maybe some of the most naturally EA minded people out there!). I had lots of great tutors too.

I thought at the time that Balliol also does well on other factors: central location, OK housing (though not as nice as the wealthiest ones); culture a bit more lefty; decent academically; good reputation etc.

If I hadn't gone to Balliol I might have gone for Wadham for the social life or one of prestigious ones with beautiful grounds (e.g. Merton, New, Magdalen, St Johns). New seems to have a good combination of features.

It's true these ones are harder to get into than the others, but in Phys Phil they used to have a good 'pool' system, so if you don't get into your first choice college, they'll assign you to another one so long as you're above the bar in general.

I attended the "Other Place" but my serious answer is the same for Oxford. Research online which colleges are richest (and most prestigious) and apply there unless you have a strong reason to prefer another college. The richest colleges have more grants, funding and opportunities available to students - you can save thousands of pounds and get access to opportunities just not available elsewhere. (For example I found out early on that my peers at another Cambridge college received a grant for books 3x larger each term than my college did - which in turn was more than another gave all year) - likely the additional additive networking opportunities are equally as valuable. Connections to well known EA's within faculty are probably overrated, unless you can be sure of a close connection to a faculty member for several years (as what Cambridge would call a Director of Studies) you won’t gain a significant advantage and you can pursue the interests outside of your formal study program.

It's been a few years since I was there, but most of the EA events were organised at the university level (aside from a small number of Balliol-specific events that were short-lived I think) and I would be surprised if that had changed.

I guess one EA-specific consideration might be proximity to the FHI office. If you can get college accommodation I guess that would favour Worcester, which is a lovely college in any case.

Balliol tends to have a lot of philosophy graduate students, and Wadham is considered to be one of the most left-wing colleges. Looking at the list of current Oxford philosophy graduate students, I noticed there are a lot at St Anne's right now as well. But this can change depending on the year, and philosophy student obviously doesn't mean EA. I would be surprised if any college reliably had a higher number of EAs. 

AlasdairGives' suggestion to consider funding options makes sense, though you should also keep in mind that the wealthiest colleges get the most applications, so if you apply to St John's, there's more of a risk they won't pick you, and then there's more randomness in the college you end up at.