1. All options maximize expected utility (EU), since the expected utility will be undefined (or infinite) regardless. There's always a nonzero chance you will end up choosing the right religion and be rewarded infinitely and a nonzero chance you will end up choosing badly and be punished infinitely, so the EU is +infinity + (-infinity) = undefined. (I got this from a paper, but I forget which one; maybe something by Alan Hájek.)
2. In response to 1, you might say that we should maximize the probability of +infinity and minimize the probability of -infinity before considering finite values. This could be justified through the application of plausible rationality axioms directly, in particular the independence axiom. This could reduce to EU maximization with some prior steps where we ignore equiprobable parts of distributions with the same values. However, infinite and unbounded values violate the continuity axiom. Furthermore, if we're allowing infinities, especially as limits of aggregates of finite values like an eternity of positive welfare, then it would be suspicious to not allow unbounded finite values at least in principle. Unbounded finite values can lead to violations of the sure-thing principle, as well as vulnerability to Dutch books and money pumps (e.g. see here, here and my reply, and here). If the bases for allowing and treating infinities this way require the violation of some plausible requirements of rationality or require ad hoc and suspicious beliefs about what kinds of values are possible (infinity is possible but finite values must be bounded), then it's at least not obvious that we're normatively required to treat infinities this way. Some other decision theory might be preferable, or we can allow decision-theoretic uncertainty.
3. There are plausible alternative decision theories that don't require (but may permit) choosing extremely low probability bets of extremely high payoffs, like EU maximization with bounded utility functions and stochastic dominance. Under decision-theoretic uncertainty assigning some credence to unbounded EU maximization with infinities, low probabilities of Heaven or Hell might still not dominate.
4. Under impartial views, conditional on a given god (or gods), you won't change the aggregate: it's already undefined, +infinity or -infinity, and adding one more person to Heaven or Hell won't make a difference to that value. Some possible responses:
   1. More complex approaches to aggregation (e.g. ignoring unaffected individuals, the Pareto principle), so that getting one more person into Heaven or keeping one more person out of Hell is still infinitely better.
   2. Maybe you can decrease the probability that Heaven will be empty, or increase the probability that Hell will be empty.
5. There might be more promising infinities we can pursue in practice, potentially
   1. Creating or preventing infinite universes or infinitely many universes.
   2. The universe is already infinite and has an infinite amount of value in expectation even on highly plausible physics, because it's very plausibly infinite in spatial extent (or temporally), and under evidential decision theory, we already acausally affect an infinite amount of value, because infinitely many agents make decisions correlated with our own.
   3. Some others are discussed here.

I included the wager below for reference since it doesn't seem to be in the original question. 

I think one problem is that belief in the existence of God is probably not sufficient for an infinite payoff (and it's not 100% clear to everyone what is sufficient). My understanding is that most major religions are meant to teach something more complex than that. Usually something to do with helping others and attaining peace by letting go of selfish desires in favor of loving and kind ones. 

But honestly, I think the reason people reject the wager is because they don't like it. Maybe because infinity is already incomprehensible and uncertainty around infinities just makes it even more difficult to deal with. We generally like certainty or at least ways to be somewhat certain about how uncertain we are and how to become more certain. 

So, it's often easier to just avoid something that doesn't clearly guarantee a payoff. And switching to either for or against God existing doesn't seem like it has a clear payoff for most people. Like many things, you can just forget about the question and then it won't seem to have much impact on your life. Same way most of the time most people just forget about the meaning of life or the possibility of nuclear war and other complex topics that seem to not have clear solutions to most people. 

Pascal’s Wager (Stanford Encyclopedia of Philosophy)

The argument:

"Either God exists or God does not exist, and you can either wager for God or wager against God. The utilities of the relevant possible outcomes are as follows, where f1, f2, and f3 are numbers whose values are not specified beyond the requirement that they be finite:

1. Rationality requires the probability that you assign to God existing to be positive, and not infinitesimal.
2. Rationality requires you to perform the act of maximum expected utility (when there is one).
3. Conclusion 1. Rationality requires you to wager for God.
4. Conclusion 2. You should wager for God."

The binary choice nature of the wager always seemed bizarre to me. The real-life choice isn’t “Christian God: yes or no,” it’s “try to pick the right option among these very many religious choices.” It also seems to me that some possibly right choices have rankings that go: my god > no religion > wrong religion. None of this necessarily means that you shouldn’t take the wager, but given the above it definitely isn’t obviously right to me.

I also disagree with all the answers that I've read but I have thought of one that works for me and hope it'll work for you too: I think there's a lot more probabilities of achieving infinite value in this universe, for me and/or for future people, making progress in this civilization ignoring religions, than the probabilities of achieving it following them + the probabilities of any other complex imaginary world after dead.

Now, this doesn't seem to work for Pascal's Mugging, so I hope to find another answer to that or to not get poor because of it lol.

Let's entertain as an axiom the claim that, in the absence of evidence, promises of utility/disutility become less likely the more is promised.

If I promise you $1 to drop off a letter at the post office for me, you'd believe me. If I promise you $1,000,000, you'd think I was joking.

More specifically, let's make our axiom the claim that, if we integrate the likelihood of a payoff over the range of utilities promised, that integral is convergent.

No matter how much utility is promised, the amount of utility received in expectation is finite.

In other words, there is no infinite expected utility.

If this is accepted, then expected utility (always finite) is controlled largely by mechanistic plausibility and empirical evidence, not just the sheer amount promised.

For example, if I have a history of making and keeping extravagent promises, you know I have billions of dollars in the bank, and you can see a reason it would be worth it to me to pay $1,00,000 to have you take my letter to the bank, you might think it's pretty likely I'll pay you as I promise. These sorts of considerations become extremely important as the amount of utility promised increases.

You don't have to accept the axiom, but if you do, then I think you end up at the common-sense position that you should reject Pascal's Wager, be open to the possibility of small utility gains on limited evidence, and require larger amounts of evidence the more utility is promised. This principle comes in handy when avoiding scams.

Why didn't Pascal, a brilliant mathematician, come up with this argument on his own? I can think of a few possibilities:

• I might be mistaken in my reasoning.
• Pascal was born 20 years before Newton, and died several years before Newton's seminal publication on calculus. He may therefore not have had access to what is now college-level knowledge on divergent and convergent integrals.
• He might have been rationalizing a decision  that was fundamentally emotionally driven.

I'm mostly a utilitarian too, and I mostly support taking gambles to maximize expected value...

...but I'm quite uncomfortable with having to make an arbitrarily large sacrifice (like submitting 3↑↑↑3 minds to torture) for an arbitrarily small probability (like 1/(3↑↑↑3)) of infinite value.

Moral or decision-theoretic uncertainty seems to make it reasonable to reject some wagers, at least.

My perspective on the issue is that by accepting the wager, you are likely to become far less effective at achieving your terminal goals, (since even if you can discount higher-probability wagers, there will eventually be a lower-probability one that you won’t be able to think your way out of and thus have to entertain on principle), and become vulnerable to adversarial attacks, leading to actions which in the vast majority of possible universes are losing moves. If your epistemics require that you spend all your money on projects that will, for all intents and purposes do nothing (and which if universally followed would lead to a clearly dystopian world where only muggers get money), then I’d wager that the epistemics are the problem. Rationalists, and EAs, should play to win, and not fall prey to obvious basilisks of our own making.

It seems quite clear that the probability of a god that matches the god of existing religions is far more likely than a god that is the opposite

Why is this clear? More broadly, why would you have a strong prior on god's nature?

You can get out of the infinite (+/-) payoffs by exponentially discounting future well-being. This assumes that, while in heaven or hell, you experience a finite amount of well-being at every point in time (that doesn't grow exponentially without bound), but you live for an infinite amount of time.

My thoughts are pretty similar to those already expressed by ryancbriggs and MichaelStJules, and some others.

What does it mean to accept Pascal's Wager? 

I understand Pascal's Wager to argue that it's more rational to believe in God than not because you end up in Heaven if you believe in God and not if you don't, and heaven is infinitely more valuable than any alternative. So even if your rational credence in God existing is very very low, it's still more rational to believe in God than otherwise. 

I take the correct core of the argument to be that sometimes it can be prudential to believe in something you think is likely false (such as the existence of God and Heaven), or pursue some outcome you think is very unlikely (getting into heaven), because the value of the reward is high enough. 

The general form of this argument is generally accepted and acted upon. People do similar sorts of things all the time when they pursue success in competitive or challenging environments with high rewards to the biggest winners, like high level professional sports, startups, tenure track academia, etc etc. 

I'm don't know if Pascal's Wager has any implications beyond this? 

In real life, there is no binary choice between 'Believe in God and have a chance of getting into heaven' and 'Don't believe in God and have no chance of getting into heaven'. There are several different major world religions, some very different sects inside the same religions, your own individual interpretation, the possibility of infinite utility from technological sources, etc. And this is without getting into how one actually achieves going to heaven in religions - sometimes it's not 'belief in our god and you'll get into heaven', but rather something involving structuring your life around the religion. And of course, many of these directly or indirectly oppose each other even apart from the opportunity cost. 

Perhaps Pascal's Wager could be an argument to set one's life up to deliberately pursue even one source of infinite utility rather than none. I don't think this is the worst argument in the world and it could work as part of a cluster of arguments, but it's a pretty weak one in a vacuum since it's completely silent about crucial matters such as how to choose between any of these possible infinite utilities or how to pursue them. 

There are many good critiques of the details of Pascal's wager. For example:

• He assumed that reason couldn't help you figure out if God existed, so presumably it was just a leap of faith. [1]
• He assumed God wouldn't mind someone brainwashing themselves into a religion they disagree with, out of pure self interest.
• He gives little to no reason to follow one religion over another, since almost all of them claim there the afterlife can be very postive or painful

but I have looked and not found  any good reason to dismiss what I think is the heart of the argument: that we should take the possiblity of going to Hell or Heaven super seriously, more than any Earthly matter.

Pascal's wager offers little insight on what to do with this information, but I think a good next step is trying to find out for sure if Heaven and Hell are fiction or not or if any/which religion proclaiming them is plausible.

P.S. Please correct me if I'm wrong about anything here or if I missed anything important.

P.S.S. My main sources here are https://plato.stanford.edu/entries/pascal-wager/#Prem1DeciMatr & https://www.iep.utm.edu/pasc-wag/ both accessed April 2020. For example the former says about [1]:

"Pascal maintains that we are incapable of knowing whether God exists or not, yet we must 'wager' one way or the other. Reason cannot settle which way we should incline"

P.S.S.S. Thanks for sharing this really interesting, and I think neglected, question.

There isn't one. To reject Pascal's Wager, you just have to conclude that you don't care about infinity. Taking Pascal's Wager is the correct utilitarian response. You probably need to weight religions both by how likely they are to be true and how likely you can "win" conditional upon them being true.

Amanda Askell has a good rundown on why most objections to Pascal's Wager are bad.

fwiw, I find an Evidentialist God (who rewards believing according to the evidence) far more credible than any of the existing world religions.