I've said previously that, in my view, what is unprovable is to that extent unbelievable. I should probably explain and defend that, at least a little. I think it's a good and useful principle for skeptical thinking.
The claim amounts to saying that it is literally impossible to form a belief whose content is a proposition which cannot be shown, even on balance of probabilities, to be true or false. So, there's three parts to this: the circumstances under which belief-formation is possible, the nature of belief-contents, and the issue of proving something.
Beliefs are formed largely involuntarily. By this I mean that, while one can put oneself in circumstances that tend to create beliefs -- e.g., listening to a speech by someone one reasonably believes to be a persuasive speaker -- one cannot simply form a belief by voluntary choice. From this it follows that the contents of one's beliefs are similarly not within one's control.
A belief has, as its content, one of at least three things: an object, a person, or a sentence (technically, a "proposition"). That is, one can believe in ghosts (object), one can believe Immanuel Kant (person), or one can believe that Immanuenl Kant, if he still exists, is a ghost (proposition).
In the first two cases, it makes no sense to think of proof or disproof. I can prove or disprove that there are ghosts, that ghosts are insubstantial, that ghosts go "woogy woogy woogy", but these are all propositions about ghosts. I can't prove or disprove the object, ghosts. (One might, loosely, talk of proving or disproving ghosts when one means proving or disproving the claim that ghosts exist.) Similarly, it makes no sense to prove or disprove a person. One might prove or disprove particular sentences a person said or wrote, or prove or disprove sentences about that person (that he is or is not trustworthy, for example), but not the person as such.
So, proof and disproof properly belong to propositions. And thus the issue of the impossibility of believing in what cannot be proven relates only to those beliefs that take propositions as their contents.
Proof can be glossed, more or less, as the offering of an argument for a claim. (I say "more or less" as explicit argument is actually rare outside limited contexts; argument is more often implied than stated. So, to be entirely accurate, I should say "directly or indirectly offering".) So, to prove something is, at least, to offer an argument for it.
Of course, not just any argument will do: the argument has to be a good one. The sense of "good" here will have to be rough, to avoid getting insanely complicated, but the usual sorts of standards should apply -- premises true or at least plausible, rules of deductive/inductive/abductive inference are followed, etc.
A proposition is thus provable when a good argument for it is available, even if no one has ever thought of it, and disprovable when a good argument is available for one of its logical contraries or its logical contradictory. (A proposition is contradictory to another when the truth of one implies the falsity of the other, and vice versa; a proposition is contrary to another when the truth of one implies the falsity of the other, but not vice versa.)
Thus, to say that one cannot believe in what is unprovable is to say that one cannot, even when putting oneself in circumstances that generally produce beliefs, genuinely have a belief whose content is a proposition for which there are no good arguments (or, saying the same thing: whose content is a proposition whose contraries and contradictory as such that there are no good arguments for them). I should add that I would probably be willing to stretch "unprovable" past its definition thus far, to include those propositions for which good arguments exist which have not yet been thought of, for which good arguments could not be thought of by humans, and so on. This would create a continuum of "provability" ranging from the utterly unprovable -- no good arguments at all -- to the currently unproven. But I digress.
The problem lies in the process of being convinced. In order for me to believe some proposition, I have to be convinced that it is true. In order to be convinced of some proposition, I have to be given some kind of an argument for it -- good, bad, or mediocre. When I form a belief based on a good argument, it is (to borrow J. S. Mill's phrase) a "living truth" for me. I grasp at least some of its nuances and subtleties. I see where the proposition connects to other things I believe. I can place it in relation to the world. And so on.
However, when I form some sort of cognitive attitude based on a bad or mediocre argument, the content of that attitude is much less clear to me. Given that the argument is bad, the proposition does not connect to other things I believe, or the world, and its subtleties are more or less opaque. Now, I might believe otherwise -- I might believe that my belief is a living truth to me -- but since this is a factual matter, I can believe wrongly.
(Aside: yes, it is possible to have false beliefs even on this view, as there are good arguments for false propositions. There were good arguments for the existenc of phlogiston, for example.)
What I believe in this case is not really a complete proposition, then. Propositions are, after all, meaningful pieces of a larger meaningful system -- that is, sentences of a language. It would be impossible to believe a proposition and not have at least some kind of connection to other propositions. A bad argument fails to establish these connections, so what I believe is not really or not fully a proposition. And since belief of the relevant sort requires a proposition as its object, this cognitive attitude is not really belief. Kant refers to it as "thought", and that seems right to me. An unprovable proposition may be thought, but it cannot be believed.
In short, when I "believe" -- that is, I seem to believe -- a proposition based on a bad or mediocre argument, the fact that the proposition is based on a bad or mediocre argument implies I cannot be convinced of, and thus believe, that proposition.
Therefore, what is unprovable cannot be believed.