We model mortgage refinancing as a bargaining game involving the borrowing household, the incumbent lender, and an outside bank. In equilibrium, the borrower’s ability to refinance depends both on the competitiveness of the local banking market and on the cost of switching banks. We find empirical support for the key predictions of our model using a unique data set containing the population of mortgages in Belgium. In particular, households’ refinancing propensities are positively correlated with the number of local branches and negatively correlated with local mortgage market concentration. Moreover, households are more likely to refinance externally if they already have a relation with more than one bank, but the effect is mitigated if their current mortgage lender has a branch locally.