First, read your contract. Laws vary from state to state. In California, standard C.A.R. purchase contracts allow for the return of the earnest money deposit to the buyer within a specified time period, by default 17 days, should the buyer elect to cancel the transaction. If, at that point, the seller refused to return the deposit without cause, the seller could end up paying a $1,000 civil penalty to the buyer.
Upon cancellation, the sellers and buyers are asked to sign mutual release instructions. If an agreement cannot be reached, the party holding the earnest money deposit will continue to hold it until an agreement is reached. If no agreement has been reached after a few years, escrow companies then send the parties a certified letter asking for mutual instructions. The letter says if nobody responds within a certain time period, then escrow will return the money to the buyer. If the seller contests the action then, after 3 years, escrow will send the money to the state of California, presumably to help balance our budget deficit.