There is no real "correct" answer here, in terms of official game rules. It is going to depend completely on the friendliness/casualness of your playgroup. In a tournament setting, such a thing would need to be clarified in the tournament rules; otherwise a judge would need to be informed of the situation and make a ruling.
My game group pretty much always tends to follow this basic rule: If no other player has taken an action that revealed new information to you after you made your move, you are still free to undo your move. Note that this action could be as simple as a "whew, glad you went there!" or an actual game-action such as playing a card.
In your case, I think the rule would apply. Because the number of cards in hand is open knowledge in the game anyway, you did not take any action that revealed new information (showing that you had no cards shouldn't be considered new information).
Basically, you are free to enforce whatever rules you decide on... often if there's an uncertainty in such situations, we also just take a vote amongst the players, and see who is ok with allowing the take-back. There's nothing "wrong" with a rule that says "you took your hand off, therefore the play is finalized", but in my opinion it decreases the general friendliness and thus the overall enjoyment of the game.