I think the easiest way to think of this is in terms of options, and the freedom to take whatever options benefit me.
In your example, you have no options: You need to win this ko, or you will lose.
However, in my case, killing you might not be my only option. I could kill you and swing things thirty points in my favor, or if I felt like it I could grab that empty corner over there worth twenty points and strengthen my position on the right, or I could go back and reinforce my moyo on the bottom to help me with the late game fighting that hasn't started yet, or I can pick a fight with those weak stones in the centre that you forgot to connect five turns ago. I have options. You don't.
As for why the ko is asymmetric in the first place, I would be weighing not only the relative value of winning the ko, but also the actual chances I have of winning it. In most "asymmetric ko" situations, you're probably the one with the first opportunity to win the ko, and I probably won't have any threats on the board big enough for me to significantly threaten you with: Nothing I do is likely to stop you from winning it, because you need to win this ko, or you will lose. If it was just a question of "I can definitely kill you and get a hundred points, or I can definitely take two hundred points over here instead while you win the ko" that's not really an asymmetric ko.
Even if there are big enough ko threats, or somehow I had the first opportunity to win the ko, depending on how close the score is and how strong our respective positions are there might still be no real need for me to be greedy; I can forego the risk and cost of the ko (maybe those high-value ko threats will be more useful later when there's a more important fight going on) and take a lesser but safer advantage elsewhere and still maintain a comfortable lead. Why should I use up valuable aji and give you the opportunity to carve out a chunk of my territory when I don't need to?
Let's take, for the sake of example, the following situation with a 70-point life-or-death ko on the board:
- I play a 70-point ko threat.
- You respond.
- I take the ko.
- You play a 60-point ko threat.
- I win the ko.
- You follow through on your threat.
Now, this whole exchange, I'm ahead by ten points, and I've used up a 70-point ko threat to get there. This…isn't terrible, but not really an incredible outcome; if I had just played a ten-point ko threat in the first place and let you ignore it, I'd be in the same place score-wise, but with less wasted aji. Or I could've played any number of other ko threats that may have given me less points but otherwise strengthened my position. Options.
You, on the other hand, don't really have the opportunity to play a ten-point ko threat since, well, I'll just take the ko: That's now a 60-point advantage for me. You're the one who needs the big threats here, not me. Whether you live or die, I'm still coming out ahead while you still have to go back and secure your tenuous life situation. I'm totally okay with this.
In other words, you're painting yourself into a corner where you give your opponent every chance to dictate which battlefield he wants to fight on. This is generally a bad idea.