Are there dice that have both positive and negative numbers, where the average result is zero? I'm imagining a die labeled something like: -2, -1, 0, +1, +2. I feel like I've seen dice like this, but I'm not sure what they'd be called or how to find them.
Probably the most commonly available ones are called Fudge dice, as used in the Fudge/FATE roleplaying games.
A Fudge die is a six-sided die with the following sides: minus, minus, blank, blank, plus, plus.
Games generally have players rolling a couple of Fudge dice together and adding them up, which creates a curved probability distribution centered around zero. 3dF, for example, gives you a range from -3 to +3, with 0 as the mean, median, and mode.
This Wikipedia article has more information.
You could take any two normal dice of different colors and denote one as the positive die and one as the negative die.