A unit that is supporting is also holding and is always eligible to receive support in its hold.
The relevant quote from the rulebook is:
A unit not ordered to move can be supported by a support order that mentions only its province. A unit that is ordered to hold, convoy, support, or not ordered at all can receive support in holding its position.
The most relevant thing to notice here is that a unit that is not moving is holding. A support or a convoy is also a hold. If a supporting unit is attacked, the unit does not revert to holding (this is a useful flavor justification but is not technically accurate). The unit was holding the whole time. All that happens is that it is no longer able to support in addition to holding as it is under attack (the support is cut).
As a result of this, two units can support each other, assuming they can each legally support the other (based on adjacency and unit type). Since each unit is supporting, they are each also holding, and thus each is eligible for support in holding its position. If one unit is attacked, it's support for the other unit will be cut, but it will still hold and the other unit will support it's hold.
An opponent needs a minimum of three units to dislodge one of your units and thus disrupt your position. They can do this by either doing a doubly supported attack on a single unit, or by doing a singly supported attack on one unit and an unsupported attack on the other unit to cut it's support.
This is consistent with the basic principle of diplomacy: equal numbers create a standoff, whereas superior numbers (eventually) win. Your two units can defend against an attack of two units but not against an attack of three units.
Note that you can also support the hold of a fleet that is convoying an army. This will help prevent the fleet from being dislodged if it is attacked while convoying.