If I cast a spell that has an "on cast" effect, does the effect still trigger even if the spell is countered? For example, if I play Ulamog, the Ceaseless Hunger and it gets countered by a counter spell, do I still get to exile 2 target permanents?
Yes. A player doesn't have a chance to cast a counterspell until after you have finished "casting" your spell, and any "when cast" triggers are triggered and put on the stack. At that point, a player could counter your spell, but the triggered ability is still there as a completely separate thing.
601.2. To cast a spell is to take it from where it is (usually the hand), put it on the stack, and pay its costs, so that it will eventually resolve and have its effect.
112.7a Once activated or triggered, an ability exists on the stack independently of its source. Destruction or removal of the source after that time won’t affect the ability.
A small point of clarification: If your opponent counters your Ulamog, the Ceaseless Hunger with Summary Dismissal it will counter both the spell and the "when you cast" triggered ability. Similarly, your opponent could use a couple spells to counter your spell (e.g., with a Ceremonious Rejection) and the triggered ability (e.g., with a Stifle or Disallow).