A complete history of scoring changes, coupled with the whys of those changes would be a complicated story. But those doubled contract penalty changes came about for a simple reason. A few pairs realized that preemptive bidding against slams was simply too much of a potential gain against slams, so the scoring change was implemented to prevent this. Compare how many tricks you can go down against a vulnerable small slam, and still gain under the two scoring systems:
Old: 100 300 500 700 900 1100 1300 1500
New: 100 300 500 800 1100 1400 1700
So if they are vulnerable, a small slam will cost you 1370, 1430, or 1440 points. If you are not vulnerable, you could go down 7 tricks (doubled) and STILL gain under that scoring. Under the new scoring ladder, even a 6 trick deficit will be too much against a minor suit slam, and a 7 trick deficit will always cost you against all slams.
The second part of the question is to identify the cause of declining standards. A big reason is seeing how important it is to open 1NT. When you CAN open a strong 1NT, you gain considerably in the bidding. It allows you to become declarer in most contracts, assuming that you use transfers. I recall reading that this is a potential gain of roughly 1/2 of a trick on average, because they must lead into your hand at trick one. As well, hiding the strong hand is a valuable thing. If your opponents know what cards you have, they can defend vastly more effectively.
So making the strong hand the declarer is a valuable thing. But how often does this happen? How often will you open 1NT with a 16-18 point hand?
Overall, the probability of picking up a 16-18 point hand (ignoring shape) are 7.28%, versus 10.09% for a 15-17 point hand. (I could run a complete simulation to get the odds for balanced hands in those ranges, but things won't change too much.) The point is though, compared to a 6-18 point 1NT, you will be able to open a 15-17 point 1NT almost 40% more often! This difference is a HUGE gain.
So I'd argue that the shift in strong NT ranges is not a declining of standards, but a realization that the slightly weaker range is much more valuable overall.
Other changes in preempt quality are due to the realization of how valuable a preempt can be. Your opponents today will often use highly accurate bidding systems. Compared to the "old" days, when the opponents more frequently got to the wrong contracts, it is simply too dangerous to allow a quality pair to bid to the optimal contract. So instead we preempt more aggressively. This makes them guess how high to bid, or force them to double us.