Lets do a test.

if both armor have same attack and defense, say 10 attack, both character have 100 HP. If you use spirit armor against earth and wind armor, normal attack is 10, extra dmg will be 5. Spirt armor will deal 15 dmg and 20 for special attack, but fire armor will deal 10 dmg and 15 special dmg. Without special spirit armor need 7 times to finish the enemy but a earth and wind armor needs 10 times.

If both armors are against each, the first strike will have the advantage, but mono armor is prone to one element disadvantage. Dual elements have higher chances to deal extra dmg and a higher chance to receive to receive more. You maybe right that mono needs to be stronger but it may cause imbalance.