High Imperial Roman army vs Late Roman army

[C Crastinus]

3. Once the lines met and engaged, the late army would have a distinct reach advantage from their spears and ability to form a "wall" with spears protruding.

I'm a little skeptical about that. Many spear thrusts would have glanced off the rounded scutum rather than penetrating it when they charged into close quarters, and once they had closed in, wouldn't the shorter thrusting gladius have had an advantage over the longer spatha? Similarly, didn't the Spartans prefer their shorter swords because they were more effective when fighting up close? Instead of a reach advantage, wouldn't it have been a reach disadvantage once the legionary had gotten past the spear point? I know the legions couldn't do much against the Macedonian phalanx, head on at least, but that was a different creature altogether. Caesar's legions didn't seem to have much trouble with the shield walls of the Gauls.

Wouldn't the gladius' lighter weight have been an advantage in a longer fight as well, since the man wielding it wouldn't tire as fast as the man wielding the heavier spatha?

Weren't the auxiliaries during the Principate armed quite similar to the Late Roman army, with spears and the round flat shield? During the civil wars, that of AD 69 for example, there must have been instances of legionaries fighting auxiliaries so armed. If the spear and shield wall proved more effective in those engagements, would the Romans have continued to arm their auxiliary units in a manner superior to their legionaries?

I haven't come across any reasons in the ancient sources as to why the auxiliaries were armed with the flat round shield and spear instead of with the same equipment as the legionaries. Was it perhaps because the auxiliaries were frequently deployed on the flanks? Since the flanks were more likely to be engaged by cavalry, were the auxiliaries perhaps armed that way so that they could better withstand cavalry attacks?

Was the change in equipment in the later period due to the fact that they were facing cavalry more often, rather than because it was superior to the earlier armament in an infantry vs. infantry scenario?