To add to the other posts. The phalanx was a very rigid formation with a weak right side as the shields were all carried on the left (This why the experienced fighters were put on the right side to compensate), so when an enemy army managed to flank that side or break through the line the phalanx usually collapsed.
The Romans used more flexible formations that were spread thinner which made flanking the phalanx easier. Additionally the Romans used the pilum/pila (Javelins), The pilum is debatably a specialized anti-shield weapon
which could be used to harass the phalanx from a longer distance than the long pointy sticks could reach.
Then when the (western) roman empire was reaching its end-of-life and after its collapse, the states that sprung up were significantly poorer and with less disciplined armies at which point the cheap pointy stick becomes much more economical and practical in the hands of the largely untrained masses. You should also be aware that an army that could rival the roman legions at their peak does not appear in Europe until well after the medieval period.