The cavalry has always been used to break up an infantry position. The horses come at the line, and the infantry either runs away, or they break ranks, and the cavalry can come in and attack smaller groups.

Take a look at the battle of Agincourt, as an example of when cavalry is at a disadvantage. It had been raining steadily, first water and then arrows, from English Longbows. The horses with armored knights on their backs became bogged down in the mud. The English archers would stick their arrows in the ground in front of them and pull them out to notch and release them. With animal and human waste all over the field, an arrow into the body could result in a dangerous infection.
The French had hired crossbowmen, whose bowstrings had become wet, and stretched from the rain, making them less effective than the longbow strings, which were left unstrung during the night.

And yes, forming a shield wall and thrusting out spears can deter cavalry charges. The Macedonians and the Romans both employed similar tactics to keep Persian heavy cavalry at bay, but that was all it was, a defensive formation, and one that depended on the discipline of he troops maintaining it. If the “pikemen” loose their formation, they become vulnerable to attack as they lose the protection of the formation. One man with a long spear is not going to deter a charging cavalrymen. Because of this, if infantry formations did not use this tactic when attacking, and that cavalry forces would employ different tactics to break up the formation.

Good cavalrymen developed horse archers in addition to what are traditional heavy cavalry. Instead of charging the shield wall, they’d come within the range of their arrows and will fire away until they have either used all their arrows or are driven off. This can tend to break up the defensive formation, and could even kill some of the defenders. The Huns and Mongols used these tactics repeatedly. Few European countries however ever had the skills needed to develop horse-archers.

There are also the vulnerablities of the formation itself. The formation is based off of the Macedonian Phalanx and is forward facing, which means they need a long line or some sort of boundary to protect their flanks. If a soldier turns while in this formation he creates a weak point in the front of the line that can be exploited. If not, cavalry will likely swing around and attack the flanks of this force. The actual battle, inaccurately depicted in “Braveheart” actually took place on a bridge where the English had no choice but to charge directly into the defenses. There is also the possibility that one force would expect you to attack them. The shield wall is impossible to use as an offensive weapon as moving the soldiers across the field could potentially break that formation leading to that formation becoming vulnerable. Most often, these sort of impasses lead to no action at all as each side waits for the other to make the first move, as was the case in the early moments of the Battle of Tours.