These vertical shoots are called water sprouts. They are fast growing branches that can often overtake the leader. Usually they appear in a tree that has been over pruned or is under stress. It is a good idea to find out the probable cause and remedy the situation. You should remove water sprouts before they get too big. They tend to have weak attachments. If you let them develop into big branches they have good chances of breaking off.
Water sprouts also suck up too much energy. The energy they use detracts from the tree’s regular growth. Unfortunately sometimes if you remove them they just grow back. If this happens, the next time you prune just take out the big ones and leave a few of the small ones.
In this case, I think that this plum tree is under some stress; the root system remove is poor. How could I tell? A sure sign is to lightly shake the tree trunk. Watch the ground. If the ground moves, there is poor root structure. The ground moved somewhat when I shook the tree. However, I don’t think the tree is in too much danger of falling over. Whoever planted the this plum tree may not have sufficiently broken up the root ball. Or, it had spent so much time growing in a small pot that it will take much longer to form a healthy root system
Trees are smart. When they get knocked around by the wind they react by growing support roots. If they are tied up to a stake then they learn to rely on the stake. If the tree can’t support itself in a windstorm then it should come out before it gets too big.
I removed the biggest water sprouts and the stakes. We’ll see how this tree fares a year from now.