Before getting an armor, just make sure you have correct clothing. You can have a beautiful helmet and a nice lorica, but it won't look anything good worn over jeans and t-shirt.
So you will at least need:
- an off-white linen tunic worn as an undergarnment
- another linen tunic, possibly decorated with clavi, orbiculi or stellae
- probably a wool tunic
- pants, eventually of Thorsberg or Damendorf pattern
- a sagum or sagulum (shorter, for cavalry) cloak, made of wool and fastened by an adapted brooch
- wool socks
- eventually legwraps or gaiters
- a cingulum belt with late roman fittings
- proper shoes, for exemple calcei or shoes based on the Deurne pattern.
To this can be added a panonnian hat, a basic bag, a knife, a wooden bowl and spoon, a woolen blanket and all things needed to perfect your impression and to do a bit of living history, at least to avoid eating in a platic plate on the camp.
It's only afterwards that you can have a look on the weaponry. Before getting heavy armor, just get a spear, shield and a helmet. Spear and javelins heads can be found quite easily. Make sure to have blunt weapons if you want to fight with it. You say you already made an accurate shield, but how it is? Late roman horsemen used both oval and round shields of various types.
For the helmet, I would say just like Jurjen. Go for a Berkasovo/Burgh Castle ridge helmet, or an intercisa, or eventually a Deurne or Der el Medineh helmet. You can get some good ones from Armamentaria for exemple, provided they will fit your head. Now if you have the option, asking a craftman about a specific command may be good.
The spatha would be the next step. Tim Noyes from Heron Armoury have a nice catalogue of both late roman and migration period swords. Alannic or hunnish swords would be different from those however, but you will have to make a special command. Don't forget the baldric and scabbard.
http://heronarmoury.co.uk/?page=swords/1-8
The armor would be the almost final step. Get a thoracomachus or subarmalis, a padded garnment you will wear under the lorica itself. Basic butted or even riveted mail is quite cheap now, but if you want a realistic armor you will have to go for a more expensive roman mail (half riveted and half plain rings), or a lorica squamata.
You want to do an heavy horseman. Mind it's a lot of work and money. Maybe you shall start as a simple footman, meanwhile you will get the equipment. A soldier of the late roman cavalry would have a good status, so that means better clothing and equipment than the average trooper. You will also have to think that your horse needs his equipment too.