There are as many ways of weathering ballast as there are types of ballast, but I expect you already know that.
My approach is to paint the sleepers and rail before the ballast is laid and wait for it all to dry hard. Anywhere between 2 and ?? days. This will give you the opportunity to establish what colour(s) to use on the ballast.
I use an airbrush to apply a thinned acrylic coat of the track grime colour chosen and wait for that to dry. 24 hours at least. This will allow you to see if there are any areas that need a further coat, places that have been missed, etc.
Final task is to apply weathering powders in various places to vary the tone of the grime. If you don’t do this you will end up with track that looks as if it has been airbrushed. Find a reference photograph of the type of track you’re weathering in the area you’re modelling and use it as a guide.