03 450fse advice on upgrades to improve starting

[hillbilly]

Bought bike 1 year ago. Reluctant/slow turn over to start when bought but ran well after bump starting. Got it cheap because of this, still on tight budget though. Read up a lot on here and have done following:
Overhauled starter, full of dirt. New brushes, blocked hole at front
New battery, Yuasa
Re shim ex valves, (this increased decomp valve lift slightly)
New plug
Repair various wiring insulation splits.
Replace faulty fuel pump relay
Keep battery connected to Optimate charger all the time.
Discovered cold start button on back of throttle twist grip.
All this resulted in bike starting more reliably, best improvment was cold start button. Press it in, dont touch throttle and starts nearly first turn. Hot start not so good but several short stabs on starter usually does it.
Recently getting more difficult again, slower to turn over, pulling battery down due to having to try more attempts. Checked valve clearances, stripped starter both of which ok, pulled plug, sparks ok.
Starts ok now but only just,still spins slower than previously. Always wary when I go out that I could get stuck somewhere if I have stopped and need more than 3/4 attempts to start and run battery flat. Even fully charged at 13v turn over slow at best and 5/6 aattempts is all you get then wait for battery to charge again
Wanting to know what is the best answer to upgrading starter system to give more powerful and faster engine turn over as this seems to be the issue as bike runs well and fires up quickly if turning over fairly quick.
Read about fitting 04 cam for more decomp, different starter, different batteries etc. Had already had sprag clutch upgrade on starter when I got it.
What is the definitive way to get these things to spin up better and to get more than 4/5 attempts? Someone must have done it or do you just change bikes and give up as I 'm close to doing. Love riding it, but takes away the fun when all you think about is can't go far in case it won't start! Based in UK by the way. Love to hear there is a master solution.