So I backed off the nut and span the wheel again, tightened it again and same happened, making me think that the axle spacer may be too long and loading the bearings.
Rather than mess around with the axle spacer I decided to quickly knock up a replacement from some stainless I had lying around.
I made it size for size and gradually reduced the length until torqueing the axle nut had no effect on the wheel rotating.
Ended taking 0.35mm off the spacer at which point tightening the axle nut had no effect on the spinning wheel.
All back together now and ready for a ride - I shall keep a close eye on things.