Ok, flipped the cooler, reapplied the thermal paste and reinstalled the card.
Idle temps:
DispIO: 46 C
MemIO: 56 C
ShaderCore: 48 C
VRM: 53
Load temp:
DispIO: 58 C
MemIO: 70 C
ShaderCore: 58 C
VRM: ~100 C (my system BSODs when it gets 1 degree hotter)
As you can see, the temperatures has improved, but I'm still fighting with the VRM ie the real problem