I have had a pair of TCX Infinity (albeit the Held branded version) for a number of years and have used them off-road, but they aren't ideal. Here are a few issues I've found with them:
1) The sole has no grip whatsoever on anything but dry ground - grass, mud or wet rocks, then your over on your arse.
2) The sole is too soft and cuts up really badly on abrasive surfaces and is ruined by serrated metal footrests (or even the standard pegs with the rubbers removed).
3) As you've mentioned, they are very supple round the ankle offering little protection. (Makes them dead comfy to walk in though!)
On the road they've been a fantastic boot - comfy, warm and waterproof.
I'm considering pulling the trigger on a pair of
these to replace them as a "do-it-all" boot. They seem to be getting good reviews and can be had at a decent price.