I'm trying to show milliseconds as seconds while also keeping the decimals e.g. I have 1234 milliseconds and I want to show this as 1.234 seconds.
Decimal duration = 1234;
NumberFormat formatter = new DecimalFormat("#0.000");
String durationStr = formatter.format(duration / 1000);
Any suggestions how I could do this?

It sounds like you should be using BigDecimal - create a BigDecimal from the long, and then scale it by 3 places:
BigDecimal bd = new BigDecimal(duration).scaleByPowerOfTen(-3);
String durationStr = formatter.format(bd);
By using BigDecimal instead of double, you know that you'll still have exactly the value you're really considering, rather than simply "the nearest approximation that double can hold".
You may well also want:
formatter.setMinimumFractionDigits(3);
formatter.setMaximumFractionDigits(3);
... to ensure that you always get exactly 5 digits. (Assuming you want 1 second to be "1.000" for example.)
See more on this question at Stackoverflow