Not only worth a visit if you're interested in military history, it's also a wonderful place to find some peace and quiet within the city. The cemetery covers a massive area, including trees and grassy walkways, and you can spend hours wandering around. Because it's located in a valley, there are no signs of the surrounding traffic and high buildings, and it really feels like you're not in Seoul.
The cemetery is located at Dongjak station, lines 9 and 4. To get there, come out of exit 8. If you arrive by line 4, you can come out of exit 2 and walk towards the main road. It's not far from the Han river so I cycled there via the riverside cycle route. It can be a little confusing to get there from the river, though, so check on a map. There's a small place to buy refreshments near the main entrance.