It may not be ringed by barbed wire fences and guard towers, but the cluster of houses in a small valley close to the remote North Korean town of Hyanghari is a prison camp nonetheless.

Uniquely, the camp is used to hold senior members of the regime who have fallen foul of the ruling Kim family – and members of the Kim clan who are perceived as posing a threat to the dictators’ rule.

And, instead of the visible trappings of one of the regime’s “kwan-li-so” political penal colonies, the camp has a large detachment of guards to keep its inmates in line, defectors say.

WATCH: The refugee life of a North Korean defector in Hong Kong

There are other reasons they do not flee and attempt to cross the Chinese border, a little more than 30 miles to the north. Their relatives would undoubtedly suffer at the hands of a vengeful regime, but arguably the biggest incentive to sit tight and hope for the best is that the inmates’ conditions are relatively good. “They are not forced to work in the fields or mines and they don’t have to produce their own food as meals are delivered to them every day,” said Lim Cheon-yong, a former member of North Korea’s special forces who defected in 2000.

“That’s why it is known as ‘the resort’,” said Lim.

Other defectors who retain contacts in the North have made similar claims, including Kang Myung-do, who was the son-in-law of the North Korean premier when he escaped to South Korea in 1994, but Lim says he learned of a camp for the regime’s elite when he served in the military with the son of an inmate.

“I used to drink a lot with Kim Sung-il [Kim Jong-il’s cousin] when we were in the military and we became firm friends during the seven years we served together,” Lim told This Week in Asia.

“He was easy to talk with and, as well as drinking, he liked to chase women. At times, he would go away for a week or more – which was unheard of in the unit – and come back with bottles of expensive spirits. I had my first taste of good whisky with Kim.”

How does North Korea’s Kim react to crisis in the South? He blows up the ‘Blue House’

After one such excursion and too much to drink, Lim claims, his colleague said he had been summoned to Pyongyang by his cousin, Kim Jong-il, the then-leader of North Korea who was keen to ensure his loyalty.

That loyalty was in question because Kim Jong-il had manipulated the incarceration of his cousin’s father, Kim Yong-ju, at the Hyanghari camp in 1975 after he lost a power struggle to assume the leadership of the nation after the death of the former supreme leader Kim Il-sung.

Kim Yong-ju is the younger brother of Kim Il-sung and was seen in the early 1970s as his likely successor after rising to the post of premier. That changed when Kim Il-sung began to groom his son for the role; Kim Yong-ju was initially replaced as premier and then imprisoned. “Kim Yong-ju was a tall man who had charisma when he talked,” Lim said. “In comparison, Kim Jong-il was short and fat. But, critically, Kim Jong-il had has father’s support and that meant there was only going to be one winner in the battle to be his successor.”

Lim said that his colleague’s mother had also been imprisoned, while his sister had been ordered to marry an academic and move to a remote part of the country.

Kim Il-sung partially rehabilitated his brother in 1993, the year before his death, and gave him a ceremonial role. Now 95, he has faded into the political background and was last seen in July 2015 voting in local elections.

An Englishman in North Korea ... Beijing-based Briton engages with hermit kingdom through tourism and filmmaking

Another detainee at the Hyanghari camp is believed to be Kim Song-ae, the second wife of Kim Il-sung and the mother of Kim Pyong-il, Kim Jong-il’s half-brother. Kim Pyong-il had a strong claim to the leadership of the nation after Kim Il-sung’s death, particularly as his mother was championing him.

Little is known of the fate of Kim Song-ae from the mid-1990s, although there were rumours that she was killed in a car accident in Beijing in 2001. Her son, Kim Pyong-il, has effectively been banished to serve as an ambassador abroad, representing the regime in Yugoslavia, Hungary, Bulgaria, Finland and Poland. Kim Pyong-il – who bears a closer resemblance to Kim Il-sung than Kim Jong-il did – is ambassador to the Czech Republic.

The once powerful aunt of Kim Jong-un, the present dictator, may also have been detained after her husband, Jang Song-taek, was denounced in December 2013 and executed for crimes against the state.

Kim Kyong-hui, 69, was rumoured to be in ill-health and has disappeared from the ranks of senior government officials attending state occasions.

“Purges have been ongoing in North Korea since the 1950s,” said Greg Scarlatiou, executive director of the Committee for Human Rights in North Korea.

“On Kim Jong-un’s watch, the purges have reached an intensity level not seen since Kim Il-sung, and possibly higher”, he told This Week in Asia. “For the first time ever, a member of the inner core of the Kim family regime, uncle Jang Song-taek, was executed. Previously, members of the inner core of the ruling family would be just exiled overseas or, in extreme cases, sent to ‘VIP detention facilities’, referred to by witnesses as ‘resorts’.”

Other detainees under Kim Jong-il were political and military leaders who displeased him. Kim Jong-un, his son, has shown a remarkable willingness to execute rather than imprison those who disagree with him.

“At its peak, I would say there would be no more than 1,000 inmates at the camp, which would include the families of the detainee,” Lim said. “The regime appears to be content to let them live and die there, while the inmates just accept a life of relative comfort.

“That’s why they call it the resort.”