The resumption of international talks on North Korea next week provides yet another opportunity to roll back Pyongyang’s nuclear programme. It would be a mistake, however, for negotiators in the six party talks – involving the two Koreas, the US, Japan, Russia and China – to assume that Kim Jong-il, North Korea’s leader, could be enticed into giving up his nuclear ambitions with economic aid and security reassurance alone. He must be confronted with a new strategic reality that forces him to choose between regime survival and nuclear weapons.

Washington needs a drastic rethink of its strategy. If the US sticks to the more-of-the-same approach, the pending talks will be futile.

So far, the US and China have engaged in tactical, but not strategic, co-operation in confronting Pyongyang. The reason is simple. While China shares America’s tactical interest in constraining Mr Kim from provocative behaviour, Beijing and Washington could not co-operate as genuine partners due to conflicting long-term strategic interests on the Korean peninsula.

The prevailing wisdom about China’s reluctance to put pressure on Pyongyang is Beijing’s fear that cutting off food and energy aid could cause North Korea’s collapse and an ensuing flood of refugees into China.

While obviously disruptive, such an influx would be manageable for a country with 100m migrants within its borders. What really haunts China’s leadership is the spectre of a US military presence beyond the 38th parallel after Pyongyang’s collapse. China’s predominant concern is a security buffer, now provided by North Korea, against forward American deployment in east Asia.

However, Washington is unwilling to rule out the option of extending its deployment north of the 38th parallel in the event of Korean reunification. Uncertain about China’s role in east Asian security, the US wants to maintain maximum flexibility in responding to the geopolitical consequences of China’s rise.

This underlying strategic reality has allowed Mr Kim to play China against the US for more than a decade. Emboldened by his calculation that China cannot afford to lose North Korea as a strategic buffer, Mr Kim has repeatedly engaged in brinksmanship tactics. He figures that even though China has coercive tools, any threat to use such options is a bluff because China would never sacrifice its own fundamental interests to appease the Americans.

Unfortunately, Mr Kim’s realpolitik thinking has been borne out. Since the ill-fated six party talks began in August 2003, China and the US, hobbled by their conflicting interests, have neither put enough carrots on the table nor wielded big enough sticks. Even the United Nations’ sanctions imposed on Pyongyang in October after its nuclear test lacked teeth.

Therefore, China and America must jointly alter Mr Kim’s underlying strategic calculations if they want to improve the odds of a breakthrough at the pending round of talks. Positive incentives may be needed but they alone cannot determine the outcome.

What will make Mr Kim think – and behave – differently this time would be to minimise the strategic value of his regime. To do this, Washington and Beijing need to reach a new understanding on the future of the Korean peninsula and settle on an endgame strategy in North Korea to force a shift in Mr Kim’s strategic thinking.

This would require the immediate commencement of a high-level dialogue between the US and China to resolve their conflicting strategic interests on the Korean peninsula.

First, China, the US and South Korea need to discuss the broad outlines for Korean reunification. America and China should both commit to a zone of neutrality on the peninsula, which necessarily requires a pledge from Washington not to move its military deployment beyond the 38th parallel in a reunified Korea.

Second, China, the US and South Korea must consider specific measures aimed at stabilising a post-collapse North Korea, including humanitarian assistance, security and economic reconstruction.

With the US and China co-operating on the endgame in North Korea, Mr Kim will see that the strategic landscape on the peninsula is shifting. Holding on to his nuclear programme will invite far more punitive sanctions and accelerate the demise of his rule. The only alternative, if he wants to remain in power, is denuclearisation.

Minxin Pei and Oriana Skylar Mastro are, respectively, senior associate and junior fellow, at the Carnegie Endowment for International Peace in Washington