For all the attention paid to America’s complex relationship with China, stability and economic growth in Asia depends just as much, if not more, on the fraught relationship between China and Japan. Not only are these countries the world’s second- and third- largest economies, they also are major military powers that represent two very different political and social systems. Although increasingly interdependent economically, they oscillate between political engagement and outright competition over influence in Asia, territorial issues and military capabilities. The inability of these two countries to find a balance in their ties shapes Asian politics and economics and also complicates Washington’s efforts to find a stable Asian policy. Japan’s new government—its sixth in five years—has indicated that it now wants to improve relations with Beijing.

The past year, in particular, has rocked Sino-Japanese ties. In September 2010, the Japanese Coast Guard seized a Chinese fishing vessel and arrested its crew after a confrontation in the disputed Senkaku/Diaoyu Islands, just north of Taiwan. Great stores of natural gas and oil are contained on the seabed around these islets, and they have been the source of an ongoing dispute over which country owns them, despite being administered by Tokyo since the 1972 reversion of Okinawa to Japan from the United States. The 2010 crisis led to a freeze in relations, retaliatory arrests of Japanese citizens by Chinese authorities and a near-complete cutoff of crucial rare-earth materials to Japan from the mainland.

While such confrontations are rare, Sino-Japanese political relations have been more negative than positive in recent years. Tokyo has watched warily as Beijing has dramatically increased its military capability as well as its presence in East Asia. Of particular concern is the growth in numbers of Chinese submarines, fighter jets, and medium-range ballistic missiles. Well-publicized transits of Chinese naval ships between Japanese islands in the East China Sea have raised Japanese fears about a possible threat to its trade routes. Beijing, for its part, fears the U.S.-Japan alliance could seek to to hinder China’s ability to move freely through international waterways. Further sparring centers on issues including consumer product safety, Chinese pollution harming Japan and continued jockeying for influence in regional and global multilateral institutions.

Japanese leaders, in particular, see this as a zero-sum game in which they lose relative to China’s seemingly unstoppable growth. Tokyo’s attempts to negotiate free-trade agreements, create tri- or quadrilateral groupings and make bold diplomatic initiatives are often interpreted as playing catch-up to Beijing’s smile diplomacy. Yet in recent years, Beijing has alienated many of its Asian neighbors, primarily over its more assertive stance on territorial disputes and claims over the South China Sea. As China’s military capability has grown, it seems more willing to bully its neighbors, reversing years of diplomatic success. While this may provide fodder for Japanese worries about Beijing’s ultimate intentions in the region, it has not resulted in Japan’s successfully improving its own standing in Asia.

In truth, both countries are 800-pound gorillas in comparison to other Asian nations. The two are the most advanced military powers in Asia, with Japan holding a diminishing qualitative edge while China’s quantitative lead steadily increases. With a combined GDP of nearly US$8 trillion, they dwarf the economies of all other regional states. Japan’s high-tech industrial sector is the world’s most advanced, and China’s production of consumer goods has transformed global markets. Each provides millions of dollars in direct or indirect aid to countries around the globe, competing for access to raw materials and markets. Just as importantly, however, China is Japan’s largest trade partner, and Japanese firms employ nearly 10 million Chinese on the mainland. As important as each is individually, they are increasingly interdependent yet remain locked in persistent competition and distrust related to each other’s intentions and the Chinese assertion that Japan has yet to fully atone for its atrocities during World War II.

Just a few years ago, many observers expected this dynamic to change. Once the Democratic Party of Japan gained power in August 2009 after a half-century of Liberal Democratic Party rule, its leaders sought to reduce tensions with Beijing. One of the party’s founders, the now-disgraced Ichiro Ozawa, while in the opposition had led delegations numbering in the hundreds to China. After taking office, then prime minister Yukio Hatoyama went so far as to propose a new East Asia community in which Japan and China, along with South Korea, would take the lead. Yet, within a year these overtures had come to naught and relations had sunk to a new low. Things only got worse this month with the selection of Yoshihiko Noda as Japan’s new prime minister, given prior controversial statements that denied the war-criminal status of some of Japan’s World War II leaders.

Now Japan’s new defense minister Yasuo Ichikawa, who was appointed by Noda, has called for improving relations with China. While careful to frame his suggestion within the context of the U.S.-Japan alliance, Ichikawa’s call may spur debate over whether Japan is in danger of accommodating China in order to get a positive response. If the Noda government relinquishes its claim to the joint exploration of the seabed around the Senkakus, hesitates to modernize Japan’s military and cuts back on missile defense activities, a conservative backlash could emerge, heightened by perceived regional impressions that Japan is weakening relative to China.

By any accounting, it makes sense for both sides to improve relations even though neither is really willing to compromise on key issues. Needlessly antagonizing each other is a lose-lose situation, yet few in Tokyo have much confidence that they will be able to narrow the gap with Beijing. While acknowledging the depth of their economic interdependence, many Japanese are becoming convinced that China’s interests threaten Japan’s long-term security. One reason to attempt to find some common ground, however, is the looming leadership change in China scheduled for 2012. By attempting to create a more positive environment now, Noda and Japanese foreign minister Koichiro Gemba may hope to have a tailwind when starting to work with Xi Jinping. Xi visited Japan in December 2009, yet his visit was marred by controversy that a rushed meeting with Emperor Hirohito showed Japan’s over-eagerness to play up to China’s leader-in-waiting.

For now, there has been no substantial follow-on from Ichikawa’s suggestion. Like his American counterparts, he hopes to increase military-to-military ties, in part through officer exchanges. However, given the bad feelings of the past year, there is little reason to think that Tokyo will rush into any actions that seem subservient to Beijing. Rather, Ichikawa has hit a moderate note, stressing that Japan’s alliance with the United States gives it the confidence to seek the right balance with China. The hard part, of course, will be maintaining that positive tone and actually accomplishing something.

The two sides might do well to discuss trade and the other economic issues that tie them together rather than try to tackle security concerns first. Continuing to talk about a China-Japan free-trade agreement may help spur some debate in Tokyo about economic reform and promote understanding in both countries of how much each depends on the other. Finding common ground on economic issues may be the way to increase confidence in the working relationship and pave the way for tackling far more sensitive subjects. It will be much harder to erase decades of mistrust in the military sphere, given China’s continuing arms buildup and Japanese concern about the increased presence—if not assertiveness—of the PLA Navy. Absent another crisis like that over the Senkakus, the two will likely muddle through without significantly improving relations. Given that seemingly irreducible chasm, the smart money will be on a strained, yet controlled, Sino-Japanese relationship for the foreseeable future.

Michael Auslin is a resident scholar in Asian studies and the director of Japan Studies at the American Enterprise Institute.