Last week, ministerial negotiations on the Trans-Pacific Partnership (TPP) between Japan and the United States ended abruptly after the two sides failed to reach an agreement on key sticking points, including the removal of tariffs on sensitive Japanese farm products. The failure of the talks disappointed both sides, including Prime Minister Shinzo Abe, who has long upheld TPP as a fundamental component of his structural reform agenda.

Few, however, were surprised. Japan after all, has always had trouble cracking open its farm sector thanks to opposition from its powerful farm lobby. While it is tempting to assume that this is yet another case of Japanese leaders succumbing to the demands of vested interests, it is important to note that more is going on behind the scenes than meets the eye. Japan’s farm lobby is still a potent force in Japanese politics, but its influence is decreasing, and in ways that should bode well for agricultural liberalization.

Until recently, Japanese agricultural politics were dominated by a web of interconnected institutions. At the center of that web was the partnership between the ruling Liberal Democratic Party (LDP) and Japan Agricultural Cooperatives (JA). The latter delivered votes and campaign workers to conservative politicians in return for a protected agricultural market. JA also nurtured a close relationship with the Ministry of Agriculture, Forestry and Fisheries (MAFF), functioning as the ministry’s semi-official arm in the implementation of farm-related policies, including the infamous rice acreage reduction (gentan) program. All the while, JA exercised a near monopoly over the provision of agricultural inputs to farmers and even controlled their access to financial services through its powerful banking and insurance arms. Although by no means omnipotent, this agricultural regime was notoriously unresponsive to demands for policy reform.

But Japan’s agricultural regime is now well past its prime. JA’s economic dominance is slipping amid mounting competition from private-sector providers of farm inputs and credit. Its capacity to gather the vote has shrunk in the wake of electoral reform and a declining farm population. Rural politicians, for their own part, are facing strong electoral incentives to diversify their base of support beyond agricultural interests and to champion market-oriented reform. Meanwhile, the MAFF’s jurisdiction has shrunk as a result of the post-Uruguay Round dismantling of the postwar rice pricing system, and its ties to JA are weakening now that it no longer fields its retired bureaucrats to run as the association’s official candidates in upper house elections. Even ordinary consumers are contributing to the regime’s decline; whereas a concern for food safety and security once incentivized consumers to support market protectionism and high prices, consumers in today’s sluggish economy have grown critical of the gross inefficiencies of Japanese farming and of the policies and cozy political relationships that perpetuate them.

Perhaps most importantly, Japanese farming itself is changing. As we observed for ourselves during recent fieldwork in the Japanese countryside, more and more full-time farmers are developing new forms of farm ownership and management. Despite lingering barriers to innovation, the rates of farm corporatization and farmland consolidation are slowly increasing. Some farmers are pursuing these changes outside of JA networks, while others are partnering with innovative local coops; in both instances, these farmers are responding more directly to market signals and in ways that benefit consumers.

Clearly mindful of the agricultural regime’s waning power, the Abe government is taking steps to accelerate market-oriented trends among local farmers and coops. It has loosened regulations governing the establishment of new coops, farmland consolidation, the corporatization of family farms, and the entry of private-sector firms into farming, and has devoted two of the country’s “national strategic special zones” to agriculture. More dramatically, in late 2013 it announced that gentan would be phased out within five years.

This is not to suggest that the tug-of-war between Abe and the agricultural regime is over. Far from it, as the fate of a recent reform initiative illustrates. In May 2014, the government’s Council on Regulatory Reform (CRR) issued a report that among other things recommended the withdrawal of coop status for Zennō, the JA organization that oversees the provision of non-financial services to farmers, and the virtual abolition of Zenchū, JA’s  “control tower.”  Anti-reformist LDP politicians led by Hiroshi Moriyama pushed back, however, and the government softened its stance, postponing JA reform to this fall.  The LDP later appointed Moriyama—a vocal critic of TPP, as chairman of its working group on TPP.

Chalk one up for the forces of resistance?

Not so fast. The Abe government may have lost ground in the battle over JA reform, but it has made significant progress in its war against the agricultural regime. As agricultural economist Kazuhito Yamashita has observed, for example, the subject of JA reform is no longer taboo. Moriyama, moreover, is not the die-hard anti-reformer that some observers think he is. As the Asahi Shimbun reported on August 13, Moriyama has been deeply mindful of Abe’s popularity among voters and loath to take a position on reform that might alienate his party at the polls.  Moreover, he has some experience with overdoing opposition to a sitting prime minister since he was banned from the LDP in 2005 after opposing then Prime Minister Koizumi’s postal reform initiative. Perhaps unwilling to risk another stint in the political wilderness, Moriyama helped craft a response to the CRR report that while critical,acknowledged the need for JA reform. Moriyama may very well strike a similar chord as head of the TPP working group.

Finally, it is important to note that for every LDP politician who opposes agricultural reform, there is at least one who supports it, including such prominent members of the Abe cabinet as Chief Cabinet Secretary Yoshihide Suga, Minister of Agriculture Koya Nishikawa, and Shigeru Ishiba, the minister in charge of those national strategic special zones.

Japanese agricultural politics has turned a corner; today, the focus of debate is no longer whether to reform agriculture but rather when and how to do it. What then, is Abe likely to do in the months ahead? Do expect him to keep pushing his TPP agenda; as he stated so unequivocally on numerous occasions during his recent visit to the United States, Abe firmly believes that the pact is key to the long-term health and prosperity of the Japanese economy.

But do not expect him to take a page from Koizumi’s playbook and stake his government’s future on the success of agricultural reform and/or TPP. He has far more on his reform plate than Koizumi ever did and is not about to risk it all for these two issues, important though they may be to his personal legacy.

Instead, Abe will continue his careful tug-of-war with those “forces of resistance,” pulling a little here, conceding a little there, so that more and more farmers and local coops can free themselves from JA’s stifling embrace and are eased, not thrown, into freer markets.

U.S. trade officials may be tempted to throw in the towel after the collapse of talks last week, but it is important to remember that the potential for change in Japanese agricultural politics is now greater than ever. The U.S. can help strengthen Abe’s hand as domestic battles play out by stepping up pressure on Japan to reach an agreement on TPP—an agreement that grants Abe the time and flexibility he will need to open domestic agricultural markets without inciting a debilitating backlash from his opponents.

Patricia L. Maclachlan is an associate professor of government and Asian studies and the Mitsubishi Heavy Industry Professor of Japanese Studies at the University of Texas at Austin.

Kay Shimizu is an assistant professor in the department of political science at Columbia University.

This piece first appeared in CFR's blog Asia Unbound here. 

Image: Creative Commons 3.0.