Why Japan is lagging behind in generative AI and creation of LLMs

Countries are racing to develop their own generative AI algorithms, but high-tech Japan has fallen behind.

Generative AI has been the most popular topic in tech since open artificial intelligence Its chatbot ChatGPT caused a stir. Breakthroughs in generative artificial intelligence have the potential to boost global GDP by 7 percent, or nearly $7 trillion, over the next decade, according to forecasts. Goldman Sachs Research.

The key to the development of generative AI is the large language model, which underpins ChatGPT and Baidu’s Ernie Bot, capable of processing massive data sets to generate text and other content. But Noriyuki Kojima, co-founder of Kotoba Technology, a Japanese LL.M. start-up, said Japan currently lags behind the US, China and the EU in developing these algorithms.

Chinese organizations, including tech giants alibaba and Tencent, In the past three years, at least 79 LL.M. programs have been opened in China, Reuters The May report cited research by a coalition of state agencies. American corporate giants such as OpenAI, Microsoft, Google and Yuan Kojima said it played an important role in driving the progress of the LL.M. in the country.

Kojima added that deep learning requires a “strong community of software engineers” to develop the necessary infrastructure and applications. However, Japan will face a shortage of 789,000 software engineers by 2030, according to the Ministry of Economy, Trade and Industry.The country is currently ranked 28th^{the th} Statistically ranked among the top 63 countries in terms of technical knowledge IMD World Digital Competitiveness Ranking.

Japan also faces hardware challenges, as the LLM requires the use of artificial intelligence supercomputers (e.g. IBM Bella and Microsoft’s Azure hosting system. But no private company in Japan has a “world-class machine” with these capabilities, Nikkei Asia The report said.

Government-controlled supercomputers, such as Fuyue As a result, Kojima explained, “holds the key to studying for an LL.M. in Japan”.

“The use of supercomputers of this magnitude has formed the backbone of the LLM’s development, as it has traditionally been the most significant bottleneck in the process,” he said.

Tokyo Institute of Technology and Tohoku University plan to collaborate with supercomputer developers Fujitsu and Riken to use Fugaku to develop an LLM based primarily on Japanese data, Fujitsu announced in May.

Fujitsu added that the organizations plan to publish their research results in 2024 to help other Japanese researchers and engineers develop LL.M.

The Japanese government will also invest 6.8 billion yen ($48.2 million), or about half of the total cost, to build a new supercomputer in Hokkaido that will be operational as early as next year, Nikkei Asian Report. Nikkei Asia said the supercomputer will be used exclusively for LL.M. training to promote the development of generative artificial intelligence in Japan.

In April, Japanese Prime Minister Fumio Kishida stated that Japan supports the industrial application of generative artificial intelligence technology. Kishida made the remarks after a meeting with OpenAI CEO Sam Altman, who said the company was looking to open an office in Japan.

Big tech companies have also joined the race to boost Japan’s position in generative artificial intelligence. in June, Softbank’s mobile Arm reportedly says it plans to develop its own generative AI platform local media. This was underscored by SoftBank Chief Executive Masayoshi Son’s announcement that the investment firm plans to shift from “defensive mode” to “offensive mode” and sharpen its focus on artificial intelligence.

“We want to be the leader of the AI ​​revolution,” Son said at the annual shareholder meeting.

SoftBank Group sold its 85 percent stake in SB Energy to Toyota Tsusho in April and recently agreed to sell its 90 percent stake in U.S. investment manager Fortress Investment Group, Nikkei Asian Report.Cutting these other investments helps SoftBank free up cash that it can use through its vision fund venture capital unit.

Arm, the chip design company owned by SoftBank, also plans to list in the United States later this year. “It will be the largest IPO in the world so far,” said Amir Anvarzadeh, Japan equity strategist at Asymmetry Advisors.

The IPO will provide huge cash for SoftBank, which reported a record loss of 4.3 trillion yen in its Vision Fund for the fiscal year ended March 31.

Arm was initially seeking to raise $8 billion to $10 billion. But with demand for semiconductor chips “soaring,” Anvarzadeh said Arm could raise as much as $50 billion to $60 billion, or “85% of SoftBank’s market capitalization.”

SoftBank’s stock price could rise, he said, though that doesn’t guarantee the success of its artificial intelligence efforts.

“Fundamentally, I don’t think SoftBank is going to change the game in Japan … they are not Japan’s AI savior,” he said.

Japan Telecom NTT Also announced plans to develop its own LLM this fiscal year, with the aim of creating a “Lightweight and efficientNTT said it would spend 8 trillion yen in growth areas such as data centers and artificial intelligence over the next five years, a 50 percent increase from previous levels of investment.

Local media reported that the digital advertising company web proxy Obtained a Master of Laws degree in May Enables companies to create artificial intelligence chatbot tools. The company says it is one of the few “models dedicated to the study of Japanese language and culture.”

Although Japan has not yet caught up in the field of generative AI, it is taking the first steps through these private sector efforts. Once a “robust infrastructure” is in place, the remaining technical challenges could be “significantly alleviated” by using open-source software and data from previous pioneers, Kojima said. bloom, falcon and red pajamas Both are open-source LL.M.s, trained on vast amounts of data that can be downloaded and studied.

However, Kojima said companies dabbling in this space should expect competition on a “relatively long time horizon.” He explained that developing an LL.M. requires significant capital investment and a workforce highly skilled in natural language processing and high-performance computing.

“Softbank and NTT joining this competition will not change the artificial intelligence landscape in the short term.”

The increased participation of Japanese technology companies in the development of generative AI coincides with the positive stance on the adoption of AI in other fields. Over 60% of Japanese companies are positive about using generative AI in their operations, while 9.1% have already done so, An investigation by Imperial Database found that.

Hitachi have Established the Generative AI Center It said in May that in order to promote the safe and effective use of the technology by employees. Drawing on the expertise of data scientists, AI researchers and related experts, the center will develop guidelines to mitigate the risks of generative AI, the group said.

Chief Cabinet Secretary Hirokazu Matsuno said Japan would even consider government adoption of artificial intelligence technologies such as ChatGPT if cybersecurity and privacy concerns are resolved.

Hiroki Habuka, a research professor at the Kyoto University School of Law, said that as Japan becomes more open to the use of generative AI, the government should develop and promote soft guidelines on its use, while assessing the need for hard regulation based on specific risks.

“Without clearer guidance on what companies should do when using generative AI, practice could become fragmented,” the professor said.