This AI model can perform end-to-end scientific research in AI and ML
Sakana AI, a Tokyo-based artificial intelligence (AI) company, on Tuesday unveiled its latest AI model that can conduct scientific research. The company claims that the AI model, called The AI Scientist, can autonomously perform an end-to-end scientific discovery process in the field of AI and machine learning (ML). This includes generating ideas, conducting experiments, writing code, collecting and analyzing results, and writing a scientific paper based on them. Sakana AI has also published a pre-peer-reviewed paper describing the new AI model.
Sakana AI introduces the AI scientist
The company announced the AI model in a after on X (formerly known as Twitter), saying: “From generating ideas, writing code, running experiments and summarizing results, to authoring entire papers and conducting peer review, The AI Scientist opens a new era of AI-powered scientific research and accelerated discovery.”
In a separate blog postThe company said that The AI Scientist can use a broad research direction and a starting codebase, such as the open-source codebase of older research on GitHub, and start its own discovery process. It also claimed that the AI model can follow the procedure of AI researchers and perform literature research, experiment planning, figure generation, manuscript review and more. It is also said that it can run in an open-ended loop to improve the next generation of ideas based on the previous generation.
“Combined with the most capable LLMs, The AI Scientist is able to produce papers that are rated ‘Weak Accept’ by our automated reviewer at a leading machine learning conference,” Sakana AI claimed.
The company didn’t share any technical details about the architecture or methodology for developing the LLM. Since the AI model isn’t currently available to the public, Gadgets 360 wasn’t able to verify any of Sakana AI’s claims. However, based on the description, it appears that the AI model can only explore existing software-based ideas and may be limited by hardware limitations.
Because The AI Scientist needs a codebase as a starting point, it cannot be said to be capable of real innovation, which is often necessary to make a scientific discovery. The efficiency and capabilities of the AI model can only be judged after it is launched.
The AI company did highlight some limitations of the current generation of The AI Scientist. The AI model lacks computer vision capabilities, which limits its ability to troubleshoot visual problems with the article. It is also prone to hallucinations and may implement ideas incorrectly or make unfair comparisons to baselines, potentially contaminating the results.
Sakana AI also highlighted that the AI model “makes critical errors when writing and evaluating results.” One specific area where the model struggles is comparing the magnitude of two numbers.
