Alibaba’s new AI model will have reasoning skills equivalent to OpenAI’s GPT-o1
Alibaba on Thursday released a new artificial intelligence (AI) model that could rival OpenAI’s GPT-o1 series models in reasoning power. The QwQ-32B large language model (LLM), launched in preview, is said to outperform GPT-o1-preview in several mathematical and logical reasoning-related benchmarks. The new AI model is available for download on Hugging Face, but is not fully open source. Recently, another Chinese AI company released an open-source AI model DeepSeek-R1, which it claimed could rival the basic reasoning-oriented models of ChatGPT maker.
Alibaba QwQ-32B AI Model
In one blog postAlibaba detailed its new reasoning-focused LLM and highlighted its capabilities and limitations. The QwQ-32B is currently available in preview. As the name suggests, it is built on 32 billion parameters and has a context window of 32,000 tokens. The model went through both pre- and post-training phases.
As for the architecture, the Chinese tech giant revealed that the AI model is based on transformer technology. For positional encoding, QwQ-32B uses Rotary Position Embeddings (RoPE), along with Switched Gated Linear Unit (SwiGLU) and Root Mean Square Normalization (RMSNorm) functions, as well as Attention Query-Key-Value Bias (Attention QKV) bias.
Like the OpenAI GPT-o1, the AI model shows its internal monologue when assessing a user question and trying to find the right answer. This internal thought process allows QwQ-32B to test different theories and fact-check itself before presenting the final answer. Alibaba claims that in internal testing, the LLM scored 90.6 percent in the MATH-500 benchmark and 50 percent in the AI Mathematical Evaluation (AIME) benchmark, outperforming OpenAI’s reasoning-focused models.
Notably, AI models with better reasoning are not evidence that models are becoming more intelligent or capable. It’s simply a new approach, also called test-time compute, that allows models to spend additional processing time completing a task. As a result, the AI can provide more accurate answers and solve more complex questions. Several industry veterans have pointed out that newer LLMs are not improving at the same pace as their older versions, suggesting that existing architectures are reaching a saturation point.
Because QwQ-32B spends extra processing time on queries, it also has several limitations. Alibaba stated that the AI model can sometimes mix or switch between languages, giving rise to issues such as language mixing and code switching. It also tends to enter reasoning loops and apart from math and reasoning skills, there are other areas that still need improvements.
It is striking that Alibaba has made the AI model available via a Hugging Face mention and both individuals and businesses can download it for personal, academic, and commercial purposes under the Apache 2.0 license. However, the company has not made the model weights and data available, meaning users cannot replicate the model or understand how the architecture functions.
