Alibaba's AI Reorganization: The Life-or-Death Sprint from 'Model Is King' to 'Token Is King'

目录

“Guys, this time it’s for real. Group CEO Wu Yongming is personally taking the helm, all AI businesses are broken up and reorganized, and the ‘Alibaba Smart Connectivity Group’ (ASI) is formed. The internal slogan is just one thing — ‘Token is king.’ What does that mean? It means the old arms race of ‘flexing muscles and comparing parameters’ is over; now what’s being compared isn’t whose model is smarter, but whose model can actually be used — called at scale, sustained.”

As someone in tech, my first reaction to this news wasn’t “oh, another round of org restructuring,” but the strong, life-or-death urgency behind it. This is by no means a simple business merger but a strategic pivot from the underlying logic to the top-level design. Today let’s take apart what kind of game a giant is playing technically, product-wise, and commercially when it shouts “Token is king.”

Background: Why Has “Token” Become the New Strategic High Ground?

To understand “Token is king,” we first need to grasp the context of “Token” here. In the large-model field, a token isn’t a blockchain coin but the basic unit by which a model processes information. You can think of it as the “building blocks” (which may be characters, words, or subwords) that a model “eats” and “spits out.” Every user question and every model generation consumes tokens. Therefore, token consumption directly and in real time measures how deeply and broadly an AI model is being used — it is the ultimate gauge of an AI product’s market penetration and commercial value.

Over the past few years, the AI track went through a “model is king” phase. Everyone was keen to top leaderboards (GLUE, MMLU, etc.), compete on parameters (100 billion, 1 trillion), and race to be first with multimodality. This was important — it’s the technical infrastructure. But problems followed: you built the world’s most advanced engine (large model) only to find there aren’t enough cars (applications) to put it in, or the cars can’t run once loaded (high cost, poor experience). The model became an expensive “technical bonsai” rather than an inclusive “productivity tool.”

Alibaba’s reorganization this time consolidates AI forces scattered across cloud intelligence, Taotian, local services, and other lines (Tongyi Qianwen, Quark, DingTalk AI, etc.) all into ASI, with an extremely clear goal: break down internal barriers, concentrate firepower, and get the best model capabilities to the broadest users and scenarios with the least friction and the fastest speed, thereby driving exponential growth in token consumption. This marks China’s AI competition entering the second half: from the “lab race” of technical breakthroughs to the “market race” of application adoption and commercialization.

Technical Breakdown: The Systems-Architecture Revolution Behind “Token Is King”

The slogan is easy; implementing it is hard. The “Token is king” strategy poses unprecedented challenges to the technical architecture. It requires the entire AI tech stack — from low-level compute scheduling to the middle-layer model service to the top-level application ecosystem — to be rebuilt around the core goal of “delivering tokens efficiently, stably, and at low cost.”

1. The core challenge: the “impossible triangle” of cost, performance, and scale

Anyone who has built enterprise systems knows that simultaneously satisfying “high concurrency with low latency,” “high stability at low cost,” and “flexible adaptation to many scenarios” is nearly impossible. Large-model services especially so:

• Cost: the GPU compute and memory consumption of a single inference of a 100-billion-parameter model is astronomical.
• Performance: users expect second-level or even millisecond-level responses, but a large model may take several seconds to generate 1,000 tokens.
• Scale: supporting billions of interactions per day for hundreds of millions of users is an extreme test of system elasticity.

2. Architecture evolution: from “one giant monolith model” to a “model-as-a-service” mesh

The traditional approach is “one model rules all,” routing all traffic to one giant service. That won’t work in the “Token is king” era. ASI must build a more refined, more intelligent model service mesh.

graph TD
    A[User request / app call] --> B(Intelligent routing gateway)
    B --> C{Request analysis engine}
    C -->|Simple Q&A / summary| D[Lightweight model pool <br/> 6B/7B params]
    C -->|Complex reasoning / creation| E[Heavyweight model pool <br/> 72B/100B+ params]
    C -->|Code / math| F[Vertical fine-tuned model pool]
    C -->|Multimodal| G[Vision / speech model pool]
    D & E & F & G --> H[Unified output and cache layer]
    H --> I[Return to user]
    
    J[Real-time monitoring and scheduling center] --> B
    J --> D
    J --> E
    J --> F
    J --> G
    
    K[Cost accounting center] -- Token consumption / cost data --> J
    J -- scaling / model hot-load commands --> D&E&F&G

The core idea of this architecture is “allocate on demand, strike precisely”:

• Intelligent routing: the gateway dynamically picks the most suitable model based on the request content, user identity, and SLA requirements. A simple weather query should never use a 100-billion-parameter model; a lightweight model suffices, and the cost difference may be a hundredfold.
• Model tiering: build a full ladder from lightweight (6B/7B), standard (dozens of B), heavyweight (100B+), to vertical-domain fine-tuned models. Use a “small-model cluster” to handle 80% of daily traffic and a “large-model vanguard” to solve the 20% of high-difficulty tasks.
• Extreme optimization: at the model-service layer, deep integration is needed:
  • Inference optimization: heavy use of model quantization (INT8/INT4), dynamic batching, continuous batching, attention optimization (such as PagedAttention), desperately squeezing the throughput of a single GPU.
  • Caching strategy: build multi-level caches (GPU VRAM → high-speed memory → SSD), caching common questions (FAQ) and similar generated results, returning them directly to avoid recomputation.
  • Elastic scheduling: based on real-time token-consumption prediction and traffic fluctuation, achieve second-level elastic scaling of the compute pool, guaranteeing experience at peak and cutting cost to the bone during troughs.

3. Flywheel effect: the closed loop of data, scenarios, and models

Another deeper meaning of “Token is king” is: a token is not only a consumable but also fuel. Every real user interaction produces valuable feedback data (which answers were good, which were corrected, which were ignored). ASI needs to build a powerful data flywheel:

Massive token consumption -> high-quality interaction data -> model fine-tuning and reinforcement learning -> improved model experience -> more users -> more token consumption

Whether this loop runs efficiently is the key to the success or failure of the “Token strategy.” This requires the technical architecture to include real-time data pipelines, an efficient annotation and evaluation platform, and a safe model-iteration process.

My View / Cool-Headed Thinking: Hidden Worries and the Essence Beneath the Carnival

Alibaba’s move is absolutely correct in direction — one might even call it a somewhat tragic “no other choice.” But I have a few cool-headed thoughts:

1. “Token is king” may degenerate into “traffic is king” or “subsidy is king.”
To pull up token numbers quickly, what’s the most direct means? Possibly “freebies.” Through subsidies, bundling, and forced traffic diversion, products within one’s own ecosystem can be made to “pump out” pretty token numbers. But this produces a lot of meaningless, low-quality interaction that not only fails to nourish the model but pollutes the training data. The real “king” should be “effective tokens” and “paid tokens.” How to define and measure “effective” is a harder problem than raising the total.

2. The technical-architecture challenge far exceeds the org-architecture adjustment.
Breaking down organizational walls is only the first step of a long march. The model mesh, intelligent routing, and cost optimization mentioned above are each world-class technical problems. Especially co-location scheduling: how do you let latency-sensitive online inference tasks and cost-sensitive offline training tasks share one huge heterogeneous compute pool (including domestic chips)? This requires super-strong low-level resource management and scheduling capability. Alibaba Cloud has the accumulation, but facing a “compute glutton” like a large model, the pressure is still enormous.

3. The scarcity of the application ecosystem is the fundamental bottleneck.
In the end, tokens must be produced by applications. Currently, apart from chatting, writing, and coding assistance, “killer apps” that truly integrate into core business processes and create irreplaceable value are still scarce. ASI consolidating entry points like DingTalk and Quark is an advantage. But the common disease of big companies is “centralized” thinking — always wanting to control everything themselves. The super-apps of the AI era will very likely be born at the edges and at intersections. Whether ASI can truly, with an open-platform mindset, empower rather than control the vast number of small and mid-sized developers and enterprises will determine the height of its ceiling.

4. From an engineering standpoint, this is essentially a “war of efficiency at scale.”
AI has developed to the point where the “weapons” (foundation models) in everyone’s hands are gradually converging in capability (though gaps remain). The next competition, like the cloud-computing war back then, is: who can deliver a unit of intelligence (a token) to any user in the world at the lowest cost and highest reliability. This is an all-out engineering-efficiency war involving chips, networks, data centers, software stacks, and algorithm optimization. Alibaba’s move is to concentrate all resources into one corps for a decisive battle.

Implications for Product Builders: How to Survive and Grow in the “Token Era”

Whether you’re a product team inside a big company or an independent entrepreneur, the “Token is king” wave will profoundly affect your product logic.

1. Redefine the value of your AI feature: is it a “toy” or a “tool”?
Ask yourself: do users try your product’s AI feature once or twice for novelty, or does it get integrated into daily workflows and used frequently every day? Only the latter produces sustained, valuable tokens. When designing an AI feature, the starting point shouldn’t be “what cool AI can we do,” but “where does the user have lasting pain, and how can AI seamlessly solve it.” For example, instead of building a general copywriting generator, build, for e-commerce operators, a tool that generates a full set of “title-detail-promo copy” at the click of a button based on product data.

2. Embrace “model-as-a-service,” but have a backup plan.
For most product teams, building a large model in-house isn’t realistic. You should actively use APIs from platforms like ASI. But the key is: don’t deeply bind to a single model or a single vendor. In your architecture, you should abstract out a “model adapter layer.”

class AIModelClient:
    def __init__(self, config):
        self.providers = {
            'ali': AliProvider(config['ali']),
            'openai': OpenAIProvider(config['openai']),
            'local': LocalModelProvider(config['local']) # lightweight local model as a fallback
        }
        self.router = SmartRouter(config) # route by cost, performance, task type
    
    def generate(self, prompt, **kwargs):
        provider_key = self.router.route(prompt, kwargs)
        return self.providers[provider_key].generate(prompt, **kwargs)

This way you can flexibly weigh cost, results, and stability, even mix different models across scenarios, always holding the initiative.

3. Carefully design interactions to capture high-quality feedback data.
Your product is the battlefield closest to the user. Every interaction is golden data for optimizing the model. Design with “data awareness”:

• Provide minimalist feedback entry points like “like/dislike.”
• For critical outputs (such as generated contracts or code), design a “manually correct and submit” flow; these correction data are worth a fortune.
• While protecting user privacy, think about how to feed these anonymous, high-quality feedback data back to your model vendor or your own fine-tuning pipeline, forming a moat.

4. Cost awareness must run through the entire product life cycle.
“A token is a cost.” At the product-design stage, you should do “token economics”:

• On average, how many tokens does one call of this feature consume?
• What’s the expected usage frequency?
• Can our gross margin cover this cost?
  Avoid building a feature that “gets praised but no one can afford to use.” You can design a reasonable cost-transfer or coverage model through quotas, membership tiers, or bundling with core high-margin features.

Conclusion

Alibaba’s AI reorganization and its cry of “Token is king” are a landmark event marking the Chinese internet’s entry into deep waters. It announces the end of the era of pure technical showmanship and the start of a brutal endurance race in commercialization and engineering.

Behind this is, in fact, a return: technology must ultimately serve people, and value must ultimately be measured by the market. The token is the most plain-spoken ruler for measuring AI’s value in this era.

As a technologist who has lived through the entire cloud-computing cycle from concept to popularization, I feel like I’m watching history repeat. Back then, everyone competed on data-center scale and virtualization technology; now, everyone competes on the scale and efficiency of intelligent computing clusters. But the core hasn’t changed: turn a once-expensive, scarce technical resource (compute/intelligence) into infrastructure as reliable and cheap as water and electricity through extreme engineering innovation.

This road is destined to be long and full of challenges. But for each of us practitioners, this is undoubtedly the best of times. Because the battlefield is clear and the rules are set: let go of the obsession with parameter scale, go deep into every concrete scenario, and win every valuable token consumption with solid engineering and refined products. This “all-out decisive battle” has only just begun.

中英对照

文章作者：阿文

文章链接： https://www.awen.me/post/919f95b8.html

版权声明：本博客所有文章除特别声明外，均采用 CC BY-NC-SA 4.0 许可协议。转载请注明来自 阿文的博客！