Hacker News
new
|
past
|
comments
|
ask
|
show
|
jobs
|
submit
login
riku_iki
9 months ago
|
parent
|
context
|
favorite
| on:
Kimi K2 Thinking, a SOTA open-source trillion-para...
its moe, each expert tower can be branched from some smaller model.
jychang
8 months ago
[–]
That's not how MoE works, you need to train the FFN directly or else the FFN gate would have no clue how to activate the expert.
Consider applying for YC's Fall 2026 batch!
Applications
are open till July 27.
Guidelines
|
FAQ
|
Lists
|
API
|
Security
|
Legal
|
Apply to YC
|
Contact
Search:
