Paper page - WorldDiT: A Unified Diffusion Architecture for World and Action Modeling

Paper page - WorldDiT: A Unified Diffusion Architecture for World and Action Modeling Tweet: <a href=\"https://x.com/bidhan/status/2082098687464046684\" rel=\"nofollow\">https://x.com/bidhan/status/2082098687464046684</a>\n","updatedAt":"2026-07-28T15:13:18.590Z","author":{"_id":"5f1158120c833276f61f1a84","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg","fullname":"Niels Rogge","name":"nielsr","type":"user","isPro":false,"isHf":true,"isHfAdmin":false,"isMod":false,"followerCount":1273,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.5650634169578552},"editors":["nielsr"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg"],"reactions":[],"isReport":false}},{"id":"6a697e3218ce35de24724eaf","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":378,"isUserFollowing":false},"createdAt":"2026-07-29T04:14:42.000Z","type":"comment","data":{"edited":false,"hidden":false,"latest":{"raw":"This is an automated message from the [Librarian Bot](https://huggingface.co/librarian-bots). I found the following papers similar to this paper. \n\nThe following papers were recommended by the Semantic Scholar API \n\n* [FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation](https://huggingface.co/papers/2607.08575) (2026)\n* [Learning 4D Geometric Priors for Inference-Efficient World Action Models](https://huggingface.co/papers/2607.05468) (2026)\n* [CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts](https://huggingface.co/papers/2607.03693) (2026)\n* [Geometric Action Model for Robot Policy Learning](https://huggingface.co/papers/2606.17046) (2026)\n* [DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation](https://huggingface.co/papers/2607.02604) (2026)\n* [Light-WAM: Efficient World Action Models with State-Fusion Action Decoding](https://huggingface.co/papers/2606.08242) (2026)\n* [World Pilot: Steering Vision-Language-Action Models with World-Action Priors](https://huggingface.co/papers/2606.12403) (2026)\n\n\n Please give a thumbs up to this comment if you found it helpful!\n\n If you want recommendations for any Paper on Hugging Face checkout [this](https://huggingface.co/spaces/librarian-bots/recommend_similar_papers) Space\n\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: `@librarian-bot recommend`","html":"This is an automated message from the <a href=\"https://huggingface.co/librarian-bots\">Librarian Bot</a>. I found the following papers similar to this paper. \nThe following papers were recommended by the Semantic Scholar API \n<ul>\n<li><a href=\"https://huggingface.co/papers/2607.08575\">FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.05468\">Learning 4D Geometric Priors for Inference-Efficient World Action Models</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.03693\">CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.17046\">Geometric Action Model for Robot Policy Learning</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2607.02604\">DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.08242\">Light-WAM: Efficient World Action Models with State-Fusion Action Decoding</a> (2026)</li>\n<li><a href=\"https://huggingface.co/papers/2606.12403\">World Pilot: Steering Vision-Language-Action Models with World-Action Priors</a> (2026)</li>\n</ul>\n Please give a thumbs up to this comment if you found it helpful!\n If you want recommendations for any Paper on Hugging Face checkout <a href=\"https://huggingface.co/spaces/librarian-bots/recommend_similar_papers\">this</a> Space\n You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: <code>@librarian-bot recommend</code>\n","updatedAt":"2026-07-29T04:14:42.492Z","author":{"_id":"63d3e0e8ff1384ce6c5dd17d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg","fullname":"Librarian Bot (Bot)","name":"librarian-bot","type":"user","isPro":false,"isHf":false,"isHfAdmin":false,"isMod":false,"followerCount":378,"isUserFollowing":false}},"numEdits":0,"identifiedLanguage":{"language":"en","probability":0.6633225679397583},"editors":["librarian-bot"],"editorAvatarUrls":["https://cdn-avatars.huggingface.co/v1/production/uploads/1674830754237-63d3e0e8ff1384ce6c5dd17d.jpeg"],"reactions":[],"isReport":false}}],"primaryEmailConfirmed":false,"paper":{"id":"2607.23909","authors":[{"_id":"6a68bc0c5dad3683ba934909","name":"Sen Wang","hidden":false},{"_id":"6a68bc0c5dad3683ba93490a","name":"R. Gnana Praveen","hidden":false},{"_id":"6a68bc0c5dad3683ba93490b","name":"Bidhan Roy","hidden":false},{"_id":"6a68bc0c5dad3683ba93490c","name":"Marcos Villagra","hidden":false}],"publishedAt":"2026-07-27T00:00:00.000Z","submittedOnDailyAt":"2026-07-28T00:00:00.000Z","title":"WorldDiT: A Unified Diffusion Architecture for World and Action Modeling","submittedOnDailyBy":{"_id":"5f1158120c833276f61f1a84","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/1608042047613-5f1158120c833276f61f1a84.jpeg","isPro":false,"fullname":"Niels Rogge","user":"nielsr","type":"user","name":"nielsr"},"summary":"Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies.","upvotes":7,"discussionId":"6a68bc0c5dad3683ba93490d","ai_summary":"WorldDiT is a compact diffusion transformer that jointly generates robot actions and predicts future visual patches, achieving strong performance without large pretrained vision-language backbones.","ai_keywords":["diffusion transformer","visual world modeling","continuous action chunks","normalized RGB patch targets","LIBERO simulation suites","Pareto frontier","sub-billion-parameter baseline"],"ai_summary_model":"thinkingmachines/Inkling-Small","organization":{"_id":"69404a2d504d98ba21c52be1","name":"bageldotcom","fullname":"Bagel Labs","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6862a65dd478e0d9d375be70/5cu7w3liwUf0OgcntR_rp.png"}},"canReadDatabase":false,"canManagePapers":false,"canSubmit":false,"hasHfLevelAccess":false,"upvoted":false,"upvoters":[{"_id":"6270324ebecab9e2dcf245de","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6270324ebecab9e2dcf245de/cMbtWSasyNlYc9hvsEEzt.jpeg","isPro":false,"fullname":"Kye Gomez","user":"kye","type":"user"},{"_id":"631e14ac473a6825f285e89d","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/631e14ac473a6825f285e89d/K-6QnoeGLg8XFvbTMMdqA.jpeg","isPro":false,"fullname":"Yury Panikov","user":"panikov","type":"user"},{"_id":"6862a65dd478e0d9d375be70","avatarUrl":"https://cdn-avatars.huggingface.co/v1/production/uploads/6862a65dd478e0d9d375be70/7t9NDd4psqeuZpvEPoazl.png","isPro":false,"fullname":"Bagel Labs","user":"bagellabs","type":"user"},{"_id":"69404343e3ae71a07d05d6ea","avatarUrl":"/avatars/8b48da2192644a768d3a60cba65c15c3.svg","isPro":false,"fullname":"Marcos Villagra","user":"marcos-bagel","type":"user"},{"_id":"698f8d51dcd4fa2ef0314a4a","avatarUrl":"/avatars/4d5bb4400b023230d4284b2eb5a528b2.svg","isPro":false,"fullname":"Ytdsgqy8sfg2i","user":"ytdsgqy8sfg2i","type":"user"},{"_id":"6a319713abd9178f9b7c25f5","avatarUrl":"/avatars/dd28a15eeb9ddb8276943c0e79df40f7.svg","isPro":false,"fullname":"Milo Banks","user":"milobanks","type":"user"},{"_id":"64834b399b352597e41816ac","avatarUrl":"/avatars/63d9d123bffa90f43186a0bdc4455cbd.svg","isPro":false,"fullname":"Shaobai Jiang","user":"shaobaij","type":"user"}],"acceptLanguages":["en","es","pt","fr","de","ja","zh","ko","ar","hi","ru","*"],"dailyPaperRank":0,"organization":{"_id":"69404a2d504d98ba21c52be1","name":"bageldotcom","fullname":"Bagel Labs","avatar":"https://cdn-avatars.huggingface.co/v1/production/uploads/6862a65dd478e0d9d375be70/5cu7w3liwUf0OgcntR_rp.png"},"markdownContentUrl":"https://huggingface.co/buckets/huggingchat/papers-content/resolve/2607/2607.23909.md","query":{}}"> Papers arxiv:2607.23909 Copy markdown WorldDiT: A Unified Diffusion Architecture for World and Action Modeling Published on Jul 27 · Submitted by Niels Rogge on Jul 28 · Bagel Labs Upvote 7 Authors: Sen Wang , R. Gnana Praveen , Bidhan Roy , Marcos Villagra Abstract WorldDiT is a compact diffusion transformer that jointly generates robot actions and predicts future visual patches, achieving strong performance without large pretrained vision-language backbones. Generated by thinkingmachines/Inkling-Small Many recent robot policies pursue stronger control by using large pretrained vision-language models (VLMs) as the action backbone. We introduce WorldDiT, a unified diffusion transformer architecture that couples action generation with visual world modeling and achieves strong performance without a large pretrained VLM action backbone. During training, a single diffusion transformer generates continuous action chunks and predicts normalized RGB patch targets from future camera frames. Across four LIBERO simulation suites, WorldDiT lies on the reported Pareto frontier for total model parameters and mean success among methods reporting all four suites. These results provide a strong sub-billion-parameter baseline for future scaling studies. View arXiv page View PDF Add to collection Community nielsr Paper submitter 16 days ago Tweet: https://x.com/bidhan/status/2082098687464046684 Reply librarian-bot 16 days ago This is an automated message from the Librarian Bot. I found the following papers similar to this paper. The following papers were recommended by the Semantic Scholar API FabriVLA: A Lightweight Vision-Language-Action Model for Precise Multi-Task Manipulation (2026) Learning 4D Geometric Priors for Inference-Efficient World Action Models (2026) CoRE-VLA: Towards Scalable and Robust Vision-Language-Action Modeling via Conditional Routing of Experts (2026) Geometric Action Model for Robot Policy Learning (2026) DynaWM: A Base-VLA-Guided World Foundation Model for Moving-Object Manipulation (2026) Light-WAM: Efficient World Action Models with State-Fusion Action Decoding (2026) World Pilot: Steering Vision-Language-Action Models with World-Action Priors (2026) Please give a thumbs up to this comment if you found it helpful! If you want recommendations for any Paper on Hugging Face checkout this Space You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend Reply EditPreview Upload images, audio, and videos by dragging in the text input, pasting, or clicking here. Tap or paste here to upload images Comment · Sign up or log …