The generation of 3D models through images and text may have been similar in the past, but now they can be restored in reality.
Recently, Magic Technology, in collaboration with Nanyang Technological University, Singapore A * Star Research Institute, Westlake University, University of Texas Austin and Zhejiang University, published a revolutionary research result – Cadcrafter. It is a system powered by AI which can directly convert ordinary photos into modifiable CAO 3D models and supports the importation of traditional frame engineering software for subsequent publishing and processing.
In the article “CAPCRAFTER: generate computer -assisted design models from undressed images”, the search team has shown how to automatically identify object structures from real photos or images rendering and generate set and modifiable CAD models. The document was officially accepted by CVPR 2025, the best conference on computer vision. The corresponding authors are Tianrun Chen, founder of Magic Technology and Fayao Liu, researcher at Singapore A *.
Context: photos of the CAD models have always been a “difficulty”
Although 3D -based 3D reconstruction technologies (such as the nerve and reconstruction of the mesh) have made great progress, most generate mesh models (mesh) or implicit volume data, which lack structure, configuration and edibility, and is difficult to meet the strict requirements of industrial design, machining or 3D printing.
CadcrafterUsability modelingAs a central objective, the result of the output is not an ordinary triangular mesh, but a B -REP style CAD modeling program – an orderly modeling instruction containing operations such as stretching, rotation and cutting. It can be directly imported into common CAD software (such as Fusion 360 and SolidWorks) for post-publishing.
Technical core: three-step modeling process + large-scale pre-training
In terms of technical architecture, Cadcrafter has introduced a certain number of cutting -edge AI modeling mechanisms to make a high precision conversion of the images into Available CAD models.
First, the system adopts a hidden space generation frame based on VAE and diffusion transformer. The team has mapped the CAD modeling instructions sequence in a structured hidden space by resulting in a variational autoencoder (VAE), and introduced a conditional diffusion model in this space. Using the diffusion transformer architecture (known as), the system can sample and generate a hidden space under image guidance, thus making a precise expression of the parameted design language.
Secondly, given the high complexity of the unique image modeling task, the team designed a distillation strategy that has migrated multi-visualities to one view. The researchers first formed the so-called model with multi-lives to fully grasp the mapping relationship between the image and the CAD space. Then, by introducing the dissemination of the distillation of the
Finally, in order to solve the problem of semantic legitimacy in the production of CAD instructions, Cadcrafter has a mechanism for verification of integrated code compilaility based on DPO (Direct optimization of preferences). During the generation process, the system determines in real time if the CAD instructions can be successfully analyzed by the CAD nucleus, such as the identification of structural errors such as uncluttered curves. At the same time, the model guided by DPO prefers to generate a compile and geometrically complete design code, which considerably improves the success rate of construction and the engineering of the final 3D model.
Experimental results: Multi-Scenario tests check its high-loyalty modeling capacities
To check the system performance, the search team has selected several standard CAD models, printed them in real objects using a Kokoni Sota 3D printer and obtained the image entry by taking photos. The experimental results show that Cadcrafter can generate CAD modeling instructions with a complete structure and clear geometric characteristics based on a single image, which is clearly better than the current mainstant image modeling methods in terms of detailed restoration.
In addition, the team also tested real objects in daily life. The results show that Cadcrafter can not only automatically identify the structure of objects, but also directly from engineering files that can be used for design and manufacturing. The generative model precisely restores the geometric characteristics of industrial products during the process of transformation and manufacturing in key details such as the surface flatness and the brightness of the edges.
Imagine that when part of your house is damaged, you no longer need to contact the manufacturer to wait for shipping. Simply take a photo and generate a printable replacement – this is exactly the future of intelligent manufacturing represented by Cadcrafter. Going further, when this technology is integrated and used by 3D printing companies, it can become the key to its “overtaking on the curve”.
