Apple published a study, in which he describes a unique process of how he learned large language model (LLM) to create high-quality user interfaces in SwiftUI – and that with virtually no human intervention. The result is a new model UICoder, which was able to improve itself thanks to automatic feedback and iterative training.
From lack of data to your own dataset
The problem that Apple solved, it was clear – in the normal training data it is UI code extremely rare, often making up less than 1% of the total. Apple started with an open-source model StarChat-Beta specialized in programming and gave him the task: to generate a user interface from the descriptions large synthetic dataset SwiftUI code.
Every output passed through the compiler Swiftto verify that it works, and then analyzed by a multimodal model GPT-4V, which compared the visual appearance of the application with the original specification. Inconsistent or duplicate examples were removed and the rest were used to fine-tune the model.
Iterative improvement and a million examples
The whole process was repeated in several rounds, with each improved model generating even better codeAfter five iterations, the result was 996 000 SwiftUI programs, which were used for the final training of UICoder. The one in the tests surpassed the original model and in many ways it came close to the quality GPT-4 – even surpassed it in compilation success.
Gallery
Interestingly, the original StarChat-Beta almost none SwiftNever saw the UI code. Due to an error in creating the TheStack dataset, Swift repositories accidentally omitted and in the OpenAssistant-Guanaco instruction set there was only one single example Swiftalready ten thousand answers. This means that the success of UICoder nestnot on memorized samples, but on completely newly created and carefully filtered data.
Possibility of wider use
Apple believes that this method – code generation, automatic code review, and iterative improvement – could also work for other programming languages and UI frameworks.Study UICoder: Finetuning Large Language Models to Generate User Interface Code through Automated Feedback is already available on arXiv and shows that AI can effectively teach itself if it has the right feedback.
This approach could mean revolution in application development – from faster prototyping to fully automated generation of custom user interfaces.
Dilemma .. if I publish my code on github, or copilot will supply me with codes from github. Who is the author? :-)