Adobe AI Can Now Reproduce A Famous Artwork In Less Than A Minute

A recent finding on Arxiv.org did reveal that the research team from University of Maryland and Adobe Research found a new machine learning system called the LPaintB which is able to reproduce artwork with the style of the greatest artists in the world like Leonardo da Vinci or Vincent van Gogh in a blink. 

The scientists then aim to build up a “self-supervised learning” which uses both unlabeled data and (a small portion of) labeled data to improve its accuracy level in order to train the agent with a limited source of references. On the other hand, the research team mathematically modeled the action states of the system, like paintbrush orientation and length as well as brush size, and replaced the goal state of failures with its final state. The result was a paired corpus showing positive rewards that they gave to the AI model, used to learn to paint reference pictures in the artistic style they wanted.  

The project didn't run smoothly at first. The team finds that the positive rewards came out from only a small part of sample actions, which they tackled by applying a reinforcement learning technique using the goal state as paired data for training a policy. However, the resulted policy was not particularly robust as the paired data used for training only included only actions with positive rewards (making it hard to recover from unwanted actions which returned negative rewards) as well as states resulting from successive series of actions. Again, the team had to use reinforcement learning to solve this problem: The technique added noise to the action which helped generalize the model, it also optimized the actions of the model with rewards.

The project then results in an AI system that is able to accomplish the painting process with parametric size, color, and information. The artwork is updated consequently with a reward function evaluating the differences between the current state and the goal state

In storing up a set of training data, the scientist team created random patches from a varying-scaled image source of a specific style and sampled it into a fixed size. Testing the algorithm with an hour of training in a 16-core processor and 1080 graphics chip PC, the model was capable of drawing an image in size of 1000 x 800px with 20,000 strokes in a minute. 

The model’s generalization is trained dependently with the training data and the method works on a basic painting condition. The efficiency and performance of the policy would then be greatly enhanced, thanks to the interpretation of the self-supervised and reinforcement learning system. This finding is leverage for more future investigations in improving stroke parameters and building up a painting simulator with a learning framework of model-based reinforcement.