AI's eyes

How The Grid was teaching its Artificial Intelligence to see and paint.

Note: I wrote this post back in the day I worked as an AI Engineer at The Grid. It was a crazy ride, with lots of interesting people and challenges. I left The Grid years ago but some ideas are still relevant to augment design using AI. The content of this post were originally published on The Grid itself and since there's no mirrors around, I'm sharing it here and I hope you enjoy it.

  • Face detection: for emotion recognition and supporting detection of salient regions
  • Delaunay triangulation: to generate artistic backgrounds and picture styling
  • Image classification: to extract information/semantics from pictures
  • Sentiment analysis: to extract the general mood of a picture
  • Color extraction: to extract/quantize colors from pictures

Saliency extraction

When we look into a picture we can naturally (and really fast!) determine what points get our attention first. A human face, a specific object, a region with higher contrast, those are both examples of salient areas. We can easily recognise those areas, but computers don't. So we have to teach them to.

Original image
Salient regions extracted from the image. Higher gray levels indicate more salient areas.

Generative images and image processing

Besides analytics, we can use extracted information from text, images or videos to synthesise new content. We can generate an infinite number of unique images using combinations of different shapes, curves and colours.

Generative images by Delaunay Triangulation
Image styling using Delaunay Triangulation
Filters created on imgflo being applied to the same image. Original picture by Sharon Mollerus.
A Flowhub graph to image processing on imgflo.

What's next?

It's more than 6 years since I wrote this post. Deep Learning happened, Computer Vision was completely changed by it and other ML methods. However, the fundamental problems on design are still there and we can leverage a lot from ML/CV to augment and automate graphical and web design.

Coder, applied physicist and artist. I create algorithms that create. CTO at Extend AI. Half of 2aliens.