If I understood it correctly, essentially you encode into a rendering problem, compute, and decode back. I'm not aware of this technique. If you have a link, would love to read about it.

The encode decode overhead is the main bottleneck, but it could work, specially considering today's mobile SoCs are pretty powerful. One thing to consider would be how uniform is the API from one device to the next.