Custom Shaders in Flutter: Visual Art in Your App

A version in Spanish could be found here.

In Flutter, if you didn’t already know, there’s an interesting way to render objects (in addition to the classic widgets and more). Custom Shaders offer another way to render objects without exhaustive use of the classic widgets we use in our day-to-day.

To understand a bit, let’s look at the rendering process in Flutter at a general level. The ultimate goal is always to “paint” on the screen exactly what we tell it to (pixels on the screen). Each pixel is associated with a color/texture. This part is handled by the GPU with what are called Fragment Shaders. In short and at a very high level, these are parallel processes to be able to “color” the screen pixel by pixel. This process occurs from the simplest app in Flutter, like the basic Counter app, to the most complex UI that we instruct the framework to display.

Canvas class - dart:ui library - Dart API

For the shaders to perform their task, the Canvas API, an extremely abstract layer that, when translated for the shaders layer, allows for declarative graphics, such as shapes or custom paths that will be “filled” with color. You can find all the details in the official documentation under Canvas.

Building on the above, this summarized graphic illustrates how from a widget, you reach the Canvas API layer (the classic path in Flutter development) or from a custom painter, which in turn requires a CustomPainter widget.

And leveraging the power of the paint from the CustomPainter, we can instruct it with a shader indicating which color to use for each pixel. Therefore, if we tell the shader what to do, and it in turn conveys this to the paint, then the paint will execute it as well. And let’s remember, as mentioned in the previous paragraph, that the GPU can carry out this process for hundreds, thousands, or millions of pixels in parallel. This is because it’s the very purpose of its existence.

In this way, with custom shaders, we can paint anything from a background of a particular color to a mega fractal, passing through gradients, modifying textures of an image, and many more things.

Building Custom Shaders in Flutter

As the title of the article suggests, we will specifically instruct the CustomPainter to use custom shaders that we will pass to it. These shaders are not written in Dart; they are written in GLSL (Open GL Shading Language), and it’s not new; it has been in the world for years and is widely used in the fields of mathematics, 2D and 3D design, among others.

For the scope of this article, we won’t start from scratch. On the contrary, we will leverage many shaders written and published by others, and visualize them in a Flutter app.

Now, to keep in mind, the syntax of a shader that we can use in Flutter must follow these basic guidelines:

1. In a .frag` file, we must include the flutter/runtime_effects.glsl file, which contains an extensive set of helpers specifically for Flutter.

#include <flutter/runtime_effect.glsl>

2. We declare the input parameters that our shader will receive from our app. To do this, we use the word uniformfollowed by the data type and then the variable name. So, if we want to include a variable that indicates time, or a specific color to use, we use: uniform float iTimeor uniform vec4 uColor respectively. In GLSL, “float” is used for numeric data, and “vec4” corresponds to a vector with RGB plus alpha for colors. uniform indicates that the variable will have the same value for each pixel calculated by the process.

uniform vec2 uSize;
uniform float iTime;

out vec4 fragColor;

3. We declare the variable that the shader will return. It must always be a vec4 which represents an RGB color plus alpha, and conventionally, it is often named fragColor.

4. The shader must have a main() method, and its sole task is to assign a value to the variable that we will return.

out vec4 fragColor;

void main() {
    // do something to assign fragColor a value
    // ...
    fragColor = vec4(finalColor, 1.0);
}

After having the shader file, for Flutter to recognize it, it must be defined under the shaders: section in the pubspec.yaml file, with the defined path of its location. In this case, we have it at:

shaders:
 - shaders/art_fractal.frag

This particular shader, we have taken from ShaderToy, a community and tool used for creating and sharing shaders. We will rely on shaders already created (although we can create our own or modify an existing one) and adjust them, so they can be visualized from our app.

How do we visualize these shaders in our app?

For our scenario, we have a stateful widget in our app called GeneralShaderPage, which receives the path of the shader to display. This shader, for the example mentioned with the fractal, takes as input parameters the screen size (a vec2 with width and height), a time variable (either second or millisecond), and returns the vec4 with the applied pixel color.

We initialize the shader with _loadShader(), which essentially means loading it as an asset and passing it to GeneralPainter, which is our CustomPainter responsible for painting each pixel in the way defined in the .frag file.

The most important aspect of our CustomPainter is that in the paint method, we set the variables that the shader receives (size and time). Then, to the Canvas that will be painted, we pass this object with those configurations:

void paint(Canvas canvas, Size size) {
 final paint = Paint();
 shader.setFloat(0, size.width);
 shader.setFloat(1, size.height);
 shader.setFloat(2, time);
 paint.shader = shader;
 canvas.drawRect(Rect.fromLTWH(0, 0, size.width, size.height), paint);
}

Let’s keep in mind that the input variable size is a vec2 with the width and height:

shader.setFloat(0, size.width);
shader.setFloat(1, size.height);

And the “time” variable (at any given moment, the pixel will have a particular color) is a float.

From the main page of the app, we call our widget GeneralShaderPage, setting its required arguments, and the final result is this beauty:

The rest is just playing around with different shaders that we could reference from ShaderToy, or implementing our own. And if, for example, a given shader requires an image as an input argument, to which we want to apply some effect directly like a blur or pixelate it, we must then do so by sending it as an image to the CustomPainter, setting it in the shader before placing it in the Canvas, and receiving it in the .frag of the shader:

void paint(Canvas canvas, Size size) {
 for (int i = 0; i < images.length; i++) {
   shader.setImageSampler(i, images[i]!);
 }
 shader.setFloat(0, size.width);
 shader.setFloat(1, size.height);
 for (int i = 0; i < uniforms.length; i++) {
   shader.setFloat(i + 2, uniforms[i]);
 }
 final paint = Paint();
 paint.shader = shader;
 canvas.drawRect(Offset.zero & size, paint);
}

And the result for a water drop-type blur looks something like this:

For more details on this implementation, you can find it in this repository, along with the shaders from ShaderToy that I used and the final outcome.

GitHub - cdmunoz/basic_shaders: A small project to play a bit with shaders in Flutter

To wrap up, You can star it, modify it, or add new interesting shaders, for example, ones that receive the X, Y position on the screen from a tap or mouse click to capture movement and enable rotation. There are many different and cool things to try.

References:

Writing and using fragment shaders

Playing with paths in Flutter

Creating Custom Shaders