To output the signals the VFD needs to scan the display, I already decided to re-use the bit of software that allowed the Raspberry Pi to send out a block of memory to the GPIO pins over and over again. It was pretty simple to whip up some code to fill that block of memory to display something: select a column, shift in the column data, latch, select the next column etc; do that quickly enough and you will have an image on the display.
I, however, also wanted to display grayscales. On the LED-board, I did that by basically PWMming each frame: for example, my LED-board had 4-bit grayscales for each pixel. To do that on hardware that can only completely turn a pixel on or off, instead of sending 1 frame every 1/60th second, I would send 16 frames that are displayed 1/16th of the time a normal frame would take. (These shorter frames are called subframes; when you use subframes, a normal frame is basically defined as the time it takes to display all the subframes.) If I wanted to light up a pixel with, say, 3/16th of its intensity, it would be on in 3 of those subframes and off for 13 of them. Because the subframes are shown after each other so quickly, to the eye it would look like the pixel is dimmed to 3/16th of its full brightness (We'll disregard the fact that the eye doesn't see the intensity of a pixel linearily for this explanation.) Basically, I was PWMming each individual pixel on the board.
Unfortunately, here that wouldn't work so well. The display has a fairly large amount of pixels, and even with an overclocked Raspberry Pi 2, I couldn't shift more than 350 frames/second out of the DMA engine. If we want the frame rate to be around 50Hz, I could have 7 subframes, which is not even enough for a measly 3BPP (3 bits per pixel - a maximum variation of just 8 grayscales) of grayscale depth.
Instead, I decided to use a different method. The previous one is derived from the well-known PWM way of dimming LEDs. There is another way of setting the brightness of a LED by using solely digital signals, and that is Binary Code Modulation, also known as Bit-Angle Modulation. While it has some drawbacks, it makes it possible to send out N bits of grayscale data using N slightly-modified subframes.
How does it work? The trick is mostly that if we can modify the duration every subframe is shown to be different, we can use the binary of the bits in the grayscale level for each pixel directly as the on/off value of that pixel in each subframe. Clear? No? Then let me explain it a bit further.
Say, we have a display that works in 4BPP, that is, every pixel can have any of 16 values, namely from 0000 (0, entirely dark) up and including to 1111 (15, entirely lit). Say, we want to use this to set the brightness of an LED. The LED will be lit from 0 to 100%, linearily going up with the value of the pixel. How much does every bit in the binary pixel value add to the intensity of the LED? We can see that by making only that bit active:
Bit Val Pct 0 0001 6.6% 1 0010 13.3% 2 0100 26.6% 3 1000 53%
Obviously, you can add these values, so for example the value 0011 turns on the led for (0001 + 0010 -> 6.6% + 13.3% = )20%.
So, what happens when we change the timing of our subframes to these values? We make sure subframe 0 lasts 6.6% of the total time of the frame, subframe 1 13.3% of the total frame time, etcetera. Now getting the intensity we want is very easy: we take the intensity value (0-15), convert it to binary (0000-1111) and for each bit, if it is 1 we turn on the pixel in the corresponding subframe. Like magic, our LEDs (or whatever else we use to display) will have the required intensity!
The good thing about this method, as stated before, is that for n BPP of grayscale data, I only need n subframes instead of the 2^n I would need for PWM. The bad thing is that I need a way to modify the timing of the subframes. I could just send out my pixels slower for the frames that require a longer duration, letting them light up for a longer time that way, but that would decrease the total frame rate: I needed the Raspberry Pi to spit out pixels at its maximum capacity in order to get a flicker-free display.
There is another option: I could change the amount of time a subframe is actively sending out light. After all, the only thing that counts is the amount of time the subframe lights up in relationship to the total all frames light up. I could do this by using the blanking input: both the row and column driver chips have this, and it basically disables the outputs of the drivers. What I could do is send out the subframes as fast as I could, but only allow them to light up for such a time that they add the correct amount to the total light output, blanking them for the rest of the time.
Unfortunately, this dims the screen a fair amount because of all the required blanking. The more bitplanes, the more loss - 4 bit more than halves the light output, 5 bit reduces it to less than 40%. It is possible to counter this effect by picking a solution somewhere in-between lengthening the subframes and blanking subframes for a time. For example, for a 5BPP solution done by sending out two frames at 100% for bit 5, one at 100% for bit 4,one at 50% for bit 3 etc costs just one extra frame and has an efficiency of 64%, way better than the 38% a solution with just 5 subframes would have given. Because there's a subframe extra, it decreases the 'real' framerate a bit, in my case (with the Raspberry Pi being able to spit out 350 subframes a second) from 70 to 58Hz, which is still acceptable, so this is the solution I chose.
There are a few more tricks I applied to get the display as smooth as possible. First of all, the above text isn't entirely correct: because of the way the VFD works, I don't send out an entire subframe and then let it stay on the screen for a certain amount of time. The VFDs data shift register can only contain 2 lines at a time which also have to be displayed separately, so the subframe timing happens per line instead of per frame. This does, however, allow me to change the phase of the subframes per line: if line x shows subframe 1, line x+1 can show subframe 2 etc. This cuts down on flicker because instead of the entire display flickering at the same time, the flickering is somewhat evenly distributed over time and space.