As previously reported, I’m on a slow track research project into randomness. Having explored the concept of random bits and built some experimental random electronics, I’m still progressing in my question to understand how randomness can be exploited both for understanding the fundamental process of creation and creativity as well as randomness as a shortcut for complex or even intractable algorithms.

I’ve learned a trick over the years: if you don’t understand what you’re doing, keep doing it. On that dirk-gently-esque premise I’ve designed and built a 4-bit analog random number generator. Below is the circuit diagram for one bit; it’s essentially the one as I used in my tests before. I have replaced the schmitt inverter by a transistor with a pull-up resistor.

Since I wanted a ‘few’ random bits, I designed a PCB that I got manufactured in China at JLCPCB. There service is excellent and the PCB quality just as good. Cheap too: 5 boards cost me 2 dollars (!) plus 16 USD shipping. It took a week from order to reception which is pretty good given that borders have been disrupted due to COVID19.

I haven’t designed circuit boards for a while and I was pleased to note that my track-laying skills are old school. I used to make complex boards with black tape rather than CAD software and autorouters were only someones dream at that time.

My soldering skills need adjusting though. In a time of SMD circuit boards, neither my eyes nor my solder iron tip are up to scratch. Still, I put it all together and after removing some solder shorts, got the circuits to work as per spec.

Lastly, I spent an unreasonable amount of time on designing a 3D printed case. Dirk Gently would agree that the design of the case is at least as important as the electronics inside. Not least so that if the ‘thing’ doesn’t work, then at least you have something to put on a shelf and look at.

## Does it work

The short answer is: yes. The 4-bit random generator produces random one and zeros. I hooked it up to a Teensy 3.5 and used it it factor numbers, using the same approach as described in the previous posts on the topic. Below are two of the four random bit streams (sorry, I don’t own a 4-channel oscilloscope)

It works; now what?

As said, I have no idea what I’m doing with this and was hoping that building the electronics would help me figure it out.

## Randomness

It turns out that one important question is: so what is randomness anyway? I found some interesting answers. Firstly, no one really knows but there are some good pointers at what it might mean. These are:

- the statistical distribution of ones and zeros. Random bits are like flipping coins, you should expect roughly half heads and half tails.

But this is not enough. A series of 1010101010… has a fair distribution but it is not random. Equally, the number pi, 3.141592653589793238462643383279… has a fair distribution of digits but it is not random. There is an interesting way of defining such randomness.

- A string of ones and zeros is random if and only if it is shorter than any computer program that can produce that string. This is known as Kolmogorov randomness or algorithmic randomness.

Since pi’s digits can be generated by a computer program, it is definitely not random. These programmes can be very short. Below is one I wrote using the Monte Carlo method. If you generate random points on a surface and count all the points that are less than distance 1 from the origin, which is a circle of radius 1, then you will effectively calculate the surface of a unit circle, which is … pi.

This isn’t very efficient: After 100 million loops, we only have the first 5 digits of pi. But it proves that pi is not algorithmically random.

There are many more definitions and measures of randomness and in looking at these, I didn’t get the impression that the last word has been said on this. Not to be undone by any of this, I shall add my own definition to this branch of mathematics. I will call it “fractal randomness”.

- A fractal structure is random if none of its branches, at any scale can be algorithmically generated.

Below is an example of a random tree fractal that would match that definition. In plain english: there is no discernable pattern.

## Analysing the random bits generator

Having done this superficial research and postulating a definition for random fractals, I now understand why I needed to build this random bits generator. My interest in randomness is about its function in creation processes in a random space. For this I only need two things: a fair distribution and reasonable unpredictability.

The random bits generator is indeed truly unpredictable since the underlying process is based on random electron avalanches, as discussed in my previous post on this. However, the distribution is strongly biased towards one outcome: ones. There are clever ways to fix this but there is another problem. My 4-bit generator only produces about 100k samples per second. The Teensy 3. 5 processor I was using needs to artificially slow down its sampling rate to avoid catching the same random bit twice or more. My MacBook on the other hand can generate random numbers and perform the same factoring calculations at least 100 times faster.

Given my stated needs, the 4-bit random generator is useless. But it is not pointless. Here is what I have achieved.

- I have refreshed my circuit design and building skills which had been untapped for more than 20 years.
- I’ve been dying to try out professional PCB services and I am now confident that I can produce commercial grade hardware again ( = potential income source).
- I have learned a lot about randomness and what I need from it.
- I designed a beautiful case that I will display in my personal museum of “things I once made” and it has given me ideas for new 3D printing projects.
- I don’t need a hardware random bits generator.

I’m a happy dude.

## Postscript on hardware vs software random numbers

Ever since graduating as an electronics engineer, it had been drummed into me that software can only produce pseudo random numbers. Whenever people use these, there is a whole crowd of (mostly unknowledgeable) purists that will immediately shout: fake! these are not random!

My little exercise in developing and using a true random number generator taught me something about these pseudo or fake software number generators: for 99.99% of applications they are absolutely fine and probably better! The upsides of these software based pseudo number generators – which incidentally have improved enormously since I graduated in 1984 – far outway the technical challenges of true hardware based random number generators. At least for me; and I know, because actually tested it.

Dirk Gently, Adam’s finest ever creation, yet known to far, far, far too few people.

LikeLike

Absolutely. I live by his wisdom

LikeLiked by 1 person

Exactly what transistors are you using in this circuit? Will it work with any general putpose NPN device such as 2n3704?

LikeLike

The 2n3704 has a emitter-base breakdown voltage of approximately 5V. The circuit would work and you could reduce the 12V supply to 6V or anything in between, as long as it is more than the EB breakdown. What I can’t tell is whether this 2n3704 transistor EB breakdown is noisy enough. I found that the higher breakdown voltage of the X1360 produced a higher noise amplitude than transistors with a lower breakdown voltage. My guess is that it will work but yu can only find out by trying.

LikeLike

I made something similar a while back and solved the bias issue with a low pass filter. Basically you take the analogue random voltage, low pass filter it at say 0.25Hz, and use that as one input of a comparator and the random voltage as the other. It will then self correct to always produce and equal number of 1s and 0s.

LikeLike

Yes, that would work… but it would favor clusters of ones and zeros. There have been other suggestions but they don’t lead to a fair distribution. When I measure the number of 0000, 0001, 0010, 0011 all the way to 1111 I should find that in the long run they appear all at a frequency of 100/16 %.. So far none of the suggestions would create such a flat distribution.

If I was only interested in a single but then such anti bias solutions would definitely work but I need very long sequences of ones and zeros with true equal distribution and so far I have only found pseudo random number generators that can guarantee that. But.. they are not truly random, though for my purpose good enough.

LikeLike

I have built it and tested it, there is no bias. Remember that the low pass filter is 0.25Hz, i.e. the average over 4 seconds or many hundreds of thousands of bits. The results pass Dieharder, NIST and ENT. Unfortunately I seem to have lost the results, I need to get it up on Github.

I did some other experiments too. The fastest true hardware RNG I found was using the temperature sensor of an XMEGA MCU. Take the LSB from that and you get a pretty decent source of entropy. I whitened it by running blocks of 8 bits through the CRC32 generator on the chip. Using that I was able to send around 8mb/sec over the USB interface (custom stack, the Atmel one is too slow) and again it passed Dieharder, NIST and ENT. The goal was to be fast but it turned out to be an excellent source of random bits anyway.

https://github.com/kuro68k/xrng/blob/master/finalAnalysisReport.txt

Looking at the curve in the datasheet I guess the temperature sensor is some kind of junction so the LSB probably is quantum noise. Whatever, it works very well.

LikeLike

Impressive result. It confirms my conclusion that analog RNG of the type that I built may be suitable for music synths but for serious random numbers, an approach like yours is far more reliable.

LikeLike

Pingback: Fish tank A 4-bit Random Number Generator – happyaquariums.com

Have to admit that it’s the best looking TRNG I’ve seen out there 🙂

Don’t worry about the bit rate. People only need a ‘little’ true randomness for cryptographic reasons. The rest can usually be handled by pseudo-random number generators like the one inside your Mac. If you want to pursue this, there are a few silly ideas @ http://www.reallyreallyrandom.com that deal with testing your output Kolmogorov entropy rate.

(Have a look at the Null Gamma random walk).

LikeLiked by 1 person