We needed to cleanup (delete) some archived keys from Redis. A lot of them.
Prepare All The Keys To Delete
The first step is to create a file with all of the keys to be deleted. We exported this from our database, and it's just a text file with one key per line, it looks like this:
The result file, which I will call all_redis_keys.txt contains 92,402,859 keys (3.1 GB).
Trying to execute one Redis request per key would be far too slow. Luckily DEL allows multiple keys to be passed to in and returns the number of keys deleted.
Before we unleash this on Redis we want to make sure that it's going to work. Fortunately we can substitute the DEL command with EXISTS command which returns the number of keys that exist (ie. the number of keys that would have been deleted).
Let's run through the first 100,000 keys in batch sizes of 1000 and output the number to another file:
The tee command will output as well as writing to count.txt. If this is too noisy for you then you could just write directly to the file with > count.txt.
The count.txt (or stdout) file looks like this:
We can add up all of these numbers:
This returns that 589,742 keys would have been deleted.
The Real Run
Now count the number of keys that were deleted:
Over 90 million keys deleted in less than 2 hours (14,526 per second).