Spring Data and Redis Keys

Spring + Redis as cache

Spring data , REDIS and Keys * command.

The KEYS command and SCAN command can search for all keys matching a certain pattern. The difference between the two is that SCAN iterates through the keyspace in batches with a cursor to prevent completely blocking the database for the length of the query.

With Redis, the KEYS command has been warned against being used in production as it can block the database while the query is being performed. SCAN was created to iterate through the keyspace in batches to minimize the effects.

Due to the Keys command it slows down the whole distributed application as all other Microservices are pending redis (single-threaded app) to finish the process this will cause random timeouts throughout the distributed application.

Redis recommends replacing the keys command with the scan command, however in the spring-data-redis source it seems that it is using the keys command.

byte[][] keys = Optional
                 .ofNullable(connection.keys(pattern))
                 .orElse(Collections.emptySet())
                 .toArray(new byte[0][]);

The next release will include a batch functionality (2.6.*) for a proper fix

For versions before batch, I decided that we could overwrite the RedisCacheWriter with another NoKeysNonLockingCacheWriter.

Mainly is to overwrite the clean redis to ensure keys * is not called. Instead We included Locking and Scan concept, sample as attached.

@Override 
     public void clean(String name, byte[] pattern) { 
         Assert.notNull(name, "Name must not be null!"); 
         Assert.notNull(pattern, "Pattern must not be null!"); 
         execute(name, connection -> { 
             boolean wasLocked = false; 
             try { 
                 if (isLockingCacheWriter()) { 
                     doLock(name, connection); 
                     wasLocked = true; 
                 } 
                 var scanOptions = ScanOptions.scanOptions() 
                         .match(new String(pattern, UTF_8)) 
                         .build(); 
                 Set<byte[]> keySet = new HashSet<>(); 
                 Cursor<byte[]> cursor = connection.scan(scanOptions); 
                 cursor.forEachRemaining(keySet::add); 
                 cursor.close(); 
                 var keys = keySet.toArray(new byte[0][]); 
                 if (keys.length > 0) { 
                     connection.del(keys); 
                 } 
             } catch (IOException e) { 
                 log.error("IOException occurs", e); 
                 e.printStackTrace(); 
             } finally { 
                 if (wasLocked && isLockingCacheWriter()) { 
                     doUnlock(name, connection); 
                 } 
             } 
             return "OK"; 
         }); 
     }

• Using Lettuce connector as we have a redis cluster.

Best regards.

The Effects of Redis SCAN on Performance and How KeyDB Improved It - DZone Database