Be able to set custom index names

[berickus]

Hi there

It would be awesome when it would be possible to set your own index names like it is possible with the key space. So that you can limit permissions by ACL for users on indexes and keys.

[berickus]

I had a look at the code. It would be perfect if there would be an index pattern bean which could be supplied into the RediSearchIndexer and if not null, instead of the indexName from the annotation or just the class name, it would take the supplied pattern, like it is possible with the key space.

[SylvainAssemat]

Hi @berickus

Does this help ?

[berickus]

Hi @SylvainAssemat

Not really.

We want to use ACL with limitations on keys and indexes. Our desired setup is: :::
The tenant is only known after an OAuth2 login. So it has to be dynamic. So we can lock the tenant with index and key.

I can manually do it with jedis which I already did, but redis-om-spring offers nice QueryDSL support for the repository and ofter nice features. So I was hoping to get this working.

I am already experimenting on my own fork with such a setup.

[bsbodden]

@berickus propose a plan and we figure it out together. @SylvainAssemat want to throw in your ideas too?

[berickus]

@bsbodden, to explain what I was thinking. This is what I did with jedis.

@Data @AllArgsConstructor public class RedisIndexDefinition { private Class<?> entityClass; private RedisIndexService.IndexSchema schema; }

`
@DaTa
@ConfigurationProperties(prefix = "app.redis")
public class RedisConfigProperties {

private String env;
private String defaultModelRevision;

public RedisConfigProperties() {
    this.defaultModelRevision = "1.0";
}

}
`

`
@service
@requiredargsconstructor
public class RedisKeyService {

private final RedisConfigProperties config;

public String generatePrefix(Class<?> clazz, String tenant) {
    return RedisUtils.getKeyspace(clazz, config, tenant);
}

public String generateIndexName(String entityName, String tenant) {
    return String.join("_", "index", tenant, entityName);
}

}
`

`
@log4j2
@component
@AllArgsConstructor
public class RedisIndexService {

private final UnifiedJedis jedis;

public void createIndex(String indexName, String keyPrefix, IndexSchema schema) {
    try {
        createIndex(indexName, keyPrefix, schema.toSchemaArray());
        LOGGER.info("Successfully created index: {}", indexName);
    } catch (Exception e) {
        if (isIndexAlreadyExistsError(e)) {
            LOGGER.info("Index already exists: {}", indexName);
        } else {
            LOGGER.error("Error creating index: {}", indexName, e);
            throw new RuntimeException("Failed to create index: " + indexName, e);
        }
    }
}

private void createIndex(String indexName, String keyPrefix, SchemaField[] schema) {
    jedis.ftCreate(indexName, FTCreateParams.createParams()
                                            .on(IndexDataType.JSON)
                                            .addPrefix(keyPrefix),
                   schema
    );
}

private boolean isIndexAlreadyExistsError(Throwable throwable) {
    if (throwable != null) {
        String message = throwable.getMessage();
        if (message != null && message.toLowerCase().contains("index already exists")) {
            return true;
        }
        return isIndexAlreadyExistsError(throwable.getCause());
    }
    return false;
}

// Helper class for building index schemas
public static class IndexSchema {
    private final java.util.List<SchemaField> fields = new java.util.ArrayList<>();

    public IndexSchema tag(String jsonPath, String alias, boolean sortable) {
        TagField field = TagField.of(jsonPath).as(alias).separator('|');
        if (sortable) field.sortable();
        fields.add(field);
        return this;
    }

    public IndexSchema numeric(String jsonPath, String alias, boolean sortable) {
        NumericField field = NumericField.of(jsonPath).as(alias);
        if (sortable) field.sortable();
        fields.add(field);
        return this;
    }

    public IndexSchema text(String jsonPath, String alias, boolean sortable) {
        TextField field = TextField.of(jsonPath).as(alias);
        if (sortable) field.sortable();
        fields.add(field);
        return this;
    }

    public SchemaField[] toSchemaArray() {
        return fields.toArray(new SchemaField[0]);
    }
}

}
`

`
@component
@requiredargsconstructor
@log4j2
public class RedisIndexInitializer {

private final RedisIndexService indexService;
private final RedisKeyService keyService;
private final List<RedisIndexDefinition> indexDefinitions;
private final TenantService tenantService;

@EventListener(ApplicationReadyEvent.class)
public void initializeIndexes() {
    Set<String> tenants = tenantService.getTenants();
    LOGGER.info("Initializing Redis indexes for tenants: {}", tenants);

    for (String tenant : tenants) {
        LOGGER.info("Initializing Redis indexes for tenant: {}", tenant);
        initializeIndexesForTenant(tenant);
    }
}

public void initializeIndexesForTenant(String tenant) {
    for (RedisIndexDefinition indexDef : indexDefinitions) {
        try {
            String indexName = keyService.generateIndexName(indexDef.getEntityClass().getSimpleName(), tenant);
            String keyPrefix = keyService.generatePrefix(indexDef.getEntityClass(), tenant);

            indexService.createIndex(indexName, keyPrefix, indexDef.getSchema());

            LOGGER.info("Successfully initialized index '{}' for tenant '{}'", indexName, tenant);
        } catch (Exception e) {
            LOGGER.error("Failed to initialize index for entity '{}' and tenant '{}': {}",
                         indexDef.getEntityClass().getSimpleName(), tenant, e.getMessage(), e);
        }
    }
}

}
`

`
@configuration
public class RedisIndexConfiguration {

@Bean
public List<RedisIndexDefinition> redisIndexDefinitions() {
    return List.of(
            new RedisIndexDefinition(
                    DeviceStateRedis.class,
                    new RedisIndexService.IndexSchema()
                            .tag("$.posUuid", "posUuid", false)
                            .tag("$.locationUuid", "locationUuid", false)
                            .tag("$.origin", "origin", false)
                            .tag("$.operationalState", "operationalState", false)
                            .numeric("$.timestamp", "timestamp", true)
                            .numeric("$.javaMemoryUsagePercentage", "javaMemoryUsagePercentage", true)
                            .numeric("$.nativeMemoryUsagePercentage", "nativeMemoryUsagePercentage", true)
                            .numeric("$.diskUsagePercentage", "diskUsagePercentage", true)
                            .numeric("$.cpuUsagePercentage", "cpuUsagePercentage", true)
            ),
            new RedisIndexDefinition(
                    DeviceStateHistoryRedis.class,
                    new RedisIndexService.IndexSchema()
                            .tag("$.posUuid", "posUuid", false)
                            .tag("$.locationUuid", "locationUuid", false)
                            .tag("$.origin", "origin", false)
                            .tag("$.operationalState", "operationalState", false)
                            .numeric("$.timestamp", "timestamp", true)
                            .numeric("$.javaMemoryUsagePercentage", "javaMemoryUsagePercentage", true)
                            .numeric("$.nativeMemoryUsagePercentage", "nativeMemoryUsagePercentage", true)
                            .numeric("$.diskUsagePercentage", "diskUsagePercentage", true)
                            .numeric("$.cpuUsagePercentage", "cpuUsagePercentage", true)
            )
    );
}

}
`

`
@configuration
@EnableRedisDocumentRepositories(basePackages = "my.package.impl.persistence")
public class RedisConfig {

@Bean
@Primary
public CacheManager getCacheManager(@Qualifier("redisOMCacheManager") CacheManager cacheManager) {
    return cacheManager;
}

@Bean
public UnifiedJedis jedis(RedisProperties redisProperties) {
    HostAndPort hostAndPort = new HostAndPort(redisProperties.getHost(), redisProperties.getPort());
    DefaultJedisClientConfig.Builder jedisClientConfig = DefaultJedisClientConfig.builder().resp3();
    if (StringUtils.isNotEmpty(redisProperties.getPassword())) {
        jedisClientConfig.credentials(new DefaultRedisCredentials(redisProperties.getUsername(), redisProperties.getPassword()));
    }
    return new UnifiedJedis(hostAndPort, jedisClientConfig.build());
}

}
`

`
public class RedisUtils {

public static String getKeyspace(Class<?> clazz, RedisConfigProperties config, Tenant tenant) {
    return getKeyspace(clazz, config, tenant.getName());
}

public static String getKeyspace(Class<?> clazz, RedisConfigProperties config, String tenant) {
    String modelRevision = getModelRevision(clazz, config);
    return String.join(":",
                       config.getEnv(),
                      'microservice-a',
                       tenant,
                       clazz.getSimpleName(),
                       modelRevision
    );
}

private static String getModelRevision(Class<?> clazz, RedisConfigProperties config) {
    try {
        var method = clazz.getDeclaredMethod("getModelRevision");
        if (java.lang.reflect.Modifier.isStatic(method.getModifiers())) {
            return (String) method.invoke(null);
        }
    } catch (Exception e) {
        // Method doesn't exist, use default
    }
    return config.getDefaultModelRevision();
}

}
`

The code is functionality over patterns ;)

The idea is to create per tenant on startup an index which is limited by the key and therefore helps me to isolate the tenant data in the queries.

as you can see the pattern is: ::::

This allows me to use ACL on redis 8.0:
user compose:microservice-a on +@ALL -@admin ~compose:microservice-a:* &compose:microservice-a:* (+@READ ~compose:microservice-b:* &compose:microservice-b:) (+@READ ~compose:microservice-c: &compose:microservice-c:*) >password

With the ACL further allowing me to limit which service can access which data. It's own to read and write and others just to read. Reducing the amount of API calls needed. It will only call the API when the entry is not available in redis. If called via API the called service will add it to redis and not being called again.

Now with the current setup, I can not create such dynamic indexes. Best would be to be able to create it on the fly when it is being requested for the first time.

The login happens via OAuth2 which then in a JWT token has the information of user, tenant and permissions. so inside of the SecurityContext of jakarta.ws have the current information of the UserPrincipal.

If you want to know more or need more details, just let me know.

[bsbodden]

@berickus Please take a look at https://github.com/redis/redis-om-spring/wiki/Dynamic-Indexing-Feature-Design and review