@Override
    public boolean isDuplicate(Request request, Task task) {
		try (Jedis jedis = pool.getResource()) {
            return jedis.sadd(getSetKey(task), request.getUrl()) == 0;
        }

    }
Use Redis as url scheduler for distributed crawlers.<br>

@author code4crafter@gmail.com <br>
@since 0.2.0
webmagic-extension/src/main/java/us/codecraft/webmagic/scheduler/RedisScheduler.java