public void stopWhenComplete(){
        this.exitWhenComplete = true;
    }
Stop when all tasks in the queue are completed and all worker threads are also completed
webmagic-core/src/main/java/us/codecraft/webmagic/Spider.java